Commit graph

2666 commits

Author SHA1 Message Date
Oleksandr Byelkin
e08f87d527 fix obvious bugs hidden by current_select assigned to builtin select 2020-07-28 08:23:57 +02:00
Oleksandr Byelkin
c6eb21cd87 MDEV-21998: Server crashes in st_select_lex::add_table_to_list upon mix of KILL and sequences
Continue support the hack of current select equal builtin select if
selects stack is empty even after subselects.
2020-07-28 08:23:57 +02:00
Marko Mäkelä
e67daa5653 Merge 10.4 into 10.5 2020-07-15 14:51:22 +03:00
Marko Mäkelä
9936cfd531 Merge 10.3 into 10.4 2020-07-15 10:17:15 +03:00
Marko Mäkelä
8a0944080c Merge 10.2 into 10.3 2020-07-14 22:59:19 +03:00
Varun Gupta
b0df247db6 MDEV-22463: Element_type &Bounds_checked_array<Item *>::operator[](size_t) [Element_type = Item *]: Assertion `n < m_size' failed.
Allocate space for fields inside the window function (arguments, PARTITION BY and ORDER BY clause)
in the ref pointer array. All fields inside the window function are part of the temporary
table that is required for the window function computation.
2020-07-13 22:04:54 +05:30
Alexander Barkov
5967dfdbbf MDEV-23154 Add a data type my_repertoire_t 2020-07-13 19:50:07 +04:00
Marko Mäkelä
1813d92d0c Merge 10.4 into 10.5 2020-07-02 09:41:44 +03:00
Aleksey Midenkov
762bf7a03b MDEV-22602 Disable UPDATE CASCADE for SQL constraints
CHECK constraint is checked by check_expression() which walks its
items and gets into Item_field::check_vcol_func_processor() to check
for conformity with foreign key list.

WITHOUT OVERLAPS is checked for same conformity in
mysql_prepare_create_table().

Long uniques are already impossible with InnoDB foreign keys. See
ER_CANT_CREATE_TABLE in test case.

2 accompanying bugs fixed (test main.constraints failed):

1. check->name.str lived on SP execute mem_root while "check" obj
itself lives on SP main mem_root. On second SP execute check->name.str
had garbage data. Fixed by allocating from thd->stmt_arena->mem_root
which is SP main mem_root.

2. CHECK_CONSTRAINT_IF_NOT_EXISTS value was mixed with
VCOL_FIELD_REF. VCOL_FIELD_REF is assigned in check_expression() and
then detected as CHECK_CONSTRAINT_IF_NOT_EXISTS in
handle_if_exists_options().

Existing cases for MDEV-16932 in main.constraints cover both fixes.
2020-06-12 11:12:40 +03:00
Alexander Barkov
0bf843cd13 MDEV-20366 Server crashes in get_current_user upon SET PASSWORD via SP
The opt_for_user subrule was incorrectly scanned before sp_create_assignment_lex(),
so the user name and the host were created on a wrong memory root.

- Reoganizing the grammar to make sure that sp_create_assignment_lex()
  is called immediately after PASSWORD_SYM is scanned, so all attributes
  are then allocated on its memory root.

- Moving the semantic code as methods to LEX, so the grammar looks as simple as possible.

- Changing text_or_password to be of the data type USER_AUTH*.
  As a side effect, the LEX::definer member is now not used when processing
  the SET PASSWORD statement. Everything is done using Bison's stack.

The bug sas introduced by this commit:
commit bf5a144e16
2020-05-30 14:00:56 +04:00
Aleksey Midenkov
431a740815 MDEV-21889 IF EXISTS clause does not work for RENAME COLUMN and RENAME INDEX 2020-04-04 00:53:36 +03:00
Monty
6a9e24d046 Added support for replication for S3
MDEV-19964 S3 replication support

Added new configure options:
s3_slave_ignore_updates
"If the slave has shares same S3 storage as the master"

s3_replicate_alter_as_create_select
"When converting S3 table to local table, log all rows in binary log"

This allows on to configure slaves to have the S3 storage shared or
independent from the master.

Other thing:
Added new session variable '@@sql_if_exists' to force IF_EXIST to DDL's.
2020-03-24 21:00:02 +02:00
Sergei Golubchik
c1c5222cae cleanup: PSI key is *always* the first argument 2020-03-10 19:24:23 +01:00
Sergei Golubchik
70e7b5095d perfschema sp instrumentation related changes 2020-03-10 19:24:23 +01:00
Sergei Golubchik
7c58e97bf6 perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00
Sergei Petrunia
0a9633ee62 Basic LEX::print function that supports UPDATEs 2020-03-07 01:26:28 +03:00
Aleksey Midenkov
fa8ad75439 MDEV-16290 ALTER TABLE ... RENAME COLUMN syntax
The existing syntax for renaming a column uses "ALTER TABLE ...  CHANGE"
command. This requires full column specification to rename the column.
This patch adds new syntax "ALTER TABLE ...  RENAME COLUMN", which do not
expect users to provide full column specification.  It means that the new
syntax would pick in-place or copy algorithm in the same way as that of
existing "ALTER TABLE ... CHANGE" command. The existing syntax
"ALTER TABLE ... CHANGE" will continue to work.

Syntax changes
==============

ALTER TABLE tbl_name
    [alter_specification [, alter_specification] ...]
    [partition_options]

Following is a new <alter_specification> added:

 | RENAME COLUMN <oldname> TO <newname>

Where <oldname> and <newname> are identifiers for old name and new
name of the column.

Related to: WL#10761
2020-03-03 13:50:32 +03:00
Alexander Barkov
83e75b39b3 MDEV-21702 Add a data type for privileges 2020-02-11 08:10:26 +04:00
Alexander Barkov
77c6382312 MDEV-21689 Add Sql_cmd for GRANT/REVOKE statements
Rewriting GRANT/REVOKE grammar to use more bison stack and use Sql_cmd_ style

1. Removing a few members from LEX:
   - uint grant, grant_to_col, which_columns
   - List<LEX_COLUMN> columns
   - bool all_privileges
2. Adding classes Grand_object_name, Lex_grant_object_name
3. Adding classes Grand_privilege, Lex_grand_privilege
4. Adding struct Lex_column_list_privilege_st, class Lex_column_list_privilege
5. Rewriting the GRANT/REVOKE grammar to use new classes and pass them through
   bison stack (rather than directly access LEX members)
6. Adding classes Sql_cmd_grant* and Sql_cmd_revoke*,
   changing GRANT/REVOKE to use LEX::m_sql_cmd.
7. Adding the "sp_handler" grammar rule and removing some duplicate grammar
   for GRANT/REVOKE for different kinds of SP objects.
8. Adding a new rule comma_separated_ident_list, reusing it in:
   - with_column_list
   - colum_list_privilege
2020-02-08 21:35:35 +04:00
Marko Mäkelä
8cc15c036d Merge 10.4 into 10.5 2019-12-27 21:17:16 +02:00
Marko Mäkelä
4c25e75ce7 Merge 10.3 into 10.4 2019-12-27 18:20:28 +02:00
Marko Mäkelä
5ab70e7f68 Merge 10.2 into 10.3 2019-12-27 15:14:48 +02:00
Oleksandr Byelkin
cc28947315 MDEV-20632: prerequisite:
Removed hack with with_list
2019-12-27 10:49:52 +01:00
Marko Mäkelä
73985d8301 Merge 10.1 into 10.2 2019-12-23 07:14:51 +02:00
Sergei Petrunia
8b9db11718 MDEV-21341: Fix optimizer-related UBSAN failures, part #2
Remove Query_tables_list::lock_tables_state - it is not used and it causes
errors like this:

sql_lex.h:1675:7: runtime error: load of value 2779096485, which is not a
valid value for type 'enum_lock_tables_state'
2019-12-18 12:56:54 +03:00
Aleksey Midenkov
0c05a2ed71 Merge 10.4 into 10.5 2019-11-25 17:24:09 +03:00
Aleksey Midenkov
33f55789d3 MDEV-18727 improve DML operation of System Versioning (10.4)
UPDATE, DELETE: replace linear search of current/historical records
with vers_setup_conds().

Additional DML cases in view.test
2019-11-25 16:01:43 +03:00
Aleksey Midenkov
daabc5cc68 MDEV-20729 Fix REFERENCES constraint in column definition
Add support of referential constraints directly in column defininions:

create table t1 (id1 int primary key);
create table t2 (id2 int references t1(id1));

Referenced field name can be omitted if equal to foreign field name:

create table t1 (id int primary key);
create table t2 (id int references t1);

Until 10.5 this syntax was understood by the parser but was silently
ignored.

In case of generated columns this syntax is disabled at parser level
by ER_PARSE_ERROR. Note that separate FOREIGN KEY clause for generated
columns is disabled at storage engine level.
2019-11-20 13:18:54 +03:00
Alexander Barkov
251c6e1726 MDEV-21073 Collect different grammar rules into a single chunk
On order to unify the two *.yy files easier,
this patch collects all different rules to the end of *.yy files,
so the rule section looks like this:

%%
common rules
different rules
2019-11-19 08:06:04 +04:00
Alexander Barkov
6d373e8b81 MDEV-21064 Add a new class sp_expr_lex and a new grammar rule expr_lex
Adding:
- new class sp_expr_lex
- new grammar rule expr_lex, which includes both reset_lex()
  and its corresponding restore_lex()

Also:
- Moving a few methods from LEX to sp_expr_lex.
- Moving the code from *.yy to new method sp_expr_lex methods
  sp_repeat_loop_finalize() and sp_if_expr().

This change makes it easier to edit the related grammar
(and makes it easier to unify sql_yacc.yy and sql_yacc_ora.yy later).
2019-11-16 08:32:15 +04:00
Oleksandr Byelkin
3ad37ed0eb Merge 10.4 into 10.5 2019-11-07 08:52:30 +01:00
Alexander Barkov
46f2f24ec4 MDEV-20985 Add LEX methods stmt_drop_{function|procedure}() and stmt_alter_{function|procedure}_start()
Adding a few helper LEX methods,
to unify sql_yacc.yy and sql_yacc_ora.yy easier
2019-11-05 22:37:45 +04:00
Marko Mäkelä
ec40980ddd Merge 10.3 into 10.4 2019-11-01 15:23:18 +02:00
Alexander Barkov
f2cff12556 MDEV-20924 Unify grammar rules: field_type_string and sp_param_field_type_string
In order to:
- unify sql_yacc.yy and sql_yacc_ora.yy easier
- move more functionality from the parser to Type_handler
  (so plugins can override the behavior)

this patch:
- removes rules sp_param_field_type_string and sp_param_field_type
  from sql_yacc_ora.yy
- adds a new virtial method Type_handler::Column_definition_set_attributes()
2019-10-30 11:58:52 +04:00
Michael Widenius
716d396bb3 Remove \n from DBUG_PRINT statements 2019-10-21 18:41:58 +03:00
Monty
b62101f84b Fixes for binary logging --read-only mode
- Any temporary tables created under read-only mode will never be logged
  to binary log.  Any usage of these tables to update normal tables, even
  after read-only has been disabled, will use row base logging (as the
  temporary table will not be on the slave).
- Analyze, check and repair table will not be logged in read-only mode.

Other things:
- Removed not used varaibles in
  MYSQL_BIN_LOG::flush_and_set_pending_rows_event.
- Set table_share->table_creation_was_logged for all normal tables.
- THD::binlog_query() now returns -1 if statement was not logged., This
  is used to update table_share->table_creation_was_logged.
- Don't log admin statements in opt_readonly is set.
- Table's that doesn't have table_creation_was_logged will set binlog format to row
  logging.
- Removed not needed/wrong setting of table->s->table_creation_was_logged
  in create_table_from_items()
2019-10-20 11:52:29 +03:00
Oleksandr Byelkin
de2186dd2f MDEV-20074: Lost connection on update trigger
Instead of checking lex->sql_command which does not corect in case of triggers
mark tables for insert.
2019-10-17 17:32:14 +02:00
Rucha Deodhar
837ad9ab97 MDEV-10014 Add RETURNING to INSERT
Closes #1384
2019-10-14 10:29:31 +02:00
Sergei Golubchik
57a09a72a3 cleanup st_select_lex_unit::explainable 2019-10-14 10:29:31 +02:00
Sergei Golubchik
721a9df751 cleanup: formatting
comments, whitespaces
2019-10-14 10:29:31 +02:00
Oleksandr Byelkin
1ae02f0e0d MDEV-18553: MDEV-16327 pre-requisits part 2: uniform of LIMIT/OFFSET handling
Now both offset and limit are stored and do not chenged during execution
(offset is decreased during processing in versions before 10.5).

(Big part of this changes made by Monty)
2019-10-13 09:40:41 +02:00
Oleksandr Byelkin
eb0804ef5e MDEV-18553: MDEV-16327 pre-requisits part 1: isolation of LIMIT/OFFSET handling 2019-10-13 09:40:41 +02:00
Alexander Barkov
d168601e83 MDEV-20734 Allow reserved keywords as user defined type names 2019-10-03 14:02:00 +04:00
Alexander Barkov
edef6a0074 Merge remote-tracking branch 'origin/10.4' into 10.5 2019-09-24 12:53:11 +04:00
Marko Mäkelä
1333da90b5 Merge 10.4 into 10.5 2019-09-24 10:07:56 +03:00
Igor Babaev
b44171428a MDEV-19956 Queries with subqueries containing UNION are not parsed
Shift-Reduce conflicts prevented parsing some queries with subqueries that
used set operations when the subqueries occurred in expressions or in IN
predicands.
The grammar rules for query expression were transformed in order to avoid
these conflicts. New grammar rules employ an idea taken from MySQL 8.0.
2019-09-23 09:57:37 -07:00
Alexander Barkov
2f88bd2da2 MDEV-20634 Report disallowed subquery errors as such (instead of parse error) 2019-09-20 10:36:20 +04:00
WayneXia
a896bebfa6 MDEV-18844 Implement EXCEPT ALL and INTERSECT ALL operations 2019-08-24 21:44:41 +08:00
Alexander Barkov
c1599821a5 Merge remote-tracking branch 'origin/10.4' into 10.5 2019-08-13 23:49:10 +04:00
Oleksandr Byelkin
2792c6e7b0 Merge branch '10.3' into 10.4 2019-07-28 13:43:26 +02:00
Oleksandr Byelkin
d97342b6f2 Merge branch '10.2' into 10.3 2019-07-26 22:42:35 +02:00
Oleksandr Byelkin
cf8c2a3c3b Merge branch '10.1' into 10.2 2019-07-26 07:03:39 +02:00
Oleksandr Byelkin
ae476868a5 Merge branch '5.5' into 10.1 2019-07-25 13:27:11 +02:00
Gleb Shchepa
7473a71a28 Bug #29419820: MEMORY LEAK IN MY_YYOVERFLOW()
Note: this patch is for 5.6.

Detected by ASAN.

The patch fixes the cleanup of parser stack pointers.

Reviewed-by: Guilhem Bichot <guilhem.bichot@oracle.com>
2019-07-24 18:32:24 +02:00
Igor Babaev
8540fa83bb MDEV-19421 Basic 3-way join queries are not parsed.
The parser returned a syntax error message for the queries with join
expressions like this t1 JOIN t2 [LEFT | RIGHT] JOIN t3 ON ... ON ... when
the second operand of the outer JOIN operation with ON clause was another
join expression with ON clause. In this expression the JOIN operator is
right-associative, i.e. expression has to be parsed as the expression
t1 JOIN (t2 [LEFT | RIGHT] JOIN t3 ON ... ) ON ...
Such join expressions are hard to parse because the outer JOIN is
left-associative if there is no ON clause for the first outer JOIN operator.
The patch implements the solution when the JOIN operator is always parsed
as right-associative and builds first the right-associative tree. If it
happens that there is no corresponding ON clause for this operator the
tree is converted to left-associative.

The idea of the solution was taken from the patch by Martin Hansson
"WL#8083: Fixed the join_table rule" from MySQL-8.0 code line.
As the grammar rules related to join expressions in MySQL-8.0 and
MariaDB-5.5+ are quite different MariaDB solution could not borrow
any code from the MySQL-8.0 solution.
2019-07-11 13:39:21 -07:00
Alexander Barkov
4dc85973b4 MDEV-19994 Add class Function_collection 2019-07-09 16:18:54 +04:00
Alexander Barkov
bf5a144e16 MDEV-19639 + MDEV-19640 fix + preparatory changes for WL#4179
This patch includes:
- MDEV-19639 sql_mode=ORACLE: Wrong SHOW PROCEDURE output for sysvar:=expr
- MDEV-19640 Wrong SHOW PROCEDURE output for SET GLOBAL sysvar1=expr, sysvar2=expr
- Preparatory refactoring for MySQL WL#4179

Detailed change list:

1. Changing sp_create_assignment_lex() to accept the position
   in the exact query buffer instead of a "bool no_lookahead".
   This actually fixes MDEV-19639.
   In the previous reduction sp_create_assignment_lex() was
   called too late, when the parser went far the from beginning
   of the statement, so only a part of the statement got into
   sp_instr_stmt.

2. Generating "SET" or "SET GLOBAL" inside sp_create_assignment_instr()
   depending on the option type.
   This fixes MDEV-19640.
   In the previous reduction the code passed (through no_lookahead)
   the position of the
   word GLOBAL inside sp_create_assignment_lex(), which
   worked only for the left-most assignment.

3. Fixing the affected rules to use:
   - ident_cli instead of ident
   - ident_cli_set_usual_case instead of ident_set_usual_case

4. Changing the input parameter in:
   - LEX::set_system_variable()
   - LEX::call_statement_start()
   - LEX::set_variable()
   from just LEX_CSTRING to Lex_ident_sys_st for stricter data type constrol:
   to make sure that noone passes an ident_cli
   (a fragment of the original query in the client character set)
   instead of server-side identifier
   (utf8 identifier allocated on THD when needed).

5. Adding Lex_ident_sys() in places where the affected functions are called.

6. Moving all calls of sp_create_assignment_lex() to the places
   just before parsing set_expr_or_default.
   This makes the grammar clearer, because
   sp_create_assignment_lex() and sp_create_assignment_instr()
   now stay near each other, so the balance of LEX's push/pop
   can be read easier.
   This will also help to WL#4179.

7. Adding class sp_lex_set_var
   Moving the initialization code from
   sp_create_assignment_lex() to the constructor of sp_lex_set_var.
   This will also help to WL#4179.

8. Moving a part of the "set" grammar rule into a separate
   rule "set_param".
   This makes the grammar easier to read and removes
   one shift/reduce conflict.
2019-06-04 06:21:22 +04:00
Marko Mäkelä
28fad39de7 Merge 10.4 into 10.5 2019-05-29 22:29:05 +03:00
Marko Mäkelä
f98bb23168 Merge 10.3 into 10.4 2019-05-29 22:17:00 +03:00
Marko Mäkelä
90a9193685 Merge 10.2 into 10.3 2019-05-29 11:32:46 +03:00
Alexander Barkov
9f23f8e598 MDEV-19599 Change db_name, table_name to LEX_CSTRING in Item_ident and Send_field 2019-05-26 09:49:13 +04:00
Alexander Barkov
8164bd24a6 MDEV-19535 sql_mode=ORACLE: 'SELECT INTO @var FOR UPDATE' does not lock the table 2019-05-21 14:22:49 +04:00
Alexander Barkov
ed39181a27 MDEV-19533 Add methods make() and append_uniq() to Row_definition_list 2019-05-21 12:30:21 +04:00
Sujatha
5a2110e7cf MDEV-19076: rpl_parallel_temptable result mismatch '-33 optimistic'
Problem:
========
The test now fails with the following trace:

CURRENT_TEST: rpl.rpl_parallel_temptable
--- /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.result
+++ /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.reject
@@ -194,7 +194,6 @@
 30    conservative
 31    conservative
 32    optimistic
-33    optimistic

Analysis:
=========
The part of test which fails with result content mismatch is given below.

CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t4 VALUES (32);
INSERT INTO t4 VALUES (33);
INSERT INTO t1 SELECT a, "optimistic" FROM t4;

slave_parallel_mode=optimistic

The expectation of the above test script is, INSERT FROM SELECT should read both
32, 33 and populate table 't1'. But this expectation fails occasionally.

All three INSERT statements are handed over to three different slave parallel
workers. Temporary tables are not safe for parallel replication. They were
designed to be visible to one thread only, so have no table locking.  Thus there
is no protection against two conflicting transactions committing in parallel and
things like that.

So anything that uses temporary tables will be serialized with anything before
it, when using parallel replication by using a "wait_for_prior_commit" function
call. This will ensure that the each transaction is executed sequentially.

But there exists a code path in which the above wait doesn't happen.  Because of
this at times INSERT from SELECT doesn't wait for the INSERT (33) to complete
and it completes its executes and enters commit stage.  Hence only row 32 is
found in those cases resulting in test failure.

The wait needs to be added within "open_temporary_table" call. The code looks
like this within "open_temporary_table".

Each thread tries to open temporary table in 3 different ways:

case 1: Find a temporary table which is already in use by using
         find_temporary_table(tl) && wait_for_prior_commit()
case 2: If above failed then try to look for temporary table which is marked for
        free for reuse. This internally calls "wait_for_prior_commit()" if table
        is found.
         find_and_use_tmp_table(tl, &table)
case 3: If none of the above open a new table handle from table share.
         if (!table && (share= find_tmp_table_share(tl)))
         { table= open_temporary_table(share, tl->get_table_name(), true); }

At present the "wait_for_prior_commit" happens only in case 1 & 2.

Fix:
====
On slave add a call for "wait_for_prior_commit" for case 3.

The above wait on slave will solve the issue. A more detailed fix would be to
mark temporary tables as not safe for parallel execution on the master side.
In order to do that, on the master side, mark the Gtid_log_event specific flag
FL_TRANSACTIONAL to be false all the time. So that they are not scheduled
parallely.
2019-05-20 15:46:26 +05:30
Oleksandr Byelkin
dafe41edea Removing of dead code. 2019-05-20 09:38:08 +02:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Oleksandr Byelkin
c51f85f882 Merge branch '10.2' into 10.3 2019-05-12 17:20:23 +02:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Vicențiu Ciorbaru
5543b75550 Update FSF Address
* Update wrong zip-code
2019-05-11 21:29:06 +03:00
Igor Babaev
09aa5d3f69 MDEV-17894 Assertion `(thd->lex)->current_select' failed in MYSQLparse(),
query with VALUES()

A table value constructor can be used in all contexts where a select
can be used. In particular an ORDER BY clause or a LIMIT clause or both
of them can be attached to a table value constructor to produce a new
query. Unfortunately execution of such queries was not supported.
This patch fixes the problem.
2019-05-08 09:45:24 -07:00
Marko Mäkelä
d3dcec5d65 Merge 10.3 into 10.4 2019-05-05 15:06:44 +03:00
Oleksandr Byelkin
8cbb14ef5d Merge branch '10.1' into 10.2 2019-05-04 17:04:55 +02:00
Varun Gupta
cb9fa1a08b MDEV-9959: A serious MariaDB server performance bug
If a derived table has SELECT DISTINCT, provide index statistics for it so that the join optimizer in the
upper select knows that ref access to the table will produce one row.
2019-04-30 21:07:25 +05:30
Sergei Golubchik
2ce52790ff Merge branch '5.5' into 10.1 2019-04-26 14:02:37 +02:00
Sergei Golubchik
5057d46375 bugfix: multi-update checked privileges on views incorrectly
it always required UPDATE privilege on views, not being able to detect
when a views was not actually updated in multi-update.

fix: instead of marking all tables as "updating" by default,
only set "updating" on tables that will actually be updated
by multi-update. And mark the view "updating" if any of the
view's tables is.
2019-04-24 11:15:38 +02:00
Alexander Barkov
c2a2e72164 MDEV-19142 sql_mode=MSSQL: Bracket identifiers 2019-04-15 09:39:28 +04:00
Igor Babaev
d2013e7328 MDEV-18982 Partition pruning with column list causes syntax error in 10.4
A syntax error was reported for any INSERT statement with explicit
partition selection it if i used a column list.
Fixed by saving the parsing place before parsing the clause for explicit
partition selection and restoring it when the clause has been parsed.
2019-04-04 16:36:26 -07:00
Galina Shalygina
ae15f91f22 MDEV-18769 Assertion `fixed == 1' failed in Item_cond_or::val_int
This bug is caused by pushdown from HAVING into WHERE.
    It appears because condition that is pushed wasn't fixed.

    It is also discovered that condition pushdown from HAVING into
    WHERE is done wrong. There is no need to build clones for some
    conditions that can be pushed. They can be simply moved from HAVING
    into WHERE without cloning.
    build_pushable_cond_for_having_pushdown(),
    remove_pushed_top_conjuncts_for_having() methods are changed.

    It is found that there is no transformation made for fields of
    pushed condition.
    field_transformer_for_having_pushdown transformer is added.

    New tests are added. Some comments are changed.
2019-04-04 18:06:56 +03:00
Alexander Barkov
2d825e97df Cleanup: removing unused type LEX_TYPE and #include 2019-04-01 10:09:24 +04:00
Marko Mäkelä
8b480df63e Merge 10.3 into 10.4 2019-03-25 17:18:15 +02:00
Eugene Kosov
53216091dd remove dead code 2019-03-22 22:33:27 +04:00
Eugene Kosov
a523444eda remove dead code 2019-03-22 22:30:36 +04:00
Sergey Vojtovich
3568427d11 MDEV-18450 Slaves wait shutdown
The patches features an optional shutdown behavior to hold on until
after all connected slaves have been sent the last binlogged event.
The connected slave is one whose START SLAVE has been acknowledged and
that was not stopped since that though it could be technically
reconnecting in background.

The solution therefore disallows killing the dump thread until is has
found EOF of the latest binlog file.  It is up to the shutdown
requester (DBA) to set up a sufficiently large shutdown timeout value
for shudown to wait patiently until lagging behind slaves have been
synchronized. On the other hand if a specific slave needs exclusion
from synchronization the DBA would have to stop it manually which
would terminate its dump thread.

`mysqladmin shutdown' is extended with a `--wait_for_all_slaves' option
which translates to `SHUTDOW WAIT FOR ALL SLAVES' sql query
to enable the feature on the client side.

The patch also performs a small refactoring of the server shutdown
around close_connections() to introduce kill thread phases which
are two as of current.
2019-03-12 17:34:48 +02:00
Oleksandr Byelkin
6d68a3464e MDEV-18701: Wrong result from query that uses INTERSECT after UNION ALL
Recalculate distinct pointer if we cut chain of SELECTs
2019-03-11 17:41:00 +01:00
Alexander Barkov
5f34513c2a MDEV-18813 PROCEDURE and anonymous blocks silently ignore FETCH GROUP NEXT ROW
Part#2 (final): rewritting the code to pass the correct enum_sp_aggregate_type
to the sp_head constructor, so sp_head never changes its aggregation type
later on. The grammar has been simplified and defragmented.
This allowed to check aggregate specific instructions right after
a routine body has been scanned, by calling new LEX methods:
  sp_body_finalize_{procedure|function|trigger|event}()

Moving some C++ code from *.yy to a few new helper methods in LEX.
2019-03-07 18:48:15 +04:00
Alexander Barkov
a71d185a9a MDEV-18813 PROCEDURE and anonymous blocks silently ignore FETCH GROUP NEXT ROW
Part#1: moving opt_if_not_exists from "sf_tail" and "sp_tail" to "create".
2019-03-07 14:35:01 +04:00
Marko Mäkelä
2a791c53ad Merge 10.3 into 10.4 2019-03-06 09:00:52 +02:00
Marko Mäkelä
446b3ebdfc Merge 10.2 into 10.3
FIXME: Properly resolve conflicts between MDEV-18883
and MDEV-7742/MDEV-8305, and record the correct result for
main.log_slow
2019-03-05 12:56:05 +02:00
Marko Mäkelä
9835f7b80f Merge 10.1 into 10.2 2019-03-04 16:46:58 +02:00
Alexander Barkov
1bcb66c597 MDEV-18806 Synchronize ALTER TABLE EXCHANGE PARTITION and PURGE grammar in sql_yacc.yy and sql_yacc_ora.yy 2019-03-04 14:38:21 +04:00
Alexander Barkov
47b7ca629f MDEV-18796 Synchronize PS grammar between sql_yacc.yy and sql_yacc_ora.yy 2019-03-03 07:20:15 +04:00
Alexander Barkov
3b47587f41 MDEV-18789 Port "MDEV-7773 Aggregate stored functions" to sql_yacc_ora.yy 2019-03-01 23:55:55 +04:00
Oleksandr Byelkin
f2e1451740 Merge branch '10.0' into 10.1 2019-03-01 15:52:06 +01:00
Alexander Barkov
01d7727b76 MDEV-18779 Port sp_suid implementation from sql_yacc_ora.yy to sql_yacc.yy 2019-03-01 17:02:58 +04:00
Oleksandr Byelkin
7b5c63856b Merge branch '5.5' into 10.0 2019-02-28 21:50:00 +01:00
Oleksandr Byelkin
cb11b3fbe9 MDEV-17055: Server crashes in find_order_in_list upon 2nd (3rd) execution of SP with UPDATE
1. Always drop merged_for_insert flag on cleanup (there could be errors which prevent TABLE to be assigned)
2. Make more precise cleanup of select parts which was touched
2019-02-28 18:14:53 +01:00
Alexander Barkov
0ad598a00b A cleanup in derived table handling: removing duplicate code from st_select_lex::handle_derived()
st_select_lex::handle_derived() and mysql_handle_list_of_derived() had
exactly the same implementations.

- Adding a new method LEX::handle_list_of_derived() instead
- Removing public function mysql_handle_list_of_derived()
- Reusing LEX::handle_list_of_derived() in st_select_lex::handle_derived()
2019-02-28 18:13:28 +04:00
Alexander Barkov
09d29dfc76 MDEV-18767 Port "MDEV-16294: INSTALL PLUGIN IF NOT EXISTS / UNINSTALL PLUGIN IF EXISTS" to sql_yacc_ora.yy 2019-02-28 07:17:47 +04:00
Sergei Golubchik
1e6210161d MDEV-7597 Expiration of user passwords
post-merge changes:
* handle password expiration on old tables like everything else -
  make changes in memory, even if they cannot be done on disk
* merge "debug" tests with non-debug tests, they don't use dbug anyway
* only run rpl password expiration in MIXED mode, it doesn't replicate
  anything, so no need to repeat it thrice
* restore update_user_table_password() prototype, it should not change
  ACL_USER, this is done in acl_user_update()
* don't parse json twice in get_password_lifetime and get_password_expired
* remove LEX_USER::is_changing_password, see if there was any auth instead
* avoid overflow in expiration calculations
* don't initialize Account_options in the constructor, it's bzero-ed later
* don't create ulong sysvars - they're not portable, prefer uint or ulonglong
* misc simplifications
2019-02-21 15:04:03 +01:00
Robert Bindar
90ad4dbd17 MDEV-7597 Expiration of user passwords
This patch adds support for expiring user passwords.
The following statements are extended:
  CREATE USER user@localhost PASSWORD EXPIRE [option]
  ALTER USER user@localhost PASSWORD EXPIRE [option]
If no option is specified, the password is expired with immediate
effect. If option is DEFAULT, global policy applies according to
the default_password_lifetime system var (if 0, password never
expires, if N, password expires every N days). If option is NEVER,
the password never expires and if option is INTERVAL N DAY, the
password expires every N days.
The feature also supports the disconnect_on_expired_password system
var and the --connect-expired-password client option.

Closes #1166
2019-02-21 15:04:03 +01:00
Sergei Golubchik
81e4b9b3bb misc cleanups 2019-02-21 14:57:10 +01:00
Nikita Malyavin
6294516a56 MDEV-16975 Application-time periods: ALTER TABLE
* implicit period constraint is hidden and cannot be dropped independently
* create...like and create...select support
2019-02-21 14:57:09 +01:00
Nikita Malyavin
47e28a94d5 MDEV-16973 Application-time periods: DELETE
* inject portion of time updates into mysql_delete main loop
* triggered case emits delete+insert, no updates
* PORTION OF `SYSTEM_TIME` is forbidden
* `DELETE HISTORY .. FOR PORTION OF ...` is forbidden as well
2019-02-21 14:48:04 +01:00
Nikita Malyavin
073c93b194 MDEV-17082 Application-time periods: CREATE
* add syntax `CREATE TABLE ... PERIOD FOR <apptime>`
* add table period entity
2019-02-21 14:48:04 +01:00
Oleksandr Byelkin
93ac7ae70f Merge branch '10.3' into 10.4 2019-02-21 14:40:52 +01:00
Igor Babaev
4de3fd4ea2 MDEV-7486: Cosmetic changes
- Removed dead code
- Renamed a function
- Removed a parameter that not needed.
- Corrected comments
2019-02-18 22:11:25 -08:00
Galina Shalygina
7a77b221f1 MDEV-7486: Condition pushdown from HAVING into WHERE
Condition can be pushed from the HAVING clause into the WHERE clause
if it depends only on the fields that are used in the GROUP BY list
or depends on the fields that are equal to grouping fields.
Aggregate functions can't be pushed down.

How the pushdown is performed on the example:

SELECT t1.a,MAX(t1.b)
FROM t1
GROUP BY t1.a
HAVING (t1.a>2) AND (MAX(c)>12);

=>

SELECT t1.a,MAX(t1.b)
FROM t1
WHERE (t1.a>2)
GROUP BY t1.a
HAVING (MAX(c)>12);

The implementation scheme:

1. Extract the most restrictive condition cond from the HAVING clause of
   the select that depends only on the fields that are used in the GROUP BY
   list of the select (directly or indirectly through equalities)
2. Save cond as a condition that can be pushed into the WHERE clause
   of the select
3. Remove cond from the HAVING clause if it is possible

The optimization is implemented in the function
st_select_lex::pushdown_from_having_into_where().

New test file having_cond_pushdown.test is created.
2019-02-17 23:38:44 -08:00
Alexander Barkov
62c0ac2da6 A cleanup for MDEV-13916 Enforce check constraint on JSON type
1. Renaming Type_handler_json to Type_handler_json_longtext
   There will be other JSON handlers soon, e.g. Type_handler_json_varchar.

2. Making the code more symmetric for data types:

- Adding a new virtual method
  Type_handler::Column_definition_validate_check_constraint()
- Moving JSON-specific code from sql_yacc.yy to
  Type_handler_json_longtext::Column_definition_validate_check_constraint()

3. Adding new files sql_type_json.cc and sql_type_json.h
   and moving Type_handler+JSON related code into these files.
2019-02-15 19:46:58 +04:00
Sergei Golubchik
e5701d8363 cleanup: Account_options
move account options from LEX to Account_options structure
namely, mqh and ssl_*

Also, use LEX_CSTRING for ssl_*/x509_* strings and move
setting of ACL_USER::account_locked where it belongs
2019-02-14 17:30:22 +01:00
Robert Bindar
6c8ce999f8 MDEV-13095 Implement User Account locking
Add server support for user account locking.
This patch extends the ALTER/CREATE USER statements for
denying a user's subsequent login attempts:
  ALTER USER
    user [, user2] ACCOUNT [LOCK | UNLOCK]
  CREATE USER
    user [, user2] ACCOUNT [LOCK | UNLOCK]
The SHOW CREATE USER statement was updated to display the
locking state of an user.

Closes #1006
2019-02-14 17:30:22 +01:00
Igor Babaev
7d9f45e072 Merge branch '10.4' into bb-10.4-mdev17096 2019-02-13 14:59:34 -08:00
Monty
0f48949439 MDEV-13916 Enforce check constraint on JSON type
When creating a field of type JSON, it will be automatically
converted to TEXT with CHECK (json_valid(`a`)), if there wasn't any
previous check for the column.

Additional things:
- Added two bug fixes that was found while testing JSON. These bug
  fixes has also been pushed to 10.3 (with a test case), but as they
  where minimal and needed to get this task done and tested, the fixes
  are repeated here.
  - CREATE TABLE ... SELECT drops constraints for columns that
    are both in the create and select part.
  - If one has both a default expression and check constraint for a
    column, one can get the error "Expression for field `a` is refering
    to uninitialized field `a`.
- Removed some duplicate MYSQL_PLUGIN_IMPORT symbols
2019-02-13 19:40:26 +02:00
Igor Babaev
27c3abde30 MDEV-17096 Pushdown of simple derived tables to storage engines
MDEV-17631 select_handler for a full query pushdown

Added comments and file headers for files introduced in these tasks.
2019-02-12 22:56:24 -08:00
Oleksandr Byelkin
65c5ef9b49 dirty merge 2019-02-07 13:59:31 +01:00
Igor Babaev
3f9040085a Merge branch '10.4' into bb-10.4-mdev17096 2019-02-06 18:01:29 -08:00
Igor Babaev
16327fc2e7 MDEV-17096 Pushdown of simple derived tables to storage engines
MDEV-17631 select_handler for a full query pushdown

Interfaces + Proof of Concept for federatedx with test cases.

The interfaces have been developed for integration of ColumnStore engine.
2019-02-06 17:02:44 -08:00
Marko Mäkelä
081fd8bfa2 Merge 10.1 into 10.2 2019-02-02 11:40:02 +02:00
Andrei Elkin
5d48ea7d07 MDEV-10963 Fragmented BINLOG query
The problem was originally stated in
  http://bugs.mysql.com/bug.php?id=82212
The size of an base64-encoded Rows_log_event exceeds its
vanilla byte representation in 4/3 times.
When a binlogged event size is about 1GB mysqlbinlog generates
a BINLOG query that can't be send out due to its size.

It is fixed with fragmenting the BINLOG argument C-string into
(approximate) halves when the base64 encoded event is over 1GB size.
The mysqlbinlog in such case puts out

    SET @binlog_fragment_0='base64-encoded-fragment_0';
    SET @binlog_fragment_1='base64-encoded-fragment_1';
    BINLOG @binlog_fragment_0, @binlog_fragment_1;

to represent a big BINLOG.
For prompt memory release BINLOG handler is made to reset the BINLOG argument
user variables in the middle of processing, as if @binlog_fragment_{0,1} = NULL
is assigned.

Notice the 2 fragments are enough, though the client and server still may
need to tweak their @@max_allowed_packet to satisfy to the fragment
size (which they would have to do anyway with greater number of
fragments, should that be desired).

On the lower level the following changes are made:

Log_event::print_base64()
  remains to call encoder and store the encoded data into a cache but
  now *without* doing any formatting. The latter is left for time
  when the cache is copied to an output file (e.g mysqlbinlog output).
  No formatting behavior is also reflected by the change in the meaning
  of the last argument which specifies whether to cache the encoded data.

Rows_log_event::print_helper()
  is made to invoke a specialized fragmented cache-to-file copying function
  which is

copy_cache_to_file_wrapped()
  that takes care of fragmenting also optionally wraps encoded
  strings (fragments) into SQL stanzas.

my_b_copy_to_file()
  is refactored to into my_b_copy_all_to_file(). The former function
  is generalized
  to accepts more a limit argument to constraint the copying and does
  not reinitialize anymore the cache into reading mode.
  The limit does not do any effect on the fully read cache.
2019-01-24 20:44:50 +02:00
Vladislav Vaintroub
4bcd7da076 fixup! Fixed compiler warnings from optimized builds 2019-01-15 16:44:13 +01:00
Michael Widenius
9990027f87 Fixed compiler warnings from optimized builds 2019-01-15 17:05:42 +02:00
Monty
c53aab974b Added syntax and implementation for BACKUP STAGE's
Part of MDEV-5336 Implement LOCK FOR BACKUP

- Changed check of Global_only_lock to also include BACKUP lock.
- We store latest MDL_BACKUP_DDL lock in thd->mdl_backup_ticket to be able
  to downgrade lock during copy_data_between_tables()
2018-12-09 22:12:27 +02:00
Marko Mäkelä
dde2ca4aa1 Merge 10.3 into 10.4 2018-11-19 20:22:33 +02:00
Alexander Barkov
62bcd74712 MDEV-17694 Add method LEX::sp_proc_stmt_statement_finalize() 2018-11-14 07:38:28 +04:00
Alexander Barkov
2a0b6de41b MDEV-17253 Oracle compatibility: The REVERSE key word for FOR loop behaves incorrectly 2018-11-13 18:03:14 +04:00
Alexander Barkov
f5855ba03d MDEV-17664 Add sql_mode specific tokens for ':' and '%' 2018-11-11 09:35:05 +04:00
Marko Mäkelä
c761b43451 Merge 10.3 into 10.4 2018-11-08 10:19:55 +02:00
Oleksandr Byelkin
54b2e1c1be MDEV-16697: Fix difference between 32bit/windows and 64bit systems in allowed select nest level 2018-11-07 09:43:12 +01:00
Marko Mäkelä
074c684099 Merge 10.3 into 10.4 2018-11-06 16:24:16 +02:00
Marko Mäkelä
df563e0c03 Merge 10.2 into 10.3
main.derived_cond_pushdown: Move all 10.3 tests to the end,
trim trailing white space, and add an "End of 10.3 tests" marker.
Add --sorted_result to tests where the ordering is not deterministic.

main.win_percentile: Add --sorted_result to tests where the
ordering is no longer deterministic.
2018-11-06 09:40:39 +02:00
Marko Mäkelä
32062cc61c Merge 10.1 into 10.2 2018-11-06 08:41:48 +02:00
Andrei Elkin
d8974ebd67 MDEV-14431 binlog.binlog_flush_binlogs_delete_domain failed in buildbot
The test and also rpl_gtid_delete_domain failed on PPC64 platform
due to an incorrectly specified actual key for searching
in a gtid domain system hash. While the correct size is 32 bits
the supplied value was 8 bytes of long int size on the platform.
The problem became evident thanks to the big endiness which
cut off the *least* significant part of the value field.

Fixed with correcting a dynamic array initialization to hold
now uint32 values as well as the values extraction for
searching in the gtid domain system hash.
A new added test ensures no overflowed values are accepted
for deletion which prevents inadvertent action. Notice though

MariaDB [test]> set @@session.gtid_domain_id=(1 << 32) + 1;
MariaDB [test]> show warnings;
+---------+------+--------------------------------------------------------+
| Level   | Code | Message                                                |
+---------+------+--------------------------------------------------------+
| Warning | 1292 | Truncated incorrect gtid_domain_id value: '4294967297' |
+---------+------+--------------------------------------------------------+
MariaDB [test]> select @@session.gtid_domain_id;
+--------------------------+
| @@session.gtid_domain_id |
+--------------------------+
|               4294967295 |
+--------------------------+
2018-10-16 12:42:57 +03:00
Marko Mäkelä
444c380ceb Merge 10.3 into 10.4 2018-10-05 08:09:49 +03:00
Sergei Golubchik
57e0da50bb Merge branch '10.2' into 10.3 2018-09-28 16:37:06 +02:00
Igor Babaev
3473e0452e MDEV-17154 Multiple selects from parametrized CTE fails with syntax error
This patch fills a serious flaw in the implementation of common table
expressions. Before this patch an attempt to prepare a statement from
a query with a parameter marker in a CTE that was used more than once
in the query ended up with a bogus error message. Similarly if a statement
in a stored procedure contained a CTE whose specification used a
local variables and this CTE was referred to more than once in the
statement then the server failed to execute the stored procedure returning
a bogus error message on a non-existing field.

The problems appeared due to incorrect handling of parameter markers /
local variables in CTEs that were referred more than once.

This patch fixes the problems by differentiating between the original
occurrences of a parameter marker / local variable used in the
specification of a CTE and the corresponding occurrences used
in copies of this specification. These copies are substituted
instead of non-first references to the CTE.

The idea of the fix and even some code were taken from the MySQL
implementation of the common table expressions.
2018-09-14 18:13:16 -07:00
Oleksandr Byelkin
31081593aa Merge branch '11.0' into 10.1 2018-09-06 22:45:19 +02:00
Sergei Golubchik
22bcfa011a cleanup: FOREIGN_KEY_INFO
instead of returning strings for CASCADE/RESTRICT
from every storage engine, use enum values

Backport of a3614d33e8
2018-09-04 08:37:44 +02:00
Oleksandr Byelkin
7e704a2308 Fixed 32bit version SELECT nesting depth. 2018-07-05 14:19:54 +02:00
Oleksandr Byelkin
de745ecf29 MDEV-11953: support of brackets in UNION/EXCEPT/INTERSECT operations 2018-07-04 19:13:55 +02:00
Alexander Barkov
e61568ee93 Merge remote-tracking branch 'origin/10.3' into 10.4 2018-07-03 14:02:05 +04:00
Sergei Golubchik
52a25d7b67 MDEV-16473 WITH statement throws 'no database selected' error
Different fix, just use NULL, not no_db,
2018-06-28 12:38:53 +02:00
Igor Babaev
7d0d934ca6 MDEV-16473 WITH statement throws 'no database selected' error
Before this patch if no default database was set the server threw
an error for any table name reference that was not fully qualified by
database name. In particular it happened for table names referenced
CTE tables. This was incorrect.
The error message was thrown at the parser stage when the names referencing
different tables were not resolved yet.
Now if no default database is set and  a with clause is used in the
processed statement  any table reference is just supplied with a dummy
database name "*none*" at the parser stage. Later after a call
of check_dependencies_in_with_clauses() when the names for CTE tables
can be resolved error messages are thrown only for those names that
refer to non-CTE tables. This is done in open_and_process_table().
2018-06-26 00:02:48 -07:00
Alexander Barkov
f5b60857f4 A cleanup for 84c55a5668 (that implemented cursor FOR loops earlier):
MDEV-10581 sql_mode=ORACLE: Explicit cursor FOR LOOP
  MDEV-12098 sql_mode=ORACLE: Implicit cursor FOR loop

Cleanup changes:
- Removing sp_lex_cursor::m_cursor_name
- Adding sp_instr_cursor_copy_struct::m_cursor (the cursor global index)
- Fixing sp_instr_cursor_copy_struct::print() to access to the cursor
  name using m_ctx and m_cursor (like other cursor related instructions do)
  instead of m_cursor_name.

This change is needed to unify sp_assignment_lex and sp_cursor_lex later,
to fix this problem easier:
 MDEV-16558 Parenthesized expression does not work as a lower FOR loop bound
2018-06-25 19:20:36 +04:00
Alexander Barkov
9c53cbdd88 MDEV-15941 Explicit cursor FOR loop does not close the cursor 2018-06-20 13:29:11 +04:00
Marko Mäkelä
0121d5a790 Merge 10.2 into 10.3 2018-06-18 15:43:59 +03:00
Galina Shalygina
ec4fdd5749 MDEV-16386: Wrong result when pushdown into the HAVING clause of the
materialized derived table/view that uses aliases is done

The problem appears when a column alias inside the materialized derived
table/view t1 definition coincides with the column name used in the
GROUP BY clause of t1. If the condition that can be pushed into t1
uses that ambiguous column name this column is determined as a column that
is used in the GROUP BY clause instead of the alias used in the projection
list of t1. That causes wrong result.
To prevent it resolve_ref_in_select_and_group() was changed.
2018-06-14 22:31:01 +02:00
Vladislav Vaintroub
4461b0f9b3 MDEV-16424 replace cmake/bison.cmake with cmake's builtin FindBison module 2018-06-08 14:30:04 +01:00
Alexander Barkov
ab297744b7 Merge remote-tracking branch 'origin/10.3' into 10.4 2018-06-05 10:50:08 +04:00
Alexander Barkov
106f0b5798 MDEV-16385 ROW SP variable is allowed in unexpected context
The problem described in the bug report happened because the code
did not test check_cols(1) after fix_fields() in a few places.

Additionally, fix_fields() could be called multiple times for SP variables,
because they are all fixed at a early stage in append_for_log().

Solution:
1. Adding a few helper methods
   - fix_fields_if_needed()
   - fix_fields_if_needed_for_scalar()
   - fix_fields_if_needed_for_bool()
   - fix_fields_if_needed_for_order_by()
  and using it in many cases instead of fix_fields() where
  the "fixed" status is not definitely known to be "false".

2. Adding DBUG_ASSERT(!fixed) into Item_splocal*::fix_fields()
   to catch double execution.

3. Adding tests.

As a good side effect, the patch removes a lot of duplicate code (~60 lines):

   if (!item->fixed &&
       item->fix_fields(..) &&
       item->check_cols(1))
     return true;
2018-06-05 10:25:39 +04:00