Commit graph

72031 commits

Author SHA1 Message Date
Marko Mäkelä
3c97097f11 Merge 10.4 into 10.5 2021-06-04 10:07:29 +03:00
Igor Babaev
0b797130c6 MDEV-25714 Join using derived with aggregation returns incorrect results
If a join query uses a derived table (view / CTE) with GROUP BY clause then
the execution plan for such join may employ split optimization. When this
optimization is employed the derived table is not materialized. Rather only
some partitions of the derived table are subject to grouping. Split
optimization can be applied only if:
- there are some indexes over the tables used in the join specifying the
  derived table whose prefixes partially cover the field items used in the
  GROUP BY list (such indexes are called splitting indexes)
- the WHERE condition of the join query contains conjunctive equalities
  between columns of the derived table that comprise major parts of
  splitting indexes and columns of the other join tables.

When the optimizer evaluates extending of a partial join by the rows of the
derived table it always considers a possibility of using split optimization.
Different splitting indexes can be used depending on the extended partial
join. At some rare conditions, for example, when there is a non-splitting
covering index for a table joined in the join specifying the derived table
usage of a splitting index to produce rows needed for grouping may be still
less beneficial than usage of such covering index without any splitting
technique. The function JOIN_TAB::choose_best_splitting() must take this
into account.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2021-06-03 14:44:03 -07:00
Monty
fa0bbff032 Fixed that compile-pentium64-valgrind-max works
- Removed Tokudb (no need to test this anymore with valgrind)
- Added __attribute__(unused)) to a few places to be able to compile even
  if valgrind/memcheck.h is not installed.

Reviewer: Marko Mäkelä <marko.makela@mariadb.com>
2021-06-02 18:54:49 +03:00
Marko Mäkelä
49ab50f882 Merge 10.4 into 10.5 2021-06-02 08:30:33 +03:00
Marko Mäkelä
d3d2c96567 Merge 10.3 into 10.4 2021-06-02 08:29:47 +03:00
Marko Mäkelä
aa70690e9a Merge 10.2 into 10.3 2021-06-02 08:29:01 +03:00
Marko Mäkelä
a8434c6c59 MDEV-25730 fixup: GCC -Og -Wmaybe-uninitialized
Silence a warning about an uninitialized variable that was
introduced by commit d8fa71a089.
2021-06-02 08:25:44 +03:00
Marko Mäkelä
9c7a456a92 Merge 10.4 into 10.5 2021-06-01 10:38:09 +03:00
Marko Mäkelä
77d8da57d7 Merge 10.3 into 10.4 2021-06-01 09:14:59 +03:00
Marko Mäkelä
950a220060 Merge 10.2 into 10.3 2021-06-01 08:40:59 +03:00
Dmitry Shulga
91bde0fb67 MDEV-25576: The statement EXPLAIN running as regular statement and as prepared statement produces different results for UPDATE with subquery
Both EXPLAIN and EXPLAIN EXTENDED statements produce different results set
in case it is run in normal way and in PS mode for the statements
UPDATE/DELETE with subquery.

The use case below reproduces the issue:
MariaDB [test]> CREATE TABLE t1 (c1 INT KEY) ENGINE=MyISAM;
Query OK, 0 rows affected (0,128 sec)

MariaDB [test]> CREATE TABLE t2 (c2 INT) ENGINE=MyISAM;
Query OK, 0 rows affected (0,023 sec)

MariaDB [test]> CREATE TABLE t3 (c3 INT) ENGINE=MyISAM;
Query OK, 0 rows affected (0,021 sec)

MariaDB [test]> EXPLAIN EXTENDED UPDATE t3 SET c3 =
    -> ( SELECT COUNT(d1.c1) FROM ( SELECT a11.c1 FROM t1 AS a11
    -> STRAIGHT_JOIN t2 AS a21 ON a21.c2 = a11.c1 JOIN t1 AS a12
    -> ON a12.c1 = a11.c1 ) d1 );
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                          |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
|    1 | PRIMARY     | t3    | ALL  | NULL          | NULL | NULL    | NULL |    0 |   100.00 |                                |
|    2 | SUBQUERY    | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | Impossible WHERE noticed after reading const tables
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
2 rows in set (0,002 sec)

MariaDB [test]> PREPARE stmt FROM
    -> EXPLAIN EXTENDED UPDATE t3 SET c3 =
    -> ( SELECT COUNT(d1.c1) FROM ( SELECT a11.c1 FROM t1 AS a11
    -> STRAIGHT_JOIN t2 AS a21 ON a21.c2 = a11.c1 JOIN t1 AS a12
    -> ON a12.c1 = a11.c1 ) d1 );
Query OK, 0 rows affected (0,000 sec)
Statement prepared

MariaDB [test]>  EXECUTE stmt;
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                          |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
|    1 | PRIMARY     | t3    | ALL  | NULL          | NULL | NULL    | NULL |    0 |   100.00 |                                |
|    2 | SUBQUERY    | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | no matching row in const table |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
2 rows in set (0,000 sec)

The reason by that different result sets are produced is that on execution
of the statement 'EXECUTE stmt' the flag SELECT_DESCRIBE not set
in the data member SELECT_LEX::options for instances of SELECT_LEX that
correspond to subqueries used in the UPDTAE/DELETE statements.

Initially, these flags were set on parsing the statement
  PREPARE stmt FROM "EXPLAIN EXTENDED UPDATE t3 SET ..."
but latter they were reset before starting real execution of
the parsed query during handling the statement 'EXECUTE stmt';

So, to fix the issue the functions mysql_update()/mysql_delete()
have been modified to set the flag SELECT_DESCRIBE forcibly
in the data member SELECT_LEX::options for the primary SELECT_LEX
of the UPDATE/DELETE statement.
2021-05-30 17:31:55 +07:00
Marko Mäkelä
ab87fc6c7a Cleanup: Remove handler::update_table_comment()
The only call of the virtual member function
handler::update_table_comment() was removed in
commit 82d28fada7 (MySQL 5.5.53)
but the implementation was not removed.

The only non-trivial implementation was for InnoDB. The information
is now returned via handler::get_foreign_key_create_info() and
ha_statistics::delete_length.
2021-05-27 09:31:19 +03:00
Sergei Golubchik
4777097fee followup: rename generated files to have distinct names 2021-05-27 00:40:23 +02:00
Sergei Golubchik
dfbeddaa11 MDEV-25726 get rid of cmake comment hack in sql_yacc.yy
and replace it with equally unsightly %ifdef/%endif hack
also, support %else, it's nice
2021-05-27 00:40:23 +02:00
Monty
aa284e0237 MDEV-17749 Kill during LOCK TABLE ; ALTER TABLE causes assert
The problem was that when LOCK TABLES where unwinded as part of
a killed connection, unlink_all_closed_tables() did not like that
there was uncommited transactions.
Fixed by doing a rollback of any open transaction in this particular case.
2021-05-26 14:35:23 +03:00
Marko Mäkelä
365cd08345 Merge 10.4 into 10.5 2021-05-26 09:47:28 +03:00
Igor Babaev
675716e1cb MDEV-23886 Reusing CTE inside a function fails with table doesn't exist
In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.

This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.

# Conflicts:
#	sql/sql_cte.cc
#	sql/sql_cte.h
#	sql/sql_lex.cc
#	sql/sql_lex.h
#	sql/sql_view.cc
#	sql/sql_yacc.yy
#	sql/sql_yacc_ora.yy
2021-05-25 21:48:54 -07:00
sjaakola
e212415690 MDEV-25551 applying crash with tables without PK
The underlying problem with MDEV-25551 turned out to be that
transactions having changes for tables with no primary key,
were not safe to apply in parallel. This is due to excessive locking
in innodb side, and even non related row modifications could end up
in lock conflict during applying.

The fix for MDEV-25551 has disabled parallel applying for tables with no PK.
This fix depends on change for wsrep-lib, where a separate PR allows
application to modify transaction flags in wsrep-lib.

This commit has also separate mtr test for verifying that transactions
modifying a table with no primary key, will not apply in parallel.
This test is a modified version of initial test created by Gabor Orosz,
the reporterr of MDEV-25551.
Another mtr test was added in galera_sr suite, for testing if modifying
tables with no primary key would causes issues for streaming replication
use cases.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-05-26 07:41:05 +03:00
nia
6d549aecf5 threadpool_generic: support future NetBSD kqueue versions
In NetBSD 9.x and prior, udata is an intptr_t, but in 10.x (current
development branch) it was changed to be a void * for compatibility
with other BSDs a year or so ago.

Unfortunately, this does not simplify the code, as NetBSD 8.x and 9.x
are still supported and will be for a few more years.

Signed-off-by: Nia Alarie <nia@NetBSD.org>
2021-05-26 11:15:33 +10:00
Marko Mäkelä
1dea7f7977 Merge 10.3 into 10.4 2021-05-25 15:38:57 +03:00
Sergei Golubchik
d8fa71a089 MDEV-25730: maria.repair test fails with valgrind
cherry-pick commit: 1fff2398ef
MDEV-22530 post push fixes from 10.6.

Followup. If the KILL happens - report it as a failure,
don't eat it up silently. Note that this has to be done after `table_name`
is populated, so that the error message could show it.
2021-05-25 15:29:50 +05:30
Igor Babaev
04de651725 MDEV-23886 Reusing CTE inside a function fails with table doesn't exist
In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.

This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.
2021-05-25 00:43:03 -07:00
Marko Mäkelä
1864a8ea93 Merge 10.2 into 10.3 2021-05-24 09:38:49 +03:00
Monty
15214a4f11 MDEV-25708 THD::cleanup(): Assertion `!mdl_context.has_locks()' failed
The reason was that WSREP code in mysql_create_or_drop_trigge() did
jump to an exit label that did not do proper cleanup of state.
Fixed by ensuring that also WSREP code goes trough the cleanup process.
2021-05-23 19:35:05 +03:00
Sergei Golubchik
d7321893d8 CONNECT: move jar files to /usr/share and include them in DEBs 2021-05-22 21:56:51 +02:00
Igor Babaev
43c9fcefc0 MDEV-23886 Reusing CTE inside a function fails with table doesn't exist
In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.

This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.
2021-05-21 16:00:35 -07:00
Sergei Petrunia
2087d47aae MDEV-22462: Item_in_subselect::create_single_in_to_exists_cond(JOIN *, Item **, Item **): Assertion `false' failed.
Item_in_subselect::create_single_in_to_exists_cond() should handle the
case where the subquery is a table-less select but it is not a result
of a UNION.

(Table-less subqueries like "(SELECT 1)" are "substituted" with their select
list, but table-less subqueries with WHERE or HAVING clause, like
"(SELECT 1 WHERE ...)" are not substituted. They are handled with regular
execution path)
2021-05-21 17:46:48 +03:00
mkaruza
5667baad5d MDEV-25562 Assertion `pause_seqno_.is_undefined() == false' failed in void wsrep::server_state::resume()
If pause() is not executed in galera and returns seqno = -1 we should
skip resume().
2021-05-21 10:35:17 +03:00
Rucha Deodhar
629449172a MDEV-25462: Assertion `m_status == DA_ERROR || m_status == DA_OK ||
m_status == DA_OK_BULK' failed in Diagnostics_area::message from
get_schema_tables_record

Analysis: SET NAMES changes character set for character_set_client,
character_set_connection, character_set_results to 'filename'. The .frm file of view
has @xx sequences in the SELECT query, which give parsing error because 'filename'
character set is not parser friendly. When we get parsing error (ER_PARSE_ERROR), we
directly return true without setting error status. This is caught later in assertion.
Fix: Disallow 'filename' character set in SET NAMES because it is not parser
friendly.
2021-05-20 09:45:27 +05:30
Daniel Black
406ce57232 MDEV-25728: mysqld --help --verbose creates a log-bin-index file
before change test:

 strace -fe trace=file -o /tmp/f.strace sql/mysqld --datadir=/tmp/d --log-bin=foo-bin                      --help --verbose && ls -la /tmp/
...
'mysqladmin variables' instead of 'mysqld --verbose --help'.

total 0
drwxrwxr-x.  2 dan  dan   60 May 19 18:05 .
drwxrwxrwt. 27 root root 640 May 19 18:03 ..
-rw-rw----.  1 dan  dan    0 May 19 18:05 foo-bin.index
2021-05-20 07:55:55 +10:00
Sergei Petrunia
e570f740cd MDEV-25629: Crash in get_sort_by_table() in subquery with order by having outer ref
In Item_field::fix_fields(): when the item was resolved to an Item_field
in the SELECT's select_list, copy the Item_field's "depended_from" field.

Failure to do so caused the item to have incorrect attributes: it pointed
to a Field in an upper select but used_tables() didn't return
OUTER_REF_TABLE_BIT.
2021-05-19 23:12:52 +03:00
Marko Mäkelä
cc2651b74c Merge 10.4 into 10.5 2021-05-18 09:21:59 +03:00
Marko Mäkelä
4240704abc Merge 10.3 into 10.4 2021-05-18 08:59:12 +03:00
Marko Mäkelä
ca3f497564 Merge 10.2 into 10.3, except MDEV-25682 2021-05-18 08:40:19 +03:00
Brandon Nesterenko
81402c1318 MDEV-25222: mysqlbinlog --base64-output wrong option default drops BINLOG from output
Problem:
=======
The ALWAYS option of the mariadb-binlog --base64-output flag
formats its output incorrectly. This option is deprecated, and
MySQL 8.0 has removed it entirely.

Solution:
========
Adhere to MySQL and remove this option from MariaDB.

Behavioral Changes:
==================
Use Case: ./mariadb-binlog --base64-output
 Previous Behavior: Sets base64-output mode to always
 New Behavior: Error message indicating incomplete argument

Use Case: ./mariadb-binlog --base64-output=always
 Previous Behavior: Sets base64-output mode to always
 New Behavior: Error message indicating invalid argument value

Reviewed By:
==========
Andrei Elkin: <andrei.elkin@mariadb.com>
2021-05-17 14:38:46 -06:00
Julius Goryavsky
740917620a MDEV-25693: SST failed due to incorrect connection address
Fixed bugs caused by inaccuracies in automatic merging
from other branches:

1) Authentication information is not removed from the connection
   address, which causes some tests to fail;
2) wsrep_debug=on should be replaced with wsrep_debug=1;
3) Added missing "connection" lines to test result file;
4) Some tests have been corrected for Galera 4.x (10.4+).
2021-05-17 20:33:55 +02:00
Julius Goryavsky
527675d53a MDEV-25669: SST scripts should check all server groups in config files
1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.
2021-05-17 20:33:06 +02:00
Julius Goryavsky
e861e057ad MDEV-25693: SST failed due to incorrect connection address
Fixed bugs caused by inaccuracies in automatic merging
from other branches:

1) Authentication information is not removed from the connection
   address, which causes some tests to fail;
2) wsrep_debug=on should be replaced with wsrep_debug=1;
3) Added missing "connection" lines to test result file;
4) Some tests have been corrected for Galera 4.x (10.4+).
2021-05-17 19:51:49 +02:00
Julius Goryavsky
f92cd0c56b MDEV-25669: SST scripts should check all server groups in config files
1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.
2021-05-17 15:08:40 +02:00
Julius Goryavsky
f9f8e33f29 MDEV-25669: SST scripts should check all server groups in config files
1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.
2021-05-17 14:58:49 +02:00
Sujatha
88c7a58ecf MDEV-22530: Aborting OPTIMIZE TABLE still logs in binary log and replicates to the Slave server.
Problem:
========
Aborting OPTIMIZE TABLE still logs in binary logs and replicates to the
Slave server. "Optimize table" command under execution, is killed by using
"Ctrl-C" as shown below.

MariaDB [test]> optimize table t2;
^CCtrl-C -- query killed. Continuing normally.

In spite of query execution being interrupted the query gets written to
binary log.

Analysis:
========
Admin command execution logic is not handling KILL command, hence it
ignores the KILL command and completes its execution.

Fix:
===
Check for thread killed notification, during admin command execution and
handle it. If thread kill occurs prior to any table modification the query
will not be written to binary log. If kill happens after at least one table
is modified then the query will be written to binary log. Ex: command in
execution is 'OPTIMIZE TABLE t1,t2' and the thread kill happens after t1
table is modified then 'OPTIMIZE TABLE t1,t2' will be written to binary log
as admin commands will not make the slave to diverge from master.
2021-05-17 16:38:58 +05:30
Sujatha
410e3c1a9a MDEV-17515: GTID Replication in optimistic mode deadlock
Problem:
=======
In slave_parallel_mode=optimistic configuration, when admin commands and
DML operation on the same table are scheduled simultaneously for execution,
it results in lock conflict and slave server either hangs due to
deadlock or goes down with an assert.

Analysis:
========
Admin commands OPTIMIZE, REPAIR and ANALYZE are written to binary log as
ordinary transactions. When 'slave_parallel_mode' is 'optimistic' DMLs are
allowed to run in parallel. But these locks are not detected by parallel
replication deadlock detection-and-handling mechanism. At times they result
in deadlock or assertion.

Fix:
===
Flag admin commands as DDL in Gtid_log_event at the time of writing to
binary log. Add a new bit EXECUTED_TABLE_ADMIN_CMD to
'm_unsafe_rollback_flags'. During 'mysql_admin_table' command execution it
accepts a list of tables to be processed and executes them in a loop. Upon
successful execution enable 'EXECUTED_TABLE_ADMIN_CMD' bit in
thd->transaction.stmt_unsafe_rollback_flags. Gtid_log_event constructor
will notice this flag and mark the current transaction with 'FL_DDL' flag.
Gtid_log_events marked as FL_DDL will not be scheduled parallel execution,
on the slave. They will execute in isolation to prevent deadlocks.

Note: Removed the call to 'trans_commit_implicit' from 'mysql_admin_table'
function as 'mysql_execute_command' will take care of invoking
'trans_commit_implicit'.
2021-05-17 16:38:58 +05:30
Alexey Botchkov
c22d567e1a MDEV-25690 Plugins can't execute sql statements with the Galera enabled.
For the plugin thread during the server start the wsrep_on should be 0.
2021-05-17 12:09:48 +04:00
Daniel Black
80ae3677e1 MDEV-25681: --relay-log{,-index} missing warning
No longer a MySQL server, "his" is the wrong pronoun
for a server.

Thanks Michael Newton for highlighting these problems

Also changed slave -> replica.
2021-05-17 09:39:43 +10:00
Julius Goryavsky
6811ed3e10 MDEV-25669: SST scripts should check all server groups in config files
1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.
2021-05-15 15:01:21 +02:00
Igor Babaev
677f1ef6f0 MDEV-25682 Explain shows an execution plan different from actually executed
If a select query contained an ORDER BY clause that followed a LIMIT clause
or an ORDER BY clause or ORDER BY with LIMIT the EXPLAIN output for the
query showed an execution plan different from that was actually executed.

Approved by Roman Nozdrin <roman.nozdrin@mariadb.com>
2021-05-14 16:43:36 -07:00
Sachin Kumar
355dc74b76 MDEV-22370 safe_mutex: Trying to lock uninitialized mutex at /data/src/10.4-bug/sql/rpl_parallel.cc, line 470 upon shutdown during FTWRL
Problem:- When we issue FTWRL with shutdown in parallel, there is race between
FTWRL and shutdown. Shutdown might destroy the mutex (pool->LOCK_rpl_thread_pool)
before FTWRL can lock it. So we can get crash on FTWRL thread

Solution:- mysql_mutex_destroy(pool->LOCK_rpl_thread_pool) should wait for
FTWRL thread to complete its work , and then destroy.
So slave_prepare_for_shutdown will just deactivate the pool, and mutex is destroyed
later in end_slave()
2021-05-14 11:49:46 +01:00
Andrei Elkin
3616640a31 MDEV-20821 parallel slave server shutdown hang
Parallel slave server shutdown found to be hanging in
close_connections() triggered by shutdown due to a slave worker thread
would not be notified to exit in case the worker was sitting idle.

Fixed with destroying the worker pool earlier that is in
slave_prepare_for_shutdown() when all their driver threads have already left.
A test file is added to simulate the bug condition as well as check
multi-sourced and not-idle worker cases.
2021-05-14 11:49:26 +01:00
Monty
0df51e610b MDEV-25651 Server crash or assertion failure in THD::update_stats upon concurrent DROP TRIGGER
The problem was that in a timeout event,
thd->lex->restore_backup_query_tables_list() was called when it should
not have been.

Patch tested with the script in MDEV-25651 (not suitable for mtr)
2021-05-11 21:06:36 +03:00
Monty
621501f38b MDEV-25606: Concurrent CREATE TRIGGER statements mix up in binlog and break replication
The bug is that we don't have a a lock on the trigger name, so it is
possible for two threads to try to create the same trigger at the same
time and both thinks that they have succeed.
Same thing can happen with drop trigger or a combinations of create and
drop trigger.

Fixed by adding a mdl lock for the trigger name for the duration of the
create/drop.

Patch tested by Elena
2021-05-10 21:11:46 +03:00