MDEV-4984: Implement MASTER_GTID_WAIT() and @@LAST_GTID.
MDEV-4726: Race in mysql-test/suite/rpl/t/rpl_gtid_stop_start.test
MDEV-5636: Deadlock in RESET MASTER
The problem is a deadlock between MYSQL_BIN_LOG::reset_logs() and
MYSQL_BIN_LOG::mark_xid_done(). The former takes LOCK_log and waits for the
latter to complete. But the latter also tries to take LOCK_log; this can lead
to a deadlock.
There was already code that tries to deal with this, with the flag
reset_master_pending. However, there was still a small opportunity for
deadlock, when an previous mark_xid_done() is still running when reset_logs()
is called and is at the precise point where it first releases LOCK_xid_list
and then re-aquires both LOCK_log and LOCK_xid_list.
Solve by setting reset_master_pending in reset_logs() before taking
LOCK_log. And also count how many invocations of LOCK_xid_list are in the
progress of releasing and re-aquiring locks, and in reset_logs() wait for that
number to drop to zero after setting reset_master_pending and before taking
LOCK_log.
Rewrite the gtid_waiting::wait_for_gtid() function.
The code was rubbish (and buggy). Now the logic is
much clearer.
Also fix a missing slave sync that could cause test failure.
MASTER_GTID_WAIT() is similar to MASTER_POS_WAIT(), but works with a
GTID position rather than an old-style filename/offset.
@@LAST_GTID gives the GTID assigned to the last transaction written
into the binlog.
Together, the two can be used by applications to obtain the GTID of
an update on the master, and then do a MASTER_GTID_WAIT() for that
position on any read slave where it is important to get results that
are caught up with the master at least to the point of the update.
The implementation of MASTER_GTID_WAIT() is implemented in a way
that tries to minimise the performance impact on the SQL threads,
even in the presense of many waiters on single GTID positions (as
from @@LAST_GTID).
Patches for server and the Innodb engine.
Server is fixed so it does nothing if no indexes left to alter.
Innodb parser is fixed so it looks for the IF [NOT] EXISTS option in a string.
Another change is that it uses the index name for the internal dictionary.
Prior to that it only used the CONSTRAINT name for it.
mysql-test/r/lowercase_table2.result:
Updated result
(The change happend because we don't try to open the table anymore as part of create table)
mysql-test/suite/rpl/r/create_or_replace_mix.result:
Fixed result file
mysql-test/suite/rpl/r/create_or_replace_row.result:
Fixed result file
mysql-test/suite/rpl/r/create_or_replace_statement.result:
Fixed result file
mysql-test/suite/rpl/t/create_or_replace.inc:
Drop open temporary table
mysys/my_delete.c:
Added missing newline
plugin/metadata_lock_info/mysql-test/metadata_lock_info/r/user_lock.result:
Fixed result
(Lock names was before off by one. Was corrected by my previous patch)
sql/sql_select.cc:
Fixed compiler warnings by adding missing casts
storage/connect/ha_connect.cc:
Fixed compiler warnings
storage/innobase/os/os0file.cc:
Fixed compiler warnings
storage/xtradb/btr/btr0btr.cc:
Fixed compiler warnings
storage/xtradb/handler/ha_innodb.cc:
removed not used function
strings/ctype-uca.c:
Fixed compiler warnings
support-files/compiler_warnings.supp:
Added suppression for warnings that are wrong or are not serious andthat we don't plan to fix.
The field JOIN::select_lex->where should be updated after the call
of remove_eq_conds() in the function make_join_statistics(). This
matters for subselects.
mysql-test/r/create_or_replace.result:
Added test case
mysql-test/t/create_or_replace.test:
Added test case
sql/sql_base.cc:
Downgrade locks in LOCK TABLES + CREATE TABLE OR REPLACE
sql/sql_table.cc:
Downgrade locks in LOCK TABLES + CREATE TABLE OR REPLACE
- CREATE TABLE is by default executed on the slave as CREATE OR REPLACE
- DROP TABLE is by default executed on the slave as DROP TABLE IF NOT EXISTS
This means that a slave will by default continue even if we try to create
a table that existed on the slave (the table will be deleted and re-created) or
if we try to drop a table that didn't exist on the slave.
This should be safe as instead of having the slave stop because of an inconsistency between
master and slave, it will fix the inconsistency.
Those that would prefer to get a stopped slave instead for the above cases can set slave_ddl_exec_mode to STRICT.
- Ensure that a CREATE OR REPLACE TABLE which dropped a table is replicated
- DROP TABLE that generated an error on master is handled as an identical DROP TABLE on the slave (IF NOT EXISTS is not added in this case)
- Added slave_ddl_exec_mode variable to decide how DDL's are replicated
New logic for handling BEGIN GTID ... COMMIT from the binary log:
- When we find a BEGIN GTID, we start a transaction and set OPTION_GTID_BEGIN
- When we find COMMIT, we reset OPTION_GTID_BEGIN and execute the normal COMMIT code.
- While OPTION_GTID_BEGIN is set:
- We don't generate implict commits before or after statements
- All tables are regarded as transactional tables in the binary log (to ensure things are executed exactly as on the master)
- We reset OPTION_GTID_BEGIN also on rollback
This will help ensuring that we don't get any sporadic commits (and thus new GTID's) on the slave and will help keep the GTID's between master and slave in sync.
mysql-test/extra/rpl_tests/rpl_log.test:
Added testing of mode slave_ddl_exec_mode=STRICT
mysql-test/r/mysqld--help.result:
New help messages
mysql-test/suite/rpl/r/create_or_replace_mix.result:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/r/create_or_replace_row.result:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/r/create_or_replace_statement.result:
Testing replication of create or replace
mysql-test/suite/rpl/r/rpl_gtid_startpos.result:
Test must be run in slave_ddl_exec_mode=STRICT as part of the test depends on that DROP TABLE should fail on slave.
mysql-test/suite/rpl/r/rpl_row_log.result:
Updated result
mysql-test/suite/rpl/r/rpl_row_log_innodb.result:
Updated result
mysql-test/suite/rpl/r/rpl_row_show_relaylog_events.result:
Updated result
mysql-test/suite/rpl/r/rpl_stm_log.result:
Updated result
mysql-test/suite/rpl/r/rpl_stm_mix_show_relaylog_events.result:
Updated result
mysql-test/suite/rpl/r/rpl_temp_table_mix_row.result:
Updated result
mysql-test/suite/rpl/t/create_or_replace.inc:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/t/create_or_replace_mix.cnf:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/t/create_or_replace_mix.test:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/t/create_or_replace_row.cnf:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/t/create_or_replace_row.test:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/t/create_or_replace_statement.cnf:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/t/create_or_replace_statement.test:
Testing of CREATE OR REPLACE TABLE with replication
mysql-test/suite/rpl/t/rpl_gtid_startpos.test:
Test must be run in slave_ddl_exec_mode=STRICT as part of the test depends on that DROP TABLE should fail on slave.
mysql-test/suite/rpl/t/rpl_stm_log.test:
Removed some lines
mysql-test/suite/sys_vars/r/slave_ddl_exec_mode_basic.result:
Testing of slave_ddl_exec_mode
mysql-test/suite/sys_vars/t/slave_ddl_exec_mode_basic.test:
Testing of slave_ddl_exec_mode
sql/handler.cc:
Regard all tables as transactional in commit if OPTION_GTID_BEGIN is set.
This is to ensure that statments are not commited too early if non transactional tables are used.
sql/log.cc:
Regard all tables as transactional in commit if OPTION_GTID_BEGIN is set.
Also treat 'direct' log events as transactional (to get them logged as they where on the master)
sql/log_event.cc:
Ensure that the new error from DROP TABLE when trying to drop a view is treated same as the old one.
Store error code that slave expects in THD.
Set OPTION_GTID_BEGIN if we find a BEGIN.
Reset OPTION_GTID_BEGIN if we find a COMMIT.
sql/mysqld.cc:
Added slave_ddl_exec_mode_options
sql/mysqld.h:
Added slave_ddl_exec_mode_options
sql/rpl_gtid.cc:
Reset OPTION_GTID_BEGIN if we record a gtid (safety)
sql/sql_class.cc:
Regard all tables as transactional in commit if OPTION_GTID_BEGIN is set.
sql/sql_class.h:
Added to THD: log_current_statement and slave_expected_error
sql/sql_insert.cc:
Ensure that CREATE OR REPLACE is logged if table was deleted.
Don't do implicit commit for CREATE if we are under OPTION_GTID_BEGIN
sql/sql_parse.cc:
Change CREATE TABLE -> CREATE OR REPLACE TABLE for slaves
Change DROP TABLE -> DROP TABLE IF EXISTS for slaves
CREATE TABLE doesn't force implicit commit in case of OPTION_GTID_BEGIN
Don't do commits before or after any statement if OPTION_GTID_BEGIN was set.
sql/sql_priv.h:
Added OPTION_GTID_BEGIN
sql/sql_show.cc:
Enhanced store_create_info() to also be able to handle CREATE OR REPLACE
sql/sql_show.h:
Updated prototype
sql/sql_table.cc:
Ensure that CREATE OR REPLACE is logged if table was deleted.
sql/sys_vars.cc:
Added slave_ddl_exec_mode
sql/transaction.cc:
Added warning if we got a GTID under OPTION_GTID_BEGIN
Port to mariadb-1.0 the following fix from mysql-5.6:
Revision ID: jorgen.loland@oracle.com-20120314131055-ml54x9deueqfsff4
BUG#13701206: WHERE A>=B DOES NOT GIVE SAME EXECUTION PLAN
AS WHERE B<=A (RANGE OPTIMIZER)
that fix didn't have a public testcase, so I created one.
Cleanups:
* remove unused HA_EXTRA_EXPORT (can be added later if needed, e.g. for Aria)
* clarify the meaning of HA_CAN_EXPORT
* make all engines that support EXPORT to announce it
* reduce code duplication
Syntax. Server support. Test cases.
InnoDB bugfixes:
* don't mess around with system sprintf's, always use my_error() for errors.
* don't use InnoDB internal error codes where OS error codes are expected.
* don't say "file not found", when it was.
Update InnoDB to 5.6.14
Apply MySQL-5.6 hack for MySQL Bug#16434374
Move Aria-only HA_RTREE_INDEX from my_base.h to maria_def.h (breaks an assert in InnoDB)
Fix InnoDB memory leak
- MDEV-5587 Server crashes in Locked_tables_list::restore_lock on CREATE OR REPLACE .. SELECT under LOCK
- MDEV-5586 Assertion `share->tdc.all_tables.is_empty() || remove_type != TDC_RT_REMOVE_ALL' fails in tdc_remove_table
- MDEV-5588 Strange error on CREATE OR REPLACE table over an existing view
mysql-test/r/create_or_replace.result:
Added test cases
mysql-test/r/lowercase_view.result:
New error message
mysql-test/r/merge.result:
New error message
mysql-test/r/multi_update.result:
New error message
mysql-test/r/subselect.result:
New error message
mysql-test/r/subselect_exists_to_in.result:
New error message
mysql-test/r/subselect_no_mat.result:
New error message
mysql-test/r/subselect_no_opts.result:
New error message
mysql-test/r/subselect_no_scache.result:
New error message
mysql-test/r/subselect_no_semijoin.result:
New error message
mysql-test/r/view.result:
New error message
mysql-test/suite/funcs_1/r/myisam_views-big.result:
New error message
mysql-test/t/create_or_replace.test:
New tests
mysql-test/t/view.test:
New error message
sql/share/errmsg-utf8.txt:
Added new error message
sql/sql_base.cc:
Updated error message
Do an automatic UNLOCK TABLES if we don't have any locked tables (safety fix)
sql/sql_db.cc:
Updated arguments
sql/sql_load.cc:
New error message
sql/sql_parse.cc:
Check that we are not using a table we are dropping and re-creating
sql/sql_table.cc:
Added parameter to mysql_rm_table_no_locks() to not automaticly do UNLOCK TABLES
Added better error message if trying to drop a view with DROP TABLE
Don't try to create something we select from
sql/sql_table.h:
Updated prototypes
Using CREATE OR REPLACE TABLE is be identical to
DROP TABLE IF EXISTS table_name;
CREATE TABLE ...;
Except that:
* CREATE OR REPLACE is be atomic (now one can create the same table between drop and create).
* Temporary tables will not shadow the table name for the DROP as the CREATE TABLE tells us already if we are using a temporary table or not.
* If the table was locked with LOCK TABLES, the new table will be locked with the same lock after it's created.
Implementation details:
- We don't anymore open the to-be-created table during CREATE TABLE, which the original code did.
- There is no need to open a table we are planning to create. It's enough to check if the table exists or not.
- Removed some of duplicated code for CREATE IF NOT EXISTS.
- Give an error when using CREATE OR REPLACE with IF NOT EXISTS (conflicting options).
- As a side effect of the code changes, we don't anymore have to internally re-prepare prepared statements with CREATE TABLE if the table exists.
- Made one code path for all testing if log table are in use.
- Better error message if one tries to create/drop/alter a log table in use
- Added back disabled rpl_row_create_table test as it now seams to work and includes a lot of interesting tests.
- Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
- Aligned CREATE OR REPLACE parsing code in sql_yacc.yy for TABLE and VIEW
- Changed interface for drop_temporary_table() to make it more reusable
- Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
- Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument. Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
- Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
- Added restart_trans_for_tables() to be able to restart a transaction.
- DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
- Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
- Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()
mysql-test/include/commit.inc:
With new code we get fewer status increments
mysql-test/r/commit_1innodb.result:
With new code we get fewer status increments
mysql-test/r/create.result:
Added testing of create or replace with timeout
mysql-test/r/create_or_replace.result:
Basic testing of CREATE OR REPLACE TABLE
mysql-test/r/partition_exchange.result:
New error message
mysql-test/r/ps_ddl.result:
Fewer reprepares with new code
mysql-test/suite/archive/discover.result:
Don't rediscover archive tables if the .frm file exists
(Sergei will look at this if there is a better way...)
mysql-test/suite/archive/discover.test:
Don't rediscover archive tables if the .frm file exists
(Sergei will look at this if there is a better way...)
mysql-test/suite/funcs_1/r/innodb_views.result:
New error message
mysql-test/suite/funcs_1/r/memory_views.result:
New error message
mysql-test/suite/rpl/disabled.def:
rpl_row_create_table should now be safe to use
mysql-test/suite/rpl/r/rpl_row_create_table.result:
Updated results after adding back disabled test
mysql-test/suite/rpl/t/rpl_create_if_not_exists.test:
Added comment
mysql-test/suite/rpl/t/rpl_row_create_table.test:
Added CREATE OR REPLACE TABLE test
mysql-test/t/create.test:
Added CREATE OR REPLACE TABLE test
mysql-test/t/create_or_replace-master.opt:
Create logs
mysql-test/t/create_or_replace.test:
Basic testing of CREATE OR REPLACE TABLE
mysql-test/t/partition_exchange.test:
Error number changed as we are now using same code for all log table change issues
mysql-test/t/ps_ddl.test:
Fewer reprepares with new code
sql/handler.h:
Moved things around a bit in a structure to get better alignment.
Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
Added 3 elements to end of HA_CREATE_INFO to be able to store state to add backs locks in case of LOCK TABLES.
sql/log.cc:
Reimplemented check_if_log_table():
- Simpler and faster usage
- Can give error messages
This gives us one code path for allmost all error messages if log tables are in use
sql/log.h:
New interface for check_if_log_table()
sql/slave.cc:
More logging
sql/sql_alter.cc:
New interface for check_if_log_table()
sql/sql_base.cc:
More documentation
Changed interface for drop_temporary_table() to make it more reusable
Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument. Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
Check for command number instead of open_strategy of CREATE TABLE was used.
Added restart_trans_for_tables() to be able to restart a transaction. This was needed in "create or replace ... select" between the drop table and the select.
sql/sql_base.h:
Added and updated function prototypes
sql/sql_class.h:
Added new prototypes to Locked_tables_list class
Added extra argument to select_create to avoid double call to eof() or send_error()
- I needed this in some edge case where the table was not created against expections.
sql/sql_db.cc:
New interface for check_if_log_table()
sql/sql_insert.cc:
Remember position to lock information so that we can reaquire table lock for LOCK TABLES + CREATE OR REPLACE TABLE SELECT. Later add back the lock by calling restore_lock().
Removed one not needed indentation level in create_table_from_items()
Ensure we don't call send_eof() or abort_result_set() twice.
sql/sql_lex.h:
Removed variable that I temporarly added in an earlier changeset
sql/sql_parse.cc:
Removed old test code (marked with QQ)
Ensure that we have open_strategy set as TABLE_LIST::OPEN_STUB in CREATE TABLE
Removed some IF NOT EXISTS code as this is now handled in create_table_table_impl().
Set OPTION_KEEP_LOGS later. This code had to be moved as the test for IF EXISTS has changed place.
DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
sql/sql_partition_admin.cc:
New interface for check_if_log_table()
sql/sql_rename.cc:
New interface for check_if_log_table()
sql/sql_table.cc:
New interface for check_if_log_table()
Moved some code in mysql_rm_table() under a common test.
- Safe as temporary tables doesn't have statistics.
- !is_temporary_table(table) test was moved out from drop_temporary_table() and merged with upper level code.
- Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
- Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()
- In mysql_create_like_table(), restore table->open_strategy() if it was changed.
- Re-test if table was a view after opening it.
sql/sql_table.h:
New prototype for mysql_create_table_no_lock()
sql/sql_yacc.yy:
Added syntax for CREATE OR REPLACE TABLE
Reuse new code for CREATE OR REPLACE VIEW
sql/table.h:
Added name for enum type
sql/table_cache.cc:
More DBUG
on server shutdown after SELECT with CONVERT_TZ
It's wrong to return my_empty_string from val_str().
Removing my_empty_string. Using make_empty_result() instead.
INSTALL PLUGIN
There was mixed lock order between LOCK_plugin, LOCK_global_system_variables
and LOCK_system_variables_hash. This patch ensures that write-lock on
LOCK_system_variables_hash doesn't intersect with LOCK_plugin.
Fixed by moving initialization/deinitialization of plugin options from
plugin_add()/plugin_del() to plugin_initialize()/plugin_deinitalize().
So that plugin options are handled without protection of LOCK_plugin.
