Using CREATE OR REPLACE TABLE is be identical to
DROP TABLE IF EXISTS table_name;
CREATE TABLE ...;
Except that:
* CREATE OR REPLACE is be atomic (now one can create the same table between drop and create).
* Temporary tables will not shadow the table name for the DROP as the CREATE TABLE tells us already if we are using a temporary table or not.
* If the table was locked with LOCK TABLES, the new table will be locked with the same lock after it's created.
Implementation details:
- We don't anymore open the to-be-created table during CREATE TABLE, which the original code did.
- There is no need to open a table we are planning to create. It's enough to check if the table exists or not.
- Removed some of duplicated code for CREATE IF NOT EXISTS.
- Give an error when using CREATE OR REPLACE with IF NOT EXISTS (conflicting options).
- As a side effect of the code changes, we don't anymore have to internally re-prepare prepared statements with CREATE TABLE if the table exists.
- Made one code path for all testing if log table are in use.
- Better error message if one tries to create/drop/alter a log table in use
- Added back disabled rpl_row_create_table test as it now seams to work and includes a lot of interesting tests.
- Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
- Aligned CREATE OR REPLACE parsing code in sql_yacc.yy for TABLE and VIEW
- Changed interface for drop_temporary_table() to make it more reusable
- Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
- Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument. Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
- Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
- Added restart_trans_for_tables() to be able to restart a transaction.
- DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
- Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
- Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()
mysql-test/include/commit.inc:
With new code we get fewer status increments
mysql-test/r/commit_1innodb.result:
With new code we get fewer status increments
mysql-test/r/create.result:
Added testing of create or replace with timeout
mysql-test/r/create_or_replace.result:
Basic testing of CREATE OR REPLACE TABLE
mysql-test/r/partition_exchange.result:
New error message
mysql-test/r/ps_ddl.result:
Fewer reprepares with new code
mysql-test/suite/archive/discover.result:
Don't rediscover archive tables if the .frm file exists
(Sergei will look at this if there is a better way...)
mysql-test/suite/archive/discover.test:
Don't rediscover archive tables if the .frm file exists
(Sergei will look at this if there is a better way...)
mysql-test/suite/funcs_1/r/innodb_views.result:
New error message
mysql-test/suite/funcs_1/r/memory_views.result:
New error message
mysql-test/suite/rpl/disabled.def:
rpl_row_create_table should now be safe to use
mysql-test/suite/rpl/r/rpl_row_create_table.result:
Updated results after adding back disabled test
mysql-test/suite/rpl/t/rpl_create_if_not_exists.test:
Added comment
mysql-test/suite/rpl/t/rpl_row_create_table.test:
Added CREATE OR REPLACE TABLE test
mysql-test/t/create.test:
Added CREATE OR REPLACE TABLE test
mysql-test/t/create_or_replace-master.opt:
Create logs
mysql-test/t/create_or_replace.test:
Basic testing of CREATE OR REPLACE TABLE
mysql-test/t/partition_exchange.test:
Error number changed as we are now using same code for all log table change issues
mysql-test/t/ps_ddl.test:
Fewer reprepares with new code
sql/handler.h:
Moved things around a bit in a structure to get better alignment.
Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
Added 3 elements to end of HA_CREATE_INFO to be able to store state to add backs locks in case of LOCK TABLES.
sql/log.cc:
Reimplemented check_if_log_table():
- Simpler and faster usage
- Can give error messages
This gives us one code path for allmost all error messages if log tables are in use
sql/log.h:
New interface for check_if_log_table()
sql/slave.cc:
More logging
sql/sql_alter.cc:
New interface for check_if_log_table()
sql/sql_base.cc:
More documentation
Changed interface for drop_temporary_table() to make it more reusable
Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument. Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
Check for command number instead of open_strategy of CREATE TABLE was used.
Added restart_trans_for_tables() to be able to restart a transaction. This was needed in "create or replace ... select" between the drop table and the select.
sql/sql_base.h:
Added and updated function prototypes
sql/sql_class.h:
Added new prototypes to Locked_tables_list class
Added extra argument to select_create to avoid double call to eof() or send_error()
- I needed this in some edge case where the table was not created against expections.
sql/sql_db.cc:
New interface for check_if_log_table()
sql/sql_insert.cc:
Remember position to lock information so that we can reaquire table lock for LOCK TABLES + CREATE OR REPLACE TABLE SELECT. Later add back the lock by calling restore_lock().
Removed one not needed indentation level in create_table_from_items()
Ensure we don't call send_eof() or abort_result_set() twice.
sql/sql_lex.h:
Removed variable that I temporarly added in an earlier changeset
sql/sql_parse.cc:
Removed old test code (marked with QQ)
Ensure that we have open_strategy set as TABLE_LIST::OPEN_STUB in CREATE TABLE
Removed some IF NOT EXISTS code as this is now handled in create_table_table_impl().
Set OPTION_KEEP_LOGS later. This code had to be moved as the test for IF EXISTS has changed place.
DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
sql/sql_partition_admin.cc:
New interface for check_if_log_table()
sql/sql_rename.cc:
New interface for check_if_log_table()
sql/sql_table.cc:
New interface for check_if_log_table()
Moved some code in mysql_rm_table() under a common test.
- Safe as temporary tables doesn't have statistics.
- !is_temporary_table(table) test was moved out from drop_temporary_table() and merged with upper level code.
- Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
- Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()
- In mysql_create_like_table(), restore table->open_strategy() if it was changed.
- Re-test if table was a view after opening it.
sql/sql_table.h:
New prototype for mysql_create_table_no_lock()
sql/sql_yacc.yy:
Added syntax for CREATE OR REPLACE TABLE
Reuse new code for CREATE OR REPLACE VIEW
sql/table.h:
Added name for enum type
sql/table_cache.cc:
More DBUG
Add a test case for killing a waiting query in parallel replication.
Fix several bugs found:
- We should not wakeup_subsequent_commits() in ha_rollback_trans(), since we
do not know the right wakeup_error() to give.
- When a wait_for_prior_commit() is killed, we must unregister from the
waitee so we do not race and get an extra (non-kill) wakeup.
- We need to deal with error propagation correctly in queue_for_group_commit
when one thread is killed.
- Fix one locking issue in queue_for_group_commit(), we could unlock the
waitee lock too early and this end up processing wakeup() with insufficient
locking.
- Fix Xid_log_event::do_apply_event; if commit fails it must not update the
in-memory @@gtid_slave_pos state.
- Fix and cleanup some things in the rpl_parallel.cc error handling.
- Add a missing check for killed in the slave sql driver thread, to avoid a
race.
Attempt to read the master-bin.state file always, even if the
binlog files (master-bin.index and master-bin.XXXXXX) have been
deleted.
This allows to easily preserve the binlog state when provisioning
a new server from a copy of an old one, without needing to copy
over the binlog files themselves.
MDEV-4725: Incorrect binlog state recovery if crash while writing event group
The binlog state was not recovered correctly if XA is not used (eg. InnoDB
disabled), or if server crashed in the middle of writing an event group to the
binlog.
With this patch, we ensure that recovery of binlog state is done even if we do
not do the full XA binlog recovery, and we ensure that we only recover fully
written event groups into the binlog state.
There were some places where insufficient locking between
parallel threads could cause invalid memory accesses and
possibly other grief.
This patch adds the missing locking, and moves the locking
into the struct rpl_binlog_state methods to make it easier
to see that proper locking is in place everywhere.
The merge is still missing a few hunks related to temporary tables and
InnoDB log file size. The associated code did not seem to exist in
10.0, so the merge of that needs more work. Until this is fixed, there
are a number of test failures as a result.
Add an error code to the wait_for_commit facility.
Now, when a transaction fails, it can signal the error to
any subsequent transaction that is waiting for it to commit.
The waiting transactions then receive the error code back from
wait_for_prior_commit() and can handle the error appropriately.
Also fix one race that could cause crash if @@slave_parallel_threads
were changed several times quickly in succession.
- Made slaves temporary table multi-thread slave safe by adding mutex around save_temporary_table usage.
- rli->save_temporary_tables is the active list of all used temporary tables
- This is copied to THD->temporary_tables when temporary tables are opened and updated when temporary tables are closed
- Added THD->lock_temporary_tables() and THD->unlock_temporary_tables() to simplify this.
- Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code.
- Added is_part_of_group() to mark functions that are part of the next function. This replaces setting IN_STMT when events are executed.
- Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code.
- If slave_skip_counter is set run things in single threaded mode. This simplifies code for skipping events.
- Updating state of relay log (IN_STMT and IN_TRANSACTION) is moved to one single function: update_state_of_relay_log()
We can't use OPTION_BEGIN to check for the state anymore as the sql_driver and sql execution threads may be different.
Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts
is_in_group() is now independent of state of executed transaction.
- Reset thd->transaction.all.modified_non_trans_table() if we did set it for single table row events.
This was mainly for keeping the flag as documented.
- Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it.
- Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
- Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
- Changed some functions to take rpl_group_info instead of Relay_log_info to make them multi-slave safe and to simplify usage
- do_shall_skip()
- continue_group()
- sql_slave_killed()
- next_event()
- Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure.
- set_thd_in_use_temporary_tables() removed as in_use is set on usage
- Added information to thd_proc_info() which thread is waiting for slave mutex to exit.
- In open_table() reuse code from find_temporary_table()
Other things:
- More DBUG statements
- Fixed the rpl_incident.test can be run with --debug
- More comments
- Disabled not used function rpl_connect_master()
mysql-test/suite/perfschema/r/all_instances.result:
Moved sleep_lock and sleep_cond to rpl_group_info
mysql-test/suite/rpl/r/rpl_incident.result:
Updated result
mysql-test/suite/rpl/t/rpl_incident-master.opt:
Not needed anymore
mysql-test/suite/rpl/t/rpl_incident.test:
Fixed that test can be run with --debug
sql/handler.cc:
More DBUG_PRINT
sql/log.cc:
More comments
sql/log_event.cc:
Added DBUG statements
do_shall_skip(), continue_group() now takes rpl_group_info param
Use is_begin(), is_commit() and is_rollback() functions instead of inspecting query string
We don't have set slaves temporary tables 'in_use' as this is now done when tables are opened.
Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
Use IN_TRANSACTION flag to test state of relay log.
In rows_event_stmt_cleanup() reset thd->transaction.all.modified_non_trans_table if we had set this before.
sql/log_event.h:
do_shall_skip(), continue_group() now takes rpl_group_info param
Added is_part_of_group() to mark events that are part of the next event. This replaces setting IN_STMT when events are executed.
Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code.
sql/log_event_old.cc:
Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
do_shall_skip(), continue_group() now takes rpl_group_info param
sql/log_event_old.h:
Added is_part_of_group() to mark events that are part of the next event.
do_shall_skip(), continue_group() now takes rpl_group_info param
sql/mysqld.cc:
Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it.
Relay_log_info::sleep_lock -> Rpl_group_info::sleep_lock
Relay_log_info::sleep_cond -> Rpl_group_info::sleep_cond
sql/mysqld.h:
Updated types and names
sql/rpl_gtid.cc:
More DBUG
sql/rpl_parallel.cc:
Updated TODO section
Set thd for event that is execution
Use new is_begin(), is_commit() and is_rollback() functions.
More comments
sql/rpl_rli.cc:
sql_thd -> sql_driver_thd
Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts.
Reset table->in_use for temporary tables as the table may have been used by another THD.
Use IN_TRANSACTION instead of OPTION_BEGIN to check state of relay log.
Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
sql/rpl_rli.h:
Changed relay log state flags to bit masks instead of bit positions (most other code we have uses bit masks)
Added IN_TRANSACTION to mark if we are in a BEGIN ... COMMIT section.
save_temporary_tables is now thread safe
Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code
is_in_group() is now independent of state of executed transaction.
sql/slave.cc:
Simplifed arguments to io_salve_killed(), sql_slave_killed() and check_io_slave_killed(); No reason to supply THD as this is part of the given structure.
set_thd_in_use_temporary_tables() removed as in_use is set on usage in sql_base.cc
sql_thd -> sql_driver_thd
More DBUG
Added update_state_of_relay_log() which will calculate the IN_STMT and IN_TRANSACTION state of the relay log after the current element is executed.
If slave_skip_counter is set run things in single threaded mode.
Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure.
Added information to thd_proc_info() which thread is waiting for slave mutex to exit.
Disabled not used function rpl_connect_master()
Updated argument to next_event()
sql/sql_base.cc:
Added mutex around usage of slave's temporary tables. The active list is always kept up to date in sql->rgi_slave->save_temporary_tables.
Clear thd->temporary_tables after query (safety)
More DBUG
When using temporary table, set table->in_use to current thd as the THD may be different for slave threads.
Some code is ifdef:ed with REMOVE_AFTER_MERGE_WITH_10 as the given code in 10.0 is not yet in this tree.
In open_table() reuse code from find_temporary_table()
sql/sql_binlog.cc:
rli->sql_thd -> rli->sql_driver_thd
Remove duplicate setting of rgi->rli
sql/sql_class.cc:
Added helper functions rgi_lock_temporary_tables() and rgi_unlock_temporary_tables()
Would have been nicer to have these inline, but there was no easy way to do that
sql/sql_class.h:
Added functions to protect slaves temporary tables
sql/sql_parse.cc:
Added DBUG_PRINT
sql/transaction.cc:
Added comment
Implement @@gtid_binlog_state. This is the internal state of the binlog
(most recent GTID logged for every domain_id and server_id). This allows
to save the state before RESET MASTER and restore it afterwards.
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
Impement options --binlog-commit-wait-count and
--binlog-commit-wait-usec.
These options permit the DBA to deliberately increase latency
of an individual commit to get more transactions in each
binlog group commit. This increases the opportunity for
parallel replication on the slave, and can also decrease I/O
load on the master.
The options also make it easier to test the parallel
replication with mysql-test-run.
- make the test output stable
- make Log_to_csv_event_handler::log_slow() to write the value
of thd->thread_id (it didn't, and so 0 was always logged).
Implement facility for the commit in one thread to wait for the commit of
another to complete first. The wait is done in a way that does not hinder
that a waiter and a waitee can group commit together with a single fsync()
in both binlog and InnoDB. The wait is done efficiently with respect to
locking.
The patch was originally made to support TaoBao parallel replication with
in-order commit; now it will be adapted to also be used for parallel
replication of group-committed transactions.
A waiter THD registers itself with a prior waitee THD. The waiter will then
complete its commit at the earliest in the same group commit of the waitee
(when using binlog). The wait can also be done explicitly by the waitee.
See also MySQL Bug #39750 and similar ones.
Fix my_delete() on Windows, to safely remvoe files on Windows, including files that are opened by another threads in the same process, antiviruses and backup applications. If file to be deleted is also opened by another thread, the file is renamed to unique name prior to deletion - this makes it possible to create file with the same name right after deletion.
With this patch my_delete_allow_opened() becomes obsolete and is replaced with my_delete().
This patch is rework of the patch http://lists.mysql.com/commits/59327 for MySQL bug#39750.
When @@GLOBAL.gtid_strict_mode=1, then certain operations result
in error that would otherwise result in out-of-order binlog files
between servers.
GTID sequence numbers are now allocated independently per domain;
this results in less/no holes in GTID sequences, increasing the
likelyhood that diverging binlogs will be caught by the slave when
GTID strict mode is enabled.
Change of user interface to be more logical and more in line with expectations
to work similar to old-style replication.
User can now explicitly choose in CHANGE MASTER whether binlog position is
taken into account (master_gtid_pos=current_pos) or not (master_gtid_pos=
slave_pos) when slave connects to master.
@@gtid_pos is replaced by three separate variables @@gtid_slave_pos (can
be set by user, replicated GTIDs only), @@gtid_binlog_pos (read only), and
@@gtid_current_pos (a combination of the two, most recent GTID within each
domain). mysql.rpl_slave_state is renamed to mysql.gtid_slave_pos to match.
This fixes MDEV-4474.
Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".
Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.
Solaris fixes:
- Fixed that wait_timeout_func and wait_timeout tests works on solaris
- We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO).
- Fixed that compile-solaris-amd64-debug works (before that we got a wrong ELF class: ELFCLASS64 on linkage)
- Added missing sync_with_master
Other bug fixes:
- Free memory for rpl_global_gtid_binlog_state before exit() to avoid 'accessing uninitalized mutex' error.
BUILD/FINISH.sh:
Fixed issues on Solaris with ksh
BUILD/compile-solaris-amd64-debug:
Added missing -m64 flag
configure.cmake:
We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO)
mysql-test/suite/rpl/t/rpl_gtid_mdev4473.test:
- Added missing sync_with_master (fix by knielsen)
sql-common/client.c:
Added () to get rid of compiler warning
sql/item_strfunc.cc:
Fixed compiler warning
sql/log.cc:
Free memory for static variable rpl_global_gtid_binlog_state before exit()
- If we are compiling with safemalloc, we would try to call sf_free() for some members after sf_terminate() was called, which would result of trying to access the uninitalized mutex 'sf_mutex'
sql/multi_range_read.cc:
Fixed compiler warnings of converting double to ulong.
sql/opt_range.cc:
Fixed compiler warnings of converting double to ulong or uint
- Better to have all variables that can be number of rows as 'ha_rows'
sql/rpl_gtid.cc:
Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/rpl_gtid.h:
Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/set_var.cc:
Fixed compiler warning
sql/sql_join_cache.cc:
Fixed compiler warnings of converting double to uint
sql/sql_show.cc:
Added cast to get rid of compiler warning
sql/sql_statistics.cc:
Remove code that didn't do anything.
(store_record() with record[0] is a no-op)
storage/xtradb/os/os0file.c:
Added __attribute__ ((unused))
support-files/compiler_warnings.supp:
Ignore warnings from atomic_add_64_nv
(was not able to fix this with a cast as the macro is a bit different between systems)
vio/viosocket.c:
Added more DBUG_PRINT
