An INSERT query log event is preceeded by an INSERT_ID intvar event if the
INSERT allocates a new auto_increment value. But if we ignore the INSERT
due to --replicate-ignore-table or similar, then the INSERT_ID event is
still executed, and the set value of INSERT_ID lingers around in the
slave sql thread THD object indefinitely until the next INSERT that
happens to need allocation of a new auto_increment value.
Normally this does not cause problems as such following INSERT would
normally come with its own INSERT_ID event. In this bug, the user had
a trigger on the slave which was missing on the master, and this
trigger had an INSERT which could be affected. In any case, it seems
better to not leave a stray INSERT_ID hanging around in the sql thread
THD indefinitely.
Note that events can also be skipped from apply_event_and_update_pos();
however it is not possible in that code to skip the INSERT without also
skipping the INSERT_ID event.
Open issues:
- A better fix for #57688; Igor is working on this
- Test failure in index_merge_innodb.test ; Igor promised to look at this
- Some Innodb tests fails (need to merge with latest xtradb) ; Kristian promised to look at this.
- Failing tests: innodb_plugin.innodb_bug56143 innodb_plugin.innodb_bug56632 innodb_plugin.innodb_bug56680 innodb_plugin.innodb_bug57255
- Werror is disabled; Should be enabled after merge with xtradb.
"Grantor" columns' data is lost when replicating mysql.tables_priv.
Slave SQL thread used its default user ''@'' as the grantor of GRANT|REVOKE
statements executing on it.
In this patch, current user is put in query log event for all GRANT and REVOKE
statement, SQL thread uses the user in query log event as grantor.
mysql-test/suite/rpl/r/rpl_do_grant.result:
Add test for this bug.
mysql-test/suite/rpl/t/rpl_do_grant.test:
Add test for this bug.
sql/log_event.cc:
Refactoring THD::current_user_used and related functions.
current_user_used is used to judge if current user should be
binlogged in query log event. So it is better to call it m_binlog_invoker.
The related functions are renamed too.
sql/sql_class.cc:
Refactoring THD::current_user_used and related functions.
current_user_used is used to judge if current user should be
binlogged in query log event. So it is better to call it m_binlog_invoker.
The related functions are renamed too.
sql/sql_class.h:
Refactoring THD::current_user_used and related functions.
current_user_used is used to judge if current user should be
binlogged in query log event. So it is better to call it m_binlog_invoker.
The related functions are renamed too.
sql/sql_parse.cc:
Call binlog_invoker() for GRANT and REVOKE statements.
Rows events were applied wrongly on the temporary table with the same name.
But rows events are generated only for base tables. As temporary
table's data never be binlogged on row mode. Normally, base table of the
same name cannot be updated if a temporary table has the same name.
But there are two cases which can generate rows events on
the base table of same name.
Case1: 'CREATE TABLE ... SELECT' statement.
In mixed format, it will generate rows events if it is unsafe.
Case2: Drop a transactional temporary table in a transaction
(happens only on 5.5+).
BEGIN;
DROP TEMPORARY TABLE t1; # t1 is a InnoDB table
INSERT INTO t1 VALUES(rand()); # t1 is a MyISAM table
COMMIT;
'DROP TEMPORARY TABLE' will be put in the transaction cache and
binlogged after the rows events generated by the 'INSERT' statement.
After this patch, slave opens only base table when applying a rows event.
replication aborts
When recieving a 'SLAVE STOP' command, slave SQL thread will roll back the
transaction and stop immidiately if there is only transactional table updated,
even through 'CREATE|DROP TEMPOARY TABLE' statement are in it. But These
statements can never be rolled back. Because the temporary tables to the user
session mapping remain until 'RESET SLAVE', Therefore it will abort SQL thread
with an error that the table already exists or doesn't exist, when it restarts
and executes the whole transaction again.
After this patch, SQL thread always waits till the transaction ends and then stops,
if 'CREATE|DROP TEMPOARY TABLE' statement are in it.
mysql-test/extra/rpl_tests/rpl_stop_slave.test:
Auxiliary file which is used to test this bug.
mysql-test/suite/rpl/t/rpl_stop_slave.test:
Test case for this bug.
sql/slave.cc:
Checking if OPTION_KEEP_LOG is set. If it is set, SQL thread should wait
until the transaction ends.
sql/sql_parse.cc:
Add a debug point for testing this bug.
After ALTER TABLE which changed only table's metadata, row-based
binlog sometimes got corrupted since the tablemap was unexpectedly
set to 0 for subsequent updates to the same table.
ALTER TABLE which changed only table's metadata always reset
table_map_id for the table share to 0. Despite the fact that
0 is a valid value for table_map_id, this step caused problems
as it could have created situation in which we had more than
one table share with table_map_id equal 0. If more than one
table with table_map_id are 0 were updated in the same statement,
updates to these different tables were written into the same
rows event. This caused slave server to crash.
This bug happens only on 5.1. It doesn't affect 5.5+.
This patch solves this problem by ensuring that ALTER TABLE
statements which change metadata only never reset table_map_id
to 0. To do this it changes reopen_table() to correctly use
refreshed table_map_id value instead of using the old one/
resetting it.
mysql-test/suite/rpl/r/rpl_alter.result:
Add test for BUG#56226
mysql-test/suite/rpl/t/rpl_alter.test:
Add test for BUG#56226
When slave executes a transaction bigger than slave's max_binlog_cache_size,
slave will crash. It is caused by the assert that server should only roll back
the statement but not the whole transaction if the error ER_TRANS_CACHE_FULL
happens. But slave sql thread always rollbacks the whole transaction when
an error happens.
Ather this patch, we always clear any error set in sql thread(it is different
from the error in 'SHOW SLAVE STATUS') and it is cleared before rolling back
the transaction.
mysql-test/suite/rpl/r/rpl_binlog_max_cache_size.result:
SET binlog_cache_size and max_binlog_cache_size for all test cases.
Add test case for bug#55375.
mysql-test/suite/rpl/t/rpl_binlog_max_cache_size-master.opt:
binlog_cache_size and max_binlog_cache_size can be set in the client connection.
so remove this option file.
mysql-test/suite/rpl/t/rpl_binlog_max_cache_size.test:
SET binlog_cache_size and max_binlog_cache_size for all test cases.
Add test case for bug#55375.
sql/log_event.cc:
Some functions don't return the error code, so it is a wrong error code.
The error should always be set into thd->main_da. So we use
slave_rows_error_report to report the right error.
sql/slave.cc:
exec_relay_log_event() need call cleanup_context() to clear context.
clearup_context() will call end_trans().
Clear thd's error before cleanup_context. It avoid to trigger the assert
which cause this bug.
- Changed to still use bcmp() in certain cases becasue
- Faster for short unaligneed strings than memcmp()
- Bettern when using valgrind
- Changed to use my_sprintf() instead of sprintf() to get higher portability for old systems
- Changed code to use MariaDB version of select->skip_record()
- Removed -%::SCCS/s.% from Makefile.am:s to remove automake warnings
'CREATE TABLE IF NOT EXISTS ... SELECT' behaviour
BUG#55474, BUG#55499, BUG#55598, BUG#55616 and BUG#55777 are fixed
in this patch too.
This is the 5.1 part.
It implements:
- if the table exists, binlog two events: CREATE TABLE IF NOT EXISTS
and INSERT ... SELECT
- Insert nothing and binlog nothing on master if the existing object
is a view. It only generates a warning that table already exists.
mysql-test/r/trigger.result:
Ather this patch, 'CREATE TABLE IF NOT EXISTS ... SELECT' will not
insert anything if the creating table already exists and is a view.
sql/sql_class.h:
Declare virtual function write_to_binlog() for select_insert.
It's used to binlog 'create select'
sql/sql_insert.cc:
Implement write_to_binlog();
Use write_to_binlog() instead of binlog_query() to binlog the statement.
if the table exists, binlog two events: CREATE TABLE IF NOT EXISTS
and INSERT ... SELECT
sql/sql_lex.h:
Declare create_select_start_with_brace and create_select_pos.
They are helpful for binlogging 'create select'
sql/sql_parse.cc:
Do nothing on master if the existing object is a view.
sql/sql_yacc.yy:
Record the relative postion of 'SELECT' in the 'CREATE ...SELECT' statement.
Record whether there is a '(' before the 'SELECT' clause.
* no automatic --loose-skip-innodb added by mtr based on the test name.
instead loose-skip-innodb is now in the default_mysqld.cnf
* have_innodb_plugin.inc is changed to give a verbose "skip" message
(instead of "require: true")
* My::Suite class. It's support in mtr, and everywhere
* support for suite.pm
* when sorting tests, take combinations into account
* support for SUITENAME_COMBINATIONS
* no special treatment for innodb_plugin in mtr_cases.pm
* two special pre-created config groups: ENV and OPT
* allow option names to start from #
* allow magic option to have an argument
* remove dead code
* fix @-substitution to works as expected
* new processes take the value of $opt_verbose automatically, no need to pass it to a constructor
* innodb_plugin suite uses suite.pm and combinations file to test as much as possible
(innodb plugin, xtradb plugin, xtradb static - whatever available)
* besides test-master.opt and test-slave.opt a test.opt file is also
loaded, both for master and slave
* .opt files for all included files are loaded too
* progress report in the xterm titlebar
mysql-test/suite/percona/percona_log_slow_slave_statements.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-control_global_slow.result:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-control_global_slow.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-log_slow_filter.result:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-log_slow_filter.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-log_slow_verbosity.result:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-log_slow_verbosity.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-long_query_time.result:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-long_query_time.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-microseconds_in_slow_query_log.result:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-microseconds_in_slow_query_log.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-min_examined_row_limit.result:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-min_examined_row_limit.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-use_global_long_query_time.result:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/percona/percona_slow_query_log-use_global_long_query_time.test:
Replaced 'grep | wc -l' with 'grep -c' as output of wc -l is not portable (for example on Mac)
mysql-test/suite/rpl/r/rpl_stm_until.result:
Updated results
mysql-test/suite/rpl/t/rpl_stm_until.test:
Fixed random failure in test suite: master could be reading data from master-bin.00001 at the same time as test suite was changing it.
With statement- or mixed-mode logging, "LOAD DATA INFILE" queries
are written to the binlog using special types of log events.
When mysqlbinlog reads such events, it re-creates the file in a
temporary directory with a generated filename and outputs a
"LOAD DATA INFILE" query where the filename is replaced by the
generated file. The temporary file is not deleted by mysqlbinlog
after termination.
To fix the problem, in mixed mode we go to row-based. In SBR, we
document it to remind user the tmpfile is left in a temporary
directory.
mysql-test/extra/rpl_tests/rpl_loaddata.test:
Updated for Bug#34283
mysql-test/suite/binlog/r/binlog_mixed_load_data.result:
Test result for BUG#34283.
mysql-test/suite/binlog/t/binlog_killed_simulate.test:
Updated for Bug#34283
mysql-test/suite/binlog/t/binlog_mixed_load_data.test:
Added the test file to verify that 'load data infile...' statement
will go to row-based in mixed mode.
mysql-test/suite/binlog/t/binlog_stm_blackhole.test:
Updated for Bug#34283
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
Updated for Bug#34283
mysql-test/suite/rpl/t/rpl_loaddata_fatal.test:
Updated for Bug#34283
mysql-test/suite/rpl/t/rpl_loaddata_map.test:
Updated for Bug#34283
mysql-test/suite/rpl/t/rpl_slave_load_remove_tmpfile.test:
Updated for Bug#34283
mysql-test/suite/rpl/t/rpl_stm_log.test:
Updated for Bug#34283
sql/sql_load.cc:
Added code to go to row-based in mixed mode for
'load data infile ...' statement
A CREATE...SELECT that fails is written to the binary log if a non-transactional
statement is updated. If the logging format is ROW, the CREATE statement and the
changes are written to the binary log as distinct events and by consequence the
created table is not rolled back in the slave.
In this patch, we opted to let the slave goes out of sync by not writting to the
binary log the CREATE statement. We do this by simply reseting the binary log's
cache.
mysql-test/suite/rpl/r/rpl_drop.result:
Added a test case.
mysql-test/suite/rpl/t/rpl_drop.test:
Added a test case.
sql/log.cc:
Introduced a function to clean up the cache.
sql/log.h:
Introduced a function to clean up the cache.
sql/sql_insert.cc:
Cleaned up the binary log cache if a CREATE...SELECT fails.
Speed up some PBXT tests by adding begin...commit around creating of testing tables.
include/my_base.h:
Fixed wrong constant
mysql-test/mysql-test-run.pl:
Print MariaDB instead of MySQL
mysql-test/r/range.result:
Move test that required partitions to parts.optimizer
mysql-test/suite/innodb_plugin/t/disabled.def:
Disable test that causes valgrind warning about not released memory in xtradb
mysql-test/suite/parts/r/optimizer.result:
Moved from range.result
mysql-test/suite/parts/t/optimizer.test:
Moved from range.test
mysql-test/suite/pbxt/r/join_nested.result:
Updated results after optimizer changes
mysql-test/suite/pbxt/r/renamedb.result:
Updated test for new error message
mysql-test/suite/pbxt/t/check.test:
Speed up test
mysql-test/suite/pbxt/t/count_distinct2.test:
Speed up test
mysql-test/suite/pbxt/t/derived.test:
Speed up test
mysql-test/suite/pbxt/t/renamedb.test:
Updated test for new error message
mysql-test/suite/rpl/r/rpl_log_pos.result:
Updated results
mysql-test/suite/rpl/t/rpl_log_pos.test:
Update test to read from a position that has 'known wrong' data.
The orignal test read a timestamp, so the error message could differ between runs.
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
Sync to slave to make test predictable
mysql-test/t/events_time_zone.test:
Extend wait to make test predictable
mysql-test/t/range.test:
Move test that required partitions to parts.optimizer
sql/sql_list.h:
Fixed compiler warning
sql/sql_load.cc:
buffer was not freed in some error conditions
tests/mysql_client_test.c:
Fixed compiler warning
/*![:version:] Query Code */, where [:version:] is a sequence of 5
digits representing the mysql server version(e.g /*!50200 ... */),
is a special comment that the query in it can be executed on those
servers whose versions are larger than the version appearing in the
comment. It leads to a security issue when slave's version is larger
than master's. A malicious user can improve his privileges on slaves.
Because slave SQL thread is running with SUPER privileges, so it can
execute queries that he/she does not have privileges on master.
This bug is fixed with the logic below:
- To replace '!' with ' ' in the magic comments which are not applied on
master. So they become common comments and will not be applied on slave.
- Example:
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /*!99999 ,(3)*/
will be binlogged as
'INSERT INTO t1 VALUES (1) /*!10000, (2)*/ /* 99999 ,(3)*/
mysql-test/suite/rpl/t/rpl_conditional_comments.test:
Test the patch for this bug.
sql/mysql_priv.h:
Rename inBuf as rawBuf and remove the const limitation.
sql/sql_lex.cc:
To replace '!' with ' ' in the magic comments which are not applied on
master.
sql/sql_lex.h:
Remove the const limitation on parameter buff, as it can be modified in the function since
this patch.
Add member function yyUnput for Lex_input_stream. It set a character back the query buff.
sql/sql_parse.cc:
Rename inBuf as rawBuf and remove the const limitation.
sql/sql_partition.cc:
Remove the const limitation on parameter part_buff, as it can be modified in the function since
this patch.
sql/sql_partition.h:
Remove the const limitation on parameter part_buff, as it can be modified in the function since
this patch.
sql/table.h:
Remove the const limitation on variable partition_info, as it can be modified since
this patch.
Problem: when SHOW BINLOG EVENTS was issued, it increased the value of
@@session.max_allowed_packet. This allowed a non-root user to increase
the amount of memory used by her thread arbitrarily. Thus, it removes
the bound on the amount of system resources used by a client, so it
presents a security risk (DoS attack).
Fix: it is correct to increase the value of @@session.max_allowed_packet
while executing SHOW BINLOG EVENTS (see BUG 30435). However, the
increase should only be temporary. Thus, the fix is to restore the value
when SHOW BINLOG EVENTS ends.
The value of @@session.max_allowed_packet is also increased in
mysql_binlog_send (i.e., the binlog dump thread). It is not clear if this
can cause any trouble, since normally the client that issues
COM_BINLOG_DUMP will not issue any other commands that would be affected
by the increased value of @@session.max_allowed_packet. However, we
restore the value just in case.
mysql-test/suite/rpl/r/rpl_packet.result:
update result file
mysql-test/suite/rpl/t/rpl_packet.test:
Add test that verifies that @@session.max_allowed_packet does not change
when issuing SHOW BINLOG EVENTS.
Make previous sub-test clean up.
Add comments listing the bugs in this test case.
sql/sql_repl.cc:
Restore the old value of thd->variables.max_allowed_packet at the
end of mysql_binlog_send and mysql_show_binlog_events.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENTbut, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, session's user will be written into query log events
if these statements call CURREN_USER() or 'ALTER EVENT' does not assign a definer.
mysql-test/include/diff_tables.inc:
Expend its abilities.
Now it can diff not only in sessions of 'master' and 'slave', but
other sessions as well.
mysql-test/include/rpl_diff_tables.inc:
Diff the same table between master and slaves.
sql/log_event.cc:
session's user will be written into Query_log_event, if is_current_user_used() is TRUE.
On slave SQL thread, Only thd->variables.current_user is written into Query_log_event,
if it exists.
sql/sql_acl.cc:
On slave SQL thread, grantor should copy from thd->variables.current_user, if it exists
sql/sql_class.h:
On slave SQL thread, thd->variables.current_user is used to store the applying event's
invoker.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENTbut, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, session's user will be written into query log events
if these statements call CURREN_USER() or 'ALTER EVENT' does not assign a definer.
mysql-test/include/diff_tables.inc:
Expend its abilities.
Now it can diff not only in sessions of 'master' and 'slave', but
other sessions as well.
sql/log_event.cc:
session's user will be written into Query_log_event, if is_current_user_used() is TRUE.
On slave SQL thread, Only thd->invoker is written into Query_log_event,
if it exists.
sql/sql_acl.cc:
On slave SQL thread, grantor should copy from thd->invoker, if it exists
sql/sql_class.h:
On slave SQL thread, thd->invoker is used to store the applying event's
invoker.
* fully support --mysqld=--plugin-load=xxxx
* uniformly support all loadable plugins, no need to hard-code
every new plugin in mtr
* autodetect MTR_VS_CONFIG on windows
When using Unique Keys with nullable parts in RBR, the slave can
choose the wrong row to update. This happens because a table with
an unique key containing nullable parts cannot strictly guarantee
uniqueness. As stated in the manual, for all engines, a UNIQUE
index allows multiple NULL values for columns that can contain
NULL.
We fix this at the slave by extending the checks before assuming
that the row found through an unique index is is the correct
one. This means that when a record (R) is fetched from the storage
engine and a key that is not primary (K) is used, the server does
the following:
- If K is unique and has no nullable parts, it returns R;
- Otherwise, if any field in the before image that is part of K
is null do an index scan;
- If there is no NULL field in the BI part of K, then return R.
A side change: renamed the existing test case file and added a
test case covering the changes in this patch.
Add code to waiting for a set of errors.
Add code to waiting for an error instead of waiting for io thread to stop, as
after 'START SLAVE', the status of io thread is still not running.
But it doesn't mean slave io thread encounters an error.
Fixed some bugs introduced in 5.1.47
Disabled some tests until we have merged with latest Xtradb
configure.in:
Added testing if valgrind/memcheck.h exists
storage/pbxt/src/ha_pbxt.cc:
LOCK_plugin is not anymore locked in init
