binlog
Mixing transactional (T) and non-transactional (N) tables on behalf of a
transaction may lead to inconsistencies among masters and slaves in STATEMENT
mode. The problem stems from the fact that although modifications done to
non-transactional tables on behalf of a transaction become immediately visible
to other connections they do not immediately get to the binary log and therefore
consistency is broken. Although there may be issues in mixing T and M tables in
STATEMENT mode, there are safe combinations that clients find useful.
In this bug, we fix the following issue. Mixing N and T tables in multi-level
(e.g. a statement that fires a trigger) or multi-table table statements (e.g.
update t1, t2...) were not handled correctly. In such cases, it was not possible
to distinguish when a T table was updated if the sequence of changes was N and T.
In a nutshell, just the flag "modified_non_trans_table" was not enough to reflect
that both a N and T tables were changed. To circumvent this issue, we check if an
engine is registered in the handler's list and changed something which means that
a T table was modified.
Check WL 2687 for a full-fledged patch that will make the use of either the MIXED or
ROW modes completely safe.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Truncate statement is wrapped in BEGIN/COMMIT.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Truncate statement is wrapped in BEGIN/COMMIT.
This is a post-push fix addressing review requests and
problems with extra warnings.
Problem 1: The sub-statement where an unsafe warning was detected was
printed as part of the warning. This was ok for statements that
were unsafe due to, e.g., calls to UUID(), but did not make
sense for statements that were unsafe because there was more than
one autoincrement column (unsafeness in this case comes from the
combination of several sub-statements).
Fix 1: Instead of printing the sub-statement, print an explanation
of why the statement is unsafe.
Problem 2:
When a recursive construct (i.e., stored proceure, stored
function, trigger, view, prepared statement) contained several
sub-statements, and at least one of them was unsafe, there would be
one unsafeness warning per sub-statement - even for safe
sub-statements.
Fix 2:
Ensure that each type of warning is printed at most once, by
remembering throughout the execution of the statement which types
of warnings have been printed.
mysql-test/extra/rpl_tests/create_recursive_construct.inc:
- Clarified comment per review request.
- Added checks for the number of warnings in each invocation.
mysql-test/extra/rpl_tests/rpl_insert_delayed.test:
Per review request, replaced @@session.binlog_format by
@@global.binlog_format, since INSERT DELAYED reads the global
variable. (In this test case, the two variables have the same
value, so the change is cosmetic.)
mysql-test/r/sp_trans.result:
updated result file
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_ps.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
updated result file
mysql-test/suite/binlog/r/binlog_unsafe.result:
Updated result file. Note that duplicate warnings are now gone.
mysql-test/suite/binlog/t/binlog_unsafe.test:
- Added tests for: (1) a statement that is unsafe in many ways;
(2) a statement that is unsafe in the same way several times.
- Use -- style to invoke mysqltest commands.
mysql-test/suite/rpl/r/rpl_stm_found_rows.result:
updated result file
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
updated result file
mysql-test/suite/rpl/t/rpl_mix_found_rows.test:
Per review request, added comment explaining what the test case
does (copied from rpl_stm_found_rows.test)
mysql-test/suite/rpl/t/rpl_stm_found_rows.test:
Clarified grammar in comment.
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
Updated result file.
sql/item_create.cc:
Made set_stmt_unsafe take one parameter, describing the
type of unsafeness.
sql/sp_head.cc:
Added unsafe_flags field and made it hold all the unsafe flags.
sql/sp_head.h:
- Removed the BINLOG_ROW_BASED_IF_MIXED flag from m_flags.
Instead, we use the new unsafe_flags field to hold the
unsafeness state of the sp.
- Made propagate_attributes() copy all unsafe flags.
sql/sql_base.cc:
- Made LEX::set_stmt_unsafe() take an extra argument.
- Made binlog_unsafe_warning_flags store the type of unsafeness.
- Per review requests, clarified comments
- Added DBUG printouts
sql/sql_class.cc:
- Made warnings be generated in issue_warnings() and call that from
binlog_query(). Wrote issue_warnings(), which prints zero or more
warnings, avoiding to print warnings more than once per statement.
- Per review request, added @todo so that we remember to assert
correct behavior in binlog_query.
sql/sql_class.h:
- Removed BINLOG_WARNING_PRINTED
- Use [set|clear]_current_stmt_binlog_row_based() instead of
modifying the flag directly.
- added issue_unsafe_warnings() (only called from binlog_unsafe)
- Per review request, improved some documentation.
sql/sql_insert.cc:
Added extra argument to LEX::set_stmt_unsafe()
sql/sql_lex.h:
- Added enum_binlog_stmt_unsafe, listing all types of unsafe
statements.
- Per review requests, improved many comments for member
functions.
- Added [get|set]_stmt_unsafe_flags(), which return/set all the
unsafe flags for a statement.
sql/sql_parse.cc:
- Renamed binlog_warning_flags to binlog_unsafe_warning_flags.
- Per review requests, improved comment.
sql/sql_view.cc:
Made views propagate all the new unsafe flags.
sql/sql_yacc.yy:
Added parameter to set_stmt_unsafe().
storage/innobase/handler/ha_innodb.cc:
Per review requests, replaced DBUG_EXECUTE_IF() by DBUG_EVALUATE_IF().
The "get_master_version_and_clock(...)" function in sql/slave.cc ignores
error and passes directly when queries fail, or queries succeed
but the result retrieved is empty.
The "get_master_version_and_clock(...)" function should try to reconnect master
if queries fail because of transient network problems, and fail otherwise.
The I/O thread should print a warning if the some system variables do not
exist on master (very old master)
mysql-test/extra/rpl_tests/rpl_get_master_version_and_clock.test:
Added test file for bug #45214
mysql-test/suite/rpl/r/rpl_get_master_version_and_clock.result:
Added test result for bug #45214
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test:
Added test file for bug #45214
sql/slave.cc:
The 'is_network_error()' function is added for checking if the error is caused by network.
Added a new return value (2) to 'get_master_version_and_clock()' function result set
to indicate transient network errors when queries fail, and the caller should
try to reconnect in this case.
NOTE: This undoes changes by BUG#42829 in sql_class.cc:binlog_query().
I will revert the change in a post-push fix (the binlog filter should
be checked in sql_base.cc:decide_logging_format()).
General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
mysql-test/extra/rpl_tests/create_recursive_construct.inc:
Added auxiliary file used by binlog_unsafe.test to create and
execute recursive constructs
(functions/procedures/triggers/views/prepared statements).
mysql-test/extra/rpl_tests/rpl_foreign_key.test:
removed unnecessary set @@session.binlog_format
mysql-test/extra/rpl_tests/rpl_insert_delayed.test:
Filter out table id from table map events in binlog listing.
Got rid of $binlog_format_statement.
mysql-test/extra/rpl_tests/rpl_ndb_apply_status.test:
disable warnings around call to unsafe procedure
mysql-test/include/rpl_udf.inc:
Disabled warnings for code that generates warnings
for some binlog formats. That would otherwise cause
inconsistencies in the result file.
mysql-test/r/mysqldump.result:
Views are now unsafe if they contain a LIMIT clause.
That fixed BUG#45831. Due to BUG#45832, a warning is
printed for the CREATE VIEW statement.
mysql-test/r/sp_trans.result:
Unsafe statements in stored procedures did not give a warning if
binlog_format=statement. This is BUG#45824. Now they do, so this
result file gets a new warning.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Error message changed.
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
INSERT DELAYED didn't generate a warning when binlog_format=STATEMENT.
That was BUG#45825. Now there is a warning, so result file needs to be
updated.
mysql-test/suite/binlog/r/binlog_stm_ps.result:
Changed error message.
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file:
- error message changed
- added test for most combinations of unsafe constructs invoked
from recursive constructs
- INSERT DELAYED now gives a warning (because BUG#45826 is fixed)
- INSERT..SELECT..LIMIT now gives a warning from inside recursive
constructs (because BUG#45785 was fixed)
- When a recursive construct (e.g., stored proc or function)
contains more than one statement, at least one of which is
unsafe, then all statements in the recursive construct give
warnings. This is a new bug introduced by this changeset.
It will be addressed in a post-push fix.
mysql-test/suite/binlog/t/binlog_innodb.test:
Changed error code for innodb updates with READ COMMITTED or
READ UNCOMMITTED transaction isolation level and
binlog_format=statement.
mysql-test/suite/binlog/t/binlog_multi_engine.test:
The error code has changed for statements where more than one
engine is involved and one of them is self-logging.
mysql-test/suite/binlog/t/binlog_unsafe-master.opt:
Since binlog_unsafe now tests unsafe-ness of UDF's, we need an extra
flag in the .opt file.
mysql-test/suite/binlog/t/binlog_unsafe.test:
- Clarified comment.
- Rewrote first part of test. Now it tests not only unsafe variables
and functions, but also unsafe-ness due to INSERT..SELECT..LIMIT,
INSERT DELAYED, insert into two autoinc columns, use of UDF's, and
access to log tables in the mysql database.
Also, in addition to functions, procedures, triggers, and prepared
statements, it now also tests views; and it constructs recursive
calls in two levels by combining these recursive constructs.
Part of the logic is in extra/rpl_tests/create_recursive_construct.inc.
- added tests for all special system variables that should not be unsafe.
- added specific tests for BUG#45785 and BUG#45825
mysql-test/suite/rpl/r/rpl_events.result:
updated result file
mysql-test/suite/rpl/r/rpl_extraColmaster_innodb.result:
updated result file
mysql-test/suite/rpl/r/rpl_extraColmaster_myisam.result:
updated result file
mysql-test/suite/rpl/r/rpl_foreign_key_innodb.result:
updated result file
mysql-test/suite/rpl/r/rpl_idempotency.result:
updated result file
mysql-test/suite/rpl/r/rpl_mix_found_rows.result:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). This file equals
the second half of the old rpl_found_rows.result, with the following
modifications:
- minor formatting changes
- additional initialization
mysql-test/suite/rpl/r/rpl_mix_insert_delayed.result:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/r/rpl_rbr_to_sbr.result:
updated result file
mysql-test/suite/rpl/r/rpl_row_idempotency.result:
Moved the second half of rpl_idempotency.test, which only
executed in row mode, to rpl_row_idempotency.test. This is
the new result file.
mysql-test/suite/rpl/r/rpl_row_insert_delayed.result:
Got rid of unnecessary explicit setting of binlog format.
mysql-test/suite/rpl/r/rpl_stm_found_rows.result:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). Changes in
this file:
- minor formatting changes
- warning is now issued for unsafe statements inside procedures
(since BUG#45824 is fixed)
- second half of file is moved to rpl_mix_found_rows.result
mysql-test/suite/rpl/r/rpl_stm_insert_delayed.result:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
error message changed
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
updated result file
mysql-test/suite/rpl/r/rpl_udf.result:
Remove explicit set of binlog format (and triplicate test execution)
and rely on test system executing the test in all binlog formats.
mysql-test/suite/rpl/t/rpl_bug31076.test:
Test is only valid in mixed or row mode since it generates row events.
mysql-test/suite/rpl/t/rpl_events.test:
Removed explicit set of binlog_format and removed duplicate testing.
Instead, we rely on the test system to try all binlog formats.
mysql-test/suite/rpl/t/rpl_extraColmaster_innodb.test:
Removed triplicate testing and instead relying on test system.
Test is only relevant for row format since statement-based replication
cannot handle extra columns on master.
mysql-test/suite/rpl/t/rpl_extraColmaster_myisam.test:
Removed triplicate testing and instead relying on test system.
Test is only relevant for row format since statement-based replication
cannot handle extra columns on master.
mysql-test/suite/rpl/t/rpl_idempotency-slave.opt:
Removed .opt file to avoid server restarts.
mysql-test/suite/rpl/t/rpl_idempotency.test:
- Moved out row-only tests to a new test file, rpl_row_idempotency.test.
rpl_idempotency now only contains tests that execute in all
binlog_formats.
- While I was here, also removed .opt file to avoid server restarts.
The slave_exec_mode is now set inside the test instead.
mysql-test/suite/rpl/t/rpl_mix_found_rows.test:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). This file
contains the second half of the original rpl_found_rows.test with the
follwing changes:
- initialization
- removed SET_BINLOG_FORMAT and added have_binlog_format_mixed.inc
- minor formatting changes
mysql-test/suite/rpl/t/rpl_mix_insert_delayed.test:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/t/rpl_rbr_to_sbr.test:
Test cannot execute in statement mode, since we no longer
switch to row format when binlog_format=statement.
Enforced mixed mode throughout the test.
mysql-test/suite/rpl/t/rpl_row_idempotency.test:
Moved the second half of rpl_idempotency.test, which only
executed in row mode, to this new file. We now rely on the
test system to set binlog format.
mysql-test/suite/rpl/t/rpl_row_insert_delayed.test:
- Got rid of unnecessary explicit setting of binlog format.
- extra/rpl_tests/rpl_insert_delayed.test does not need the
$binlog_format_statement variable any more, so that was
removed.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
The test switches binlog_format internally and master generates both
row and statement events. Hence, the slave must be able to log in both
statement and row format. Hence test was changed to only execute in
mixed mode.
mysql-test/suite/rpl/t/rpl_stm_found_rows.test:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). Changes in
this file:
- minor formatting changes
- added have_binlog_format_statement and removed SET BINLOG_FORMAT.
- second half of file is moved to rpl_mix_found_rows.test
- added cleanup code
mysql-test/suite/rpl/t/rpl_stm_insert_delayed.test:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/t/rpl_switch_stm_row_mixed.test:
The test switches binlog_format internally and master generates both
row and statement events. Hence, the slave must be able to log in both
statement and row format. Hence test was changed to only execute in
mixed mode on slave.
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
Removed explicit set of binlog format. Instead, the test now only
executes in row mode.
mysql-test/suite/rpl/t/rpl_udf.test:
Remove explicit set of binlog format (and triplicate test execution)
and rely on test system executing the test in all binlog formats.
mysql-test/suite/rpl_ndb/combinations:
Added combinations file for rpl_ndb.
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
new result file
mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result:
updated result file
mysql-test/suite/rpl_ndb/t/rpl_ndb_2innodb.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_2myisam.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_basic.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-master.opt:
new option file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-slave.opt:
new option file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
New test case to verify all errors and warnings generated by
decide_logging_format.
mysql-test/suite/rpl_ndb/t/rpl_ndb_blob.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_blob2.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_circular.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_circular_simplex.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
While I was here, also made the test clean up after itself.
mysql-test/suite/rpl_ndb/t/rpl_ndb_commit_afterflush.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_ctype_ucs2_def.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_delete_nowhere.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_do_db.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_do_table.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_func003.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_innodb_trans.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_insert_ignore.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_mixed_engines_transactions.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_multi_update3.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_rep_ignore.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_row_001.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_sp003.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_sp006.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_trig004.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/t/partition_innodb_stmt.test:
Changed error code for innodb updates with READ COMMITTED or
READ UNCOMMITTED transaction isolation level and
binlog_format=statement.
sql/event_db_repository.cc:
Use member function to read current_stmt_binlog_row_based.
sql/events.cc:
Use member function to read current_stmt_binlog_row_based.
sql/ha_ndbcluster_binlog.cc:
reset_current_stmt_binlog_row_based() is not a no-op for the ndb_binlog
thread any more. Instead, the ndb_binlog thread now forces row mode both
initially and just after calling mysql_parse. (mysql_parse() is the only
place where reset_current_stmt_binlog_row_based() may be called from
the ndb_binlog thread, so these are the only two places that need to
change.)
sql/ha_partition.cc:
Use member function to read current_stmt_binlog_row_based.
sql/handler.cc:
Use member function to read current_stmt_binlog_row_based.
sql/item_create.cc:
Added DBUG_ENTER to some functions, to be able to trace when
set_stmt_unsafe is called.
sql/log.cc:
Use member function to read current_stmt_binlog_row_based.
sql/log_event.cc:
- Moved logic for changing to row format out of do_apply_event (and into
decide_logging_format).
- Added @todo comment for post-push cleanup.
sql/log_event_old.cc:
Move logic for changing to row format out of do_apply_event (and into
decide_logging_format).
sql/mysql_priv.h:
Make decide_logging_format() a member of the THD class, for two reasons:
- It is natural from an object-oriented perspective.
- decide_logging_format() needs to access private members of THD
(specifically, the new binlog_warning_flags field).
sql/rpl_injector.cc:
Removed call to set_current_stmt_binlog_row_based().
From now on, only decide_logging_fromat is allowed to modify
current_stmt_binlog_row_based. This call is from the ndb_binlog
thread, mostly executing code in ha_ndbcluster_binlog.cc.
This call can be safely removed, because:
- current_stmt_binlog_row_based is initialized for the ndb_binlog
thread's THD object when the THD object is created. So we're
not going to read uninitialized memory.
- The behavior of ndb_binlog thread does not use the state of the
current_stmt_binlog_row_based. It is conceivable that the
ndb_binlog thread would rely on the current_stmt_binlog_format
in two situations:
(1) when it calls mysql_parse;
(2) when it calls THD::binlog_query.
In case (1), it always clears THD::options&OPTION_BIN_LOG (because
run_query() in ha_ndbcluster_binlog.cc is only called with
disable_binlogging = TRUE).
In case (2), it always uses qtype=STMT_QUERY_TYPE.
sql/set_var.cc:
Added @todo comment for post-push cleanup.
sql/share/errmsg.txt:
Added new error messages and clarified ER_BINLOG_UNSAFE_STATEMENT.
sql/sp.cc:
Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
Got rid of MYSQL_QUERY_TYPE: it was equivalent to STMT_QUERY_TYPE.
sql/sp_head.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sp_head.h:
Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
sql/sql_acl.cc:
Got rid of MYSQL_QUERY_TYPE: it was equivalent to STMT_QUERY_TYPE.
sql/sql_base.cc:
- Made decide_logging_format take care of all logic for deciding the
logging format, and for determining the related warnings and errors.
See comment above decide_logging_format for details.
- Made decide_logging_format a member function of THD, since it needs
to access private members of THD and since its purpose is to update
the state of a THD object.
- Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
sql/sql_class.cc:
- Moved logic for determining unsafe warnings away from THD::binlog_query
(and into decide_logging_format()). Now, it works like this:
1. decide_logging_format detects that the current statement shall
produce a warning, if it ever makes it to the binlog
2. decide_logging_format sets a flag of THD::binlog_warning_flags.
3. THD::binlog_query reads the flag. If the flag is set, it generates
a warning.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_class.h:
- Added THD::binlog_warning_flags (see sql_class.cc for explanation).
- Made decide_logging_format() and reset_for_next_command() member
functions of THD (instead of standalone functions). This was needed
for two reasons: (1) the functions need to access the private member
THD::binlog_warning_flags; (2) the purpose of these functions is to
update the staet of a THD object, so from an object-oriented point
of view they should be member functions.
- Encapsulated current_stmt_binlog_row_based, so it is now private and
can only be accessed from a member function. Also changed the
data type to an enumeration instead of a bool.
- Removed MYSQL_QUERY_TYPE, because it was equivalent to
STMT_QUERY_TYPE anyways.
- When reset_current_stmt_binlog_row_based was called from the
ndb_binlog thread, it would behave as a no-op. This special
case has been removed, and the behavior of
reset_current_stmt_binlog_row_based does not depend on which thread
calls it any more. The special case did not serve any purpose,
since the ndb binlog thread did not take the
current_stmt_binlog_row_based flag into account anyways.
sql/sql_delete.cc:
- Moved logic for setting row format for DELETE..LIMIT away from
mysql_prepare_delete.
(Instead, we mark the statement as unsafe at parse time (sql_yacc.yy)
and rely on decide_logging_format() (sql_class.cc) to set row format.)
This is part of the fix for BUG#45831.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_insert.cc:
- Removed unnecessary calls to thd->lex->set_stmt_unsafe() and
thd->set_current_stmt_binlog_row_based_if_mixed() from
handle_delayed_insert(). The calls are unnecessary because they
have already been made; they were made in the constructor of
the `di' object.
- Since decide_logging_format() is now a member function of THD, code
that calls decide_logging_format() had to be updated.
- Added DBUG_ENTER call, to be able to trace when set_stmt_unsafe is
called.
- Moved call to set_stmt_unsafe() for INSERT..SELECT..LIMIT away from
mysql_insert_select_prepare() (and into decide_logging_format).
This is part of the fix for BUG#45831.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_lex.h:
- Added the flag BINLOG_STMT_FLAG_ROW_INJECTION to enum_binlog_stmt_flag.
This was necessary so that a statement can identify itself as a row
injection.
- Added appropriate setter and getter functions for the new flag.
- Added or clarified some comments.
- Added DBUG_ENTER()
sql/sql_load.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_parse.cc:
- Made mysql_reset_thd_for_next_command() clear thd->binlog_warning_flags.
- Since thd->binlog_warning_flags is private, it must be set in a
member function of THD. Hence, moved the body of
mysql_reset_thd_for_next_command() to the new member function
THD::reset_thd_for_next_command(), and made
mysql_reset_thd_for_next_command() call
THD::reset_thd_for_next_command().
- Removed confusing comment.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_repl.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_table.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_udf.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_update.cc:
Moved logic for setting row format for UPDATE..LIMIT away from
mysql_prepare_update.
(Instead, we mark the statement as unsafe at parse time (sql_yacc.yy)
and rely on decide_logging_format() (sql_class.cc) to set row format.)
This is part of the fix for BUG#45831.
sql/sql_yacc.yy:
Made INSERT DELAYED, INSERT..SELECT..LIMIT, UPDATE..LIMIT, and
DELETE..LIMIT mark themselves as unsafe at parse time (instead
of at execution time).
This is part of the fixes BUG#45831 and BUG#45825.
storage/example/ha_example.cc:
Made exampledb accept inserts. This was needed by the new test case
rpl_ndb_binlog_format_errors, because it needs an engine that
is statement-only (and accepts inserts).
storage/example/ha_example.h:
Made exampledb a statement-only engine instead of a row-only engine.
No existing test relied exampledb's row-only capabilities. The new
test case rpl_ndb_binlog_format_errors needs an engine that is
statement-only.
storage/innobase/handler/ha_innodb.cc:
- Changed error error code and message given by innodb when
binlog_format=STATEMENT and transaction isolation level is
READ COMMITTED or READ UNCOMMITTED.
- While I was here, also simplified the condition for
checking when to give the error.
The test case added failed sporadically on PB. This is due to the
fact that the user thread in some cases is waiting for slave IO
to stop and then check the error number. Thence, sometimes the
user thread would race for the error number with IO thread.
This post push fix addresses this by replacing the wait for slave
io to stop with a wait for slave io error (as it seems it was
added in 6.0 also after patch on which this is based was
pushed). This implied backporting wait_for_slave_io_error.inc
from 6.0 also.
The server was not cleaning the last IO error and error number when
resetting slave.
This patch addresses this issue by backporting into 5.1 part of the
patch in BUG 34654. A fix for this issue had already been pushed into
6.0 as part of the aforementioned bug, however the patch also included
some refactoring. The fix for 5.1 does not take into account the
refactoring part.
mysql-test/extra/rpl_tests/rpl_reset_slave.test:
Backported the test case and improved with deploying include/start_slave.inc
in relevant spots.
sql/slave.cc:
Backported part of patch from 6.0 that includes cleaning
mi->clear_error() at:
1. beginning of handle_slave_io
2. on successful connection
Also, backported the assertion added in the original patch.
sql/sql_repl.cc:
Backported the call to mi->clear_error() on reset_slave().
1. Test case was rewritten completely.
2. Test covers 3 cases:
a) do deadlock on slave, wait retries of transaction, unlock slave before lock
timeout;
b) do deadlock on slave and wait error 'lock timeout exceed' on slave;
c) same as b) but if of max relay log size = 0;
3. Added comments inline.
4. Updated result file.
LOAD_FILE
LOAD_FILE is not safe to replicate in STATEMENT mode, because it
depends on a file (which is loaded on master and may not exist in
slave(s)). This leads to scenarios on which the slave replicates the
statement with 'load_file' and it will try to load the file from local
file system. Given that the file may not exist in the slave filesystem
the operation will not succeed (probably returning NULL), causing
master and slave(s) to diverge. However, when using MIXED mode
replication, this can be made to work, if the statement including
LOAD_FILE is marked as unsafe, triggering a switch to ROW mode,
meaning that the contents of the file are written to binlog as row
events. Consequently, the contents from the file in the master will
reach the slave via the binlog.
This patch addresses this bug by marking the load_file function as
unsafe. When in mixed mode and when LOAD_FILE is issued, there will be
a switch to row mode. Furthermore, when in statement mode, the
LOAD_FILE will raise a warning that the statement is unsafe in that
mode.
mysql-test/extra/rpl_tests/rpl_loadfile.test:
Extra file that is "sourced" on both rpl_loadfile and rpl_stm_loadfile
test files.
mysql-test/suite/rpl/r/rpl_loadfile.result:
Updated with the results from the test case added to this file.
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
Result file for rpl_loadfile test split with the warnings in statement
mode.
mysql-test/suite/rpl/t/rpl_loadfile.test:
After splitting the original rpl_loadfile file, this one is only
required to be executed in mixed or row format.
Appended the test for 39701 to this file.
mysql-test/suite/rpl/t/rpl_stm_loadfile.test:
Split the original rpl_loadfile test because load_file now raises
a warning when in statement mode. The goal of this split is
two-fold: i) make the test case more resilient; ii) assert that
warnings are indeed raised when in statement mode.
sql/item_create.cc:
Added the set_stmt_unsafe call to lex.
TRUNCATE TABLE fails to replicate when stmt-based binlogging is not supported.
There were two separate problems with the code, both of which are fixed with
this patch:
1. An error was printed by InnoDB for TRUNCATE TABLE in statement mode when
the in isolation levels READ COMMITTED and READ UNCOMMITTED since InnoDB
does permit statement-based replication for DML statements. However,
the TRUNCATE TABLE is not transactional, but is a DDL, and should therefore
be allowed to be replicated as a statement.
2. The statement was not logged in mixed mode because of the error above, but
the error was not reported to the client.
This patch fixes the problem by treating TRUNCATE TABLE a DDL, that is, it is
always logged as a statement and not reporting an error from InnoDB for TRUNCATE
TABLE.
mysql-test/extra/binlog_tests/binlog_truncate.test:
Adding new test to check that TRUNCATE TABLE is written correctly
to the binary log.
mysql-test/extra/rpl_tests/rpl_truncate.test:
Removing redundant testing by eliminating settings of BINLOG_FORMAT.
mysql-test/extra/rpl_tests/rpl_truncate_helper.test:
Replacing slave and master reset code with include file.
Removing settings of BINLOG_FORMAT.
Replacing printing of table contents to compare master and slave
with diff_tables.inc.
mysql-test/suite/binlog/t/binlog_truncate_innodb.test:
Adding test for testing that TRUNCATE TABLE is logged correctly for InnoDB
in all isolation levels.
mysql-test/suite/binlog/t/binlog_truncate_myisam.test:
Adding test for testing that TRUNCATE TABLE is logged correctly for MyISAM.
mysql-test/suite/binlog/t/disabled.def:
Disabling binlog_truncate_innodb since it does not work (yet).
sql/sql_base.cc:
Correcting setting of capabilities flags to make the comparison with 0
later in the code work correctly.
sql/sql_delete.cc:
Re-organizing code to ensure that TRUNCATE TABLE is logged in statement
format and that row format is not used unless there are rows to log (which
there are not when delete_all_rows() is called, so this has to be logged
as a statement).
Documented behaviour was broken by the patch for bug 33699
that actually is not a bug.
This fix reverts patch for bug 33699 and reverts the
UPDATE of NOT NULL field with NULL query to old
behavior.
mysql-test/extra/rpl_tests/rpl_extraMaster_Col.test:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/include/ps_modify.inc:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/auto_increment.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/csv_not_null.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/null.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/ps_2myisam.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/ps_3innodb.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/ps_4heap.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/ps_5merge.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/r/warnings.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/suite/ndb/r/ps_7ndb.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/suite/rpl/r/rpl_extraColmaster_innodb.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/suite/rpl/r/rpl_extraColmaster_myisam.result:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/suite/rpl/t/rpl_err_ignoredtable.test:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/t/auto_increment.test:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/t/csv_not_null.test:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/t/null.test:
Bug #39265: fix for the bug 33699 should be reverted
mysql-test/t/warnings.test:
Bug #39265: fix for the bug 33699 should be reverted
sql/sql_update.cc:
Bug #39265: fix for the bug 33699 should be reverted
Remove size of binlog file from SHOW BINARY LOGS.
Changing size of binlog file is an affect of adding or removing events to/from
binlog and it can be checked in next command of test: SHOW BINLOG EVENTS.
For SHOW BINARY LOGS statement enough to show the list of file names.
conflicts:
Text conflict in client/mysqltest.cc
Text conflict in mysql-test/include/wait_until_connected_again.inc
Text conflict in mysql-test/lib/mtr_report.pm
Text conflict in mysql-test/mysql-test-run.pl
Text conflict in mysql-test/r/events_bugs.result
Text conflict in mysql-test/r/log_state.result
Text conflict in mysql-test/r/myisam_data_pointer_size_func.result
Text conflict in mysql-test/r/mysqlcheck.result
Text conflict in mysql-test/r/query_cache.result
Text conflict in mysql-test/r/status.result
Text conflict in mysql-test/suite/binlog/r/binlog_index.result
Text conflict in mysql-test/suite/binlog/r/binlog_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_packet.result
Text conflict in mysql-test/suite/rpl/t/rpl_packet.test
Text conflict in mysql-test/t/disabled.def
Text conflict in mysql-test/t/events_bugs.test
Text conflict in mysql-test/t/log_state.test
Text conflict in mysql-test/t/myisam_data_pointer_size_func.test
Text conflict in mysql-test/t/mysqlcheck.test
Text conflict in mysql-test/t/query_cache.test
Text conflict in mysql-test/t/rpl_init_slave_func.test
Text conflict in mysql-test/t/status.test
The next number (AUTO_INCREMENT) field of the table for write
rows events are not initialized, and cause some engines (innodb)
not correctly update the tables's auto_increment value.
This patch fixed this problem by honor next number fields if present.
mysql-test/extra/rpl_tests/rpl_auto_increment.test:
Add test code for BUG#41986
mysql-test/suite/rpl/r/rpl_auto_increment.result:
update test result file for BUG#41986
sql/log_event.cc:
set next_number_field before writing rows, and reset next_number_field after finished writing rows
Problem 1: The test waits for an error in the slave sql thread,
then resolves the error and issues 'start slave'. However, there
is a gap between when the error is reported and the slave sql
thread stops. If this gap was long, the slave would still be
running when 'start slave' happened, so 'start slave' would fail
and cause a test failure.
Fix 1: Made wait_for_slave_sql_error wait for the slave to stop
instead of wait for error in the IO thread. After stopping, the
error code is verified. If the error code is wrong, debug info
is printed. To print debug info, the debug printing code in
wait_for_slave_param.inc was moved out to a new file,
show_rpl_debug_info.inc.
Problem 2: rpl_stm_mystery22 is a horrible name, the comments in
the file didn't explain anything useful, the test was generally
hard to follow, and the test was essentially duplicated between
rpl_stm_mystery22 and rpl_row_mystery22.
Fix 2: The test is about conflicts in the slave SQL thread,
hence I renamed the tests to rpl_{stm,row}_conflicts. Refactored
the test so that the work is done in
extra/rpl_tests/rpl_conflicts.inc, and
rpl.rpl_{row,stm}_conflicts merely sets some variables and then
sourced extra/rpl_tests/rpl_conflicts.inc.
The tests have been rewritten and comments added.
Problem 3: When calling wait_for_slave_sql_error.inc, you always
want to verify that the sql thread stops because of the expected
error and not because of some other error. Currently,
wait_for_slave_sql_error.inc allows the caller to omit the error
code, in which case all error codes are accepted.
Fix 3: Made wait_for_slave_sql_error.inc fail if no error code
is given. Updated rpl_filter_tables_not_exist accordingly.
Problem 4: rpl_filter_tables_not_exist had a typo, the dollar
sign was missing in a 'let' statement.
Fix 4: Added dollar sign.
Problem 5: When replicating from other servers than the one named
'master', the wait_for_slave_* macros were unable to print debug
info on the master.
Fix 5: Replace parameter $slave_keep_connection by
$master_connection.
mysql-test/extra/rpl_tests/rpl_conflicts.test:
rpl_stm_mystery22 and rpl_row_mystery22 have now been refactored and renamed:
The two test cases rpl.rpl_stm_conflicts.test and rpl.rpl_row_conflicts.test
just set some parameters, and then source include/rpl_tests/rpl_conflicts.test.
Also, cleaned up the test case a bit, and fixed BUG#37718.
mysql-test/include/show_rpl_debug_info.inc:
Factored out the debug printing code from wait_for_slave_param.inc to
a new file, show_rpl_debug_info.inc.
Also removed the $slave_keep_connection parameter, and replaced it by
$master_connection. This allows printing debug info on the master, no
matter what the name of the master connection is.
mysql-test/include/start_slave.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/stop_slave.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/sync_slave_io_with_master.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/wait_for_slave_io_to_start.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/wait_for_slave_io_to_stop.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/wait_for_slave_param.inc:
Factored out the debug printing code from wait_for_slave_param.inc to
a new file, show_rpl_debug_info.inc.
Also removed the $slave_keep_connection parameter, and replaced it by
$master_connection. This allows printing debug info on the master, no
matter what the name of the master connection is.
Had to move the printing of debug info out of the while loop because
of BUG number 41913.
mysql-test/include/wait_for_slave_sql_error.inc:
Made it wait until the slave sql thread has stopped. This
takes very short time and avoids race condition bugs in
test cases (e.g., fixes BUG#37718).
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/wait_for_slave_sql_error_and_skip.inc:
Since wait_for_slave_sql_error now waits for the slave sql
thread to stop too, wait_for_slave_sql_error_and_skip does not
have to wait for the slave sql thread to stop.
Also, since wait_for_slave_sql_error now requires the parameter
$slave_sql_errno to be set, wait_for_slave_sql_error_and_skip
requires that as well: updated the usage instructions.
mysql-test/include/wait_for_slave_sql_to_start.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/wait_for_slave_sql_to_stop.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/wait_for_slave_to_start.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/include/wait_for_slave_to_stop.inc:
Replaced $slave_keep_connection by $master_connection.
mysql-test/suite/rpl/r/rpl_row_conflicts.result:
update result file
mysql-test/suite/rpl/r/rpl_stm_conflicts.result:
update result file
mysql-test/suite/rpl/t/rpl_dual_pos_advance.test:
Replaced $slave_keep_connection by $master_connection.
mysql-test/suite/rpl/t/rpl_filter_tables_not_exist.test:
Set $slave_sql_errno, since it is now required.
Add dollar sign to $show_sql_error (without dollar sign,
mtr makes it an environment variable).
mysql-test/suite/rpl/t/rpl_row_conflicts.test:
rpl_stm_mystery22 and rpl_row_mystery22 have now been refactored and renamed:
The two test cases rpl.rpl_stm_conflicts.test and rpl.rpl_row_conflicts.test
just set some parameters, and then source include/rpl_tests/rpl_conflicts.test.
Also, cleaned up the test case a bit, and fixed BUG#37718.
mysql-test/suite/rpl/t/rpl_row_mystery22.test:
rpl_stm_mystery22 and rpl_row_mystery22 have now been refactored and renamed:
The two test cases rpl.rpl_stm_conflicts.test and rpl.rpl_row_conflicts.test
just set some parameters, and then source include/rpl_tests/rpl_conflicts.test.
Also, cleaned up the test case a bit, and fixed BUG#37718.
mysql-test/suite/rpl/t/rpl_stm_conflicts.test:
rpl_stm_mystery22 and rpl_row_mystery22 have now been refactored and renamed:
The two test cases rpl.rpl_stm_conflicts.test and rpl.rpl_row_conflicts.test
just set some parameters, and then source include/rpl_tests/rpl_conflicts.test.
Also, cleaned up the test case a bit, and fixed BUG#37718.
where timeout can happen:
1. Added waiting start/stop slave to make sure that slave works properly.
2. Added cleanup for slave.
3. Updated related result files.
Problem: Many test cases don't clean up after themselves (fail
to drop tables or fail to reset variables). This implies that:
(1) check-testcase in the new mtr that currently lives in
5.1-rpl failed. (2) it may cause unexpected results in
subsequent tests.
Fix: make all tests clean up.
Also: cleaned away unnecessary output in rpl_packet.result
Also: fixed bug where rpl_log called RESET MASTER with a running
slave. This is not supposed to work.
Also: removed unnecessary code from rpl_stm_EE_err2 and made it
verify that an error occurred.
Also: removed unnecessary code from rpl_ndb_ctype_ucs2_def.
mysql-test/extra/rpl_tests/rpl_loaddata.test:
Made test clean up after itself.
mysql-test/extra/rpl_tests/rpl_stm_EE_err2.test:
Made test clean up after itself.
Also removed unnecessary save_master_pos.
Also made test verify that an error occurred.
mysql-test/extra/rpl_tests/rpl_truncate_helper.test:
Made test clean up after itself.
mysql-test/include/query_cache.inc:
Made test clean up after itself.
mysql-test/suite/binlog/t/binlog_auto_increment_bug33029.test:
Made test clean up after itself.
mysql-test/suite/rpl/t/rpl_alter_db.test:
Made test clean up after itself.
Easiest to refactor the test a little, to use an auxiliary database instead of mysql:
Then we don't need to read the collation at the start of the test and reset it at the
end of the test. We can just drop the database.
mysql-test/suite/rpl/t/rpl_drop_db.test:
Made test clean up after itself.
mysql-test/suite/rpl/t/rpl_drop_view.test:
Made test clean up after itself.
mysql-test/suite/rpl/t/rpl_events.test:
Made test clean up after itself.
mysql-test/suite/rpl/t/rpl_idempotency.test:
Made test clean up after itself.
mysql-test/suite/rpl/t/rpl_ignore_table.test:
Made test clean up after itself.
The existing cleanup code didn't work, because the --replicate-*
options prevented the "drop table" from being replicated. Moved
"drop table t4" to a statement of its own.
mysql-test/suite/rpl/t/rpl_packet.test:
Made test clean up after itself.
Also replaced SHOW SLAVE STATUS by query_get_value(SHOW SLAVE STATUS...),
since only one column of the result was interesting.
mysql-test/suite/rpl/t/rpl_row_max_relay_size.test:
Made test clean up after itself.
Instead of setting binlog format at the beginning of the test, we just
do 'source include/have_binlog_format_row.inc' like all other tests.
mysql-test/suite/rpl_ndb/t/rpl_ndb_2other.test:
Made test clean up after itself.
mysql-test/suite/rpl_ndb/t/rpl_ndb_ctype_ucs2_def.test:
Made test clean up after itself.
Also removed unnecessary 'create table; drop table'
Also removed unnecessary use of --command
mysql-test/suite/rpl_ndb/t/rpl_ndb_multi.test:
Made test clean up after itself.
mysql-test/t/innodb-autoinc-optimize.test:
Made test clean up after itself.
mysql-test/t/innodb.test:
Made test clean up after itself.
mysql-test/t/log_tables.test:
Made test clean up after itself.
mysql-test/t/mysql_client_test.test:
Made test clean up after itself.
mysql-test/t/partition.test:
Made test clean up after itself.
mysql-test/t/status.test:
Made test clean up after itself.
a test on the bug page does not reveal the problem with the latest trees.
Adding the test to the rpl suite in order to monitor regression.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
a regression test for the bug#38230.
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
results changed.
mysql-test/extra/rpl_tests/rpl_log.test:
Using RESET MASTER in the middle of a sequence of statements cause
a race condition between the dump thread sending the preceeding
statements to the slave and the RESET MASTER.
mysql-test/suite/federated/federated.test:
Fixing include paths.
mysql-test/suite/federated/federated_bug_25714.test:
Removing extreneous lines added by merge.
mysql-test/suite/rpl/t/rpl_circular_for_4_hosts.test:
Correcting typo (the spaces following the connection name was included as part of the name).
mysql-test/t/distinct.test:
Correcting paths.
In certain situations, a scan of the table will return the error
code HA_ERR_RECORD_DELETED, and this error code is not
correctly caught in the Rows_log_event::find_row() function, which
causes an error to be returned for this case.
This patch fixes the problem by adding code to either ignore the
record and continuing with the next one, the the event of a table
scan, or change the error code to HA_ERR_KEY_NOT_FOUND, in the event
that a key lookup is attempted.
The code to get read the value of a system variable was extracting its value
on PREPARE stage and was substituting the value (as a constant) into the parse tree.
Note that this must be a reversible transformation, i.e. it must be reversed before
each re-execution.
Unfortunately this cannot be reliably done using the current code, because there are
other non-reversible source tree transformations that can interfere with this
reversible transformation.
Fixed by not resolving the value at PREPARE, but at EXECUTE (as the rest of the
functions operate). Added a cache of the value (so that it's constant throughout
the execution of the query). Note that the cache also caches NULL values.
Updated an obsolete related test suite (variables-big) and the code to test the
result type of system variables (as per bug 74).
mysql-test/extra/rpl_tests/rpl_insert_id.test:
Bug #32124: removed ambiguous testcase
mysql-test/r/innodb_data_home_dir_basic.result:
Bug #32124: fixed wrong test case
mysql-test/r/innodb_flush_method_basic.result:
Bug #32124: fixed wrong test case
mysql-test/r/ps_11bugs.result:
Bug #32124: test case
mysql-test/r/ssl_capath_basic.result:
Bug #32124: fixed wrong test case
mysql-test/r/ssl_cipher_basic.result:
Bug #32124: fixed wrong test case
mysql-test/r/variables.result:
Bug #32124: system vars are shown as such in EXPLAIN EXTENDED, not as constants.
mysql-test/suite/rpl/r/rpl_insert_id.result:
Bug #32124: removed ambiguous testcase
mysql-test/t/ps_11bugs.test:
Bug #32124: test case
sql/item.cc:
Bug #32124: placed the code to convert string to longlong or double
to a function (so that it can be reused)
sql/item.h:
Bug #32124: placed the code to convert string to longlong or double
to a function (so that it can be reused)
sql/item_func.cc:
Bug #32124: moved the evaluation of system variables at runtime (val_xxx).
sql/item_func.h:
Bug #32124: moved the evaluation of system variables at runtime (val_xxx).
sql/set_var.cc:
Bug #32124: removed the code that calculated the system variable's value
at PREPARE
sql/set_var.h:
Bug #32124: removed the code that calculated the system variable's value
at PREPARE
tests/mysql_client_test.c:
Bug #32124 : removed the reading of the system variable, because its max
length is depended on the system charset and client charset and can't be
easily calculated.
mysql-test/extra/rpl_tests/rpl_log.test:
Removing extreneous start slave.
Added missing drop of table t1.
mysql-test/suite/rpl/t/rpl_binlog_query_filter_rules.test:
Replaced $MYSQLTEST_VARDIR/log with $MYSQLD_DATADIR
mysql-test/suite/rpl/t/rpl_misc_functions.test:
Using temporary table instead to remove need for drop table.
The Blackhole engine did not support row-based replication
since the delete_row(), update_row(), and the index and range
searching functions were not implemented.
This patch adds row-based replication support for the
Blackhole engine by implementing the two functions mentioned
above, and making the engine pretend that it has found the
correct row to delete or update when executed from the slave
SQL thread by implementing index and range searching functions.
It is necessary to only pretend this for the SQL thread, since
a SELECT executed on the Blackhole engine will otherwise never
return EOF, causing a livelock.
mysql-test/extra/binlog_tests/blackhole.test:
Blackhole now handles row-based replication.
mysql-test/extra/rpl_tests/rpl_blackhole.test:
Test helper file for testing that blackhole actually
writes something to the binary log on the slave.
mysql-test/suite/binlog/t/binlog_multi_engine.test:
Replication now handles row-based replcation.
mysql-test/suite/rpl/t/rpl_blackhole.test:
Test that Blackhole works with primary key, index, or none.
sql/log_event.cc:
Correcting code to only touch filler bits and leave
all other bits alone. It is necessary since there is
no guarantee that the engine will be able to fill in
the bits correctly (e.g., the blackhole engine).
storage/blackhole/ha_blackhole.cc:
Adding definitions for update_row() and delete_row() to return OK
when executed from the slave SQL thread with thd->query == NULL
(indicating that row-based replication events are being processed).
Changing rnd_next(), index_read(), index_read_idx(), and
index_read_last() to return OK when executed from the slave SQL
thread (faking that the row has been found so that processing
proceeds to update/delete the row).
storage/blackhole/ha_blackhole.h:
Enabling row capabilities for engine.
Defining write_row(), update_row(), and delete_row().
Making write_row() private (as it should be).
The assertion indicates that some data was left in the transaction
cache when the server was shut down, which means that a previous
statement did not commit or rollback correctly.
What happened was that a bug in the rollback of a transactional
table caused the transaction cache to be emptied, but not reset.
The error can be triggered by having a failing UPDATE or INSERT,
on a transactional table, causing an implicit rollback.
Fixed by always flushing the pending event to reset the state
properly.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Testing that a failed update (that writes some rows to the
transaction cache) does not cause the transaction cache to
hold on to the data or forget to reset the transaction cache.
sql/log.cc:
Added call to remove pending event when the transaction cache
is emptied instead of written to binary log. The call will also
clear the outstanding table map count so that the cache is not
left it in a state of "empty but not reset".
Added function MYSQL_BIN_LOG::remove_pending_rows_event().
sql/log.h:
Added function MYSQL_BIN_LOG::remove_pending_rows_event().
sql/sql_class.cc:
Adding function THD::binlog_remove_pending_rows_event().
sql/sql_class.h:
Adding function THD::binlog_remove_pending_rows_event().
manually resolved conflicts:
Text conflict in client/mysqltest.c
Contents conflict in mysql-test/include/have_bug25714.inc
Text conflict in mysql-test/include/have_ndbapi_examples.inc
Text conflict in mysql-test/mysql-test-run.pl
Text conflict in mysql-test/suite/parts/inc/partition_check_drop.inc
Text conflict in mysql-test/suite/parts/inc/partition_layout.inc
Text conflict in mysql-test/suite/parts/inc/partition_layout_check1.inc
Text conflict in mysql-test/suite/parts/inc/partition_layout_check2.inc
Text conflict in mysql-test/suite/parts/r/partition_alter1_1_2_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter1_1_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter1_2_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter2_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter3_innodb.result
Text conflict in mysql-test/suite/parts/r/partition_alter3_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_basic_innodb.result
Text conflict in mysql-test/suite/parts/r/partition_basic_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_basic_symlink_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_engine_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_syntax_myisam.result
Text conflict in mysql-test/suite/rpl_ndb/t/disabled.def
Text conflict in mysql-test/t/disabled.def
BUG#37884: rpl_row_basic_2myisam and rpl_row_basic_3innodb fail sporadically in pushbuild
These have been fixed in 5.1-rpl. Re-applying fix for BUG#37884
in 5.1-bugteam, and disabling rpl_flushlog_loop for BUG#37733 in
5.1-bugteam.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Missing sync_slave_with_master added.
mysql-test/suite/rpl/t/disabled.def:
Disabling rpl_flushlog_loop until the fixed version gets
merged from 5.1-rpl
Problem 1: tests often fail in pushbuild with a timeout when waiting
for the slave to start/stop/receive error.
Fix 1: Updated the wait_for_slave_* macros in the following way:
- The timeout is increased by a factor ten
- Refactored the macros so that wait_for_slave_param does the work for
the other macros.
Problem 2: Tests are often incorrectly written, lacking a
source include/wait_for_slave_to_[start|stop].inc.
Fix 2: Improved the chance to get it right by adding
include/start_slave.inc and include/stop_slave.inc, and updated tests
to use these.
Problem 3: The the built-in test language command
wait_for_slave_to_stop is a misnomer (does not wait for the slave io
thread) and does not give as much debug info in case of failure as
the otherwise equivalent macro
source include/wait_for_slave_sql_to_stop.inc
Fix 3: Replaced all calls to the built-in command by a call to the
macro.
Problem 4: Some, but not all, of the wait_for_slave_* macros had an
implicit connection slave. This made some tests confusing to read,
and made it more difficult to use the macro in circular replication
scenarios, where the connection named master needs to wait.
Fix 4: Removed the implicit connection slave from all
wait_for_slave_* macros, and updated tests to use an explicit
connection slave where necessary.
Problem 5: The macros wait_slave_status.inc and wait_show_pattern.inc
were unused. Moreover, using them is difficult and error-prone.
Fix 5: remove these macros.
Problem 6: log_bin_trust_function_creators_basic failed when running
tests because it assumed @@global.log_bin_trust_function_creators=1,
and some tests modified this variable without resetting it to its
original value.
Fix 6: All tests that use this variable have been updated so that
they reset the value at end of test.
mysql-test/extra/rpl_tests/rpl_extraMaster_Col.test:
Replaced wait_for_slave_to_stop by include/wait_for_slave_sql_to_stop.inc
mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
Replaced wait_for_slave_to_stop by include/wait_for_slave_sql_to_stop.inc
Added connection slave since includ/wait_for_slave_sql_to_stop.inc
does not do that anymore.
mysql-test/extra/rpl_tests/rpl_log.test:
Replaced start slave+wait_slave_status by start_slave.inc
mysql-test/include/reset_master_and_slave.inc:
replaced start/stop slave by start_slave.inc/stop_slave.inc
mysql-test/include/sync_slave_io_with_master.inc:
Improved comments and error message.
mysql-test/include/wait_for_slave_io_to_stop.inc:
Refactored to use wait_for_slave_param.inc.
Removed connection slave.
mysql-test/include/wait_for_slave_param.inc:
- Improved usage instructions
- Added more debug info in case of timeout
- Added parameters $slave_param_comparison, $slave_timeout,
$slave_keep_connection, $slave_error_message
mysql-test/include/wait_for_slave_sql_error.inc:
Refactored to use wait_for_slave_param.inc.
Removed connection slave.
mysql-test/include/wait_for_slave_sql_to_start.inc:
Refactored to use wait_for_slave_param.inc.
Removed connection slave.
mysql-test/include/wait_for_slave_sql_to_stop.inc:
Refactored to use wait_for_slave_param.inc.
Removed connection slave.
mysql-test/include/wait_for_slave_to_start.inc:
Refactored to use wait_for_slave_param.inc.
Removed connection slave.
mysql-test/include/wait_for_slave_to_stop.inc:
Refactored to use wait_for_slave_param.inc.
Removed connection slave.
mysql-test/include/wait_show_pattern.inc:
Removed unused (and error-prone) file
mysql-test/include/wait_slave_status.inc:
Removed unused (and error-prone) file
mysql-test/suite/binlog/t/binlog_auto_increment_bug33029.test:
Renamed $keep_connection to $slave_keep_connection.
mysql-test/suite/rpl/t/rpl_bug26395.test:
Replace stop slave by stop_slave.inc
mysql-test/suite/rpl/t/rpl_circular_for_4_hosts.test:
Replace start/stop slave by start_slave.inc/stop_slave.inc.
Replace wait_for_slave_param by wait_for_slave_sql_to_stop.inc.
mysql-test/suite/rpl/t/rpl_dual_pos_advance.test:
Renamed $keep_connection to $slave_keep_connection.
mysql-test/suite/rpl/t/rpl_flushlog_loop.test:
Replace wait_slave_status by start_slave.inc
mysql-test/suite/rpl/t/rpl_idempotency.test:
Added connection slave since wait_for_slave_sql_to_stop.inc does not
do that any more.
mysql-test/suite/rpl/t/rpl_incident.test:
Replaced wait_for_slave_to_stop by wait_for_slave_sql_to_stop.inc
mysql-test/suite/rpl/t/rpl_init_slave.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
Replaced save_master_pos;connection slave;sync_with_master by
sync_slave_with_master.
mysql-test/suite/rpl/t/rpl_log_pos.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
Replaced wait_for_slave_param by other wait_for_slave_* macros.
mysql-test/suite/rpl/t/rpl_packet.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/suite/rpl/t/rpl_row_inexist_tbl.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/suite/rpl/t/rpl_row_stop_middle_update.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/suite/rpl/t/rpl_row_until.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
Replaced save_master_pos;connection slave;sync_with_master by
sync_slave_with_master.
mysql-test/suite/rpl/t/rpl_server_id1.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/suite/rpl/t/rpl_slave_grp_exec.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/suite/rpl/t/rpl_slave_status.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/suite/rpl/t/rpl_sp.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/suite/rpl/t/rpl_sp_effects.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/suite/rpl/t/rpl_stm_until.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
Replaced save_master_pos;connection slave;sync_with_master by
sync_slave_with_master.
mysql-test/suite/rpl_ndb/t/rpl_ndb_basic.test:
Replaced start slave by start_slave.inc.
Added explicit connection slave since wait_for_slave_sql_to_stop.inc
does not do that anymore.
mysql-test/t/disabled.def:
Disabled failing test.
mysql-test/t/func_time.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/t/grant.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/t/grant2.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/t/innodb_notembedded.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/t/log_bin_trust_function_creators_func.test:
Restore @@global.log_bin_trust_function_creators at end of test.
Clean up at end of test by dropping the created user.
mysql-test/t/query_cache.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/t/query_cache_notembedded.test:
Restore @@global.log_bin_trust_function_creators at end of test.
mysql-test/t/rpl_init_slave_func.test:
Replaced start/stop slave by start_slave.inc/stop_slave.inc.
mysql-test/t/timezone2.test:
Restore @@global.log_bin_trust_function_creators at end of test.
In order to handle CHAR() fields, 8 bits were reserved for
the size of the CHAR field. However, instead of denoting the
number of characters in the field, field_length was used which
denotes the number of bytes in the field.
Since UTF-8 fields can have three bytes per character (and
has been extended to have four bytes per character in 6.0),
an extra two bits have been encoded in the field metadata
work for fields of type Field_string (i.e., CHAR fields).
Since the metadata word is filled, the extra bits have been
encoded in the upper 4 bits of the real type (the most
significant byte of the metadata word) by computing the
bitwise xor of the extra two bits. Since the upper 4 bits
of the real type always is 1111 for Field_string, this
means that for fields of length <256, the encoding is
identical to the encoding used in pre-5.1.26 servers, but
for lengths of 256 or more, an unrecognized type is formed,
causing an old slave (that does not handle lengths of 256
or more) to stop.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Adding test cases for replicating UTF-8 fields of lengths
of 256 or more (bytes).
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Result file change.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Adding tests to trigger check that an error is generated when replicating from a
5.1.25 server for tables with a CHAR(128) but not when replicating a table with a
CHAR(63). Although the bug indicates that the limit is 83, we elected to use CHAR(63)
since 6.0 uses 4-byte UTF-8, and anything exceeding 63 would then cause the test to fail
when the patch is merged to 6.0.
mysql-test/suite/bugs/combinations:
Adding combinations file to run all bug reports in all binlog modes (where
applicable).
mysql-test/suite/bugs/r/rpl_bug37426.result:
Result file change.
mysql-test/suite/bugs/t/rpl_bug37426.test:
Added test for reported bug.
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
Result file change.
sql/field.cc:
Encoding an extra two bits in the most significant nibble (4 bits)
of the metadata word. Adding assertions to ensure that no attempt
is made to use lengths longer than supported.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
Field_string::compatible_field_size() now reports an error if field
size for a CHAR is >255.
sql/field.h:
Field length is now computed from most significant 4 bits
of metadata word, or is equal to the row pack length if
there is no metadata.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
sql/rpl_utility.cc:
Adding relay log parameter to compatible_field_size().
Minor refactoring to eliminate duplicate code.
sql/slave.cc:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
sql/slave.h:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
Also removing gratuitous default argument.
sql/sql_insert.cc:
Changing calls to rpl_master_has_bug() to adapt to changed signature.
