Let
- T be a transactional table and N non-transactional table.
- B be begin, C commit and R rollback.
- N be a statement that accesses and changes only N-tables.
- T be a statement that accesses and changes only T-tables.
In RBR, changes to N-tables that happen early in a transaction are not immediately flushed
upon committing a statement. This behavior may, however, break consistency in the presence
of concurrency since changes done to N-tables become immediately visible to other
connections. To fix this problem, we do the following:
. B N N T C would log - B N C B N C B T C.
. B N N T R would log - B N C B N C B T R.
Note that we are not preserving history from the master as we are introducing a commit that
never happened. However, this seems to be more acceptable than the possibility of breaking
consistency in the presence of concurrency.
Let
- T be a transactional table and N non-transactional table.
- B be begin, C commit and R rollback.
- M be a mixed statement, i.e. a statement that updates both T and N.
- M* be a mixed statement that fails while updating either T or N.
This patch restore the behavior presented in 5.1.37 for rows either produced in
the RBR or MIXED modes, when a M* statement that happened early in a transaction
had their changes written to the binary log outside the boundaries of the
transaction and wrapped in a BEGIN/ROLLBACK. This was done to keep the slave
consistent with with the master as the rollback would keep the changes on N and
undo them on T. In particular, we do what follows:
. B M* T C would log - B M* R B T C.
Note that, we are not preserving history from the master as we are introducing a
rollback that never happened. However, this seems to be more acceptable than
making the slave diverge. We do not fix the following case:
. B T M* C would log B T M* C.
The slave will diverge as the changes on T tables that originated from the M
statement are rolled back on the master but not on the slave. Unfortunately, we
cannot simply rollback the transaction as this would undo any uncommitted
changes on T tables.
SBR is not considered in this patch because a failing statement is written to
the binary along with the error code and a slave executes and then rolls back
the statement when it has an associated error code, thus undoing the effects
on T. In RBR and MBR, a full-fledged fix will be pushed after the WL 2687.
In RBR, 'DROP TEMPORARY TABLE IF EXISTS...' statement is binlogged when the table
does not exist.
In fact, 'DROP TEMPORARY TABLE ...' statement should never be binlogged in RBR
no matter if the table exists or not.
This patch addresses this by checking whether we are dropping a
temporary table or not, when building the custom drop statement.
This test case uses mysqlbinlog to dump the content of master-bin.000001,
but the content of master-bin.000001 is not that this test needs.
MTR runs a lot of test cases on one server, so when this test starts, the current binlog file
might not be master-bin.000001, or there are other events are written by tests before.
'RESET MASTER' command must be called at the begin, it ensures that binlog of this test
is wrote to master-bin.000001 correctly.
Three other tests have the same problem, They were fixed together.
mysqlbinlog-cp932
binlog_incident
binlog_tmp_table
binlog
Mixing transactional (T) and non-transactional (N) tables on behalf of a
transaction may lead to inconsistencies among masters and slaves in STATEMENT
mode. The problem stems from the fact that although modifications done to
non-transactional tables on behalf of a transaction become immediately visible
to other connections they do not immediately get to the binary log and therefore
consistency is broken. Although there may be issues in mixing T and M tables in
STATEMENT mode, there are safe combinations that clients find useful.
In this bug, we fix the following issue. Mixing N and T tables in multi-level
(e.g. a statement that fires a trigger) or multi-table table statements (e.g.
update t1, t2...) were not handled correctly. In such cases, it was not possible
to distinguish when a T table was updated if the sequence of changes was N and T.
In a nutshell, just the flag "modified_non_trans_table" was not enough to reflect
that both a N and T tables were changed. To circumvent this issue, we check if an
engine is registered in the handler's list and changed something which means that
a T table was modified.
Check WL 2687 for a full-fledged patch that will make the use of either the MIXED or
ROW modes completely safe.
If using statement based replication (SBR), repeatedly calling
statements which are unsafe for SBR will cause a warning message
to be written to the error for each statement. This might lead
to filling up the error log and there is no way to disable this
behavior.
The solution is to only log these message (about statements unsafe
for statement based replication) if the log_warnings option is set.
For example:
SET GLOBAL LOG_WARNINGS = 0;
INSERT INTO t1 VALUES(UUID());
SET GLOBAL LOG_WARNINGS = 1;
INSERT INTO t1 VALUES(UUID());
In this case the message will be printed only once:
[Warning] Statement may not be safe to log in statement format.
Statement: INSERT INTO t1 VALUES(UUID())
"create as select" (innodb table)
Problem: code constructing "CREATE TABLE..." statement
doesn't take into account that current database is not set
in some cases. That may lead to a server crash.
Fix: check if current database is set.
format." warnings
Despite the fact that a statement would be filtered out from binlog, a
warning would still be thrown if it was issued with the LIMIT.
This patch addresses this issue by checking the filtering rules before
printing out the warning.
mysqlbinlog --database parameter was being ignored when processing
row events. As such no event filtering would take place.
This patch addresses this by deploying a call to shall_skip_database
when table_map_events are handled (as these contain also the name of
the database). All other rows events referencing the table id for the
filtered map event, will also be skipped.
"freeing items"
The calculation of the table map log event in the event constructor
was one byte shorter than what would be actually written. This would
lead to a mismatch between the number of bytes written and the event
end_log_pos, causing bad event alignment in the binlog (corrupted
binlog) or in the transaction cache while fixing positions
(MYSQL_BIN_LOG::write_cache). This could lead to impossible to read
binlog or even infinite loops in MYSQL_BIN_LOG::write_cache.
This patch addresses this issue by correcting the expected event
length in the Table_map_log_event constructor, when the field metadata
size exceeds 255.
In the output from mysqlbinlog, incident log events were
represented as just a comment. Since the incident log event
represents an incident that could cause the contents of the
database to change without being logged to the binary log,
it means that if the SQL is applied to a server, it could
potentially lead to that the databases are out of sync.
In order to handle that, this patch adds the statement "RELOAD
DATABASE" to the SQL output for the incident log event. This will
require a DBA to edit the file and handle the case as apropriate
before applying the output to a server.
Mysql server crashes because unsafe statements warning is wrongly elevated to error,
which is set the error status of Diagnostics_area of the thread in THD::binlog_query().
Yet the caller believes that binary logging shouldn't touch the status, so it will
set the status also later by my_ok(), my_error() or my_message() seperately
according to the execution result of the statement or transaction.
But the status of Diagnostics_area of the thread is allowed to set only once.
Fixed to clear the error wrongly set by binary logging, but keep the warning message.
The reason of the bug is in that the test makes a trick with relay log files and
did not reset fully at the end.
If mtr does not restart the test the new SQL thread tried to work with the old time
session data.
Fixed with deploying RESET slave at the clean-up.
When binlog_format is STATEMENT and the statement is unsafe before,
the unsafe warning/error message was issued without checking
whether the SQL_LOG_BIN was turned on or not.
Fixed with adding a sql_log_bin_toplevel flag in THD to check
whether SQL_LOG_BIN is ON in current session whatever the current is in sp or not.
Set wrong sql_mode when creating a procedure.
So that the sql_mode can't be writen into binary log correctly.
Restore the current session sql_mode right before generating the binlog event
when creating a procedure.
TRUNCATE TABLE fails to replicate when stmt-based binlogging is not supported.
Correcting some tests that was failing in pushbuild as well as fixing result
file for some tests that are not executed in the default MTR run.
The problem is that a unfiltered user query was being passed as
the format string parameter of sql_print_warning which later
performs printf-like formatting, leading to crashes if the user
query contains formatting instructions (ie: %s). Also, it was
using THD::query as the source of the user query, but this
variable is not meaningful in some situations -- in a delayed
insert, it points to the table name.
The solution is to pass the user query as a parameter for the
format string and use the function parameter query_arg as the
source of the user query.
TRUNCATE TABLE fails to replicate when stmt-based binlogging is not supported.
There were two separate problems with the code, both of which are fixed with
this patch:
1. An error was printed by InnoDB for TRUNCATE TABLE in statement mode when
the in isolation levels READ COMMITTED and READ UNCOMMITTED since InnoDB
does permit statement-based replication for DML statements. However,
the TRUNCATE TABLE is not transactional, but is a DDL, and should therefore
be allowed to be replicated as a statement.
2. The statement was not logged in mixed mode because of the error above, but
the error was not reported to the client.
This patch fixes the problem by treating TRUNCATE TABLE a DDL, that is, it is
always logged as a statement and not reporting an error from InnoDB for TRUNCATE
TABLE.
The test case proposed by the bugfix fails in bugteam trees after merging new
mtr from main. The failure is due to the fact that the binlog file location has
changed and is no more under $MYSQLTEST_VARDIR/log.
This patch fixes the test failure by setting the correct path to the binlog
file.
When using CREATE TEMPORARY TABLE LIKE to create a temporary table,
or using TRUNCATE to delete all rows of a temporary table, they
did not set the tmp_table_used flag, and cause the omission of
"SET @@session.pseudo_thread_id" when dumping binlog with mysqlbinlog,
and cause error when replay the statements.
This patch fixed the problem by setting tmp_table_used in these two
cases. (Done by He Zhenxing 2009-01-12)
conflicts:
Text conflict in client/mysqltest.cc
Text conflict in mysql-test/include/wait_until_connected_again.inc
Text conflict in mysql-test/lib/mtr_report.pm
Text conflict in mysql-test/mysql-test-run.pl
Text conflict in mysql-test/r/events_bugs.result
Text conflict in mysql-test/r/log_state.result
Text conflict in mysql-test/r/myisam_data_pointer_size_func.result
Text conflict in mysql-test/r/mysqlcheck.result
Text conflict in mysql-test/r/query_cache.result
Text conflict in mysql-test/r/status.result
Text conflict in mysql-test/suite/binlog/r/binlog_index.result
Text conflict in mysql-test/suite/binlog/r/binlog_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_packet.result
Text conflict in mysql-test/suite/rpl/t/rpl_packet.test
Text conflict in mysql-test/t/disabled.def
Text conflict in mysql-test/t/events_bugs.test
Text conflict in mysql-test/t/log_state.test
Text conflict in mysql-test/t/myisam_data_pointer_size_func.test
Text conflict in mysql-test/t/mysqlcheck.test
Text conflict in mysql-test/t/query_cache.test
Text conflict in mysql-test/t/rpl_init_slave_func.test
Text conflict in mysql-test/t/status.test
Added cleanup of status variables to the end of binlog_database.
Re-recorded .result file to account for cleanup statement.
NOTE: binlog.binlog_innodb also has had an FLUSH STATUS; statement added to it as well, but
adding this cleanup as a preventative measure.
The binlog_innodb test was sensitive to what tests ran before it. Now run
FLUSH STATUS before performing operations that need to be checked.
sys_var_thd_ulong::update() was improperly casting an option value from
ulonglong to ulong before comparing it to the max allowed value. On systems
where ulong and ulonglong are of different size, this caused values greater
than ULONG_MAX to wrap around (not be truncated to ULONG_MAX, which appears to
have been the intention of the original coder), and caused some checks to work
incorrectly. This wasn't generally visible to the user, because later checks
would prevent the wrapped-around value from being used. But it caused warning
messages to differ between 32- and 64-bit platforms. Fix is to just remove the
cast. Also added a DBUG_ASSERT to ensure that the value really is capped
properly before finally stuffing it into the ulong.
binlog_row_mix_innodb_myisam resutls are corrected.
The last operations prior the dup error is
TRUNCATE table t2;
Therefore after
--error ER_DUP_ENTRY
INSERT INTO t2 select * from t1;
table t2 must be empty, and that is what the updated results confirm.
after rollback on master
When starting a transaction with a statement containing changes
to both transactional tables and non-transactional tables, the
statement is considered as non-transactional and is therefore
written directly to the binary log. This behaviour was present
in 5.0, and has propagated to 5.1.
If a trigger containing a change of a non-transactional table is
added to a transactional table, any changes to the transactional
table is "tainted" as non-transactional.
This patch solves the problem by removing the existing "hack" that
allows non-transactional statements appearing first in a transaction
to be written directly to the binary log. Instead, anything inside
a transaction is treaded as part of the transaction and not written
to the binary log until the transaction is committed.
A transaction could result in having an extra event after a query that
errored e.g because of a dup key. Such a query is rolled back in
innodb, as specified, but has not been in binlog.
It appeares that the binlog engine did not always register for a query
(statement) because the previous query had not reset at its statement
commit time. Because of that fact there was no roll-back to the
trx_data->before_stmt_pos position and a the pending event of the
errorred query could become flushed to the binlog file.
Fixed with deploying the reset of trx_data->before_stmt_pos at the end
of the query processing.
Problem: Many test cases don't clean up after themselves (fail
to drop tables or fail to reset variables). This implies that:
(1) check-testcase in the new mtr that currently lives in
5.1-rpl failed. (2) it may cause unexpected results in
subsequent tests.
Fix: make all tests clean up.
Also: cleaned away unnecessary output in rpl_packet.result
Also: fixed bug where rpl_log called RESET MASTER with a running
slave. This is not supposed to work.
Also: removed unnecessary code from rpl_stm_EE_err2 and made it
verify that an error occurred.
Also: removed unnecessary code from rpl_ndb_ctype_ucs2_def.
The test failed originally -- did not reset binlogging - for the reason
identified by bug@15580.
However it never can be run on the embedded platfrom for yet another cause -
the embedded can not KILL query.
Comments added to the test particularly relating `reset master'
to the mentioned bug.
The Blackhole engine did not support row-based replication
since the delete_row(), update_row(), and the index and range
searching functions were not implemented.
This patch adds row-based replication support for the
Blackhole engine by implementing the two functions mentioned
above, and making the engine pretend that it has found the
correct row to delete or update when executed from the slave
SQL thread by implementing index and range searching functions.
It is necessary to only pretend this for the SQL thread, since
a SELECT executed on the Blackhole engine will otherwise never
return EOF, causing a livelock.
