BUG#32407: Impossible to do point-in-time recovery from older binlog
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
2007-12-14 19:02:02 +01:00
|
|
|
# This test case verifies that the mysqlbinlog --base64-output=X flags
|
|
|
|
# work as expected, and that BINLOG statements with row events fail if
|
|
|
|
# they are not preceded by BINLOG statements with Format description
|
|
|
|
# events.
|
|
|
|
#
|
|
|
|
# See also BUG#32407.
|
|
|
|
|
|
|
|
|
2008-02-09 19:43:32 +01:00
|
|
|
# BINLOG statement does not work in embedded mode.
|
|
|
|
source include/not_embedded.inc;
|
|
|
|
|
2008-03-28 13:16:41 +01:00
|
|
|
disable_warnings;
|
|
|
|
DROP TABLE IF EXISTS t1;
|
|
|
|
enable_warnings;
|
2008-02-09 19:43:32 +01:00
|
|
|
|
BUG#32407: Impossible to do point-in-time recovery from older binlog
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
2007-12-14 19:02:02 +01:00
|
|
|
# Test to show BUG#32407. This reads a binlog created with the
|
|
|
|
# mysql-5.1-telco-6.1 tree, specifically at the tag
|
|
|
|
# mysql-5.1.15-ndb-6.1.23, and applies it to the database. The test
|
|
|
|
# should fail before BUG#32407 was fixed and succeed afterwards.
|
|
|
|
--echo ==== Test BUG#32407 ====
|
|
|
|
|
|
|
|
# The binlog contains row events equivalent to:
|
|
|
|
# CREATE TABLE t1 (a int) engine = myisam
|
|
|
|
# INSERT INTO t1 VALUES (1), (1)
|
2008-02-07 12:30:52 +01:00
|
|
|
exec $MYSQL_BINLOG suite/binlog/std_data/bug32407.001 | $MYSQL;
|
BUG#32407: Impossible to do point-in-time recovery from older binlog
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
2007-12-14 19:02:02 +01:00
|
|
|
# The above line should succeed and t1 should contain two ones
|
|
|
|
select * from t1;
|
|
|
|
|
|
|
|
|
|
|
|
# Test that a BINLOG statement encoding a row event fails unless a
|
|
|
|
# Format_description_event as been supplied with an earlier BINLOG
|
|
|
|
# statement.
|
|
|
|
--echo ==== Test BINLOG statement w/o FD event ====
|
|
|
|
|
|
|
|
# This is a binlog statement consisting of one Table_map_log_event and
|
|
|
|
# one Write_rows_log_event. Together, they correspond to the
|
|
|
|
# following query:
|
|
|
|
# INSERT INTO TABLE test.t1 VALUES (2)
|
|
|
|
|
|
|
|
error ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENT;
|
|
|
|
BINLOG '
|
|
|
|
SVtYRxMBAAAAKQAAADQBAAAAABAAAAAAAAAABHRlc3QAAnQxAAEDAAE=
|
|
|
|
SVtYRxcBAAAAIgAAAFYBAAAQABAAAAAAAAEAAf/+AgAAAA==
|
|
|
|
';
|
|
|
|
# The above line should fail and 2 should not be in the table
|
|
|
|
select * from t1;
|
|
|
|
|
|
|
|
|
|
|
|
# Test that it works to read a Format_description_log_event with a
|
|
|
|
# BINLOG statement, followed by a row-event in base64 from the same
|
|
|
|
# version.
|
|
|
|
--echo ==== Test BINLOG statement with FD event ====
|
|
|
|
|
|
|
|
# This is a binlog statement containing a Format_description_log_event
|
|
|
|
# from the same version as the Table_map and Write_rows_log_event.
|
|
|
|
BINLOG '
|
|
|
|
ODdYRw8BAAAAZgAAAGoAAAABAAQANS4xLjIzLXJjLWRlYnVnLWxvZwAAAAAAAAAAAAAAAAAAAAAA
|
|
|
|
AAAAAAAAAAAAAAAAAAA4N1hHEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
|
|
|
|
';
|
|
|
|
|
|
|
|
# This is a Table_map_log_event+Write_rows_log_event corresponding to:
|
|
|
|
# INSERT INTO TABLE test.t1 VALUES (3)
|
|
|
|
BINLOG '
|
|
|
|
TFtYRxMBAAAAKQAAAH8BAAAAABAAAAAAAAAABHRlc3QAAnQxAAEDAAE=
|
|
|
|
TFtYRxcBAAAAIgAAAKEBAAAQABAAAAAAAAEAAf/+AwAAAA==
|
|
|
|
';
|
|
|
|
# The above line should succeed and 3 should be in the table
|
|
|
|
select * from t1;
|
|
|
|
|
|
|
|
|
|
|
|
# Test that mysqlbinlog stops with an error message when the
|
|
|
|
# --base64-output=never flag is used on a binlog with base64 events.
|
|
|
|
--echo ==== Test --base64-output=never on a binlog with row events ====
|
|
|
|
|
|
|
|
# mysqlbinlog should fail
|
BUG#31168: @@hostname does not replicate
Problem: in mixed and statement mode, a query that refers to a
system variable will use the slave's value when replayed on
slave. So if the value of a system variable is inserted into a
table, the slave will differ from the master.
Fix: mark statements that refer to a system variable as "unsafe",
meaning they will be replicated by row in mixed mode and produce a warning
in statement mode. There are some exceptions: some variables are actually
replicated. Those should *not* be marked as unsafe.
BUG#34732: mysqlbinlog does not print default values for auto_increment variables
Problem: mysqlbinlog does not print default values for some variables,
including auto_increment_increment and others. So if a client executing
the output of mysqlbinlog has different default values, replication will
be wrong.
Fix: Always print default values for all variables that are replicated.
I need to fix the two bugs at the same time, because the test cases would
fail if I only fixed one of them.
include/m_ctype.h:
Added definition of ILLEGAL_CHARSET_INFO_NUMBER. We just need a symbol
for a number that will never be used by any charset. ~0U should be safe
since charset numbers are sequential, starting from 0.
mysql-test/include/commit.inc:
Upated test to avoid making statements unsafe.
mysql-test/r/commit_1innodb.result:
Updated test needs updated result file.
mysql-test/r/mysqlbinlog.result:
Updated result file.
mysql-test/r/mysqlbinlog2.result:
Updated result file.
mysql-test/r/user_var-binlog.result:
Updated result file.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Updated result file.
mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result:
Updated result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
Modified test file needs modified result file.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Need to filter out pseudo_thread_id from result since it is
nondeterministic.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Add tests that using variables is unsafe. The 'CREATE VIEW' tests didn't
make sense, so I removed them. SHOW WARNINGS is not necessary either,
because we get warnings for each statement in the result file.
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file.
mysql-test/suite/rpl/r/rpl_skip_error.result:
Updated result file.
mysql-test/suite/rpl/t/rpl_skip_error.test:
The test used @@server_id, which is not safe to replicate, so it would
have given a warning. The way it used @@server_id was hackish (issue a
query on master that removes rows only on master), so I replaced it by a
more robust way to do the same thing (connect to slave and insert the
rows only there).
Also clarified what the test case does.
mysql-test/t/mysqlbinlog2.test:
Use --short-form instead of manually filtering out nondeterministic stuff
from mysqlbinlog (because we added the nondeterministic @@pseudo_thread_id
to the output).
sql/item_func.cc:
Added method of Item_func_get_system_var that indicates whether the given
system variable will be written to the binlog or not.
sql/item_func.h:
Added method of Item_func_get_system_var that indicates whether the given
system variable will be written to the binlog or not.
sql/log_event.cc:
- auto_increment_offset was not written to the binlog if
auto_increment_increment=1
- mysqlbinlog did not output default values for some variables
(BUG#34732). In st_print_event_info, we remember for each variable whether
it has been printed or not. This is achieved in different ways for
different variables:
- For auto_increment_*, lc_time_names, charset_database_number,
we set the default values in st_print_event_info to something
illegal, so that it will look like they have changed the first time
they are seen.
- For charset, sql_mode, pseudo_thread_id, we add a flag to
st_print_event_info which indicates whether the variable has been
printed.
- Since pseudo_thread_id is now printed more often, and its value is
not guaranteed to be constant across different runs of the same
test script, I replaced it by a constant if --short-form is used.
- Moved st_print_event_info constructor from log_event.h to log_event.cc,
since it now depends on ILLEGAL_CHARSET_NUMBER, which is defined in
m_ctype.h, which is better to include from a .cc file than from a header
file.
sql/log_event.h:
Added fields to st_print_event_info that indicate whether some of the
variables have been written or not. Since the initialization of
charset_database_number now depends on ILLEGAL_CHARSET_INFO_NUMBER, which
is defined in a header file, which we'd better not include from this
header file -- I moved the constructor from here to log_event.cc.
sql/set_var.cc:
System variables now have a flag binlog_status, which indicates if they
are written to the binlog. If nothing is specified, all variables are
marked as not written to the binlog (NOT_IN_BINLOG) when created. In this
file, the variables that are written to the binlog are marked with
SESSION_VARIABLE_IN_BINLOG.
sql/set_var.h:
Added flag binlog_status to class sys_var. Added a getter and a
constructor parameter that sets it.
Since I had to change sys_var_thd_enum constructor anyways, I simplified
it to use default values of arguments instead of three copies of the
constructor.
sql/sql_yacc.yy:
Mark statements that refer to a system variable as "unsafe",
meaning they will be replicated by row in mixed mode. Added comment to
explain strange piece of code just above.
mysql-test/include/diff_tables.inc:
New auxiliary test file that tests whether two tables (possibly one on
master and one on slave) differ.
mysql-test/suite/rpl/r/rpl_variables.result:
New test case needs new result file.
mysql-test/suite/rpl/r/rpl_variables_stm.result:
New test file needs new result file.
mysql-test/suite/rpl/t/rpl_variables.test:
Test that INSERT of @@variables is replicated correctly (by switching to
row-based mode).
mysql-test/suite/rpl/t/rpl_variables_stm.test:
Test that replication of @@variables which are replicated explicitly works
as expected in statement mode (without giving warnings).
2008-03-07 13:59:36 +01:00
|
|
|
--replace_regex /#[0-9][0-9][0-9][0-9][0-9][0-9] .*/<#>/ /SET \@\@session.pseudo_thread_id.*/<#>/
|
BUG#32407: Impossible to do point-in-time recovery from older binlog
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
2007-12-14 19:02:02 +01:00
|
|
|
error 1;
|
2008-02-07 12:30:52 +01:00
|
|
|
exec $MYSQL_BINLOG --base64-output=never suite/binlog/std_data/bug32407.001;
|
BUG#32407: Impossible to do point-in-time recovery from older binlog
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
2007-12-14 19:02:02 +01:00
|
|
|
# the above line should output the query log event and then stop
|
|
|
|
|
|
|
|
|
|
|
|
# Test that the following fails cleanly: "First, read a
|
|
|
|
# Format_description event which has N event types. Then, read an
|
|
|
|
# event of type M>N"
|
|
|
|
--echo ==== Test non-matching FD event and Row event ====
|
|
|
|
|
|
|
|
# This is the Format_description_log_event from
|
2008-02-07 12:30:52 +01:00
|
|
|
# bug32407.001, encoded in base64. It contains only the old
|
BUG#32407: Impossible to do point-in-time recovery from older binlog
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
2007-12-14 19:02:02 +01:00
|
|
|
# row events (number of event types is 22)
|
|
|
|
BINLOG '
|
|
|
|
4CdYRw8BAAAAYgAAAGYAAAAAAAQANS4xLjE1LW5kYi02LjEuMjQtZGVidWctbG9nAAAAAAAAAAAA
|
|
|
|
AAAAAAAAAAAAAAAAAADgJ1hHEzgNAAgAEgAEBAQEEgAATwAEGggICAg=
|
|
|
|
';
|
|
|
|
|
|
|
|
# The following is a Write_rows_log_event with event type 23, i.e.,
|
|
|
|
# not supported by the Format_description_log_event above. It
|
|
|
|
# corresponds to the following query:
|
|
|
|
# INSERT INTO t1 VALUES (5)
|
|
|
|
error 1149;
|
|
|
|
BINLOG '
|
|
|
|
Dl1YRxMBAAAAKQAAADQBAAAAABAAAAAAAAAABHRlc3QAAnQxAAEDAAE=
|
|
|
|
Dl1YRxcBAAAAIgAAAFYBAAAQABAAAAAAAAEAAf/+BQAAAA==
|
|
|
|
';
|
|
|
|
# the above line should fail and 5 should not be in the binlog.
|
|
|
|
select * from t1;
|
|
|
|
|
BUG#37426: RBR breaks for CHAR() UTF-8 fields > 85 chars
In order to handle CHAR() fields, 8 bits were reserved for
the size of the CHAR field. However, instead of denoting the
number of characters in the field, field_length was used which
denotes the number of bytes in the field.
Since UTF-8 fields can have three bytes per character (and
has been extended to have four bytes per character in 6.0),
an extra two bits have been encoded in the field metadata
work for fields of type Field_string (i.e., CHAR fields).
Since the metadata word is filled, the extra bits have been
encoded in the upper 4 bits of the real type (the most
significant byte of the metadata word) by computing the
bitwise xor of the extra two bits. Since the upper 4 bits
of the real type always is 1111 for Field_string, this
means that for fields of length <256, the encoding is
identical to the encoding used in pre-5.1.26 servers, but
for lengths of 256 or more, an unrecognized type is formed,
causing an old slave (that does not handle lengths of 256
or more) to stop.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Adding test cases for replicating UTF-8 fields of lengths
of 256 or more (bytes).
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Result file change.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Adding tests to trigger check that an error is generated when replicating from a
5.1.25 server for tables with a CHAR(128) but not when replicating a table with a
CHAR(63). Although the bug indicates that the limit is 83, we elected to use CHAR(63)
since 6.0 uses 4-byte UTF-8, and anything exceeding 63 would then cause the test to fail
when the patch is merged to 6.0.
mysql-test/suite/bugs/combinations:
Adding combinations file to run all bug reports in all binlog modes (where
applicable).
mysql-test/suite/bugs/r/rpl_bug37426.result:
Result file change.
mysql-test/suite/bugs/t/rpl_bug37426.test:
Added test for reported bug.
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
Result file change.
sql/field.cc:
Encoding an extra two bits in the most significant nibble (4 bits)
of the metadata word. Adding assertions to ensure that no attempt
is made to use lengths longer than supported.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
Field_string::compatible_field_size() now reports an error if field
size for a CHAR is >255.
sql/field.h:
Field length is now computed from most significant 4 bits
of metadata word, or is equal to the row pack length if
there is no metadata.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
sql/rpl_utility.cc:
Adding relay log parameter to compatible_field_size().
Minor refactoring to eliminate duplicate code.
sql/slave.cc:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
sql/slave.h:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
Also removing gratuitous default argument.
sql/sql_insert.cc:
Changing calls to rpl_master_has_bug() to adapt to changed signature.
2008-06-30 22:11:18 +02:00
|
|
|
# Test that BUG#37426 is triggered.
|
BUG#32407: Impossible to do point-in-time recovery from older binlog
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
2007-12-14 19:02:02 +01:00
|
|
|
|
BUG#37426: RBR breaks for CHAR() UTF-8 fields > 85 chars
In order to handle CHAR() fields, 8 bits were reserved for
the size of the CHAR field. However, instead of denoting the
number of characters in the field, field_length was used which
denotes the number of bytes in the field.
Since UTF-8 fields can have three bytes per character (and
has been extended to have four bytes per character in 6.0),
an extra two bits have been encoded in the field metadata
work for fields of type Field_string (i.e., CHAR fields).
Since the metadata word is filled, the extra bits have been
encoded in the upper 4 bits of the real type (the most
significant byte of the metadata word) by computing the
bitwise xor of the extra two bits. Since the upper 4 bits
of the real type always is 1111 for Field_string, this
means that for fields of length <256, the encoding is
identical to the encoding used in pre-5.1.26 servers, but
for lengths of 256 or more, an unrecognized type is formed,
causing an old slave (that does not handle lengths of 256
or more) to stop.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Adding test cases for replicating UTF-8 fields of lengths
of 256 or more (bytes).
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Result file change.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Adding tests to trigger check that an error is generated when replicating from a
5.1.25 server for tables with a CHAR(128) but not when replicating a table with a
CHAR(63). Although the bug indicates that the limit is 83, we elected to use CHAR(63)
since 6.0 uses 4-byte UTF-8, and anything exceeding 63 would then cause the test to fail
when the patch is merged to 6.0.
mysql-test/suite/bugs/combinations:
Adding combinations file to run all bug reports in all binlog modes (where
applicable).
mysql-test/suite/bugs/r/rpl_bug37426.result:
Result file change.
mysql-test/suite/bugs/t/rpl_bug37426.test:
Added test for reported bug.
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
Result file change.
sql/field.cc:
Encoding an extra two bits in the most significant nibble (4 bits)
of the metadata word. Adding assertions to ensure that no attempt
is made to use lengths longer than supported.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
Field_string::compatible_field_size() now reports an error if field
size for a CHAR is >255.
sql/field.h:
Field length is now computed from most significant 4 bits
of metadata word, or is equal to the row pack length if
there is no metadata.
Extending compatible_field_size() function with an extra parameter
holding a Relay_log_instace for error reporting.
sql/rpl_utility.cc:
Adding relay log parameter to compatible_field_size().
Minor refactoring to eliminate duplicate code.
sql/slave.cc:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
sql/slave.h:
Extending rpl_master_has_bug() with a single-argument predicate function and
a parameter to the predicate function. The predicate function can be used to
test for extra conditions for the bug before writing an error message.
Also removing gratuitous default argument.
sql/sql_insert.cc:
Changing calls to rpl_master_has_bug() to adapt to changed signature.
2008-06-30 22:11:18 +02:00
|
|
|
CREATE TABLE char128_utf8 (
|
|
|
|
i1 INT NOT NULL,
|
|
|
|
c CHAR(128) CHARACTER SET utf8 NOT NULL,
|
|
|
|
i2 INT NOT NULL);
|
|
|
|
CREATE TABLE char63_utf8 (
|
|
|
|
i1 INT NOT NULL,
|
|
|
|
c CHAR(63) CHARACTER SET utf8 NOT NULL,
|
|
|
|
i2 INT NOT NULL);
|
|
|
|
|
|
|
|
#
|
|
|
|
# This is the format description log event
|
|
|
|
#
|
|
|
|
|
|
|
|
BINLOG '
|
|
|
|
MuNkSA8BAAAAZgAAAGoAAAAAAAQANS4xLjI1LXJjLWRlYnVnLWxvZwAAAAAAAAAAAAAAAAAAAAAA
|
|
|
|
AAAAAAAAAAAAAAAAAAAy42RIEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
|
|
|
|
';
|
|
|
|
|
|
|
|
# ... this event corresponding to
|
|
|
|
#
|
|
|
|
# INSERT INTO char63_utf8 VALUES ( 1, "123", 1 )
|
|
|
|
#
|
|
|
|
# The binlog event below shall not trigger the bug check
|
|
|
|
|
|
|
|
BINLOG '
|
|
|
|
3u9kSBMBAAAANgAAAJYBAAAAABAAAAAAAAAABHRlc3QAC2NoYXI2M191dGY4AAMD/gMC/r0A
|
|
|
|
3u9kSBcBAAAAKgAAAMABAAAQABAAAAAAAAEAA//4AQAAAAMxMjMBAAAA
|
|
|
|
';
|
|
|
|
SELECT * FROM char63_utf8;
|
|
|
|
|
|
|
|
# ... and this is an event corresponding to
|
|
|
|
#
|
|
|
|
# INSERT INTO char128_utf8 VALUES ( 1, "123", 1 )
|
|
|
|
#
|
|
|
|
# The binlog event below shall trigger the bug check and produce an error
|
|
|
|
#
|
|
|
|
|
|
|
|
error ER_UNKNOWN_ERROR;
|
|
|
|
BINLOG '
|
|
|
|
iONkSBMBAAAANwAAAJkBAAAAABAAAAAAAAAABHRlc3QADGNoYXIxMjhfdXRmOAADA/4DAv6AAA==
|
|
|
|
iONkSBcBAAAAKwAAAMQBAAAQABAAAAAAAAEAA//4AQAAAAMAMTIzAQAAAA==
|
|
|
|
';
|
|
|
|
|
|
|
|
drop table t1, char63_utf8, char128_utf8;
|
2010-06-18 21:32:23 +04:00
|
|
|
|
2013-05-03 01:54:47 +03:00
|
|
|
call mtr.add_suppression("Slave SQL.*master suffers from this bug: http:..bugs.mysql.com.bug.php.id=37426.* error.* 1105");
|
|
|
|
call mtr.add_suppression("Slave SQL.*Table definition on master and slave does not match: Column 1 size mismatch.* error.* 1535");
|
|
|
|
call mtr.add_suppression("Slave SQL.*Column 1 of table .test.char128_utf8. cannot be converted.* error.* 1677");
|
2010-06-18 21:32:23 +04:00
|
|
|
|
|
|
|
--echo #
|
|
|
|
--echo # Bug #54393: crash and/or valgrind errors in
|
|
|
|
--echo # mysql_client_binlog_statement
|
|
|
|
--echo #
|
|
|
|
--error ER_SYNTAX_ERROR
|
|
|
|
BINLOG '';
|
2013-09-23 18:58:33 +04:00
|
|
|
--error ER_BASE64_DECODE_ERROR
|
2010-06-18 21:32:23 +04:00
|
|
|
BINLOG '123';
|
2013-09-23 18:58:33 +04:00
|
|
|
--error ER_BASE64_DECODE_ERROR
|
2010-06-18 21:32:23 +04:00
|
|
|
BINLOG '-2079193929';
|
2013-09-23 18:58:33 +04:00
|
|
|
--error ER_BASE64_DECODE_ERROR
|
2010-06-18 21:32:23 +04:00
|
|
|
BINLOG 'xç↓%~∙D╒ƒ╡';
|