Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
into hezx.(none):/media/hda5/work/mysql/bkwork/bug#30998/merge-mysql-5.1-new-rpl
sql/sql_string.cc:
Auto merged
BitKeeper/deleted/.del-compile-solaris-amd64:
Auto merged
sql/sql_view.cc:
Manual merged
When executing drop view statement on the master, the statement is written
into bin-log without checking for possible errors, so the statement would
always be bin-logged with error code cleared even if some error might occur,
for example, some of the views being dropped does not exist. This would cause
failure on the slave.
Writing bin-log after check for errors, if at least one view has been dropped
the query is bin-logged possible with an error.
sql/sql_view.cc:
Writing bin-log after check for errors, if at least one view has been dropped
the query is bin-logged possible with an error.
mysql-test/r/rpl_drop_view.result:
Add test result for bug#30998
mysql-test/t/rpl_drop_view.test:
Add test for bug#30998
into dl145h.mysql.com:/data0/mkindahl/mysql-5.1-new-rpl
mysql-test/lib/mtr_report.pl:
Auto merged
mysql-test/r/mysqlbinlog.result:
Auto merged
mysql-test/t/mysqlbinlog.test:
Auto merged
sql/log_event.cc:
Auto merged
mysql-test/mysql-test-run.pl:
Manual merge
mysql-test/lib/mtr_cases.pl:
Manual merge, using remote code.
- Additional patch to fix compiler warnings
client/mysql.cc:
Initialize warnings to 0 to avid compiler warning
Call 'print_warnings' also when error occured
mysql-test/r/mysql.result:
Change test to make is possible to see that second set of warnings
are from second invocation of mysql
mysql-test/t/mysql.test:
Change test to make is possible to see that second set of warnings
are from second invocation of mysql
The client program 'mysqlbinlog' crashed when trying to print a User_var_log_event holding
a floating-point value since the format specifier for my_b_printf() does not support
floating-point format specifiers.
This patch prints the floating-point number to an internal buffer, and then writes
that buffer to the output instead.
mysql-test/r/mysqlbinlog.result:
Result file change.
mysql-test/t/mysqlbinlog.test:
Adding test that mysqlbinlog can write and read back User_var_log_event
for real, decimal, integer, and string. These are the only types supported
for user variables.
sql/log_event.cc:
Using my_sprintf() to print floating-point value of User_var_log_event value to a
character buffer and then to the real output, since my_b_printf() does not
support floating-point format. Also adding macro to give buffer size needed
for printing floating-point numbers in %g format.
into dl145h.mysql.com:/data0/mkindahl/mysql-5.1-rpl-merge
client/mysql.cc:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/ctype_ucs.result:
Auto merged
mysql-test/r/mysql.result:
Auto merged
mysql-test/suite/ndb/r/ndb_dd_basic.result:
Auto merged
mysql-test/suite/rpl/r/rpl_extraCol_innodb.result:
Auto merged
mysql-test/suite/rpl/r/rpl_extraCol_myisam.result:
Auto merged
mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result:
Auto merged
mysql-test/t/ctype_uca.test:
Auto merged
mysql-test/t/ctype_ucs.test:
Auto merged
mysql-test/t/innodb.test:
Auto merged
mysql-test/t/mysql.test:
Auto merged
mysql-test/t/partition.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
sql/field.cc:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/slave.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
mysql-test/r/innodb.result:
Manual merge.
In several cases, an error when processing the query would cause mysql to
return to the top level without printing warnings. Fix is to always
print any available warnings before returning to the top level.
client/mysql.cc:
In com_go(), ensure that warnings are printed even if an error
occurred when processing the query. Before this patch, an error
in several places would return to the top level without printing
associated warnings.
Attempt to avoid printing the warning, though, if it's a duplicate
of mysql_error() for the connection handle.
mysql-test/r/mysql.result:
Add test for bug 25146
mysql-test/t/mysql.test:
Add test for bug 25146
mysql-test/lib/mtr_cases.pl:
updated the code for combination
mysql-test/mysql-test-run.pl:
added soft switching of binlog format
mysql-test/lib/mtr_misc.pl:
added new sub mtr_diff_opts
mysql-test/suite/rpl/r/rpl_invoked_features.result:
updated result
mysql-test/suite/rpl/t/rpl_invoked_features.test:
updated test
mysql-test/suite/rpl/t/rpl_misc_functions.test:
updated test
mysql-test/suite/rpl/t/rpl_session_var.test:
updated test
into dl145h.mysql.com:/data0/mkindahl/mysql-5.1-rpl-merge
mysql-test/r/mysqlbinlog.result:
Auto merged
mysql-test/t/mysqlbinlog.test:
Auto merged
sql/log_event.cc:
Auto merged
BitKeeper/deleted/.del-compile-solaris-amd64:
Delete: BUILD/compile-solaris-amd64
tests adjusted for Windows
mysql-test/r/log_tables.result:
Be more conservative in tests (for Windows)
mysql-test/t/log_tables.test:
Be more conservative in tests (for Windows)
into trift2.:/MySQL/M51/push-5.1
include/my_sys.h:
Auto merged
sql/CMakeLists.txt:
Manual merge: Keep the workaround for the cmake 2.4.6 bug, enclose it in "if".
into mysql.com:/home/gluh/MySQL/Merge/5.1-opt
BitKeeper/etc/ignore:
auto-union
mysql-test/r/events_bugs.result:
Auto merged
mysql-test/r/partition.result:
Auto merged
mysql-test/r/ps_2myisam.result:
Auto merged
mysql-test/r/ps_3innodb.result:
Auto merged
mysql-test/r/ps_4heap.result:
Auto merged
mysql-test/r/ps_5merge.result:
Auto merged
mysql-test/r/sp.result:
Auto merged
mysql-test/t/partition.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/field.cc:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_partition.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_udf.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/share/errmsg.txt:
Auto merged
tests/mysql_client_test.c:
Auto merged
mysql-test/r/select.result:
manual merge
mysql-test/t/select.test:
manual merge
into mysql.com:/home/gluh/MySQL/Merge/5.0-opt
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
mysql-test/r/select.result:
manual merge
mysql-test/t/select.test:
manual merge
into dl145h.mysql.com:/data0/mkindahl/mysql-5.1-new-rpl
mysql-test/extra/rpl_tests/rpl_extraMaster_Col.test:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/suite/rpl/include/rpl_mixed_dml.inc:
Auto merged
mysql-test/suite/rpl/r/rpl_bug31076.result:
Auto merged
mysql-test/suite/rpl/t/rpl_bug31076.test:
Auto merged
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Auto merged
mysql-test/suite/rpl_ndb/t/disabled.def:
Auto merged
sql/field.cc:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/slave.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
mysql-test/t/partition.test:
Manual merge.
BitKeeper/deleted/.del-rpl_row_extraColmaster_ndb.result~a2c64bae75b49d2:
Manual merge.
mysql-test/suite/rpl/r/rpl_extraColmaster_innodb.result:
Manual merge.
mysql-test/suite/rpl/r/rpl_extraColmaster_myisam.result:
Manual merge.
mysql-test/suite/rpl/t/disabled.def:
Manual merge.
sql/sql_delete.cc:
Manual merge.
storage engine system variables was not validated and
unexpected value was assigned.
The check_func_enum function used subtraction from the uint
value with the probably negative result. That result of
type uint was compared with 0 after casting to signed long
type. On architectures where long type is longer than int
type the result of comparison was unexpected.
storage/example/ha_example.cc:
Fixed bug #32034.
Sample system variable example_enum_var has been added to the
EXAMPLE storage to test ENUM variables.
sql/sql_plugin.cc:
Fixed bug #32034.
The check_func_enum function used subtraction from the uint
value with the probably negative result. That result of
type uint was compared with 0 after casting to signed long
type. On architectures where long type is longer than int
type the result of comparison was unexpected.
uint value has been casted to long type before subtraction.
mysql-test/t/plugin.test:
Added test case for bug #32034.
mysql-test/r/plugin.result:
Added test case for bug #32034.
into magare.gmz:/home/kgeorge/mysql/work/B31562-5.1-opt
sql/item.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_select.cc:
Auto merged
mysql-test/r/lowercase_view.result:
merge bug 31562 5.0->5.1: changed explain format
into moonbone.local:/work/30081-bug-5.1-opt-mysql
client/mysql.cc:
Auto merged
mysql-test/r/information_schema.result:
Auto merged
mysql-test/r/log_tables.result:
Auto merged
mysql-test/t/information_schema.test:
Auto merged
sql/sql_show.cc:
Auto merged
command and reported to a client.
The fact that a timestamp field will be set to NO on UPDATE wasn't shown
by the SHOW COMMAND and reported to a client through connectors. This led to
problems in the ODBC connector and might lead to a user confusion.
A new filed flag called ON_UPDATE_NOW_FLAG is added.
Constructors of the Field_timestamp set it when a field should be set to NOW
on UPDATE.
The get_schema_column_record function now reports whether a timestamp field
will be set to NOW on UPDATE.
mysql-test/t/information_schema.test:
A test case adjusted after fixing the bug#30081.
mysql-test/r/type_timestamp.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/type_ranges.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/show_check.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/ps_5merge.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/ps_4heap.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/ps_3innodb.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/ps_2myisam.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/metadata.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/log_tables.result:
Adjusted a test case after fixing bug#30081.
mysql-test/r/information_schema.result:
A test case adjusted after fixing the bug#30081.
mysql-test/r/grant.result:
Adjusted a test case after fixing bug#30081.
tests/mysql_client_test.c:
A test case adjusted after fixing the bug#30081.
sql/sql_show.cc:
Bug#30081: "ON UPDATE CURRENT_TIMESTAMP" wasn't shown by the SHOW FIELDS
command and reported to a client.
The get_schema_column_record function now reports whether a timestamp field
will be set to NOW on UPDATE.
sql/field.cc:
Bug#30081: "ON UPDATE CURRENT_TIMESTAMP" wasn't shown by the SHOW FIELDS
command and reported to a client.
Constructors of the Field_timestamp set the ON_UPDATE_NOW_FLAG on a field when
it should be set to NOW on UPDATE.
include/mysql_com.h:
Bug#30081: "ON UPDATE CURRENT_TIMESTAMP" wasn't shown by the SHOW FIELDS
command and reported to a client.
A new filed flag called ON_UPDATE_NOW_FLAG is added.
client/mysql.cc:
Bug#30081: "ON UPDATE CURRENT_TIMESTAMP" wasn't shown by the SHOW FIELDS
command and reported to a client.
The fieldflag2str function is adjusted to print the ON_UPDATE_NOW_FLAG.
Post pushbuild fix
Disabled test on windows due to bug#30459
(DATA/INDEX DIR for partitions not working on windows)
Patch from Mattias Jonsson.
mysql-test/r/partition_mgm.result:
Bug#32091: Security breach via directory changes
fixed non-windows lines.
mysql-test/t/partition_mgm.test:
Bug#32091: Security breach via directory changes
fixed non-windows lines.
mysql-test/t/partition_symlink.test:
Bug#32091: Security breach via directory changes
Added no_windows, since it is affected of bug#30459
The columns in HAVING can reference the GROUP BY and
SELECT columns. There can be "table" prefixes when
referencing these columns. And these "table" prefixes
in HAVING use the table alias if available.
This means that table aliases are subject to the same
storage rules as table names and are dependent on
lower_case_table_names in the same way as the table
names are.
Fixed by :
1. Treating table aliases as table names
and make them lowercase when printing out the SQL
statement for view persistence.
2. Using case insensitive comparison for table
aliases when requested by lower_case_table_names
mysql-test/r/lowercase_view.result:
Bug #31562: test case
mysql-test/t/lowercase_view.test:
Bug #31562: test case
sql/item.cc:
Bug #31562: lower_case_table_name contious comparison
when searching in GROUP BY
sql/sql_base.cc:
Bug #31562: lower_case_table_name contious comparison
when searching in SELECT
sql/sql_select.cc:
Bug #31562: treat table aliases as table names
and make them lowercase when printing
into mattiasj-laptop.(none):/home/mattiasj/clones/mysql-5.1-last_with_main
libmysqld/lib_sql.cc:
Auto merged
mysql-test/include/mix1.inc:
Auto merged
mysql-test/r/innodb_mysql.result:
Auto merged
sql/event_scheduler.cc:
Auto merged
sql/events.cc:
Auto merged
sql/ha_ndbcluster_binlog.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/slave.cc:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_connect.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_yacc.yy:
Auto merged
Partition handler fails updating tables with partitioning
based on timestamp field, as it calculates the timestamp field
AFTER it calculates the number of partition of a record.
Fixed by adding timestamp_field->set_time() call and disabling
such consequent calls
mysql-test/r/partition.result:
Bug #32067 Partitions: crash with timestamp column.
test result
mysql-test/t/partition.test:
Bug #32067 Partitions: crash with timestamp column.
test case
sql/ha_partition.cc:
Bug #32067 Partitions: crash with timestamp column.
do timestamp_field->set_time() in the ha_partition::update_row()
Buffer overflow due to wrong key length in partitioning
Changed to the correct key_length function.
sql/opt_range.cc:
Bug#31705 Partitions: crash if varchar length > 65530
Problem: buffer overflow due to wrong key-length
Fix: Using correct key_length function
mysql-test/r/partition_datatype.result:
Bug#31705 Partitions: crash if varchar length > 65530
New test-result case for testing all column types
used in key-partitioning.
(For verifying correct key-length use)
mysql-test/t/partition_datatype.test:
Bug#31705 Partitions: crash if varchar length > 65530
New test case for testing all column types
used in key-partitioning.
(For verifying correct key-length used)
into mysql.com:/home/kent/bk/bug30069/mysql-5.1-build
mysql-test/r/bigint.result:
Auto merged
mysql-test/t/bigint.test:
Auto merged
strings/ctype-simple.c:
Auto merged
Merge fix
partition_mgm did not require have_symlink.
Moved the test case to partition_symlink, which
require have_symlink, and should work on both *nix and
Windows
mysql-test/r/partition_mgm.result:
Bug#32091: Security breach via directory changes
Moved the test case to partition_symlink.
mysql-test/t/partition_mgm.test:
Bug#32091: Security breach via directory changes
Moved the test case to partition_symlink.
mysql-test/r/partition_symlink.result:
Bug#32091: Security breach via directory changes
Moved the test case to partition_symlink. It requires
have_symlink.
mysql-test/t/partition_symlink.test:
Bug#32091: Security breach via directory changes
Moved the test case to partition_symlink. It requires
have_symlink.
