mariadb/mysql-test/suite/rpl/r/rpl_loaddata_map.result

32 lines
1.3 KiB
Text
Raw Normal View History

BUG#30435 loading large LOAD DATA INFILE breaks slave with read_buffer_size set on master BUG#33413 show binlog events fails if binlog has event size of close to max_allowed_packet The size of Append_block replication event was determined solely by read_buffer_size whereas the rest of replication code deals with max_allowed_packet. When the former parameter was set to larger than the latter there were two artifacts: the master could not read events from binlog; show master events did not show. Fixed with - fragmenting the used io-cached buffer into pieces each size of less than max_allowed_packet (bug#30435) - incrementing show-binlog-events handling thread's max_allowed_packet with the max estimated for the replication header size include/my_sys.h: accessor-macros added in order not to mess with the io cache's implementation details in code that merely exploits the io-cache. sql/sql_repl.cc: BUG#33413: incrementing thd->variables.max_allowed_packet with the max estimation for the replication header size (from bug#19402); refactoring log_loaded_block() to fragment the io_cache buffer in case read_buffer_size > max_allowed_packet. mysql-test/r/rpl_loaddata_map.result: New BitKeeper file ``mysql-test/r/rpl_loaddata_map.result'' mysql-test/t/rpl_loaddata_map-master.opt: specific options to trigger BUG#30435, BUG#33413 situations mysql-test/t/rpl_loaddata_map-slave.opt: max_allowed_packet to be compatible with the master's version. mysql-test/t/rpl_loaddata_map.test: regression tests for two bugs.
2007-12-21 22:34:43 +02:00
stop slave;
drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
reset master;
reset slave;
drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
start slave;
==== Create a big file ====
==== Load our big file into a table ====
BUG#30435 loading large LOAD DATA INFILE breaks slave with read_buffer_size set on master BUG#33413 show binlog events fails if binlog has event size of close to max_allowed_packet The size of Append_block replication event was determined solely by read_buffer_size whereas the rest of replication code deals with max_allowed_packet. When the former parameter was set to larger than the latter there were two artifacts: the master could not read events from binlog; show master events did not show. Fixed with - fragmenting the used io-cached buffer into pieces each size of less than max_allowed_packet (bug#30435) - incrementing show-binlog-events handling thread's max_allowed_packet with the max estimated for the replication header size include/my_sys.h: accessor-macros added in order not to mess with the io cache's implementation details in code that merely exploits the io-cache. sql/sql_repl.cc: BUG#33413: incrementing thd->variables.max_allowed_packet with the max estimation for the replication header size (from bug#19402); refactoring log_loaded_block() to fragment the io_cache buffer in case read_buffer_size > max_allowed_packet. mysql-test/r/rpl_loaddata_map.result: New BitKeeper file ``mysql-test/r/rpl_loaddata_map.result'' mysql-test/t/rpl_loaddata_map-master.opt: specific options to trigger BUG#30435, BUG#33413 situations mysql-test/t/rpl_loaddata_map-slave.opt: max_allowed_packet to be compatible with the master's version. mysql-test/t/rpl_loaddata_map.test: regression tests for two bugs.
2007-12-21 22:34:43 +02:00
create table t2 (id int not null primary key auto_increment);
select @@session.read_buffer_size - @@session.max_allowed_packet > 0 ;
@@session.read_buffer_size - @@session.max_allowed_packet > 0
1
load data infile 'MYSQLTEST_VARDIR/tmp/bug30435_5k.txt' into table t2;
BUG#30435 loading large LOAD DATA INFILE breaks slave with read_buffer_size set on master BUG#33413 show binlog events fails if binlog has event size of close to max_allowed_packet The size of Append_block replication event was determined solely by read_buffer_size whereas the rest of replication code deals with max_allowed_packet. When the former parameter was set to larger than the latter there were two artifacts: the master could not read events from binlog; show master events did not show. Fixed with - fragmenting the used io-cached buffer into pieces each size of less than max_allowed_packet (bug#30435) - incrementing show-binlog-events handling thread's max_allowed_packet with the max estimated for the replication header size include/my_sys.h: accessor-macros added in order not to mess with the io cache's implementation details in code that merely exploits the io-cache. sql/sql_repl.cc: BUG#33413: incrementing thd->variables.max_allowed_packet with the max estimation for the replication header size (from bug#19402); refactoring log_loaded_block() to fragment the io_cache buffer in case read_buffer_size > max_allowed_packet. mysql-test/r/rpl_loaddata_map.result: New BitKeeper file ``mysql-test/r/rpl_loaddata_map.result'' mysql-test/t/rpl_loaddata_map-master.opt: specific options to trigger BUG#30435, BUG#33413 situations mysql-test/t/rpl_loaddata_map-slave.opt: max_allowed_packet to be compatible with the master's version. mysql-test/t/rpl_loaddata_map.test: regression tests for two bugs.
2007-12-21 22:34:43 +02:00
select count(*) from t2 /* 5 000 */;
count(*)
5000
show binlog events from <binlog_start>;
BUG#30435 loading large LOAD DATA INFILE breaks slave with read_buffer_size set on master BUG#33413 show binlog events fails if binlog has event size of close to max_allowed_packet The size of Append_block replication event was determined solely by read_buffer_size whereas the rest of replication code deals with max_allowed_packet. When the former parameter was set to larger than the latter there were two artifacts: the master could not read events from binlog; show master events did not show. Fixed with - fragmenting the used io-cached buffer into pieces each size of less than max_allowed_packet (bug#30435) - incrementing show-binlog-events handling thread's max_allowed_packet with the max estimated for the replication header size include/my_sys.h: accessor-macros added in order not to mess with the io cache's implementation details in code that merely exploits the io-cache. sql/sql_repl.cc: BUG#33413: incrementing thd->variables.max_allowed_packet with the max estimation for the replication header size (from bug#19402); refactoring log_loaded_block() to fragment the io_cache buffer in case read_buffer_size > max_allowed_packet. mysql-test/r/rpl_loaddata_map.result: New BitKeeper file ``mysql-test/r/rpl_loaddata_map.result'' mysql-test/t/rpl_loaddata_map-master.opt: specific options to trigger BUG#30435, BUG#33413 situations mysql-test/t/rpl_loaddata_map-slave.opt: max_allowed_packet to be compatible with the master's version. mysql-test/t/rpl_loaddata_map.test: regression tests for two bugs.
2007-12-21 22:34:43 +02:00
Log_name Pos Event_type Server_id End_log_pos Info
master-bin.000001 # Query # # use `test`; create table t2 (id int not null primary key auto_increment)
master-bin.000001 # Begin_load_query # # ;file_id=#;block_len=#
master-bin.000001 # Append_block # # ;file_id=#;block_len=#
master-bin.000001 # Append_block # # ;file_id=#;block_len=#
Bug#43746: YACC return wrong query string when parse 'load data infile' sql statement "load data" statements were written to the binlog as a mix of the original statement and bits recreated from parse-info. This relied on implementation details and broke with IGNORE_SPACES and versioned comments. We now completely resynthesize the query for LOAD DATA for binlog (which among other things normalizes them somewhat with regard to case, spaces, etc.). We have already parsed the query properly, so we make use of that rather than mix-and-match string literals and parsed items. This should make us safe with regard to versioned comments, even those spanning multiple tokens. Also no longer affected by IGNORE_SPACES. mysql-test/r/mysqlbinlog.result: LOAD DATA INFILE normalized mysql-test/suite/binlog/r/binlog_killed_simulate.result: LOAD DATA INFILE normalized mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result: LOAD DATA INFILE normalized mysql-test/suite/binlog/r/binlog_stm_blackhole.result: LOAD DATA INFILE normalized mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result: LOAD DATA INFILE normalized mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result: LOAD DATA INFILE normalized mysql-test/suite/rpl/r/rpl_loaddata.result: LOAD DATA INFILE normalized mysql-test/suite/rpl/r/rpl_loaddata_fatal.result: LOAD DATA INFILE normalized; offsets adjusted to reflect that mysql-test/suite/rpl/r/rpl_loaddata_map.result: LOAD DATA INFILE normalized mysql-test/suite/rpl/r/rpl_loaddatalocal.result: test for #43746 - trying to break LOAD DATA part of parser mysql-test/suite/rpl/r/rpl_stm_log.result: LOAD DATA INFILE normalized mysql-test/suite/rpl/t/rpl_loaddatalocal.test: try to break the LOAD DATA part of the parser (test for #43746) mysql-test/t/mysqlbinlog.test: LOAD DATA INFILE normalized; adjust offsets to reflect that sql/log_event.cc: clean up Load_log_event::print_query and friends so they don't print excess spaces. add support for printing charset names to print_query. sql/log_event.h: We already have three places where we synthesize LOAD DATA queries. Better use one of those! sql/sql_lex.h: When binlogging LOAD DATA statements, we make up the statement to be logged (from the parse-info, rather than substrings of the original query) now. Consequently, we no longer need (string-) pointers into the original query. sql/sql_load.cc: Completely rewrote write_execute_load_query_log_event() to synthesize the LOAD DATA statement wholesale, rather than piece it together from synthesized bits and literal excerpts from the original query. This will not only give us a nice, normalized statement (all uppercase, no excess spaces, etc.), it will also handle comments, including versioned comments right, which is certainly more than we can say about the previous incarnation. sql/sql_yacc.yy: We're no longer assembling LOAD DATA statements from bodyparts of the original query, so some bookkeeping in the parser can go.
2009-09-28 05:41:10 -07:00
master-bin.000001 # Execute_load_query # # use `test`; LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/bug30435_5k.txt' INTO TABLE `t2` FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\' LINES TERMINATED BY '\n' (id) ;file_id=#
==== Verify results on slave ====
[on slave]
BUG#30435 loading large LOAD DATA INFILE breaks slave with read_buffer_size set on master BUG#33413 show binlog events fails if binlog has event size of close to max_allowed_packet The size of Append_block replication event was determined solely by read_buffer_size whereas the rest of replication code deals with max_allowed_packet. When the former parameter was set to larger than the latter there were two artifacts: the master could not read events from binlog; show master events did not show. Fixed with - fragmenting the used io-cached buffer into pieces each size of less than max_allowed_packet (bug#30435) - incrementing show-binlog-events handling thread's max_allowed_packet with the max estimated for the replication header size include/my_sys.h: accessor-macros added in order not to mess with the io cache's implementation details in code that merely exploits the io-cache. sql/sql_repl.cc: BUG#33413: incrementing thd->variables.max_allowed_packet with the max estimation for the replication header size (from bug#19402); refactoring log_loaded_block() to fragment the io_cache buffer in case read_buffer_size > max_allowed_packet. mysql-test/r/rpl_loaddata_map.result: New BitKeeper file ``mysql-test/r/rpl_loaddata_map.result'' mysql-test/t/rpl_loaddata_map-master.opt: specific options to trigger BUG#30435, BUG#33413 situations mysql-test/t/rpl_loaddata_map-slave.opt: max_allowed_packet to be compatible with the master's version. mysql-test/t/rpl_loaddata_map.test: regression tests for two bugs.
2007-12-21 22:34:43 +02:00
select count(*) from t2 /* 5 000 */;
count(*)
5000
==== Clean up ====
[on master]
drop table t2;