Replication: new code to not modify in-memory log positions until the COMMIT
is executed, even if the transaction spans on >=2 relay logs (bug #53).
New variable relay_log_purge =0|1
New test to verify bug #53
sql/log.cc:
Now we purge a relay log only when we are sure we won't need it,
i.e. we have executed the final query (if autocommit=1) or the COMMIT.
sql/log_event.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/mysql_priv.h:
new option relay_log_purge (the user can now decide himself
if he wants his relay logs to be automatically purged or not,
we don't make unsafe guesses like before)
sql/mysqld.cc:
new option --innodb (replaces --skip-innodb).
Useful for the test suite : we have skip-innodb in mysql-test-run,
but we can ('-opt.info' file) choose to start the server with
InnoDB for this test only.
New option --bdb
sql/repl_failsafe.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/set_var.cc:
new variable relay_log_purge
sql/slave.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
Now we purge a relay log only when we are sure we won't need it,
i.e. we have executed the final query (if autocommit=1) or the COMMIT
sql/slave.h:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/sql_class.h:
prototypes change
sql/sql_parse.cc:
removed thd argument (was not used in the function's body)
sql/sql_repl.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
Turn relay_log_purge silently off when someone does CHANGE
MASTER TO RELAY_LOG_*
2003-04-24 15:29:25 +02:00
|
|
|
# When the relay log gets rotated while the I/O thread
|
|
|
|
# is reading a transaction, the transaction spans on two or more
|
|
|
|
# relay logs. If STOP SLAVE occurs while the SQL thread is
|
|
|
|
# executing a part of the transaction in the non-first relay logs,
|
|
|
|
# we test if START SLAVE will resume in the beginning of the
|
|
|
|
# transaction (i.e., step back to the first relay log)
|
|
|
|
|
|
|
|
# The slave is started with max_binlog_size=16384 bytes,
|
|
|
|
# to force many rotations (approximately 30 rotations)
|
|
|
|
|
|
|
|
# If the master or slave does not support InnoDB, this test will pass
|
|
|
|
|
|
|
|
source include/master-slave.inc;
|
|
|
|
connection slave;
|
|
|
|
stop slave;
|
|
|
|
connection master;
|
2003-06-11 17:07:34 +02:00
|
|
|
--disable_warnings
|
2003-12-10 05:31:42 +01:00
|
|
|
create table t1 (a int) engine=innodb;
|
2003-06-11 17:07:34 +02:00
|
|
|
--enable_warnings
|
Replication: new code to not modify in-memory log positions until the COMMIT
is executed, even if the transaction spans on >=2 relay logs (bug #53).
New variable relay_log_purge =0|1
New test to verify bug #53
sql/log.cc:
Now we purge a relay log only when we are sure we won't need it,
i.e. we have executed the final query (if autocommit=1) or the COMMIT.
sql/log_event.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/mysql_priv.h:
new option relay_log_purge (the user can now decide himself
if he wants his relay logs to be automatically purged or not,
we don't make unsafe guesses like before)
sql/mysqld.cc:
new option --innodb (replaces --skip-innodb).
Useful for the test suite : we have skip-innodb in mysql-test-run,
but we can ('-opt.info' file) choose to start the server with
InnoDB for this test only.
New option --bdb
sql/repl_failsafe.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/set_var.cc:
new variable relay_log_purge
sql/slave.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
Now we purge a relay log only when we are sure we won't need it,
i.e. we have executed the final query (if autocommit=1) or the COMMIT
sql/slave.h:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/sql_class.h:
prototypes change
sql/sql_parse.cc:
removed thd argument (was not used in the function's body)
sql/sql_repl.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
Turn relay_log_purge silently off when someone does CHANGE
MASTER TO RELAY_LOG_*
2003-04-24 15:29:25 +02:00
|
|
|
let $1=8000;
|
|
|
|
disable_query_log;
|
|
|
|
begin;
|
|
|
|
while ($1)
|
|
|
|
{
|
|
|
|
# eval means expand $ expressions
|
|
|
|
eval insert into t1 values( $1 );
|
|
|
|
dec $1;
|
|
|
|
}
|
|
|
|
commit;
|
|
|
|
# This will generate a 500kB master's binlog,
|
|
|
|
# which corresponds to 30 slave's relay logs.
|
|
|
|
enable_query_log;
|
|
|
|
save_master_pos;
|
|
|
|
connection slave;
|
|
|
|
reset slave;
|
|
|
|
start slave;
|
|
|
|
# We wait 1 sec for the SQL thread to be somewhere in
|
|
|
|
# the middle of the transaction, hopefully not in
|
|
|
|
# the first relay log, and hopefully before the COMMIT.
|
|
|
|
# Usually it stops when the SQL thread is around the 15th relay log.
|
|
|
|
# We cannot use MASTER_POS_WAIT() as master's position
|
|
|
|
# increases only when the slave executes the COMMIT.
|
2003-10-03 22:14:23 +02:00
|
|
|
# Note that except when using Valgrind, 1 second is enough for the I/O slave
|
|
|
|
# thread to fetch the whole master's binlog.
|
2003-08-20 16:32:00 +02:00
|
|
|
sleep 1;
|
Replication: new code to not modify in-memory log positions until the COMMIT
is executed, even if the transaction spans on >=2 relay logs (bug #53).
New variable relay_log_purge =0|1
New test to verify bug #53
sql/log.cc:
Now we purge a relay log only when we are sure we won't need it,
i.e. we have executed the final query (if autocommit=1) or the COMMIT.
sql/log_event.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/mysql_priv.h:
new option relay_log_purge (the user can now decide himself
if he wants his relay logs to be automatically purged or not,
we don't make unsafe guesses like before)
sql/mysqld.cc:
new option --innodb (replaces --skip-innodb).
Useful for the test suite : we have skip-innodb in mysql-test-run,
but we can ('-opt.info' file) choose to start the server with
InnoDB for this test only.
New option --bdb
sql/repl_failsafe.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/set_var.cc:
new variable relay_log_purge
sql/slave.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
Now we purge a relay log only when we are sure we won't need it,
i.e. we have executed the final query (if autocommit=1) or the COMMIT
sql/slave.h:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
sql/sql_class.h:
prototypes change
sql/sql_parse.cc:
removed thd argument (was not used in the function's body)
sql/sql_repl.cc:
Better tracking of the relay log's name and position
lastly executed, even if we are in a transaction which spans on
2 or more relay logs.
Turn relay_log_purge silently off when someone does CHANGE
MASTER TO RELAY_LOG_*
2003-04-24 15:29:25 +02:00
|
|
|
stop slave;
|
|
|
|
# We suppose the SQL thread stopped before COMMIT.
|
|
|
|
# If so the transaction was rolled back
|
|
|
|
# and the table is now empty.
|
|
|
|
# Now restart
|
|
|
|
start slave;
|
|
|
|
# And see if the table contains '8000'
|
|
|
|
# which proves that the transaction restarted at
|
|
|
|
# the right place.
|
|
|
|
# We must wait for the transaction to commit before
|
|
|
|
# reading, MASTER_POS_WAIT() will do it for sure
|
|
|
|
# (the only statement with position>=3000 is COMMIT).
|
2003-10-03 20:07:08 +02:00
|
|
|
select master_pos_wait('master-bin.001',3000)>=0;
|
This is the final commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
It contains a few bugfixes (which I made when running the testsuite).
I carefully updated the results of the testsuite (i.e. I checked for every one,
if the difference between .reject and .result could be explained).
Apparently mysql-test-run --manager is broken in 4.1 and 5.0 currently,
so I could neither run the few tests which require --manager, nor check
that they pass nor modify their .result. But for builds, we don't run
with --manager.
Apart from --manager, the full testsuite passes, with Valgrind too (no errors).
I'm going to push in the next minutes. Remains: update the manual.
Note: by chance I saw that (in 4.1, in 5.0) rpl_get_lock fails when run alone;
this is normal at it makes assumptions on thread ids. I will fix this one day
in 4.1.
mysql-test/r/rpl000015.result:
result update
mysql-test/r/rpl_change_master.result:
result update
mysql-test/r/rpl_error_ignored_table.result:
result update
mysql-test/r/rpl_flush_log_loop.result:
result update
mysql-test/r/rpl_flush_tables.result:
result update
mysql-test/r/rpl_loaddata.result:
result update
mysql-test/r/rpl_loaddata_rule_m.result:
result update
mysql-test/r/rpl_loaddata_rule_s.result:
result update
mysql-test/r/rpl_log.result:
result update
mysql-test/r/rpl_log_pos.result:
result update
mysql-test/r/rpl_max_relay_size.result:
result update
mysql-test/r/rpl_relayrotate.result:
result update
mysql-test/r/rpl_replicate_do.result:
result update
mysql-test/r/rpl_reset_slave.result:
result update
mysql-test/r/rpl_rotate_logs.result:
result update
mysql-test/r/rpl_session_var.result:
result update
mysql-test/r/rpl_temporary.result:
result update
mysql-test/r/rpl_trunc_binlog.result:
result update
mysql-test/r/rpl_until.result:
result update
mysql-test/r/rpl_user_variables.result:
result update
mysql-test/t/rpl000010-slave.opt:
need to wait 2 events, because now we receive a Format_desc on top of the Rotate,
when replication starts.
mysql-test/t/rpl000015.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_change_master.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_empty_master_crash.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_error_ignored_table.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_flush_log_loop.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_loaddata.test:
position update
mysql-test/t/rpl_loaddata_rule_m.test:
position update
mysql-test/t/rpl_loaddata_rule_s.test:
position update
mysql-test/t/rpl_log.test:
position update
mysql-test/t/rpl_log_pos.test:
position update
mysql-test/t/rpl_max_relay_size.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_openssl.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_redirect.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_relayrotate-slave.opt:
better options for this test
mysql-test/t/rpl_relayrotate.test:
using max() is better for debugging (it shows at which place the slave
SQL thread resumed)
mysql-test/t/rpl_replicate_do.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_reset_slave.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_rotate_logs.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_session_var.test:
100 because password() is longer than 10 chars
mysql-test/t/rpl_trunc_binlog.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_until.test:
position update
mysql-test/t/rpl_user_variables.test:
position update
sql/log.cc:
Ensure that the Format_desc propagated on next relay logs does not trigger
undue actions (like incrementing some positions or clearing some files).
sql/log_event.cc:
* When the slave SQL thread finds a Rotate/Format_desc/Stop in the middle of
a transaction (then these were written by the slave itself to its relay log),
it should not increment rli->group* variables, but only rli->event* ones.
* When the slave SQL thread finds a Format_desc not to be ignored (not
the same server id as the slave's), if it has log_pos==0 it must not trigger
"unfinished transaction in master's binlog" (log_pos==0 is always a marker
in the relay log to mean "this event was not at this place in the master's
binlog": it's for fake Rotate events, and for Format_description events which the master had
to send us for replication to start).
* In the Query_log_event on disk, catalog is now terminated by '\0'.
* thd->catalog must be set to 0 when some exec_event() terminate (otherwise
double free).
sql/slave.cc:
* Fixes for a few bugs when ignoring events in the slave SQL thread:
- do not decrement rli->slave_skip_counter if the event is an event
related to the binlog or relay log itself (FORMAT_DESCRIPTION,
ROTATE, STOP) because these events should never be skipped (or the slave
will be confused). Usually the user wants to skip a query, not a Rotate...
- when we (re)connect to the master, we must free description_event_for_queue
(otherwise memory leak when we reconnect).
* Changed a bit the code where we change description_event_for_queue,
to make it look "safer".
* Moved 'created=0' to log.cc where it is safer.
* When the slave SQL thread finds a Rotate/Format_desc/Stop in the middle of
a transaction (then these were written by the slave itself to its relay log),
it should not increment rli->group* variables, but only rli->event* ones.
sql/sql_class.h:
a warning comment
sql/sql_repl.cc:
A mistake: I had passed a char* instead of char**
2003-12-19 22:40:23 +01:00
|
|
|
select max(a) from t1;
|
|
|
|
--replace_column 1 # 8 # 9 # 23 # 33 #
|
2003-12-21 18:41:34 +01:00
|
|
|
--replace_result $MASTER_MYPORT MASTER_MYPORT
|
This is the final commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
It contains a few bugfixes (which I made when running the testsuite).
I carefully updated the results of the testsuite (i.e. I checked for every one,
if the difference between .reject and .result could be explained).
Apparently mysql-test-run --manager is broken in 4.1 and 5.0 currently,
so I could neither run the few tests which require --manager, nor check
that they pass nor modify their .result. But for builds, we don't run
with --manager.
Apart from --manager, the full testsuite passes, with Valgrind too (no errors).
I'm going to push in the next minutes. Remains: update the manual.
Note: by chance I saw that (in 4.1, in 5.0) rpl_get_lock fails when run alone;
this is normal at it makes assumptions on thread ids. I will fix this one day
in 4.1.
mysql-test/r/rpl000015.result:
result update
mysql-test/r/rpl_change_master.result:
result update
mysql-test/r/rpl_error_ignored_table.result:
result update
mysql-test/r/rpl_flush_log_loop.result:
result update
mysql-test/r/rpl_flush_tables.result:
result update
mysql-test/r/rpl_loaddata.result:
result update
mysql-test/r/rpl_loaddata_rule_m.result:
result update
mysql-test/r/rpl_loaddata_rule_s.result:
result update
mysql-test/r/rpl_log.result:
result update
mysql-test/r/rpl_log_pos.result:
result update
mysql-test/r/rpl_max_relay_size.result:
result update
mysql-test/r/rpl_relayrotate.result:
result update
mysql-test/r/rpl_replicate_do.result:
result update
mysql-test/r/rpl_reset_slave.result:
result update
mysql-test/r/rpl_rotate_logs.result:
result update
mysql-test/r/rpl_session_var.result:
result update
mysql-test/r/rpl_temporary.result:
result update
mysql-test/r/rpl_trunc_binlog.result:
result update
mysql-test/r/rpl_until.result:
result update
mysql-test/r/rpl_user_variables.result:
result update
mysql-test/t/rpl000010-slave.opt:
need to wait 2 events, because now we receive a Format_desc on top of the Rotate,
when replication starts.
mysql-test/t/rpl000015.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_change_master.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_empty_master_crash.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_error_ignored_table.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_flush_log_loop.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_loaddata.test:
position update
mysql-test/t/rpl_loaddata_rule_m.test:
position update
mysql-test/t/rpl_loaddata_rule_s.test:
position update
mysql-test/t/rpl_log.test:
position update
mysql-test/t/rpl_log_pos.test:
position update
mysql-test/t/rpl_max_relay_size.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_openssl.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_redirect.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_relayrotate-slave.opt:
better options for this test
mysql-test/t/rpl_relayrotate.test:
using max() is better for debugging (it shows at which place the slave
SQL thread resumed)
mysql-test/t/rpl_replicate_do.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_reset_slave.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_rotate_logs.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_session_var.test:
100 because password() is longer than 10 chars
mysql-test/t/rpl_trunc_binlog.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_until.test:
position update
mysql-test/t/rpl_user_variables.test:
position update
sql/log.cc:
Ensure that the Format_desc propagated on next relay logs does not trigger
undue actions (like incrementing some positions or clearing some files).
sql/log_event.cc:
* When the slave SQL thread finds a Rotate/Format_desc/Stop in the middle of
a transaction (then these were written by the slave itself to its relay log),
it should not increment rli->group* variables, but only rli->event* ones.
* When the slave SQL thread finds a Format_desc not to be ignored (not
the same server id as the slave's), if it has log_pos==0 it must not trigger
"unfinished transaction in master's binlog" (log_pos==0 is always a marker
in the relay log to mean "this event was not at this place in the master's
binlog": it's for fake Rotate events, and for Format_description events which the master had
to send us for replication to start).
* In the Query_log_event on disk, catalog is now terminated by '\0'.
* thd->catalog must be set to 0 when some exec_event() terminate (otherwise
double free).
sql/slave.cc:
* Fixes for a few bugs when ignoring events in the slave SQL thread:
- do not decrement rli->slave_skip_counter if the event is an event
related to the binlog or relay log itself (FORMAT_DESCRIPTION,
ROTATE, STOP) because these events should never be skipped (or the slave
will be confused). Usually the user wants to skip a query, not a Rotate...
- when we (re)connect to the master, we must free description_event_for_queue
(otherwise memory leak when we reconnect).
* Changed a bit the code where we change description_event_for_queue,
to make it look "safer".
* Moved 'created=0' to log.cc where it is safer.
* When the slave SQL thread finds a Rotate/Format_desc/Stop in the middle of
a transaction (then these were written by the slave itself to its relay log),
it should not increment rli->group* variables, but only rli->event* ones.
sql/sql_class.h:
a warning comment
sql/sql_repl.cc:
A mistake: I had passed a char* instead of char**
2003-12-19 22:40:23 +01:00
|
|
|
show slave status;
|
2004-04-01 22:47:20 +02:00
|
|
|
connection master;
|
|
|
|
|
|
|
|
# binlog_cache_use and binlog_cache_disk_use status vars test
|
|
|
|
# This test uses the previous test. Namely, it needs the long
|
|
|
|
# transaction that adds 8000 lines to the t1 table.
|
|
|
|
|
|
|
|
show status like "binlog_cache_use";
|
|
|
|
show status like "binlog_cache_disk_use";
|
|
|
|
|
|
|
|
# transaction which should not be flushed to disk and so should not
|
|
|
|
# increase binlog_cache_disk_use
|
|
|
|
begin;
|
|
|
|
delete from t1;
|
|
|
|
commit;
|
|
|
|
|
|
|
|
show status like "binlog_cache_use";
|
|
|
|
show status like "binlog_cache_disk_use";
|
2003-05-22 22:29:44 +02:00
|
|
|
|
|
|
|
# The following DROP is a very important cleaning task:
|
|
|
|
# imagine the next test is run with --skip-innodb: it will do
|
|
|
|
# DROP TABLE IF EXISTS t1; but this will delete the frm and leave
|
|
|
|
# some data in the InnoDB datafile (because at that time mysqld
|
|
|
|
# does not know about InnoDB : --skip-innodb). So if later in the
|
|
|
|
# test suite a test wants to create an InnoDB table called t1, it
|
|
|
|
# will fail with
|
|
|
|
# InnoDB: Error: table t1 already exists in InnoDB internal
|
|
|
|
# InnoDB: data dictionary. Have you deleted the .frm file etc
|
|
|
|
drop table t1;
|
2004-04-01 22:47:20 +02:00
|
|
|
# wait until this drop is executed on slave
|
|
|
|
save_master_pos;
|
|
|
|
connection slave;
|
|
|
|
sync_with_master;
|