mirror of
https://github.com/MariaDB/server.git
synced 2025-02-05 13:22:17 +01:00
7af50e4df4
rpl_semi_sync_slave_enabled_consistent.test and the first part of
the commit message comes from Brandon Nesterenko.
A test to show how to induce the "Read semi-sync reply magic number
error" message on a primary. In short, if semi-sync is turned on
during the hand-shake process between a primary and replica, but
later a user negates the rpl_semi_sync_slave_enabled variable while
the replica's IO thread is running; if the io thread exits, the
replica can skip a necessary call to kill_connection() in
repl_semisync_slave.slave_stop() due to its reliance on a global
variable. Then, the replica will send a COM_QUIT packet to the
primary on an active semi-sync connection, causing the magic number
error.
The test in this patch exits the IO thread by forcing an error;
though note a call to STOP SLAVE could also do this, but it ends up
needing more synchronization. That is, the STOP SLAVE command also
tries to kill the VIO of the replica, which makes a race with the IO
thread to try and send the COM_QUIT before this happens (which would
need more debug_sync to get around). See THD::awake_no_mutex for
details as to the killing of the replica’s vio.
Notes:
- The MariaDB documentation does not make it clear that when one
enables semi-sync replication it does not matter if one enables
it first in the master or slave. Any order works.
Changes done:
- The rpl_semi_sync_slave_enabled variable is now a default value for
when semisync is started. The variable does not anymore affect
semisync if it is already running. This fixes the original reported
bug. Internally we now use repl_semisync_slave.get_slave_enabled()
instead of rpl_semi_sync_slave_enabled. To check if semisync is
active on should check the @@rpl_semi_sync_slave_status variable (as
before).
- The semisync protocol conflicts in the way that the original
MySQL/MariaDB client-server protocol was designed (client-server
send and reply packets are strictly ordered and includes a packet
number to allow one to check if a packet is lost). When using
semi-sync the master and slave can send packets at 'any time', so
packet numbering does not work. The 'solution' has been that each
communication starts with packet number 1, but in some cases there
is still a chance that the packet number check can fail. Fixed by
adding a flag (pkt_nr_can_be_reset) in the NET struct that one can
use to signal that packet number checking should not be done. This
is flag is set when semi-sync is used.
- Added Master_info::semi_sync_reply_enabled to allow one to configure
some slaves with semisync and other other slaves without semisync.
Removed global variable semi_sync_need_reply that would not work
with multi-master.
- Repl_semi_sync_master::report_reply_packet() can now recognize
the COM_QUIT packet from semisync slave and not give a
"Read semi-sync reply magic number error" error for this case.
The slave will be removed from the Ack listener.
- On Windows, don't stop semisync Ack listener just because one
slave connection is using socket_id > FD_SETSIZE.
- Removed busy loop in Ack_receiver::run() by using
"Self-pipe trick" to signal new slave and stop Ack_receiver.
- Changed some Repl_semi_sync_slave functions that always returns 0
from int to void.
- Added Repl_semi_sync_slave::slave_reconnect().
- Removed dummy_function Repl_semi_sync_slave::reset_slave().
- Removed some duplicate semisync notes from the error log.
- Add test of "if (get_slave_enabled() && semi_sync_need_reply)"
before calling Repl_semi_sync_slave::slave_reply().
(Speeds up the code as we can skip all initializations).
- If epl_semisync_slave.slave_reply() fails, we disable semisync
for that connection.
- We do not call semisync.switch_off() if there are no active slaves.
Instead we check in Repl_semi_sync_master::commit_trx() if there are
no active threads. This simplices the code.
- Changed assert() to DBUG_ASSERT() to ensure that the DBUG log is
flushed in case of asserts.
- Removed the internal rpl_semi_sync_slave_status as it is not needed
anymore. The @@rpl_semi_sync_slave_status status variable is now
mapped to rpl_semi_sync_enabled.
- Removed rpl_semi_sync_slave_enabled as it is not needed anymore.
Repl_semi_sync_slave::get_slave_enabled() contains the active status.
- Added checking that we do not add a slave twice with
Ack_receiver::add_slave(). This could happen with old code.
- Removed Repl_semi_sync_master::check_and_switch() as it is not
needed anymore.
- Ensure that when we call Ack_receiver::remove_slave() that the slave
is removed from the listener before function returns.
- Call listener.listen_on_sockets() outside of mutex for better
performance and less contested mutex.
- Ensure that listening is ignoring newly added slaves when checking for
responses.
- Fixed the master ack_receiver listener is not killed if there are no
connected slaves (and thus stop semisync handling of future
connections). This could happen if all slaves sockets where would be
marked as unreliable.
- Added unlink() to base_ilist_iterator and remove() to
I_List_iterator. This enables us to remove 'dead' slaves in
Ack_recever::run().
- kill_zombie_dump_threads() now does killing of dump threads properly.
- It can now kill several threads (should be impossible but could
happen if IO slaves reconnects very fast).
- We now wait until the dump thread is done before starting the
dump.
- Added an error if kill_zombie_dump_threads() fails.
- Set thd->variables.server_id before calling
kill_zombie_dump_threads(). This simplies the code.
- Added a lot of comments both in code and tests.
- Removed DBUG_EVALUATE_IF "failed_slave_start" as it is not used.
Test changes:
- rpl.rpl_session_var2 added which runs rpl.rpl_session_var test with
semisync enabled.
- Some timings changed slight with startup of slave which caused
rpl_binlog_dump_slave_gtid_state_info.text to fail as it checked the
error log file before the slave had started properly. Fixed by
adding wait_for_pattern_in_file.inc that allows waiting for the
pattern to appear in the log file.
- Tests have been updated so that we first set
rpl_semi_sync_master_enabled on the master and then set
rpl_semi_sync_slave_enabled on the slaves (this is according to how
the MariaDB documentation document how to setup semi-sync).
- Error text "Master server does not have semi-sync enabled" has been
replaced with "Master server does not support semi-sync" for the
case when the master supports semi-sync but semi-sync is not
enabled.
Other things:
- Some trivial cleanups in Repl_semi_sync_master::update_sync_header().
- We should in 11.3 changed the default value for
rpl-semi-sync-master-wait-no-slave from TRUE to FALSE as the TRUE
does not make much sense as default. The main difference with using
FALSE is that we do not wait for semisync Ack if there are no slave
threads. In the case of TRUE we wait once, which did not bring any
notable benefits except slower startup of master configured for
using semisync.
Co-author: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
This solves the problem reported in MDEV-32960 where a new
slave may not be registered in time and the master disables
semi sync because of that.
195 lines
8 KiB
Text
195 lines
8 KiB
Text
include/master-slave.inc
|
|
[connection master]
|
|
call mtr.add_suppression("Unsafe statement written to the binary log using statement format");
|
|
connection slave;
|
|
call mtr.add_suppression("Unsafe statement written to the binary log using statement format");
|
|
connection master;
|
|
[on master]
|
|
CREATE TABLE t1 (a VARCHAR(100), b INT);
|
|
INSERT INTO t1 VALUES ("zero", 0);
|
|
==== Normal setup ====
|
|
[on slave]
|
|
connection slave;
|
|
include/stop_slave.inc
|
|
# CHANGE MASTER TO MASTER_DELAY = 2*T
|
|
include/start_slave.inc
|
|
# Asserted this: SHOW SLAVE STATUS should return the same delay that we set with CHANGE MASTER
|
|
[on master]
|
|
connection master;
|
|
INSERT INTO t1 VALUES ('normal setup', 1);
|
|
connection master;
|
|
[on slave]
|
|
include/sync_slave_io_with_master.inc
|
|
# sleep 1*T
|
|
# Asserted this: Query 1 should not be executed
|
|
# Asserted this: Status should be 'Waiting until MASTER_DELAY...'
|
|
# sleep 1*T
|
|
# sync with master (with timeout 1*T)
|
|
include/wait_for_slave_param.inc [Relay_Master_Log_File]
|
|
include/wait_for_slave_param.inc [Exec_Master_Log_Pos]
|
|
# Asserted this: Query 1 should be executed
|
|
# Asserted this: Status should be 'Has read all relay log...'
|
|
include/check_slave_is_running.inc
|
|
==== Slave lags "naturally" after master ====
|
|
[on master]
|
|
connection master;
|
|
# CREATE FUNCTION delay_on_slave(time_units INT) RETURNS INT BEGIN IF @@GLOBAL.server_id = 2 THEN RETURN SLEEP(time_units * T); ELSE RETURN 0; END IF; END
|
|
INSERT INTO t1 SELECT delay_on_slave(3), 2;
|
|
Warnings:
|
|
Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system variable that may have a different value on the slave
|
|
Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system function that may return a different value on the slave
|
|
INSERT INTO t1 VALUES ('slave is already lagging: this statement should execute immediately', 3);
|
|
INSERT INTO t1 SELECT delay_on_slave(2), 4;
|
|
Warnings:
|
|
Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system variable that may have a different value on the slave
|
|
Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system function that may return a different value on the slave
|
|
[on slave]
|
|
include/sync_slave_io_with_master.inc
|
|
# sleep 1*T
|
|
# Asserted this: No query executed
|
|
# Asserted this: Status should be 'Waiting until MASTER_DELAY...'
|
|
# wait for first query to execute
|
|
# sleep 1*T
|
|
# Asserted this: Second query executed
|
|
# Asserted this: Status should be executing third query (i.e., 'User sleep')
|
|
# sleep 2*T
|
|
# Asserted this: Third query executed
|
|
# Asserted this: Status should be 'Has read all relay log...'
|
|
==== Seconds_Behind_Master ====
|
|
# Bring slave to sync.
|
|
include/stop_slave.inc
|
|
CHANGE MASTER TO MASTER_DELAY = 0;
|
|
include/start_slave.inc
|
|
connection master;
|
|
INSERT INTO t1 VALUES ('Syncing slave', 5);
|
|
connection slave;
|
|
include/stop_slave.inc
|
|
# CHANGE MASTER TO MASTER_DELAY = 2*T
|
|
include/start_slave.inc
|
|
connection master;
|
|
INSERT INTO t1 VALUES ('Syncing slave', 5);
|
|
connection slave;
|
|
connection master;
|
|
INSERT INTO t1 VALUES (delay_on_slave(1), 6);
|
|
Warnings:
|
|
Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system variable that may have a different value on the slave
|
|
Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it uses a system function that may return a different value on the slave
|
|
connection slave;
|
|
# sleep 1*T
|
|
# Asserted this: Seconds_Behind_Master should be between 0 and the 2*T
|
|
# sleep 1*T
|
|
# Asserted this: Seconds_Behind_Master should be at least 2*T
|
|
==== STOP SLAVE / START SLAVE + DML ====
|
|
include/stop_slave.inc
|
|
# CHANGE MASTER TO MASTER_DELAY = 3*T
|
|
include/start_slave.inc
|
|
[on master]
|
|
connection master;
|
|
INSERT INTO t1 VALUES ('stop slave and start slave: DML', 7);
|
|
[on slave]
|
|
connection slave;
|
|
# sleep 1*T
|
|
include/stop_slave.inc
|
|
# Asserted this: STOP SLAVE should finish quickly, not wait for the ongoing sleep to finish
|
|
# Asserted this: SQL thread position should not increase after STOP SLAVE
|
|
# Asserted this: Query should not be executed after STOP SLAVE
|
|
# Asserted this: Status should be '' after STOP SLAVE
|
|
include/start_slave.inc
|
|
# Asserted this: START SLAVE should finish quickly
|
|
connection master;
|
|
[on slave]
|
|
include/sync_slave_io_with_master.inc
|
|
# sleep 1*T
|
|
# Asserted this: Query 7 should not be executed
|
|
# Asserted this: Status should be 'Waiting until MASTER_DELAY...'
|
|
# sleep 1*T
|
|
# sync with master (with timeout 1*T)
|
|
include/wait_for_slave_param.inc [Relay_Master_Log_File]
|
|
include/wait_for_slave_param.inc [Exec_Master_Log_Pos]
|
|
# Asserted this: Query 7 should be executed
|
|
# Asserted this: Status should be 'Has read all relay log...'
|
|
include/check_slave_is_running.inc
|
|
==== STOP SLAVE / START SLAVE + DDL ====
|
|
This verifies BUG#56442
|
|
[on master]
|
|
connection master;
|
|
CREATE TABLE t_check_dml_not_executed_prematurely (a INT);
|
|
include/save_master_pos.inc
|
|
[on slave]
|
|
connection slave;
|
|
# sleep 1*T
|
|
include/stop_slave.inc
|
|
# Asserted this: STOP SLAVE should finish quickly, not wait for the ongoing sleep to finish
|
|
# Asserted this: SQL thread position should not increase after STOP SLAVE
|
|
# Asserted this: Query should not be executed after STOP SLAVE
|
|
# Asserted this: Status should be '' after STOP SLAVE
|
|
include/start_slave.inc
|
|
# Asserted this: START SLAVE should finish quickly
|
|
# sleep 1*T
|
|
# Asserted this: DDL Query should not be executed after START SLAVE
|
|
# Asserted this: Status should be 'Waiting until MASTER_DELAY...'
|
|
# sleep 1*T
|
|
# sync with master (with timeout 1*T)
|
|
include/wait_for_slave_param.inc [Relay_Master_Log_File]
|
|
include/wait_for_slave_param.inc [Exec_Master_Log_Pos]
|
|
# Asserted this: DDL Query should be executed
|
|
# Asserted this: Status should be 'Has read all relay log...'
|
|
include/check_slave_is_running.inc
|
|
==== Change back to no delay ====
|
|
[on slave]
|
|
connection slave;
|
|
include/stop_slave.inc
|
|
CHANGE MASTER TO MASTER_DELAY = 0;
|
|
# Asserted this: Delay should be 0 when we set it to 0
|
|
include/start_slave.inc
|
|
[on master]
|
|
connection master;
|
|
INSERT INTO t1 VALUES ('change back to no delay', 8);
|
|
[on slave]
|
|
include/sync_slave_io_with_master.inc
|
|
# sleep 1*T
|
|
# Asserted this: Query should be executed
|
|
# Asserted this: Status should be 'Slave has read all relay log...'
|
|
==== Reset delay with RESET SLAVE ====
|
|
include/stop_slave.inc
|
|
CHANGE MASTER TO MASTER_DELAY = 71;
|
|
include/start_slave.inc
|
|
# Asserted this: Delay should be 71 when we set it to 71
|
|
include/stop_slave.inc
|
|
RESET SLAVE;
|
|
[on master]
|
|
connection master;
|
|
RESET MASTER;
|
|
[on slave]
|
|
connection slave;
|
|
include/start_slave.inc
|
|
# Asserted this: Delay should be 0 after RESET SLAVE
|
|
==== Set an invalid value for the delay ====
|
|
include/stop_slave.inc
|
|
# Expect error for setting negative delay
|
|
CHANGE MASTER TO MASTER_DELAY = -1;
|
|
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '-1' at line 1
|
|
# Expect that it's ok to set delay of 2^31-1
|
|
CHANGE MASTER TO MASTER_DELAY = 2147483647;
|
|
# Expect error for setting delay between 2^31 and 2^32-1
|
|
CHANGE MASTER TO MASTER_DELAY = 2147483648;
|
|
ERROR HY000: The requested value 2147483648 for the master delay exceeds the maximum 2147483647
|
|
# Expect error for setting delay to nonsense
|
|
CHANGE MASTER TO MASTER_DELAY = blah;
|
|
ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'blah' at line 1
|
|
CHANGE MASTER TO MASTER_DELAY = 0;
|
|
include/start_slave.inc
|
|
==== Clean up ====
|
|
[on master]
|
|
connection master;
|
|
DROP TABLE t1, t_check_dml_not_executed_prematurely;
|
|
DROP FUNCTION delay_on_slave;
|
|
[on slave]
|
|
connection slave;
|
|
SELECT @@GLOBAL.slave_parallel_mode;
|
|
@@GLOBAL.slave_parallel_mode
|
|
optimistic
|
|
SELECT @@GLOBAL.slave_parallel_threads;
|
|
@@GLOBAL.slave_parallel_threads
|
|
0
|
|
include/rpl_end.inc
|