The rpl_ndb/combinations file was introduced as part of the fix.
The file contained an error: ndb suites shall not run with
binlog_format=mixed. Removed that combination.
Post-push fix.
Problem: After the original bugfix, if a statement is unsafe,
binlog_format=mixed, and engine is statement-only, a warning was
generated and the statement executed. However, it is a fundamental
principle of binlogging that binlog_format=mixed should guarantee
correct logging, no compromise. So correct behavior is to generate
an error and don't execute the statement.
Fix: Generate error instead of warning.
Since issue_unsafe_warnings can only generate one error message,
this allows us to simplify the code a bit too:
decide_logging_format does not have to save the error code for
issue_unsafe_warnings
Use ev_offset instead of 1 as the packet header offset when getting
log position from events for heartbeat
call reset_transmit_packet before calling send_heartbeat_event
Add an option to control whether the master should keep waiting
until timeout when it detected that there is no semi-sync slave
available.
The bool option 'rpl_semi_sync_master_wait_no_slave' is 1 by
defalt, and will keep waiting until timeout. When set to 0, the
master will switch to asynchronous replication immediately when
no semi-sync slave is available.
Semi-sync status were not reset by FLUSH STATUS, this was because
all semi-sync status variables are defined as SHOW_FUNC and FLUSH
STATUS could only reset SHOW_LONG type variables.
This problem is fixed by change all status variables that should
be reset by FLUSH STATUS from SHOW_FUNC to SHOW_LONG.
After the fix, the following status variables will be reset by
FLUSH STATUS:
Rpl_semi_sync_master_yes_tx
Rpl_semi_sync_master_no_tx
Note: normally, FLUSH STATUS itself will be written into binlog
and be replicated, so after FLUSH STATS, one of
Rpl_semi_sync_master_yes_tx
Rpl_semi_sync_master_no_tx
can be 1 dependent on the semi-sync status. So it's recommended
to use FLUSH NO_WRITE_TO_BINLOG STATUS to avoid this.
Errors when send reply to master should never cause the IO thread
to stop, because master can fall back to async replication if it
does not get reply from slave.
The problem is fixed by deliberately ignoring the return value of
slaveReply.
Semi-sync uses an extra connection from slave to master to send
replies, this is a normal client connection, and used a normal
SET query to set the reply information on master, which is visible
to user and may cause some confusion and complaining.
This problem is fixed by using the method of sending reply by
using the same connection that is used by master dump thread to
send binlog to slave. Since now the semi-sync plugins are integrated
with the server code, it is not a problem to use the internal net
interfaces to do this.
The master dump thread will mark the event requires a reply and
wait for the reply when the event just sent is the last event
of a transaction and semi-sync status is ON; And the slave will
send a reply to master when it received such an event that requires
a reply.
The issue appears when number of heartbeat events non-zero before start of test
block. But really we need to check that no new events has received during test block.
So I did following:
1. Replace absolute values by diff of values
2. Increase heartbeat period from 1.5 to 5 sec
CHANGE MASTER TO command required the value for RELAY_LOG_FILE to
be an absolute path, which was different from the requirement of
MASTER_LOG_FILE.
This patch fixed the problem by changing the value for RELAY_LOG_FILE
to be the basename of the log file as that for MASTER_LOG_FILE.
Options loaded from config files were added before command line
arguments, and they were parsed together, which could interprete
the following:
option-a
option-b
as --option-a=--option-b if 'option-a' requires a value, and
caused confusing.
Because all options that requires a value are always given in
the form '--option=value', so it's an error if there is no
'=value' part for such an option read from config file.
This patch added a separator to separate the arguments from
config files and that from command line, so that they can be
handled differently. And report an error for options loaded
from config files that requires a value and is not given in the
form '--option=value'.
There were two memory leaks in mysqlbinlog command, one was already
fixed by previous patches, another one was that defaults_argv was
set to the value of argv after parse_args, in which called
handle_options after calling load_defaults and changed the value
of argv, and caused the memory allocated for defaults arguments
not freed.
Fixed the problem by setting defaults_argv right after calling
load_defaults.
There are three internal status for slave I/O thread, both
MYSQL_SLAVE_RUN_NOT_CONNECT and MYSQL_SLAVE_NOT_RUN are reported
as 'No' for Slave_IO_running of command SHOW SLAVE STATUS.
Change MYSQL_SLAVE_RUN_NOT_CONNECT to be reported as 'Connecting'.
When a storage engine failed to initialize before allocated slot number,
the slot number would be 0, and when later finalizing this plugin, it would
accidentally unplug the storage engine currently uses slot 0.
This patch fixed this problem by add a new macro value HA_SLOT_UNDEF to
distinguish undefined slot number from slot 0.
NOTE: Backporting the patch to next-mr.
The slave was crashing while failing to execute the init_slave() function.
The issue stems from two different reasons:
1 - A failure while allocating the master info structure generated a
segfault due to a NULL pointer.
2 - A failure while recovering generated a segfault due to a non-initialized
relay log file. In other words, the mi->init and rli->init were both set to true
before executing the recovery process thus creating an inconsistent state as the
relay log file was not initialized.
To circumvent such problems, we refactored the recovery process which is now executed
while initializing the relay log. It is ensured that the master info structure is
created before accessing it and any error is propagated thus avoiding to set mi->init
and rli->init to true when for instance the relay log is not initialized or the relay
info is not flushed.
The changes related to the refactory are described below:
1 - Removed call to init_recovery from init_slave.
2 - Changed the signature of the function init_recovery.
3 - Removed flushes. They are called while initializing the relay log and master
info.
4 - Made sure that if the relay info is not flushed the mi-init and rli-init are not
set to true.
In this patch, we also replaced the exit(1) in the fault injection by DBUG_ABORT()
to make it compliant with the code guidelines.
rpl_slave_skip fails randomly on PB2. This patch fixes the failure by
setting explicit wait for SQL thread to stop, instead of the
wait_for_slave_to_stop mysqltest command, after a start until command
is executed.
NOTE: Backporting the patch to next-mr.
The fix proposed in BUG#35542 and BUG#31665 introduces a performance issue
when fsyncing the master.info, relay.info and relay-log.bin* after #th events.
Although such solution has been proposed to reduce the probability of corrupted
files due to a slave-crash, the performance penalty introduced by it has
made the approach impractical for highly intensive workloads.
In a nutshell, the option --syn-relay-log proposed in BUG#35542 and BUG#31665
simultaneously fsyncs master.info, relay-log.info and relay-log.bin* and
this is the main source of performance issues.
This patch introduces new options that give more control to the user on
what should be fsynced and how often:
1) (--sync-master-info, integer) which syncs the master.info after #th event;
2) (--sync-relay-log, integer) which syncs the relay-log.bin* after #th
events.
3) (--sync-relay-log-info, integer) which syncs the relay.info after #th
transactions.
To provide both performance and increased reliability, we recommend the following
setup:
1) --sync-master-info = 0 eventually the operating system will fsync it;
2) --sync-relay-log = 0 eventually the operating system will fsync it;
3) --sync-relay-log-info = 1 fsyncs it after every transaction;
Notice, that the previous setup does not reduce the probability of
corrupted master.info and relay-log.bin*. To overcome the issue, this patch also
introduces a recovery mechanism that right after restart throws away relay-log.bin*
retrieved from a master and updates the master.info based on the relay.info:
4) (--relay-log-recovery, boolean) which enables a recovery mechanism that
throws away relay-log.bin* after a crash.
However, it can only recover the incorrect binlog file and position in master.info,
if other informations (host, port password, etc) are corrupted or incorrect,
then this recovery mechanism will fail to work.
BUG#31665 sync_binlog should cause relay logs to be synchronized
NOTE: Backporting the patch to next-mr.
Add sync_relay_log option to server, this option works for relay log
the same as option sync_binlog for binlog. This option also synchronize
master info to disk when set to non-zero value.
Original patches from Sinisa and Mark, with some modifications
vs not null
NOTE: Backporting the patch to next-mr.
The replication was generating corrupted data, warning messages on Valgrind
and aborting on debug mode while replicating a "null" to "not null" field.
Specifically the unpack_row routine, was considering the slave's table
definition and trying to retrieve a field value, where there was nothing to be
retrieved, ignoring the fact that the value was defined as "null" by the master.
To fix the problem, we proceed as follows:
1 - If it is not STRICT sql_mode, implicit default values are used, regardless
if it is multi-row or single-row statement.
2 - However, if it is STRICT mode, then a we do what follows:
2.1 If it is a transactional engine, we do a rollback on the first NULL that is
to be set into a NOT NULL column and return an error.
2.2 If it is a non-transactional engine and it is the first row to be inserted
with multi-row, we also return the error. Otherwise, we proceed with the
execution, use implicit default values and print out warning messages.
Unfortunately, the current patch cannot mimic the behavior showed by the master
for updates on multi-tables and multi-row inserts. This happens because such
statements are unfolded in different row events. For instance, considering the
following updates and strict mode:
(master)
create table t1 (a int);
create table t2 (a int not null);
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
t1 would have (10) and t2 would have (0) as this would be handled as a
multi-row update. On the other hand, if we had the following updates:
(master)
create table t1 (a int);
create table t2 (a int);
(slave)
create table t1 (a int);
create table t2 (a int not null);
(master)
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
On the master t1 would have (10) and t2 would have (NULL). On
the slave, t1 would have (10) but the update on t1 would fail.