Currently several places use description_event->common_header_len instead of
LOG_EVENT_MINIMAL_HEADER_LEN when parsing events with "frozen" headers (such
as Start_event_v3 and its subclasses such as Format_description_log_event, as
well as Rotate_event). This causes events with extra headers (which would otherwise
be valid and those headers ignored) to be corrupted due to over-reading or skipping
into the data portion of the log events.
It is rewritten in some details patch of Jeremy Cole (See MDEV):
- The virtual function returns length to avoid IFs (and only one call of the virtual function made)
- Printing function avoids printing strings
mysql-test/suite/rpl/r/last_insert_id.result:
Test case for last_insert_id
mysql-test/suite/rpl/t/last_insert_id.cnf:
Test case for last_insert_id
mysql-test/suite/rpl/t/last_insert_id.test:
Test case for last_insert_id
sql/log_event.cc:
Added DBUG_PRINT
Set thd->first_successful_insert_id_in_prev_stmt_for_binlog when setting thd->first_successful_insert_id_in_prev_stmt.
This is required to get last_insert_id() replicated.
This is analog to how read_first_successful_insert_id_in_prev_stmt() works.
sql/rpl_utility.cc:
Added DBUG_PRINT
The ignored events are not written to the relay log, but instead a fake
Rotate event is generated to handle update of position.
Extend this for Gtid so we similarly generate a fake Gtid_list event
to update the GTID position.
Also fix an unrelated test issue that got triggered by the added test cases.
- Make query plan be re-saved after the first join execution
(saving it after JOIN::cleanup is too late because EXPLAIN output
is currently produced before that)
- Handle QPF allocation/deallocation for edge cases, like unsuccessful
BINLOG command.
- Work around the problem with UNION's direct subselects not being visible.
- Update test results ("Using temporary; Using filesort" are now always printed
last in the Extra column)
- This cset gets rid of memory leaks/crashes. Some result mismatches still remain.
In record_gtid(), too many rows were deleted from the slave position
hash - we need to always keep on to the most recent committed row,
so we have a valid slave position at all times.
See also MySQL Bug #39750 and similar ones.
Fix my_delete() on Windows, to safely remvoe files on Windows, including files that are opened by another threads in the same process, antiviruses and backup applications. If file to be deleted is also opened by another thread, the file is renamed to unique name prior to deletion - this makes it possible to create file with the same name right after deletion.
With this patch my_delete_allow_opened() becomes obsolete and is replaced with my_delete().
This patch is rework of the patch http://lists.mysql.com/commits/59327 for MySQL bug#39750.
When @@GLOBAL.gtid_strict_mode=1, then certain operations result
in error that would otherwise result in out-of-order binlog files
between servers.
GTID sequence numbers are now allocated independently per domain;
this results in less/no holes in GTID sequences, increasing the
likelyhood that diverging binlogs will be caught by the slave when
GTID strict mode is enabled.
The problem was the Gtid_list event which is logged to the binlog in
10.0 and is not understood by the 5.5 server.
This event is supposed to be replaced with a dummy event for 5.5
servers. But the very first event logged in the very first binlog
has an empty list of GTID, which makes the event too short to be
replacable with an empty event.
The fix is to pad the empty Gtid_list event to be big enough to
be replacable by a dummy event.
Change of user interface to be more logical and more in line with expectations
to work similar to old-style replication.
User can now explicitly choose in CHANGE MASTER whether binlog position is
taken into account (master_gtid_pos=current_pos) or not (master_gtid_pos=
slave_pos) when slave connects to master.
@@gtid_pos is replaced by three separate variables @@gtid_slave_pos (can
be set by user, replicated GTIDs only), @@gtid_binlog_pos (read only), and
@@gtid_current_pos (a combination of the two, most recent GTID within each
domain). mysql.rpl_slave_state is renamed to mysql.gtid_slave_pos to match.
This fixes MDEV-4474.
Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".
Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.
MDEV-4489 "Replication of big5, cp932, gbk, sjis strings makes wrong values on slave"
has been fixed.
Problem:
String constants of some Asian charsets (big5,cp932,gbk,sjis)
can have backslash '\' (0x5C) in the second byte of multi-byte characters.
Replicating of such constants using the standard '\'-escaping is dangerous.
Therefore, constants of these charsets are replicated using hex notation:
INSERT INTO t1 (a) VALUES (0x815C);
However, 0xHHHH constants do not work well in some cases,
because they can behave as strings and as numbers, depending on context
(for example, depending on the data type of the column in an INSERT statement).
This SQL script was not replicated correctly with statement-based replication:
SET NAMES gbk;
PREPARE STMT FROM 'INSERT INTO t1 (a) VALUES (?)';
SET @a = '1';
EXECUTE STMT USING @a;
The INSERT statement was replicated as:
INSERT INTO t1 (a) VALUES (0x31);
'1' was correctly converted to the number 1 on master.
But the 0x31 constant was treated as number 49 on slave.
Fix:
1. Binary log now uses X'HHHH' instead of 0xHHHH constants.
2. The X'HHHH' constants now work always as strings, in all contexts.
This is the SQL standard compliant behaviour.
After the fix, the above statement is replicated as:
INSERT INTO t1 (a) VALUES (X'31');
X'31' is treated as string '1' on slave, and is correctly converted to 1.
modified:
@ mysql-test/r/ctype_cp932_binlog_stm.result
@ mysql-test/r/select.result
@ mysql-test/r/select_jcl6.result
@ mysql-test/r/select_pkeycache.result
@ mysql-test/r/user_var-binlog.result
@ mysql-test/r/varbinary.result
@ mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result
@ mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
@ mysql-test/suite/rpl/r/rpl_charset_sjis.result
@ mysql-test/suite/rpl/r/rpl_mdev382.result
@ mysql-test/suite/rpl/t/rpl_charset_sjis.test
@ mysql-test/t/ctype_cp932_binlog_stm.test
@ mysql-test/t/select.test
@ mysql-test/t/varbinary.test
Adding and updating tests
@ sql/item.cc
@ sql/item.h
@ sql/sql_yacc.yy
@ sql/sql_lex.cc
Splitting the implementations of X'HH' and 0xHH constants into two
separate classes. Fixing the parser to distinguish the two syntaxes.
@ sql/log_event.cc
Using X'HH' instead of 0xHH for binary logging for string constants
of the "dangerous" charsets.
@ sql/sql_string.h
Adding a helped method String::append_hex().
- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name
Other bug fixes:
- Enlarge buffer for connection names with 'special characters' one can't store in filenames
Optimization:
- Don't do 'lower case' of connection names. We can use cmp_connection_name, where we already have the connection name in lower case.
mysql-test/suite/multi_source/load_data.result:
Test case for MDEV-4352
mysql-test/suite/multi_source/load_data.test:
Test case for MDEV-4352
sql/log_event.cc:
Fixed: MDEV-4352
- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name
The fix was to add the connection name (if one exists) to all slave temporary files used by LOAD DATA
sql/rpl_mi.cc:
Enlarge buffer for connection names with 'special characters' one can't store in filenames
Use mi->cmp_connection_name for connection file names.
sql/rpl_rli.cc:
Use mi->cmp_connection_name for connection file names.
sql/slave.cc:
Removed not needed empty line
sql/sql_const.h:
Added MAX_FILENAME_MBWIDTH to be able to calculate buffer length for connection_names stored in file names
sql/sql_repl.cc:
Use mi->cmp_connection_name for connection file names.
When the slave connects, the master skips binlog event groups
until it reaches the position requested by the slave. To
identify event groups, it needs to detect COMMIT events. But
this detection did not correctly handle binlog checksums, so
could incorrectly skip extra groups due to not detecting the
end of an event group.
Server shutdown timeout of 10 seconds in test cases is too little for heavily
loaded test servers.
Fix innodb_bug12902967 to not fail with wrong error log output if we have
warnings about too few AIO handles for InnoDB.
Fix typo which could lead to unnecessarily replacing GTID event with dummy
event.
Users can set different repplication filter rules for each replication connection, in my.cnf or command line.
But the rules set online will not record in master.info, it means if users restart MySQL, these rules will lose.
So if users wantn't their replication filter rules lose, they should write the rules in my.cnf.
Users can set rules by 2 ways:
1. Online SET command, "SET connection_name.replication_filter_settings = rules;".
2. In my.cnf, "connection_name.replication_filter_settings = rules".
If no connection_name in my.cnf, this rule will apply for ALL replication connection.
If no connetion_name in SET statement, this rull will apply for default_connection_name.
Merge of 10.0-mdev26 feature tree into 10.0-base.
Global transaction ID is prepended to each event group in the binlog.
Slave connect can request to start from GTID position instead of specifying
file name/offset of master binlog. This facilitates easy switch to a new
master.
Slave GTID state is stored in a table mysql.rpl_slave_state, which can be
InnoDB to get crash-safe slave state.
GTID includes a replication domain ID, allowing to keep track of distinct
positions for each of multiple masters.
* replace pointer acrobatics with a struct
* make sorting explicit: MY_DONT_SORT -> MY_WANT_SORT
(if you want something to be done - say it. fixes all places where
my_dir() was used without thinking)
* typo s/number_off_files/number_of_files/
* directory_file_name() doesn't need to be extern
* remove #ifdef __BORLANDC__
* ignore '.' and '..' entries
NO ERRORS REPORTED
Problem:
=======
Errors from my_b_fill are ignored. MYSQL_BIN_LOG::write_cache
code assumes that 0 returned from my_b_fill always means
end-of-cache, but that is incorrect. It can result in error
and the error is ignored. Other callers of my_b_fill don't
check for error: my_b_copy_to_file, maybe my_b_gets.
Fix:
===
An error handler is already present to check the "cache"
error that is reported during "MYSQL_BIN_LOG::write_cache"
call. Hence error handlers are added for "my_b_copy_to_file"
and "my_b_gets".
During my_b_fill() function call, when the cache read fails
info->error= -1 is set. Hence a check for "info->error"
is added for the above to callers upon their return.
mysys/mf_iocache2.c:
Added a check for "cache->error" and simulation of cache read failure
mysys/my_read.c:
Simulation of read failure
sql/log_event.cc:
Added debug simulation
sql/sql_repl.cc:
Added a check for cache error
At logging a first Query referring a user var, the slave missed to log the user var.
It appears that at execution of a Uservar event the slaver applier
thought of the variable as already logged.
The reason of misjudgement is in coincidence of query id:s: of one that the thread
holds at Uservar execution and another one that the thread sees at the Query applying.
While the two are naturally different in the regular execution branch (as two computational
events are separated as individual events), in the deferred applying case the User var execution
effectively belongs to its Query processing.
Fixed with storing the Uservar parsing time (where desicion to defer is taken) query id
to temporarily substitute with it the actual query id at the Uservar execution time
(along with its query).
Such manipulation mimics behaviour of the regular applying branch.
sql/log_event.cc:
Storing the Uservar parsing time query id into a new member of the event
to to temporarily substitute
with it the actual thread id at the Uservar execution time.
sql/log_event.h:
Storage for keeping query-id in User-var intance is added.
Adjust full test suite to work with GTID.
Huge patch, mainly due to having to update .result file for all SHOW BINLOG
EVENTS and mysqlbinlog outputs, where the new GTID events pop up.
Everything was painstakingly checked to be still correct and valid .result
file updates.
The output of mysqlbinlog (with "-v --base64-output=DECODE-ROWS" flags) can not always be read or parsed correctly
when string columns contain single-quotes or backslash characters.
The fix for this bug is to escape single-quote and backslash characters on output, so that the result is both more
readable and more easily parse-able.
Note that this is just for comments, so it doesn't affect the replication.
sql/log_event.cc:
Escape \ and ' properly for mysqlbin user comments.
Improvements to record_gtid():
- Check for correct table definition of mysql.rpl_slave_state
- Use autocommit, to save one call to ha_commit_trans()
- Slightly more efficient way to set table->write_set
- Use ha_index_read_map() to locate rows to support any storage engine.
Fix that CHANGE MASTER ... MASTER_GTID_POS="" works to start from the very
beginning of the binary log (with test case).
Fix that not finding the requested GTID position in master binlog results in
fatal error, not endless connect retry.
