~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
DISCONNECT CON1 AND CON2
Problem:
The test suite/binlog/t/binlog_killed.test makes the connections
con1 and con2 but forgets to disconnect them + wait till that
operation is finished at test end.
This mistake has the potential to harm subsequent tests in
case these tests depend on the content of the processlist.
Solution:
Added disconnect + wait_until_disconnected.inc
within the test cleanup.
WRITTEN WHILE ROWS REMAINS
Problem:
========
When truncate table fails while using transactional based
engines even though the operation errors out we still
continue and log it to binlog. Because of this master has
data but the truncate will be written to binary log which
will cause inconsistency.
Analysis:
========
Truncate table can happen either through drop and create of
table or by deleting rows. In the second case the existing
code is written in such a way that even if an error occurs
the truncate statement will always be binlogged. Which is not
correct.
Binlogging of TRUNCATE TABLE statement should check whether
truncate is executed "transactionally or not". If the table
is transaction based we log the TRUNCATE TABLE only on
successful completion.
If table is non transactional there are possibilities that on
error we could have partial changes done hence in such cases
we do log in spite of errors as some of the lines might have
been removed, so the statement has to be sent to slave.
Fix:
===
Using table handler whether truncate table is being executed
in transaction based mode or not is identified and statement
is binlogged accordingly.
mysql-test/suite/binlog/r/binlog_truncate_kill.result:
Added test case to test the fix for Bug#17942050.
mysql-test/suite/binlog/t/binlog_truncate_kill.test:
Added test case to test the fix for Bug#17942050.
sql/sql_truncate.cc:
Check if truncation is successful or not and retun appropriate
return values so that binlogging can be done based on that.
sql/sql_truncate.h:
Added a new enum.
IN TIME RECOVERY FAILURE ON SLAVES
Problem:
DROP TEMP TABLE IF EXISTS commands can cause point
in time recovery (re-applying binlog) failures.
Analyses:
In RBR, 'DROP TEMPORARY TABLE' commands are
always binlogged by adding 'IF EXISTS' clauses.
Also, the slave SQL thread will not check replicate.* filter
rules for "DROP TEMPORARY TABLE IF EXISTS" queries.
If log-slave-updates is enabled on slave, these queries
will be binlogged in the format of "USE `db`;
DROP TEMPORARY TABLE IF EXISTS `t1`;" irrespective
of filtering rules and irrespective of the `db` existence.
When users try to recover slave from it's own binlog,
use `db` command might fail if `db` is not present on slave.
Fix:
At the time of writing the 'DROP TEMPORARY TABLE
IF EXISTS' query into the binlog, 'use `db`' will not be
present and the table name in the query will be a fully
qualified table name.
Eg:
'USE `db`; DROP TEMPORARY TABLE IF EXISTS `t1`;'
will be logged as
'DROP TEMPORARY TABLE IF EXISTS `db`.`t1`;'.
BY BINLOG_KILLED_SIMULATE.TEST
'mysqbinlog' tool creates a temporary file while
preparing LOAD DATA QUERY. These files needs to be deleted
at the end of the test script otherwise these files are
left out in the daily-run machines, causing
"no space on device issues"
Fix:
Delete them at the end of these test scripts
1) execute mysqlbinlog with --local-load option to
create these files in a specified tmpdir
2) delete the tmpdir at the end of the test script
MDEV-4489 "Replication of big5, cp932, gbk, sjis strings makes wrong values on slave"
has been fixed.
Problem:
String constants of some Asian charsets (big5,cp932,gbk,sjis)
can have backslash '\' (0x5C) in the second byte of multi-byte characters.
Replicating of such constants using the standard '\'-escaping is dangerous.
Therefore, constants of these charsets are replicated using hex notation:
INSERT INTO t1 (a) VALUES (0x815C);
However, 0xHHHH constants do not work well in some cases,
because they can behave as strings and as numbers, depending on context
(for example, depending on the data type of the column in an INSERT statement).
This SQL script was not replicated correctly with statement-based replication:
SET NAMES gbk;
PREPARE STMT FROM 'INSERT INTO t1 (a) VALUES (?)';
SET @a = '1';
EXECUTE STMT USING @a;
The INSERT statement was replicated as:
INSERT INTO t1 (a) VALUES (0x31);
'1' was correctly converted to the number 1 on master.
But the 0x31 constant was treated as number 49 on slave.
Fix:
1. Binary log now uses X'HHHH' instead of 0xHHHH constants.
2. The X'HHHH' constants now work always as strings, in all contexts.
This is the SQL standard compliant behaviour.
After the fix, the above statement is replicated as:
INSERT INTO t1 (a) VALUES (X'31');
X'31' is treated as string '1' on slave, and is correctly converted to 1.
modified:
@ mysql-test/r/ctype_cp932_binlog_stm.result
@ mysql-test/r/select.result
@ mysql-test/r/select_jcl6.result
@ mysql-test/r/select_pkeycache.result
@ mysql-test/r/user_var-binlog.result
@ mysql-test/r/varbinary.result
@ mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result
@ mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
@ mysql-test/suite/rpl/r/rpl_charset_sjis.result
@ mysql-test/suite/rpl/r/rpl_mdev382.result
@ mysql-test/suite/rpl/t/rpl_charset_sjis.test
@ mysql-test/t/ctype_cp932_binlog_stm.test
@ mysql-test/t/select.test
@ mysql-test/t/varbinary.test
Adding and updating tests
@ sql/item.cc
@ sql/item.h
@ sql/sql_yacc.yy
@ sql/sql_lex.cc
Splitting the implementations of X'HH' and 0xHH constants into two
separate classes. Fixing the parser to distinguish the two syntaxes.
@ sql/log_event.cc
Using X'HH' instead of 0xHH for binary logging for string constants
of the "dangerous" charsets.
@ sql/sql_string.h
Adding a helped method String::append_hex().
PROPERLY QUOTED IN BINLOG FILE
Problem: In load data file query, User variables are allowed
inside "Into_list" and "Set_list". These user variables used
inside these two lists are not properly guarded with backticks
while server is writting into binlog. Hence user variable names
like a` cannot be used in this context.
Fix: Properly quote these variables while
writting into binlog
mysql-test/r/func_compress.result:
changing result file
mysql-test/r/variables.result:
changing result file
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
changing result file
sql/item_func.cc:
Quote the user variable items
The test, binlog.binlog_spurious_ddl_errors was failing on pb2 at the statement
"UNINSTALL PLUGIN example;" with this warning:
"Warning 1620 Plugin is busy and will be uninstalled on shutdown "
Fix
Spurious warnings occur in the test since we do not empty the Query cache,
used by the example plugin at the time of creating tables using the plugin.
Hence, the query chache is flushed before uninstalling the plugin.
Also, as part of running the test across platforms, the plugin installation
script is changed.
When a binlog is replayed into a server, e.g.:
$ mysqlbinlog binlog.000001 | mysql
it sets a pseudo slave mode on the client connection in order to server
be able to read binlog events, there is, a format description event is
needed to correctly read following events.
Also this pseudo slave mode applies to the current connection
replication rules that are needed to correctly apply binlog events.
If a binlog dump is sourced on a connection, this pseudo slave mode will
remains after it, what will apply unexpected rules from customer
perspective to following commands.
Added a new SET statement to binlog dump that will unset pseudo slave
mode at the end of dump file.
main.mysqlbinlog_row_innodb are skipped by mtr
=== Problem ===
The following tests are wrongly placed in main suite and as a
result these are not run with proper binlog format combinations.
Some are always skipped by mtr.
1) mysqlbinlog_row_myisam
2) mysqlbinlog_row_innodb
3) mysqlbinlog_row.test
4) mysqlbinlog_row_trans.test
5) mysqlbinlog-cp932
6) mysqlbinlog2
7) mysqlbinlog_base64
=== Background ===
mtr runs the tests placed in main suite with binlog format=stmt.
Those that need to be tested against binlog format=row or mixed
or more than one binlog format and require only one mysql server
are placed in binlog suite. mtr runs tests in binlog suite with
all three binlog formats(stmt,row and mixed).
=== Fix ===
1) Moved the test listed in problem section above to binlog suite.
2) Added prefix "binlog_" to the name of each test case moved.
Renamed the coresponding result files and option files accordingly.
mysql-test/extra/binlog_tests/mysqlbinlog_row_engine.inc:
include file for mysqlbinlog_row_myisam.test and
mysqlbinlog_row_myisam.test which are being moved to
binlog suite.
mysql-test/suite/binlog/r/binlog_mysqlbinlog-cp932.result:
result file for mysqlbinlog-cp932.test which is being moved to
binlog suite.
mysql-test/suite/binlog/r/binlog_mysqlbinlog2.result:
result file for mysqlbinlog2.test which is being moved to
binlog suite.
mysql-test/suite/binlog/r/binlog_mysqlbinlog_base64.result:
result file for mysqlbinlog_base64.test which is being moved to
binlog suite.
mysql-test/suite/binlog/r/binlog_mysqlbinlog_row.result:
result file for mysqlbinlog_row.test which is being moved to
binlog suite.
mysql-test/suite/binlog/r/binlog_mysqlbinlog_row_innodb.result:
result file for mysqlbinlog_row_innodb.test which is being moved to
binlog suite.
mysql-test/suite/binlog/r/binlog_mysqlbinlog_row_myisam.result:
result file for mysqlbinlog_row_myisam.test which is being moved to
binlog suite.
mysql-test/suite/binlog/r/binlog_mysqlbinlog_row_trans.result:
result file for mysqlbinlog_row_trans.test which is being moved to
binlog suite.
mysql-test/suite/binlog/t/binlog_mysqlbinlog-cp932-master.opt:
option file for mysqlbinlog-cp932.test which is being moved to
binlog suite.
mysql-test/suite/binlog/t/binlog_mysqlbinlog-cp932.test:
the test requires binlog format=stmt or mixed. Since, it was placed in
main suite earlier, it was only run with binlog format=stmt, and hence
this test was never run with binlog format=mixed.
mysql-test/suite/binlog/t/binlog_mysqlbinlog2.test:
the test requires binlog format=stmt or mixed. Since, it was placed in
main suite earlier, it was only run with binlog format=stmt, and hence
this test was never run with binlog format=mixed.
mysql-test/suite/binlog/t/binlog_mysqlbinlog_base64.test:
the test requires binlog format=row. Since, it was placed in main
suite earlier, it was only run with binlog format=stmt, and hence
this test was always skipped by mtr.
mysql-test/suite/binlog/t/binlog_mysqlbinlog_row.test:
the test requires binlog format=row. Since, it was placed in main
suite earlier, it was only run with binlog format=stmt, and hence
this test was always skipped by mtr.
mysql-test/suite/binlog/t/binlog_mysqlbinlog_row_innodb.test:
the test requires binlog format=row. Since, it was placed in main
suite earlier, it was only run with binlog format=stmt, and hence
this test was always skipped by mtr.
mysql-test/suite/binlog/t/binlog_mysqlbinlog_row_myisam.test:
the test requires binlog format=row. Since, it was placed in main
suite earlier, it was only run with binlog format=stmt, and hence
this test was always skipped by mtr.
mysql-test/suite/binlog/t/binlog_mysqlbinlog_row_trans.test:
the test requires binlog format=row. Since, it was placed in main
suite earlier, it was only run with binlog format=stmt, and hence
this test was always skipped by mtr.
QUOTING IN REPLICATION
Problem: Misquoting or unquoted identifiers may lead to
incorrect statements to be logged to the binary log.
Fix: we use specialized functions to append quoted identifiers in
the statements generated by the server.
create table t1 (a smallint primary key auto_increment);
insert into t1 values(32767);
insert into t1 values(NULL);
ERROR 1062 (23000): Duplicate entry '32767' for key 'PRIMARY
Now on always gets error HA_ERR_AUTOINC_RANGE=167 "Out of range value for column", independent of
store engine, SQL Mode or number of inserted rows. This is an unique error that is easier to test for in replication.
Another bug fix is that we now get an error when trying to insert a too big auto-generated value, even in non-strict mode.
Before one get insted the max column value inserted.
This patch also fixes some issues with inserting negative numbers in an auto-increment column.
Fixed the ER_DUP_ENTRY and HA_ERR_AUTOINC_ERANGE are compared the same between master and slave.
This ensures that replication works between an old server to a new slave for auto-increment overflow errors.
Added SQLSTATE errors for handler errors
Smaller bug fixes:
* Added warnings for duplicate key errors when using INSERT IGNORE
* Fixed bug when using --skip-log-bin followed by --log-bin, which did set log-bin to "0"
* Allow one to see how cmake is called by using --just-print --just-configure
BUILD/FINISH.sh:
--just-print --just-configure now shows how cmake would be invoked. Good for understanding parameters to cmake.
cmake/configure.pl:
--just-print --just-configure now shows how cmake would be invoked. Good for understanding parameters to cmake.
include/CMakeLists.txt:
Added handler_state.h
include/handler_state.h:
SQLSTATE for handler error messages.
Required for HA_ERR_AUTOINC_ERANGE, but solves also some other cases.
mysql-test/extra/binlog_tests/binlog.test:
Fixed old wrong behaviour
Added more tests
mysql-test/extra/binlog_tests/binlog_insert_delayed.test:
Reset binary log to only print what's necessary in show_binlog_events
mysql-test/extra/rpl_tests/rpl_auto_increment.test:
Update to new error codes
mysql-test/extra/rpl_tests/rpl_insert_delayed.test:
Ignore warnings as this depends on how the test is run
mysql-test/include/strict_autoinc.inc:
On now gets an error on overflow
mysql-test/r/auto_increment.result:
Update results after fixing error message
mysql-test/r/auto_increment_ranges_innodb.result:
Test new behaviour
mysql-test/r/auto_increment_ranges_myisam.result:
Test new behaviour
mysql-test/r/commit_1innodb.result:
Added warnings for duplicate key error
mysql-test/r/create.result:
Added warnings for duplicate key error
mysql-test/r/insert.result:
Added warnings for duplicate key error
mysql-test/r/insert_select.result:
Added warnings for duplicate key error
mysql-test/r/insert_update.result:
Added warnings for duplicate key error
mysql-test/r/mix2_myisam.result:
Added warnings for duplicate key error
mysql-test/r/myisam_mrr.result:
Added warnings for duplicate key error
mysql-test/r/null_key.result:
Added warnings for duplicate key error
mysql-test/r/replace.result:
Update to new error codes
mysql-test/r/strict_autoinc_1myisam.result:
Update to new error codes
mysql-test/r/strict_autoinc_2innodb.result:
Update to new error codes
mysql-test/r/strict_autoinc_3heap.result:
Update to new error codes
mysql-test/r/trigger.result:
Added warnings for duplicate key error
mysql-test/r/xtradb_mrr.result:
Added warnings for duplicate key error
mysql-test/suite/binlog/r/binlog_innodb_row.result:
Updated result
mysql-test/suite/binlog/r/binlog_row_binlog.result:
Out of range data for auto-increment is not inserted anymore
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
Updated result
mysql-test/suite/binlog/r/binlog_stm_binlog.result:
Out of range data for auto-increment is not inserted anymore
mysql-test/suite/binlog/r/binlog_unsafe.result:
Updated result
mysql-test/suite/innodb/r/innodb-autoinc.result:
Update to new error codes
mysql-test/suite/innodb/r/innodb-lock.result:
Updated results
mysql-test/suite/innodb/r/innodb.result:
Updated results
mysql-test/suite/innodb/r/innodb_bug56947.result:
Updated results
mysql-test/suite/innodb/r/innodb_mysql.result:
Updated results
mysql-test/suite/innodb/t/innodb-autoinc.test:
Update to new error codes
mysql-test/suite/maria/maria3.result:
Updated result
mysql-test/suite/maria/mrr.result:
Updated result
mysql-test/suite/optimizer_unfixed_bugs/r/bug43617.result:
Updated result
mysql-test/suite/rpl/r/rpl_auto_increment.result:
Update to new error codes
mysql-test/suite/rpl/r/rpl_insert_delayed,stmt.rdiff:
Updated results
mysql-test/suite/rpl/r/rpl_loaddatalocal.result:
Updated results
mysql-test/t/auto_increment.test:
Update to new error codes
mysql-test/t/auto_increment_ranges.inc:
Test new behaviour
mysql-test/t/auto_increment_ranges_innodb.test:
Test new behaviour
mysql-test/t/auto_increment_ranges_myisam.test:
Test new behaviour
mysql-test/t/replace.test:
Update to new error codes
mysys/my_getopt.c:
Fixed bug when using --skip-log-bin followed by --log-bin, which did set log-bin to "0"
sql/handler.cc:
Ignore negative values for signed auto-increment columns
Always give an error if we get an overflow for an auto-increment-column (instead of inserting the max value)
Ensure that the row number is correct for the out-of-range-value error message.
******
Fixed wrong printing of column namn for "Out of range value" errors
Fixed that INSERT_ID is correctly replicated also for out-of-range autoincrement values
Fixed that print_keydup_error() can also be used to generate warnings
******
Return HA_ERR_AUTOINC_ERANGE (167) instead of ER_WARN_DATA_OUT_OF_RANGE for auto-increment overflow
sql/handler.h:
Allow INSERT IGNORE to continue also after out-of-range inserts.
Fixed that print_keydup_error() can also be used to generate warnings
sql/log_event.cc:
Added DBUG_PRINT
Fixed the ER_AUTOINC_READ_FAILED, ER_DUP_ENTRY and HA_ERR_AUTOINC_ERANGE are compared the same between master and slave.
This ensures that replication works between an old server to a new slave for auto-increment overflow errors.
sql/sql_insert.cc:
Add warnings for duplicate key errors when using INSERT IGNORE
sql/sql_state.c:
Added handler errors
sql/sql_table.cc:
Update call to print_keydup_error()
storage/innobase/handler/ha_innodb.cc:
Fixed increment handling of auto-increment columns to be consistent with rest of MariaDB.
storage/xtradb/handler/ha_innodb.cc:
Fixed increment handling of auto-increment columns to be consistent with rest of MariaDB.
two tests still fail:
main.innodb_icp and main.range_vs_index_merge_innodb
call records_in_range() with both range ends being open
(which triggers an assert)
mysql_rm_table_no_locks() function was modified.
When we construct log record for the DROP TABLE, now we
look if there's a comment before the first table name and
add it to the record if so.
per-file comments:
sql/sql_table.cc
MDEV-340 Save replication comments for DROP TABLE.
comment_length() function implemented to find comments in the query,
call it in mysql_rm_table_no_locks() and use the result to form log record.
mysql-test/suite/binlog/r/binlog_drop_if_exists.result
MDEV-340 Save replication comments for DROP TABLE.
test result updated.
mysql-test/suite/binlog/t/binlog_drop_if_exists.test
MDEV-340 Save replication comments for DROP TABLE.
test case added.
the new file is fully synced to disk and binlog index. This fixes a window
where a crash would leave next server restart unable to detect that a crash
occured, causing recovery to fail.
