timeout
In STMT and MIXED modes, a statement that changes both non-transactional and
transactional tables must be written to the binary log whenever there are
changes to non-transactional tables. This means that the statement gets into the
binary log even when the changes to the transactional tables fail. In particular
, in the presence of a failure such statement is annotated with the error number
and wrapped in a begin/rollback. On the slave, while applying the statement, it
is expected the same failure and the rollback prevents the transactional changes
to be persisted.
Unfortunately, statements that fail due to concurrency issues (e.g. deadlocks,
timeouts) are logged in the same way causing the slave to stop as the statements
are applied sequentially by the SQL Thread. To fix this bug, we automatically
ignore concurrency failures on the slave. Specifically, the following failures
are ignored: ER_LOCK_WAIT_TIMEOUT, ER_LOCK_DEADLOCK and ER_XA_RBDEADLOCK.
Creating an IBMDB2I table with the macce character set
is successful, but any attempt to insert data into the
table was failing.
This was happening because the character set name "macce"
is not a valid iconv descriptor for IBM i PASE. This patch
adds an override to convertTextDesc to use the equivalent
valid iconv descriptor "IBM-1282" instead.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_45793.result:
Bug#45793 macce charset causes error with IBMDB2I
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_45793.test:
Bug#45793 macce charset causes error with IBMDB2I
Add a testcase for the macce charater set.
storage/ibmdb2i/db2i_charsetSupport.cc:
Bug#45793 macce charset causes error with IBMDB2I
The character set name "macce" is not a valid iconv
descriptor for IBM i PASE. Add an override to convertTextDesc
to use the equivalent valid iconv descriptor "IBM-1282"
instead.
Some collations--including cp1250_czech_cs,latin2_czech_cs,
ucs2/utf8_czech_ci, ucs2/utf8_danish_ci--are not being
sorted correctly by the IBMDB2I storage engine. This
was being caused because the sort order used by DB2 is
incompatible with the order expected by MySQL.
This patch removes support for the cp1250_czech_cs and
latin2_czech_cs collations because it has been determined
that the sort order used by DB2 is incompatible with the
order expected by MySQL. Users needing a czech collation
with IBMDB2I are encouraged to use a Unicode-based collation
instead of these single-byte collations. This patch also
modifies the DB2 sort sequence used for ucs2/utf8_czech_ci
and ucs2/utf8_danish_ci collations to better match the
sorting expected by MySQL. This will only affect indexes
or tables that are newly created through the IBMDB2I storage
engine. Existing IBMDB2I tables will retain the old sort
sequence until recreated.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_45196.result:
Bug#45196 Some collations do not sort correctly with IBMDB2I
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_45196.test:
Bug#45196 Some collations do not sort correctly with IBMDB2I
Adding tests for testing the sort order with the modified collations.
storage/ibmdb2i/db2i_collationSupport.cc:
Bug#45196 Some collations do not sort correctly with IBMDB2I
Remove the support for the cp1250_czech_cs and latin2_czech_cs
collations because it has been determined that the sort order
used by DB2 is incompatible with the order expected by MySQL.
Users needing a czech collation with IBMDB2I are encouraged to
use a Unicode-based collation instead of these single-byte
collations. This patch also modifies the DB2 sort sequence
used for ucs2/utf8_czech_ci and ucs2/utf8_danish_ci collations
to better match the sorting expected by MySQL. This will only
affect indexes or tables that are newly created through the
IBMDB2I storage engine. Existing IBMDB2I tables will retain
the old sort sequence until recreated.
format." warnings
Despite the fact that a statement would be filtered out from binlog, a
warning would still be thrown if it was issued with the LIMIT.
This patch addresses this issue by checking the filtering rules before
printing out the warning.
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning-master.opt:
Parameter to filter out database: "b42851".
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
Added a new test case.
sql/sql_class.cc:
Added filtering rules check to condition used to decide whether to
printout warning or not.
Details:
- Limit the queries to character sets and collations
which are most probably available in all build types.
But try to preserve the intention of the tests.
- Remove the variants adjusted to some build types.
Note:
1. The results of the review by Bar are included.
2. I am not able to check the correctness of this patch
on any existing build type and any MySQL version.
So it could happen that the new test fails somewhere.
Large transactions and statements may corrupt the binary log if the size of the
cache, which is set by the max_binlog_cache_size, is not enough to store the
the changes.
In a nutshell, to fix the bug, we save the position of the next character in the
cache before starting processing a statement. If there is a problem, we simply
restore the position thus removing any effect of the statement from the cache.
Unfortunately, to avoid corrupting the binary log, we may end up loosing changes
on non-transactional tables if they do not fit in the cache. In such cases, we
store an Incident_log_event in order to stop the slave and alert users that some
changes were not logged.
Precisely, for every non-transactional changes that do not fit into the cache,
we do the following:
a) the statement is *not* logged
b) an incident event is logged after committing/rolling back the transaction,
if any. Note that if a failure happens before writing the incident event to
the binary log, the slave will not stop and the master will not have reported
any error.
c) its respective statement gives an error
For transactional changes that do not fit into the cache, we do the following:
a) the statement is *not* logged
b) its respective statement gives an error
To work properly, this patch requires two additional things. Firstly, callers to
MYSQL_BIN_LOG::write and THD::binlog_query must handle any error returned and
take the appropriate actions such as undoing the effects of a statement. We
already changed some calls in the sql_insert.cc, sql_update.cc and sql_insert.cc
modules but the remaining calls spread all over the code should be handled in
BUG#37148. Secondly, statements must be either classified as DDL or DML because
DDLs that do not get into the cache must generate an incident event since they
cannot be rolled back.
Needed for substitution in some tests.
mysql-test/suite/funcs_1/t/ndb_storedproc_06.tes:
Remove unused file.
mysql-test/suite/funcs_1/t/ndb_storedproc_08.tes:
Remove unused file.
mysql-test/suite/ndb/my.cnf:
Export the socket path.
the --big-test flag is supplied. Test is too resource intensive
under normal valgrind runs (takes more than 30min on powerful
hardware).
mysql-test/include/no_valgrind_without_big.inc:
Add MTR prerequisite file by Matthias Leich.
mysql-test/suite/funcs_1/t/myisam_views.test:
Test is too resource intensive under "Valgrind".
mysqlbinlog --database parameter was being ignored when processing
row events. As such no event filtering would take place.
This patch addresses this by deploying a call to shall_skip_database
when table_map_events are handled (as these contain also the name of
the database). All other rows events referencing the table id for the
filtered map event, will also be skipped.
client/mysqlbinlog.cc:
Added shall_skip_database call to the part of the code that handles
Table_map_log_events. It inspects the database name and decides whether
to filter the event or not. Furthermore, if table map event is filtered
next events referencing the table id in the table map event, will also
be filtered.
mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_db_filter.test:
Test case that checks if row events are actually filtered out.
sql/log_event.h:
Added a map for holding the currently ignored table map events.
Table map events are inserted when they shall be skipped and removed
once the last row event in the statement is processed.
The server was not cleaning the last IO error and error number when
resetting slave.
This patch addresses this issue by backporting into 5.1 part of the
patch in BUG 34654. A fix for this issue had already been pushed into
6.0 as part of the aforementioned bug, however the patch also included
some refactoring. The fix for 5.1 does not take into account the
refactoring part.
mysql-test/extra/rpl_tests/rpl_reset_slave.test:
Backported the test case and improved with deploying include/start_slave.inc
in relevant spots.
sql/slave.cc:
Backported part of patch from 6.0 that includes cleaning
mi->clear_error() at:
1. beginning of handle_slave_io
2. on successful connection
Also, backported the assertion added in the original patch.
sql/sql_repl.cc:
Backported the call to mi->clear_error() on reset_slave().
assertion .\filesort.cc, line 797
A query with the "ORDER BY @@some_system_variable" clause,
where @@some_system_variable is NULL, causes assertion
failure in the filesort procedures.
The reason of the failure is in the value of
Item_func_get_system_var::maybe_null: it was unconditionally
set to false even if the value of a variable was NULL.
mysql-test/r/variables.result:
Added test case for bug #42778.
mysql-test/suite/sys_vars/r/innodb_data_home_dir_basic.result:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/r/innodb_flush_method_basic.result:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/r/ssl_capath_basic.result:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/r/ssl_cipher_basic.result:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/t/innodb_data_home_dir_basic.test:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/t/innodb_flush_method_basic.test:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/t/ssl_capath_basic.test:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/suite/sys_vars/t/ssl_cipher_basic.test:
Updated test case for bug #42778:
system variables were NOT NULL, now they are nullable.
mysql-test/t/variables.test:
Added test case for bug #42778.
sql/item.cc:
Bug #42778: delete order by null global variable causes
assertion .\filesort.cc, line 797
The longlong_from_string_with_check function has been modified
to skip unwanted warnings: now it uses the THD::no_errors
flag to suppress warnings.
The Item_func_get_system_var::update_null_value method
sets the no_error flag.
sql/item_func.cc:
Bug #42778: delete order by null global variable causes
assertion .\filesort.cc, line 797
1. The Item_func_get_system_var::fix_length_and_dec method
has been modified to make system variables truly nullable.
2. The Item_func_get_system_var::update_null_value method
method has been overloaded with a simple wrapper (like
Item_field::update_null_value) to suppress unwanted warnings
from Item_func_get_system_var::val_int() calls on non-numeric
variable values: the Item_func_get_system_var::update_null_value
method sets and restores THD::no_errors flag for a nested
call of the longlong_from_string_with_check function.
sql/item_func.h:
Bug #42778: delete order by null global variable causes
assertion .\filesort.cc, line 797
The Item_func_get_system_var::update_null_value method
method has been overloaded.
Detail:
The results for the "normal" server testcase variants
were already adjusted to the modified information_schema
content. Therefore just do the same for the embedded
server variants.
Disabling these two tests as they are affected by this bug / causing PB2 failures
on Windows platforms. Can always disable via include/not_windows.inc if
the bug fix looks like it will take some time.
In order to better support the usage of
IBMDB2I tables from within RPG programs,
the storage engine should ensure that the
RCDFMT name is consistent and predictable
for DB2 tables.
This patch appends a "RCDFMT <name>"
clause to the CREATE TABLE statement
that is passed to DB2. <name> is
generated from the original name of
the table itself. This ensures a
consistent and deterministic mapping
from the original table.
For the sake of simplicity only
the alpha-numeric characters are
preserved when generating the new
name, and these are upper-cased;
other characters are replaced with
an underscore (_). Following DB2
system identifier rules, the name
always begins with an alpha-character
and has a maximum of ten characters.
If no usable characters are found in
the table name, the name X is used.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_44610.result:
Bug#44610 RCDFMT clause requested when creating DB2 table
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_44610.test:
Bug#44610 RCDFMT clause requested when creating DB2 table
Test case that confirms that the names that are being generated are valid.
storage/ibmdb2i/ha_ibmdb2i.cc:
Bug#44610 RCDFMT clause requested when creating DB2 table
This patch appends a "RCDFMT <name>"
clause to the CREATE TABLE statement
that is passed to DB2. <name> is
generated from the original name of
the table itself. This ensures a
consistent and deterministic mapping
from the original table.
storage/ibmdb2i/ha_ibmdb2i.h:
Bug#44610 RCDFMT clause requested when creating DB2 table
This patch appends a "RCDFMT <name>"
clause to the CREATE TABLE statement
that is passed to DB2. <name> is
generated from the original name of
the table itself. This ensures a
consistent and deterministic mapping
from the original table.
checking in a test case that will reproduce
the error on v5r4.
mysql-test/suite/ibmdb2i/include/have_i54.inc:
Bug#44232 Error msg should be improved when collation not supported.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_44232.result:
Bug#44232 Error msg should be improved when collation not supported.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_44232.test:
Bug#44232 Error msg should be improved when collation not supported.
"freeing items"
The calculation of the table map log event in the event constructor
was one byte shorter than what would be actually written. This would
lead to a mismatch between the number of bytes written and the event
end_log_pos, causing bad event alignment in the binlog (corrupted
binlog) or in the transaction cache while fixing positions
(MYSQL_BIN_LOG::write_cache). This could lead to impossible to read
binlog or even infinite loops in MYSQL_BIN_LOG::write_cache.
This patch addresses this issue by correcting the expected event
length in the Table_map_log_event constructor, when the field metadata
size exceeds 255.
sql/log_event.cc:
Added the extra byte as net_store_length imposes.
In the output from mysqlbinlog, incident log events were
represented as just a comment. Since the incident log event
represents an incident that could cause the contents of the
database to change without being logged to the binary log,
it means that if the SQL is applied to a server, it could
potentially lead to that the databases are out of sync.
In order to handle that, this patch adds the statement "RELOAD
DATABASE" to the SQL output for the incident log event. This will
require a DBA to edit the file and handle the case as apropriate
before applying the output to a server.
mysql-test/suite/binlog/t/binlog_incident-master.opt:
Options file to cause server to generate an incident log
event when executing a REPLACE.
mysql-test/suite/binlog/t/binlog_incident.test:
Test to check that the incident log event is represented
correctly in the output from mysqlbinlog.
sql/log_event.cc:
The incident log event now ouput a "RELOAD DATABASE" instead
of just a comment. RELOAD DATABASE is not an existing command
and will generate a syntax error.
Certain multi-updates gave different results on InnoDB from
to MyISAM, due to on-the-fly updates being used on the former and
the update order matters.
Fixed by turning off on-the-fly updates when update order
dependencies are present.
mysql-test/r/innodb_mysql.result:
Bug#43580: Test result.
mysql-test/suite/rpl/r/rpl_slave_skip.result:
Bug#43580: Changed test result. The InnoDB result is now what it would have been on MyISAM.
mysql-test/t/innodb_mysql.test:
Bug#43580: Test case.
sql/sql_base.cc:
Bug#43580: Added a word of caution about using tmp_set here.
sql/sql_update.cc:
Bug#43580: Fix.
Calls to TABLE::mark_columns_needed_for_update() are moved
from mysql_multi_update_prepare() and right before the decison
to do on-the-fly updates to the place where we do (or don't do)
on-the-fly updates.
1. Replace waiting of SQL thread stop by waiting of SQL error on slave and stopped
SQL thread.
2. Remove debug code because it already implemented in MTR2.
Respectively, replaced "--exec diff" by "--diff_files" which is a mysqltest command to run a
non-operating system specific diff. Removed the file rpl_000015-slave.sh as it is not
necessary in the new MTR.
Turned off autocommit at the start of this test per Innobase recommendation.
Noted significant reduction in run time for this test w/ a minor increase in other tests' run-times.
In order to define the --slave-load-tmpdir, the init_relay_log_file()
was calling fn_format(MY_PACK_FILENAME) which internally was indirectly
calling strmov_overlapp() (through pack_dirname) and the following
warning message was being printed out while running in Valgrind:
"source and destination overlap in strcpy".
We fixed the issue by removing the flag MY_PACK_FILENAME as it was not
necessary. In a nutshell, with this flag the function fn_format() tried
to replace a directory by either "~", "." or "..". However, we wanted
exactly to remove such strings.
In this patch, we also refactored the functions init_relay_log_file()
and check_temp_dir(). The former was refactored to call the fn_format()
with the flag MY_SAFE_PATH along with the MY_RETURN_REAL_PATH, in order
to avoid issues with long directories and return an absolute path,
respectively. The flag MY_SAFE_UNPACK_FILENAME was removed too as it was
responsible for removing "~", "." or ".." only from the file parameter
and we wanted to remove such strings from the directory parameter in
the fn_format(). This result is stored in an rli variable, which is then
processed by the other function in order to verify if the directory exists
and if we are able to create files in it.
mysql-test/suite/rpl/t/rpl_slave_load_tmpdir_not_exist.test:
Changed the output to make it consistent among different runs.
mysys/mf_format.c:
Replaced a return for DBUG_RETURN.
