to wrong result
When using MIXED mode and issuing 'CREATE TEMPORARY TABLE t_tmp',
the statement is logged if the current binlogging mode is
STATEMENT. This causes the slave to replay the instruction and
create the temporary table as well. If there is no switch to ROW
mode, and later on a 'DROP TEMPORARY TABLE t_tmp' is issued, then
this statement will also be logged and the slave will
remove/close the temporary table.
However, if there is a switch to ROW mode between the CREATE and
DROP TEMPORARY table, the DROP statement will not be logged,
leaving the slave with a dangling temporary table.
This patch addresses this, by always logging a DROP TEMPORARY
TABLE IF EXISTS when in mixed mode and a drop statement is issued
for temporary table(s).
mysql-test/suite/rpl/r/rpl_temp_table_mix_row.result:
Updated result file.
mysql-test/suite/rpl/t/rpl_temp_table_mix_row.test:
Added test case.
sql/sql_table.cc:
When dropping table(s) in mixed mode and current statement
logging is ROW, builds an extra DROP TEMPORARY TABLE IF
EXISTS for temporary tables that are being dropped. Later,
it logs the extra drop statement.
binlog
The fix for BUG 43929 introduced a regression issue. In a nutshell, when a
statement that changes a non-transactional table fails, it is written to the
binary log with the error code appended. Unfortunately, after BUG 43929, this
failure was flushing the transactional chace causing mismatch between execution
and logging histories. To fix this issue, we avoid flushing the transactional
cache when a commit or rollback is not issued.
The "get_master_version_and_clock(...)" function in sql/slave.cc ignores
error and passes directly when queries fail, or queries succeed
but the result retrieved is empty.
The "get_master_version_and_clock(...)" function should try to reconnect master
if queries fail because of transient network problems, and fail otherwise.
The I/O thread should print a warning if the some system variables do not
exist on master (very old master)
mysql-test/extra/rpl_tests/rpl_get_master_version_and_clock.test:
Added test file for bug #45214
mysql-test/suite/rpl/r/rpl_get_master_version_and_clock.result:
Added test result for bug #45214
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test:
Added test file for bug #45214
sql/slave.cc:
The 'is_network_error()' function is added for checking if the error is caused by network.
Added a new return value (2) to 'get_master_version_and_clock()' function result set
to indicate transient network errors when queries fail, and the caller should
try to reconnect in this case.
"create as select" (innodb table)
Problem: code constructing "CREATE TABLE..." statement
doesn't take into account that current database is not set
in some cases. That may lead to a server crash.
Fix: check if current database is set.
mysql-test/extra/binlog_tests/binlog.test:
Fix for bug#45998: database crashes when running
"create as select" (innodb table)
- test case.
mysql-test/suite/binlog/r/binlog_row_binlog.result:
Fix for bug#45998: database crashes when running
"create as select" (innodb table)
- test result.
sql/sql_show.cc:
Fix for bug#45998: database crashes when running
"create as select" (innodb table)
- added check if there's current database set.
There is an inconsistency with DROP DATABASE|TABLE|EVENT IF EXISTS and
CREATE DATABASE|TABLE|EVENT IF NOT EXISTS. DROP IF EXISTS statements are
binlogged even if either the DB, TABLE or EVENT does not exist. In
contrast, Only the CREATE EVENT IF NOT EXISTS is binlogged when the EVENT
exists.
This patch fixes the following cases for all the replication formats:
CREATE DATABASE IF NOT EXISTS.
CREATE TABLE IF NOT EXISTS,
CREATE TABLE IF NOT EXISTS ... LIKE,
CREAET TABLE IF NOT EXISTS ... SELECT.
sql/sql_insert.cc:
Part of the code was moved from the create_table_from_items to select_create::prepare.
When replication is row based, CREATE TABLE IF NOT EXISTS.. SELECT is binlogged if the table exists.
the auto_increment value
This is an alternative patch that instead of allowing RECREATE TABLE
on TRUNCATE TABLE it implements reset_auto_increment that is called
after delete_all_rows.
Note: this bug was fixed by Mattias Jonsson:
Pusing this patch: http://lists.mysql.com/commits/70370
mysql-test/suite/parts/r/partition_auto_increment_memory.result:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
mysql-test/suite/parts/r/partition_auto_increment_myisam.result:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
sql/ha_partition.cc:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate.
storage/heap/ha_heap.cc:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate
storage/heap/ha_heap.h:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate
storage/myisam/ha_myisam.cc:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate.
storage/myisam/ha_myisam.h:
Bug#35111: Truncate a MyISAM partitioned table does not reset
the auto_increment value
Added reset_auto_increment, to be used after delete_all_rows
to simulate truncate.
With ibmdb2i_create_index_option set to 1, creating an IBMDB2I table
with a primary key should produce an additional index that uses EBCDIC
hexadecimal sorting. However, this does not work. Adding indexes that
are not primary keys does work. The ibmdb2i_create_index_option should
be honoured when creating a table with a primary key.
This patch adds code to the create() function to check for the value
of the ibmdb2i_create_index_option variable and, when appropriate, to
generate a *HEX-based shadow index in DB2 for the primary key. Previously
this behavior was limited to secondary indexes.
Additionally, this patch restricts the creation of shadow indexes to
cases in which a non-*HEX sort sequence is used, as the documentation
for ibmdb2i_create_index_option describes. Previously, the shadow index
would in some cases be created even when the MySQL-specific index used
*HEX sorting, leading to redundant indexes.
Finally, the code used to generate the list of fields for indexes
and the code used to generate the SQL statement for the shadow
indexes has been refactored into individual functions.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_45983.result:
Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_45983.test:
Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
Add tests to verify that the ibmdb2i_create_index_option is being honoured
when creating a table with a primary key.
storage/ibmdb2i/ha_ibmdb2i.cc:
Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
- Add code to the create() function to check for the value of the
ibmdb2i_create_index_option variable and, when appropriate, to
generate a *HEX-based shadow index in DB2 for the primary key.
- Restrict the creation of shadow indexes to cases in which a
non-*HEX sort sequence is used.
- Refractor code used to generate the list of fields for indexes
and the code used to generate the SQL statement for the shadow
indexes into individual functions.
storage/ibmdb2i/ha_ibmdb2i.h:
Bug#45983 ibmdb2i_create_index_option=1 not working for primary key
Add function prototypes for the functions that.
- Generate the list of fields for indexes
- Generate the SQL statement for the shadow
indexes
Had attempted to disable this test on Windows only, but the nature of this bug
does not allow for this. The master.opt file is processed before anything in
in the actual test. As a result, we must use disabled.def files to ensure
these tests are skipped on the problematic platforms.
Removed Windows-only code and updated the proper disabled.def files accordingly.
Some collations were causing IBMDB2I to report
inaccurate key range estimations to the optimizer
for LIKE clauses that select substrings. This can
be seen by running EXPLAIN. This problem primarily
affects multi-byte and unicode character sets.
This patch involves substantial changes to several
modules. There are a number of problems with the
character set and collation handling. These problems
have been or are being fixed, and a comprehensive
test has been included which should provide much
better coverage than there was before. This test
is enabled only for IBM i 6.1, because that version
has support for the greatest number of collations.
mysql-test/suite/ibmdb2i/r/ibmdb2i_collations.result:
Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
result file for test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_collations.test:
Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
Tests for character sets and collations. This test
is enabled only for IBM i 6.1, because that version
has support for the greatest number of collations.
storage/ibmdb2i/db2i_conversion.cc:
Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
- Added support in convertFieldChars to enable records_in_range
to determine how many substitute characters were inserted and
to suppress conversion warnings.
- Fixed bug which was causing all multi-byte and Unicode fields
to be created as UTF16 (CCSID 1200) fields in DB2. The corrected
code will now create UCS2 fields as UCS2 (CCSID 13488), UTF8
fields (except for utf8_general_ci) as UTF8 (CCSID 1208), and
all other multi-byte or Unicode fields as UTF16. This will only
affect tables that are newly created through the IBMDB2I storage
engine. Existing IBMDB2I tables will retain the original CCSID
until recreated. The existing behavior is believed to be
functionally correct, but it may negatively impact performance
by causing unnecessary character conversion. Additionally, users
accessing IBMDB2I tables through DB2 should be aware that mixing
tables created before and after this change may require extra type
casts or other workarounds. For this reason, users who have
existing IBMDB2I tables using a Unicode collation other than
utf8_general_ci are encouraged to recreate their tables (e.g.
ALTER TABLE t1 ENGINE=IBMDB2I) in order to get the updated CCSIDs
associated with their DB2 tables.
- Improved error reporting for unsupported character sets by forcing
a check for the iconv conversion table at table creation time,
rather than at data access time.
storage/ibmdb2i/db2i_myconv.h:
Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
Fix to set errno when iconv fails.
storage/ibmdb2i/db2i_rir.cc:
Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
Significant improvements were made to the records_in_range code
that handles partial length string data in keys for optimizer plan
estimation. Previously, to obtain an estimate for a partial key
value, the implementation would perform any necessary character
conversion and then attempt to determine the unpadded length of
the partial key by searching for the minimum or maximum sort
character. While this algorithm was sufficient for most single-byte
character sets, it did not treat Unicode and multi-byte strings
correctly. Furthermore, due to an operating system limitation,
partial keys having UTF8 collations (ICU sort sequences in DB2)
could not be estimated with this method.
With this patch, the code no longer attempts to explicitly determine
the unpadded length of the key. Instead, the entire key is converted
(if necessary), including padding, and then passed to the operating
system for estimation. Depending on the source and target character
sets and collations, additional logic is required to correctly
handle cases in which MySQL uses unconvertible or differently
-weighted values to pad the key. The bulk of the patch exists
to implement this additional logic.
storage/ibmdb2i/ha_ibmdb2i.h:
Bug#45803 Inaccurate estimates for partial key values with IBMDB2I
The convertFieldChars declaration was updated to support additional
optional behaviors.
timeout
In STMT and MIXED modes, a statement that changes both non-transactional and
transactional tables must be written to the binary log whenever there are
changes to non-transactional tables. This means that the statement gets into the
binary log even when the changes to the transactional tables fail. In particular
, in the presence of a failure such statement is annotated with the error number
and wrapped in a begin/rollback. On the slave, while applying the statement, it
is expected the same failure and the rollback prevents the transactional changes
to be persisted.
Unfortunately, statements that fail due to concurrency issues (e.g. deadlocks,
timeouts) are logged in the same way causing the slave to stop as the statements
are applied sequentially by the SQL Thread. To fix this bug, we automatically
ignore concurrency failures on the slave. Specifically, the following failures
are ignored: ER_LOCK_WAIT_TIMEOUT, ER_LOCK_DEADLOCK and ER_XA_RBDEADLOCK.
Creating an IBMDB2I table with the macce character set
is successful, but any attempt to insert data into the
table was failing.
This was happening because the character set name "macce"
is not a valid iconv descriptor for IBM i PASE. This patch
adds an override to convertTextDesc to use the equivalent
valid iconv descriptor "IBM-1282" instead.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_45793.result:
Bug#45793 macce charset causes error with IBMDB2I
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_45793.test:
Bug#45793 macce charset causes error with IBMDB2I
Add a testcase for the macce charater set.
storage/ibmdb2i/db2i_charsetSupport.cc:
Bug#45793 macce charset causes error with IBMDB2I
The character set name "macce" is not a valid iconv
descriptor for IBM i PASE. Add an override to convertTextDesc
to use the equivalent valid iconv descriptor "IBM-1282"
instead.
Some collations--including cp1250_czech_cs,latin2_czech_cs,
ucs2/utf8_czech_ci, ucs2/utf8_danish_ci--are not being
sorted correctly by the IBMDB2I storage engine. This
was being caused because the sort order used by DB2 is
incompatible with the order expected by MySQL.
This patch removes support for the cp1250_czech_cs and
latin2_czech_cs collations because it has been determined
that the sort order used by DB2 is incompatible with the
order expected by MySQL. Users needing a czech collation
with IBMDB2I are encouraged to use a Unicode-based collation
instead of these single-byte collations. This patch also
modifies the DB2 sort sequence used for ucs2/utf8_czech_ci
and ucs2/utf8_danish_ci collations to better match the
sorting expected by MySQL. This will only affect indexes
or tables that are newly created through the IBMDB2I storage
engine. Existing IBMDB2I tables will retain the old sort
sequence until recreated.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_45196.result:
Bug#45196 Some collations do not sort correctly with IBMDB2I
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_45196.test:
Bug#45196 Some collations do not sort correctly with IBMDB2I
Adding tests for testing the sort order with the modified collations.
storage/ibmdb2i/db2i_collationSupport.cc:
Bug#45196 Some collations do not sort correctly with IBMDB2I
Remove the support for the cp1250_czech_cs and latin2_czech_cs
collations because it has been determined that the sort order
used by DB2 is incompatible with the order expected by MySQL.
Users needing a czech collation with IBMDB2I are encouraged to
use a Unicode-based collation instead of these single-byte
collations. This patch also modifies the DB2 sort sequence
used for ucs2/utf8_czech_ci and ucs2/utf8_danish_ci collations
to better match the sorting expected by MySQL. This will only
affect indexes or tables that are newly created through the
IBMDB2I storage engine. Existing IBMDB2I tables will retain
the old sort sequence until recreated.
format." warnings
Despite the fact that a statement would be filtered out from binlog, a
warning would still be thrown if it was issued with the LIMIT.
This patch addresses this issue by checking the filtering rules before
printing out the warning.
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning-master.opt:
Parameter to filter out database: "b42851".
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
Added a new test case.
sql/sql_class.cc:
Added filtering rules check to condition used to decide whether to
printout warning or not.
mysql-test/lib/mtr_cases.pm:
Expand test cases list with test cases to test
InnoDB plugin.
mysql-test/mysql-test-run.pl:
Added "innodb" suite to default list of suites.
mysql-test/suite/innodb/include/have_innodb_plugin.inc:
This file determines if innodb plugin is available.
mysql-test/suite/innodb/my.cnf:
Removed temporary my.cnf, which was added for testing. Not
needed anymore - options are substituted by mtr.
mysql-test/suite/innodb/t/innodb-analyze.test:
This test can only be run with InnoDB plugin.
mysql-test/suite/innodb/t/innodb-timeout.test:
This test can only be run with InnoDB plugin.
mysql-test/suite/innodb/t/innodb-use-sys-malloc-master.opt:
Use "loose" prefix with mysqld options. This makes this
test skipped properly when no innodb is available.
mysql-test/suite/innodb/t/innodb-use-sys-malloc.test:
This test can only be run with InnoDB plugin.
mysql-test/suite/innodb/t/innodb-zip.test:
This test can only be run with InnoDB plugin.
mysql-test/suite/innodb/t/innodb_bug36169.test:
This test can only be run with InnoDB plugin.
mysql-test/suite/innodb/t/innodb_bug36172.test:
This test can only be run with InnoDB plugin.
mysql-test/suite/innodb/t/innodb_information_schema.test:
This test can only be run with InnoDB plugin.
Details:
- Limit the queries to character sets and collations
which are most probably available in all build types.
But try to preserve the intention of the tests.
- Remove the variants adjusted to some build types.
Note:
1. The results of the review by Bar are included.
2. I am not able to check the correctness of this patch
on any existing build type and any MySQL version.
So it could happen that the new test fails somewhere.
Large transactions and statements may corrupt the binary log if the size of the
cache, which is set by the max_binlog_cache_size, is not enough to store the
the changes.
In a nutshell, to fix the bug, we save the position of the next character in the
cache before starting processing a statement. If there is a problem, we simply
restore the position thus removing any effect of the statement from the cache.
Unfortunately, to avoid corrupting the binary log, we may end up loosing changes
on non-transactional tables if they do not fit in the cache. In such cases, we
store an Incident_log_event in order to stop the slave and alert users that some
changes were not logged.
Precisely, for every non-transactional changes that do not fit into the cache,
we do the following:
a) the statement is *not* logged
b) an incident event is logged after committing/rolling back the transaction,
if any. Note that if a failure happens before writing the incident event to
the binary log, the slave will not stop and the master will not have reported
any error.
c) its respective statement gives an error
For transactional changes that do not fit into the cache, we do the following:
a) the statement is *not* logged
b) its respective statement gives an error
To work properly, this patch requires two additional things. Firstly, callers to
MYSQL_BIN_LOG::write and THD::binlog_query must handle any error returned and
take the appropriate actions such as undoing the effects of a statement. We
already changed some calls in the sql_insert.cc, sql_update.cc and sql_insert.cc
modules but the remaining calls spread all over the code should be handled in
BUG#37148. Secondly, statements must be either classified as DDL or DML because
DDLs that do not get into the cache must generate an incident event since they
cannot be rolled back.
mysql-test/suite/innodb/my.cnf:
Added temporary my.cnf for initial testing.
mysql-test/suite/innodb/t/disabled.def:
Disabled innodb-index test until we know how to fix it.
An error message appears in the error log, which is likely
needs to be suppressed.
mysql-test/suite/innodb/t/innodb-analyze.test:
Cleanup variables before returning.
mysql-test/suite/innodb/t/innodb-index.test:
Include file from proper location.
mysql-test/suite/innodb/t/innodb-index_ucs2.test:
Include file from proper location.
mysql-test/suite/innodb/t/innodb_bug36169.test:
Cleanup variables before returning.
mysql-test/suite/innodb/t/innodb_bug36172.test:
Cleanup variables before returning.
the mysql test suite.
Tests removed:
1. innodb_trx_weight.test
2. innodb_bug35220.test
Include files removed:
1. have_innodb.inc
2. ctype_innodb_like.inc
3. innodb_trx_weight.inc
Also add the missing opt file for the test innodb-use-sys-malloc.test
Created a test suite 'innodb' under mysql-test/suite/innodb for the innodb plugin tests.
test suite 'innodb' has tests only which are not under any other mysql-test suites.
Total 14 testcases are added to the test suite.
Note: the patches in storage/innodb_plugin/mysql-test/patches are not applied yet
Needed for substitution in some tests.
mysql-test/suite/funcs_1/t/ndb_storedproc_06.tes:
Remove unused file.
mysql-test/suite/funcs_1/t/ndb_storedproc_08.tes:
Remove unused file.
mysql-test/suite/ndb/my.cnf:
Export the socket path.
the --big-test flag is supplied. Test is too resource intensive
under normal valgrind runs (takes more than 30min on powerful
hardware).
mysql-test/include/no_valgrind_without_big.inc:
Add MTR prerequisite file by Matthias Leich.
mysql-test/suite/funcs_1/t/myisam_views.test:
Test is too resource intensive under "Valgrind".
mysqlbinlog --database parameter was being ignored when processing
row events. As such no event filtering would take place.
This patch addresses this by deploying a call to shall_skip_database
when table_map_events are handled (as these contain also the name of
the database). All other rows events referencing the table id for the
filtered map event, will also be skipped.
client/mysqlbinlog.cc:
Added shall_skip_database call to the part of the code that handles
Table_map_log_events. It inspects the database name and decides whether
to filter the event or not. Furthermore, if table map event is filtered
next events referencing the table id in the table map event, will also
be filtered.
mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_db_filter.test:
Test case that checks if row events are actually filtered out.
sql/log_event.h:
Added a map for holding the currently ignored table map events.
Table map events are inserted when they shall be skipped and removed
once the last row event in the statement is processed.
The server was not cleaning the last IO error and error number when
resetting slave.
This patch addresses this issue by backporting into 5.1 part of the
patch in BUG 34654. A fix for this issue had already been pushed into
6.0 as part of the aforementioned bug, however the patch also included
some refactoring. The fix for 5.1 does not take into account the
refactoring part.
mysql-test/extra/rpl_tests/rpl_reset_slave.test:
Backported the test case and improved with deploying include/start_slave.inc
in relevant spots.
sql/slave.cc:
Backported part of patch from 6.0 that includes cleaning
mi->clear_error() at:
1. beginning of handle_slave_io
2. on successful connection
Also, backported the assertion added in the original patch.
sql/sql_repl.cc:
Backported the call to mi->clear_error() on reset_slave().
