* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
If the log_bin_trust_function_creators option is not defined, creating a stored
function requires either one of the modifiers DETERMINISTIC, NO SQL, or READS
SQL DATA. Executing a stored function should also follows the same rules if in
STATEMENT mode. However, this was not happening and a wrong error was being
printed out: ER_BINLOG_ROW_RBR_TO_SBR.
The patch makes the creation and execution compatible and prints out the correct
error ER_BINLOG_UNSAFE_ROUTINE when a stored function without one of the modifiers
above is executed in STATEMENT mode.
to wrong result
When using MIXED mode and issuing 'CREATE TEMPORARY TABLE t_tmp',
the statement is logged if the current binlogging mode is
STATEMENT. This causes the slave to replay the instruction and
create the temporary table as well. If there is no switch to ROW
mode, and later on a 'DROP TEMPORARY TABLE t_tmp' is issued, then
this statement will also be logged and the slave will
remove/close the temporary table.
However, if there is a switch to ROW mode between the CREATE and
DROP TEMPORARY table, the DROP statement will not be logged,
leaving the slave with a dangling temporary table.
This patch addresses this, by always logging a DROP TEMPORARY
TABLE IF EXISTS when in mixed mode and a drop statement is issued
for temporary table(s).
mysql-test/suite/rpl/r/rpl_temp_table_mix_row.result:
Updated result file.
mysql-test/suite/rpl/t/rpl_temp_table_mix_row.test:
Added test case.
sql/sql_table.cc:
When dropping table(s) in mixed mode and current statement
logging is ROW, builds an extra DROP TEMPORARY TABLE IF
EXISTS for temporary tables that are being dropped. Later,
it logs the extra drop statement.
binlog
The fix for BUG 43929 introduced a regression issue. In a nutshell, when a
statement that changes a non-transactional table fails, it is written to the
binary log with the error code appended. Unfortunately, after BUG 43929, this
failure was flushing the transactional chace causing mismatch between execution
and logging histories. To fix this issue, we avoid flushing the transactional
cache when a commit or rollback is not issued.
The "get_master_version_and_clock(...)" function in sql/slave.cc ignores
error and passes directly when queries fail, or queries succeed
but the result retrieved is empty.
The "get_master_version_and_clock(...)" function should try to reconnect master
if queries fail because of transient network problems, and fail otherwise.
The I/O thread should print a warning if the some system variables do not
exist on master (very old master)
mysql-test/extra/rpl_tests/rpl_get_master_version_and_clock.test:
Added test file for bug #45214
mysql-test/suite/rpl/r/rpl_get_master_version_and_clock.result:
Added test result for bug #45214
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test:
Added test file for bug #45214
sql/slave.cc:
The 'is_network_error()' function is added for checking if the error is caused by network.
Added a new return value (2) to 'get_master_version_and_clock()' function result set
to indicate transient network errors when queries fail, and the caller should
try to reconnect in this case.
Had attempted to disable this test on Windows only, but the nature of this bug
does not allow for this. The master.opt file is processed before anything in
in the actual test. As a result, we must use disabled.def files to ensure
these tests are skipped on the problematic platforms.
Removed Windows-only code and updated the proper disabled.def files accordingly.
timeout
In STMT and MIXED modes, a statement that changes both non-transactional and
transactional tables must be written to the binary log whenever there are
changes to non-transactional tables. This means that the statement gets into the
binary log even when the changes to the transactional tables fail. In particular
, in the presence of a failure such statement is annotated with the error number
and wrapped in a begin/rollback. On the slave, while applying the statement, it
is expected the same failure and the rollback prevents the transactional changes
to be persisted.
Unfortunately, statements that fail due to concurrency issues (e.g. deadlocks,
timeouts) are logged in the same way causing the slave to stop as the statements
are applied sequentially by the SQL Thread. To fix this bug, we automatically
ignore concurrency failures on the slave. Specifically, the following failures
are ignored: ER_LOCK_WAIT_TIMEOUT, ER_LOCK_DEADLOCK and ER_XA_RBDEADLOCK.
Large transactions and statements may corrupt the binary log if the size of the
cache, which is set by the max_binlog_cache_size, is not enough to store the
the changes.
In a nutshell, to fix the bug, we save the position of the next character in the
cache before starting processing a statement. If there is a problem, we simply
restore the position thus removing any effect of the statement from the cache.
Unfortunately, to avoid corrupting the binary log, we may end up loosing changes
on non-transactional tables if they do not fit in the cache. In such cases, we
store an Incident_log_event in order to stop the slave and alert users that some
changes were not logged.
Precisely, for every non-transactional changes that do not fit into the cache,
we do the following:
a) the statement is *not* logged
b) an incident event is logged after committing/rolling back the transaction,
if any. Note that if a failure happens before writing the incident event to
the binary log, the slave will not stop and the master will not have reported
any error.
c) its respective statement gives an error
For transactional changes that do not fit into the cache, we do the following:
a) the statement is *not* logged
b) its respective statement gives an error
To work properly, this patch requires two additional things. Firstly, callers to
MYSQL_BIN_LOG::write and THD::binlog_query must handle any error returned and
take the appropriate actions such as undoing the effects of a statement. We
already changed some calls in the sql_insert.cc, sql_update.cc and sql_insert.cc
modules but the remaining calls spread all over the code should be handled in
BUG#37148. Secondly, statements must be either classified as DDL or DML because
DDLs that do not get into the cache must generate an incident event since they
cannot be rolled back.
Use MY_MUTEX_INIT_FAST for pool mutex
mysql-test/mysql-test-run.pl:
Added option --staging-run
Added information about --parallell=# to help message
mysql-test/suite/federated/federated_server.test:
Slow test, don't run with --staging-run
mysql-test/suite/maria/t/maria-preload.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_optimize.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_relayrotate.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_001.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_mysqlbinlog.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_sp003.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
Slow test, don't run with --staging-run
mysql-test/t/compress.test:
Slow test, don't run with --staging-run
mysql-test/t/count_distinct3.test:
Slow test, don't run with --staging-run
mysql-test/t/index_merge_innodb.test:
Slow test, don't run with --staging-run
mysql-test/t/information_schema_all_engines.test:
Slow test, don't run with --staging-run
mysql-test/t/innodb_mysql.test:
Slow test, don't run with --staging-run
mysql-test/t/pool_of_threads.test:
Slow test, don't run with --staging-run
mysql-test/t/preload.test:
Slow test, don't run with --staging-run
mysql-test/t/ssl.test:
Slow test, don't run with --staging-run
mysql-test/t/ssl_compress.test:
Slow test, don't run with --staging-run
mysql-test/valgrind.supp:
Suppress warnings from SuSE 11.1 on x86
sql/scheduler.cc:
Use MY_MUTEX_INIT_FAST for pool mutex
The server was not cleaning the last IO error and error number when
resetting slave.
This patch addresses this issue by backporting into 5.1 part of the
patch in BUG 34654. A fix for this issue had already been pushed into
6.0 as part of the aforementioned bug, however the patch also included
some refactoring. The fix for 5.1 does not take into account the
refactoring part.
mysql-test/extra/rpl_tests/rpl_reset_slave.test:
Backported the test case and improved with deploying include/start_slave.inc
in relevant spots.
sql/slave.cc:
Backported part of patch from 6.0 that includes cleaning
mi->clear_error() at:
1. beginning of handle_slave_io
2. on successful connection
Also, backported the assertion added in the original patch.
sql/sql_repl.cc:
Backported the call to mi->clear_error() on reset_slave().
Disabling these two tests as they are affected by this bug / causing PB2 failures
on Windows platforms. Can always disable via include/not_windows.inc if
the bug fix looks like it will take some time.
Certain multi-updates gave different results on InnoDB from
to MyISAM, due to on-the-fly updates being used on the former and
the update order matters.
Fixed by turning off on-the-fly updates when update order
dependencies are present.
mysql-test/r/innodb_mysql.result:
Bug#43580: Test result.
mysql-test/suite/rpl/r/rpl_slave_skip.result:
Bug#43580: Changed test result. The InnoDB result is now what it would have been on MyISAM.
mysql-test/t/innodb_mysql.test:
Bug#43580: Test case.
sql/sql_base.cc:
Bug#43580: Added a word of caution about using tmp_set here.
sql/sql_update.cc:
Bug#43580: Fix.
Calls to TABLE::mark_columns_needed_for_update() are moved
from mysql_multi_update_prepare() and right before the decison
to do on-the-fly updates to the place where we do (or don't do)
on-the-fly updates.
1. Replace waiting of SQL thread stop by waiting of SQL error on slave and stopped
SQL thread.
2. Remove debug code because it already implemented in MTR2.
Respectively, replaced "--exec diff" by "--diff_files" which is a mysqltest command to run a
non-operating system specific diff. Removed the file rpl_000015-slave.sh as it is not
necessary in the new MTR.
In order to define the --slave-load-tmpdir, the init_relay_log_file()
was calling fn_format(MY_PACK_FILENAME) which internally was indirectly
calling strmov_overlapp() (through pack_dirname) and the following
warning message was being printed out while running in Valgrind:
"source and destination overlap in strcpy".
We fixed the issue by removing the flag MY_PACK_FILENAME as it was not
necessary. In a nutshell, with this flag the function fn_format() tried
to replace a directory by either "~", "." or "..". However, we wanted
exactly to remove such strings.
In this patch, we also refactored the functions init_relay_log_file()
and check_temp_dir(). The former was refactored to call the fn_format()
with the flag MY_SAFE_PATH along with the MY_RETURN_REAL_PATH, in order
to avoid issues with long directories and return an absolute path,
respectively. The flag MY_SAFE_UNPACK_FILENAME was removed too as it was
responsible for removing "~", "." or ".." only from the file parameter
and we wanted to remove such strings from the directory parameter in
the fn_format(). This result is stored in an rli variable, which is then
processed by the other function in order to verify if the directory exists
and if we are able to create files in it.
mysql-test/suite/rpl/t/rpl_slave_load_tmpdir_not_exist.test:
Changed the output to make it consistent among different runs.
mysys/mf_format.c:
Replaced a return for DBUG_RETURN.
Bug#319 if while a non-transactional slave is replicating a transaction possible problem
only testing related: addressing reviewers' comments.
mysql-test/suite/bugs/r/rpl_bug38205.result:
new results.
mysql-test/suite/bugs/t/rpl_bug38205.test:
changing output per reviewer's offering.
mysql-test/suite/rpl/r/rpl_start_stop_slave.result:
new results.
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
refining tests for Bug #38205, Bug#319 to start slave with the necessary synchronization;
changing output per reviewer's offering;
removing redundant wait for stop.
The result set for multi-row statements is not the same between STMT and
RBR and among different versions. Thus to avoid test failures, we are not
printing out such result sets. Note, however, that this does not have
impact on coverage and accuracy since the execution is able to continue
without further issues when an error is found on the master and such error
is set to be skipped.
Fix mysql-test-run.pl to not terminate early when warnings in error logs are detected during
server shutdown. Instead, give a nice summary report at the end of the failures.
Fix code to make 100% sure no failures will go undetected.
Revert earlier wrong change.
Fix race with port allocation semaphore file permissions.
Adjust testsuite to copy with new PBXT engine now in the tree. The PBXT engine causes an
extra table to appear in the INFORMATION_SCHEMA. This causes different output for a few
test cases.
dbug/dbug.c:
If DbugParse() is called multiple times, the stack->keywords for the
top stack frame could be overwritten without being freed, causing a
memory leak reported by Valgrind.
include/my_global.h:
Add useful macro for different values when Valgrind (HAVE_purify) and not.
mysql-test/extra/rpl_tests/rpl_auto_increment.test:
Omit pbxt variables from show variables output.
mysql-test/include/have_pbxt.inc:
Add facility to disable test if PBXT engine is not available.
mysql-test/lib/mtr_report.pm:
Give a nice summary report at the end of tests of any warnings seen in logs during
server shutdowns.
mysql-test/lib/mtr_unique.pm:
Move chmod 777 to greatly reduce the risk of leaving the port semaphore file unaccessible
bu other users.
mysql-test/mysql-test-run.pl:
Don't abort in case of warnings detected, instead give a nice summary report.
Fix code to make 100% sure no failures will go undetected.
Revert earlier wrong change when master disconnects early.
mysql-test/r/information_schema.result:
Omit PBXT INFORMATION_SCHEMA table from output.
Move part of test to information_schema_all_engines.
mysql-test/r/information_schema_all_engines.result:
New file for information_schema tests that depend on which engines are available.
mysql-test/r/information_schema_db.result:
Move part of test to information_schema_all_engines.
mysql-test/r/innodb-autoinc.result:
Omit pbxt variables from show variables output.
mysql-test/r/mysqlshow.result:
Move part of test to information_schema_all_engines.
mysql-test/suite/rpl/r/rpl_auto_increment.result:
Omit pbxt variables from show variables output.
mysql-test/t/information_schema.test:
Omit PBXT INFORMATION_SCHEMA table from output.
Move part of test to information_schema_all_engines.
mysql-test/t/information_schema_all_engines.test:
New file for information_schema tests that depend on which engines are available.
mysql-test/t/information_schema_db.test:
Move part of test to information_schema_all_engines.
mysql-test/t/innodb-autoinc.test:
Omit pbxt variables from show variables output.
mysql-test/t/mysqlshow.test:
Move part of test to information_schema_all_engines.
mysql-test/valgrind.supp:
Add variant suppression (different system library versions).
Add suppression for problem with inet_ntoa().
sql/mysqld.cc:
Fix missing DBUG_RETURN.
Fix uninitialised thd->connect_utime, likely introduced by pool_of_threads.
sql/set_var.cc:
Fix one-byte buffer overflow in several places.
Fix unsafe use of String::c_ptr() of stack-allocated String buffer.
sql/sql_select.cc:
Silence valgrind warning due to GCC bug.
sql/sql_string.h:
Document potential problem with String::c_ptr() and String() constructor with caller-supplied buffer.
storage/archive/azio.c:
Silence Valgrind false warning for libz.
RBR was not considering the option --slave-skip-errors.
To fix the problem, we are reporting the ignored ERROR(s) as warnings thus avoiding
stopping the SQL Thread. Besides, it fixes the output of "SHOW VARIABLES LIKE
'slave_skip_errors'" which was showing nothing when the value "all" was assigned
to --slave-skip-errors.
@sql/log_event.cc
skipped rbr errors when the option skip-slave-errors is set.
@sql/slave.cc
fixed the output of for SHOW VARIABLES LIKE 'slave_skip_errors'"
@test-cases
fixed the output of rpl.rpl_idempotency
updated the test case rpl_skip_error
1. Test case was rewritten completely.
2. Test covers 3 cases:
a) do deadlock on slave, wait retries of transaction, unlock slave before lock
timeout;
b) do deadlock on slave and wait error 'lock timeout exceed' on slave;
c) same as b) but if of max relay log size = 0;
3. Added comments inline.
4. Updated result file.
Mysql server crashes because unsafe statements warning is wrongly elevated to error,
which is set the error status of Diagnostics_area of the thread in THD::binlog_query().
Yet the caller believes that binary logging shouldn't touch the status, so it will
set the status also later by my_ok(), my_error() or my_message() seperately
according to the execution result of the statement or transaction.
But the status of Diagnostics_area of the thread is allowed to set only once.
Fixed to clear the error wrongly set by binary logging, but keep the warning message.
mysql-test/suite/binlog/r/binlog_stm_ps.result:
Change unsafe warning to NOTE level
mysql-test/suite/binlog/r/binlog_unsafe.result:
Test case result for unsafe statements to ensure mysql sever don't crash
mysql-test/suite/binlog/t/binlog_unsafe.test:
Test case for unsafe statements to ensure mysql sever don't crash
mysql-test/suite/rpl/r/rpl_skip_error.result:
Change unsafe warning to NOTE level
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
Change unsafe warning to NOTE level
mysql-test/suite/rpl/r/rpl_udf.result:
Change unsafe warning to NOTE level
sql/sql_class.cc:
the error status of the thread is cleared When a warning is elevated to an error
because of unsafe warning of binary log.
Test was flakey on some machines and showed spurious
reds for races.
New-and-improved test makes do with fewer statements,
no mysqltest-variables, and no backticks. Should hope-
fully be more robust. Heck, it's debatable whether we
should have a test for this, anyway.
mysql-test/suite/rpl/r/rpl_temporary.result:
streamlined
mysql-test/suite/rpl/t/rpl_temporary.test:
streamlined
Bug#319 if while a non-transactional slave is replicating a transaction possible problem
It is impossible to roll back a mixed engines transaction when one of the engine is
non-transaction. In replication that fact is crucial because the slave can not safely
re-apply a transction that was interrupted with STOP SLAVE.
Fixed with making STOP SLAVE not be effective immediately in the case the current
group of replication events has modified a non-transaction table. In order for slave to leave
either the group needs finishing or the user issues KILL QUERY|CONNECTION slave_thread_id.
mysql-test/suite/bugs/r/rpl_bug38205.result:
bug#38205 non-deterministic part of tests results.
mysql-test/suite/bugs/t/rpl_bug38205.test:
bug#38205 non-deterministic part of tests.
mysql-test/suite/rpl/r/rpl_start_stop_slave.result:
bug#38205 deterministic part of tests results.
mysql-test/suite/rpl/t/rpl_start_stop_slave-slave.opt:
increasing `innodb_lock_wait_timeout' to make the test pass on slow env w/o
timeout expired issue.
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
bug#38205 deterministic part of tests.
sql/log_event.cc:
Augmenting row-based events applying with the notion of
thd->transaction.{all,stmt}.modified_non_trans_table.
The pair is set and reset according to its specification
for the mixed transaction processing.
Particualry, once `modified_non_trans_table' is set in the row-events
processing loop, it will remain till the commit of the transaction.
sql/slave.cc:
Consulting `thd->transaction.all.modified_non_trans_table' to decide
whether to terminate by the sql thread or to continue even though
the sql thread might have been STOP-ed (rli->abort_slave).
