We found that there are some tests that are not cleaning
up properly:
1. rpl_tmp_table_and_DDL
2. rpl_do_grant
3. rpl_sync
For #1 and #2 we found that the slave would not, for some
cases, replicate all the instructions the master processed
in the cleanup section. We fix these by deploying some
synchronization commands in the test cases so that slave
processes all clean up instructions.
As for #3, this is tracked as part of another bug
(BUG@50442).
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_slow_query_log.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/options.cc
Text conflict in sql/mysqld.cc
mysql-test/t/disabled.def:
Restore disabled ssl tests: SSL certificates were updated.
Disable sp_sync.test, the test case can't work in next-4284.
mysql-test/t/partition_innodb.test:
Disable parsing of the test case for Bug#47343,
the test can not work in next-4284.
mysql-test/t/ps_ddl.test:
Update results (CREATE TABLE IF NOT EXISTS takes
into account existence of the temporary table).
into slow log
While processing a statement, down the mysql_parse execution
stack, the thd->enable_slow_log can be assigned to
opt_log_slow_admin_statements, depending whether one is executing
administrative statements, such as ALTER TABLE, OPTIMIZE,
ANALYZE, etc, or not. This can have an impact on slow logging for
statements that are executed after an administrative statement
execution is completed.
When executing statements directly from the user this is fine
because, the thd->enable_slow_log is reset right at the beginning
of the dispatch_command function, ie, everytime a new statement
is set is set to execute.
On the other hand, for slave SQL thread (sql_thd) the story is a
bit different. When in SBR the sql_thd applies statements by
calling mysql_parse. Right after, it calls log_slow_statement
function to log them if they take too long. Calling mysql_parse
directly is fine, but also means that dispatch_command function
is bypassed. As a consequence, thd->enable_slow_log does not get
a chance to be reset before the next statement to be executed by
the sql_thd. If the statement just executed by the sql_thd was an
administrative statement and logging of admin statements was
disabled, this means that sql_thd->enable_slow_log will be set to
0 (disabled) from that moment on. End result: sql_thd stops
logging slow statements.
We fix this by resetting the value of sql_thd->enable_slow_log to
the value of opt_log_slow_slave_statements right after
log_slow_stement is called by the sql_thd.
After merge fixes.
Adjust replication test cases.
mysql-test/suite/rpl/r/rpl_mixed_row_innodb.result:
Update results with a new test.
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
Add a warning, which I believe is an expected one.
mysql-test/suite/rpl/t/rpl_killed_ddl.test:
Sort results to avoid test failurs under load.
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
Update results (next-4284 merge).
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
Adjust test output to the new table opening scheme: decide_logging_format() is now called in CREATE/DROP trigger.
To 5.x Release
Notes
=====
This is a backport of BUG#23300 into 5.1 GA.
Original cset revid (in betony):
luis.soares@sun.com-20090929140901-s4kjtl3iiyy4ls2h
Description
===========
When using replication, the slave will not log any slow query
logs queries replicated from the master, even if the
option "--log-slow-slave-statements" is set and these take more
than "log_query_time" to execute.
In order to log slow queries in replicated thread one needs to
set the --log-slow-slave-statements, so that the SQL thread is
initialized with the correct switch. Although setting this flag
correctly configures the slave thread option to log slow queries,
there is an issue with the condition that is used to check
whether to log the slow query or not. When replaying binlog
events the statement contains the SET TIMESTAMP clause which will
force the slow logging condition check to fail. Consequently, the
slow query logging will not take place.
This patch addresses this issue by removing the second condition
from the log_slow_statements as it prevents slow queries to be
binlogged and seems to be deprecated.
fails in PB sporadically)
The IO thread can concurrently access the relay log IO_CACHE
while another thread is performing an FLUSH LOGS procedure.
FLUSH LOGS closes and reopens the relay log and while doing so it
(re)initializes its IO_CACHE. During this procedure the IO_CACHE
mutex is also reinitialized, which can cause problems if some
other thread (namely the IO THREAD) is concurrently accessing it
at the time .
This patch fixes the problem by extending the interface of the
flush_master_info function to also include a second paramater,
"need_relay_log_lock", stating whether the thread should grab the
relay log lock or not before actually flushing the relay log.
Also, IO thread now calls flush_master_info with this flag set
when it flushes master info with in the event read_event loop.
Finally, we also increase loop time in rpl_heartbeat_basic test
case, so that the number of calls to flush logs doubles, stressing
this part of the code a little more.
mysql-test/suite/rpl/t/rpl_heartbeat_basic.test:
Doubled the number of iterations on the FLUSH LOGS loop by
doubling the time available to perform all iterations.
sql/repl_failsafe.cc:
Updating flush_master_info call so that it uses two parameters
instead of one.
sql/rpl_mi.cc:
Updating flush_master_info call so that it uses two parameters
instead of one.
sql/rpl_mi.h:
Changed flush_master_info interface. Now takes a second parameter
instead of just one. The second parameter is: need_lock_relay_log.
sql/rpl_rli.cc:
Small fix in comment.
sql/slave.cc:
Updating flush_master_info call so that it uses two parameters
instead of one.
sql/sql_repl.cc:
Updating flush_master_info call so that it uses two parameters
instead of one.
Fix Bug#50555 "handler commands crash server in my_hash_first()"
as a post-merge fix (the new handler tests are not passing
otherwise).
- in hash.c, don't call calc_hash if ! my_hash_inited().
- add tests and results for the test case for Bug#50555
mysys/hash.c:
Assert that the hash is initialized when it's used.
sql/set_var.cc:
Check that the hash is initalized before using it (Bug#50555)
Original revision:
------------------------------------------------------------
revision-id: li-bing.song@sun.com-20100130124925-o6sfex42b6noyc6x
parent: joro@sun.com-20100129145427-0n79l9hnk0q43ajk
committer: <Li-Bing.Song@sun.com>
branch nick: mysql-5.1-bugteam
timestamp: Sat 2010-01-30 20:49:25 +0800
message:
Bug #48321 CURRENT_USER() incorrectly replicated for DROP/RENAME USER;
REVOKE/GRANT; ALTER EVENT.
The following statements support the CURRENT_USER() where a user is needed.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENT
but, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, All above statements are rewritten when they are binlogged.
The CURRENT_USER() is expanded to the real user's name and host.
------------------------------------------------------------
Linux x86_64 debug
Two test cases fail because the suppression for the unsafe
warning needs to be updated (BUG@39934 refactored this part and
these changes are only in mysql-next-mr - this is why we notice
them now when merging in next-mr). This is the case for
rpl_nondeterministic_functions and rpl_misc_functions test
cases. rpl_stm_binlog_direct test case is not needed in version >
5.1. The rpl_heartbeat_basic test case fails because patch for
BUG@50397 removed the CHANGE MASTER in the slave that would set
it's period to 1/10 of the master. This would cause the test
assertion to fail.
The fixes for the issues described above are:
- rpl_misc_functions - updated suppression message
- rpl_nondeterministic_functions - updated suppression message
- rpl_stm_binlog_direct - removed the test case (it is not
needed in versions > 5.1)
- rpl_heartbeat_basic - deployed instruction:
CHANGE MASTER TO MASTER_HEARTBEAT_PERIOD=0.1;
Conflicts:
- mysql-test/r/mysqld--help-win.result
- sql/sys_vars.cc
Original revsion (in next-mr-bugfixing):
------------------------------------------------------------
revno: 2971 [merge]
revision-id: alfranio.correia@sun.com-20100121210527-rbuheu5rnsmcakh1
committer: Alfranio Correia <alfranio.correia@sun.com>
branch nick: mysql-next-mr-bugfixing
timestamp: Thu 2010-01-21 21:05:27 +0000
message:
BUG#46364 MyISAM transbuffer problems (NTM problem)
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
Besides, it is used to enable the WL#2687 which is disabled by default.
------------------------------------------------------------
revno: 2970.1.1
revision-id: alfranio.correia@sun.com-20100121131034-183r4qdyld7an5a0
parent: alik@sun.com-20100121083914-r9rz2myto3tkdya0
committer: Alfranio Correia <alfranio.correia@sun.com>
branch nick: mysql-next-mr-bugfixing
timestamp: Thu 2010-01-21 13:10:34 +0000
message:
BUG#46364 MyISAM transbuffer problems (NTM problem)
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
Besides, it is used to enable the WL#2687 which is disabled by default.
Add a wait-for graph based deadlock detector to the
MDL subsystem.
Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and
bug #37346 "innodb does not detect deadlock between update and
alter table".
The first bug manifested itself as an unwarranted abort of a
transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER
statement, when this transaction tried to repeat use of a
table, which it has already used in a similar fashion before
ALTER started.
The second bug showed up as a deadlock between table-level
locks and InnoDB row locks, which was "detected" only after
innodb_lock_wait_timeout timeout.
A transaction would start using the table and modify a few
rows.
Then ALTER TABLE would come in, and start copying rows
into a temporary table. Eventually it would stumble on
the modified records and get blocked on a row lock.
The first transaction would try to do more updates, and get
blocked on thr_lock.c lock.
This situation of circular wait would only get resolved
by a timeout.
Both these bugs stemmed from inadequate solutions to the
problem of deadlocks occurring between different
locking subsystems.
In the first case we tried to avoid deadlocks between metadata
locking and table-level locking subsystems, when upgrading shared
metadata lock to exclusive one.
Transactions holding the shared lock on the table and waiting for
some table-level lock used to be aborted too aggressively.
We also allowed ALTER TABLE to start in presence of transactions
that modify the subject table. ALTER TABLE acquires
TL_WRITE_ALLOW_READ lock at start, and that block all writes
against the table (naturally, we don't want any writes to be lost
when switching the old and the new table). TL_WRITE_ALLOW_READ
lock, in turn, would block the started transaction on thr_lock.c
lock, should they do more updates. This, again, lead to the need
to abort such transactions.
The second bug occurred simply because we didn't have any
mechanism to detect deadlocks between the table-level locks
in thr_lock.c and row-level locks in InnoDB, other than
innodb_lock_wait_timeout.
This patch solves both these problems by moving lock conflicts
which are causing these deadlocks into the metadata locking
subsystem, thus making it possible to avoid or detect such
deadlocks inside MDL.
To do this we introduce new type-of-operation-aware metadata
locks, which allow MDL subsystem to know not only the fact that
transaction has used or is going to use some object but also what
kind of operation it has carried out or going to carry out on the
object.
This, along with the addition of a special kind of upgradable
metadata lock, allows ALTER TABLE to wait until all
transactions which has updated the table to go away.
This solves the second issue.
Another special type of upgradable metadata lock is acquired
by LOCK TABLE WRITE. This second lock type allows to solve the
first issue, since abortion of table-level locks in event of
DDL under LOCK TABLES becomes also unnecessary.
Below follows the list of incompatible changes introduced by
this patch:
- From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those
statements that acquire TL_WRITE_ALLOW_READ lock)
wait for all transactions which has *updated* the table to
complete.
- From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE
(i.e. all statements which acquire TL_WRITE table-level lock) wait
for all transaction which *updated or read* from the table
to complete.
As a consequence, innodb_table_locks=0 option no longer applies
to LOCK TABLES ... WRITE.
- DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort
statements or transactions which use tables being dropped or
renamed, and instead wait for these transactions to complete.
- Since LOCK TABLES WRITE now takes a special metadata lock,
not compatible with with reads or writes against the subject table
and transaction-wide, thr_lock.c deadlock avoidance algorithm
that used to ensure absence of deadlocks between LOCK TABLES
WRITE and other statements is no longer sufficient, even for
MyISAM. The wait-for graph based deadlock detector of MDL
subsystem may sometimes be necessary and is involved. This may
lead to ER_LOCK_DEADLOCK error produced for multi-statement
transactions even if these only use MyISAM:
session 1: session 2:
begin;
update t1 ... lock table t2 write, t1 write;
-- gets a lock on t2, blocks on t1
update t2 ...
(ER_LOCK_DEADLOCK)
- Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE
was abandoned.
LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same
priority as the usual LOCK TABLE ... WRITE.
SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in
the wait queue.
- We do not take upgradable metadata locks on implicitly
locked tables. So if one has, say, a view v1 that uses
table t1, and issues:
LOCK TABLE v1 WRITE;
FLUSH TABLE t1; -- (or just 'FLUSH TABLES'),
an error is produced.
In order to be able to perform DDL on a table under LOCK TABLES,
the table must be locked explicitly in the LOCK TABLES list.
mysql-test/include/handler.inc:
Adjusted test case to trigger an execution path on which bug 41110
"crash with handler command when used concurrently with alter
table" and bug 41112 "crash in mysql_ha_close_table/get_lock_data
with alter table" were originally discovered. Left old test case
which no longer triggers this execution path for the sake of
coverage.
Added test coverage for HANDLER SQL statements and type-aware
metadata locks.
Added a test for the global shared lock and HANDLER SQL.
Updated tests to take into account that the old simple deadlock
detection heuristics was replaced with a graph-based deadlock
detector.
mysql-test/r/debug_sync.result:
Updated results (see debug_sync.test).
mysql-test/r/handler_innodb.result:
Updated results (see handler.inc test).
mysql-test/r/handler_myisam.result:
Updated results (see handler.inc test).
mysql-test/r/innodb-lock.result:
Updated results (see innodb-lock.test).
mysql-test/r/innodb_mysql_lock.result:
Updated results (see innodb_mysql_lock.test).
mysql-test/r/lock.result:
Updated results (see lock.test).
mysql-test/r/lock_multi.result:
Updated results (see lock_multi.test).
mysql-test/r/lock_sync.result:
Updated results (see lock_sync.test).
mysql-test/r/mdl_sync.result:
Updated results (see mdl_sync.test).
mysql-test/r/sp-threads.result:
SHOW PROCESSLIST output has changed due to the fact that waiting
for LOCK TABLES WRITE now happens within metadata locking
subsystem.
mysql-test/r/truncate_coverage.result:
Updated results (see truncate_coverage.test).
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
SELECT FROM I_S.PROCESSLIST output has changed due to fact that
waiting for LOCK TABLES WRITE now happens within metadata locking
subsystem.
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
SELECT FROM I_S.PROCESSLIST output has changed due to fact that
waiting for LOCK TABLES WRITE now happens within metadata locking
subsystem.
mysql-test/suite/rpl/t/rpl_sp.test:
Updated to a new SHOW PROCESSLIST state name.
mysql-test/t/debug_sync.test:
Use LOCK TABLES READ instead of LOCK TABLES WRITE as the latter
no longer allows to trigger execution path involving waiting on
thr_lock.c lock and therefore reaching debug sync-point covered
by this test.
mysql-test/t/innodb-lock.test:
Adjusted test case to the fact that innodb_table_locks=0 option is
no longer supported, since LOCK TABLES WRITE handles all its
conflicts within MDL subsystem.
mysql-test/t/innodb_mysql_lock.test:
Added test for bug #37346 "innodb does not detect deadlock between
update and alter table".
mysql-test/t/lock.test:
Added test coverage which checks the fact that we no longer support
DDL under LOCK TABLES on tables which were locked implicitly.
Adjusted existing test cases accordingly.
mysql-test/t/lock_multi.test:
Added test for bug #46272 "MySQL 5.4.4, new MDL: unnecessary
deadlock". Adjusted other test cases to take into account the
fact that waiting for LOCK TABLES ... WRITE now happens within MDL
subsystem.
mysql-test/t/lock_sync.test:
Since LOCK TABLES ... WRITE now takes SNRW metadata lock for
tables locked explicitly we have to implicitly lock InnoDB tables
(through view) to trigger the table-level lock conflict between
TL_WRITE and TL_WRITE_ALLOW_WRITE.
mysql-test/t/mdl_sync.test:
Added basic test coverage for type-of-operation-aware metadata
locks. Also covered with tests some use cases involving HANDLER
statements in which a deadlock could arise.
Adjusted existing tests to take type-of-operation-aware MDL into
account.
mysql-test/t/multi_update.test:
Update to a new SHOW PROCESSLIST state name.
mysql-test/t/truncate_coverage.test:
Adjusted test case after making LOCK TABLES WRITE to wait until
transactions that use the table to be locked are completed.
Updated to the changed name of DEBUG_SYNC point.
sql/handler.cc:
Global read lock functionality has been
moved into a class.
sql/lock.cc:
Global read lock functionality has been
moved into a class.
Updated code to use the new MDL API.
sql/mdl.cc:
Introduced new type-of-operation aware metadata locks.
To do this:
- Changed MDL_lock to use one list for waiting requests and one
list for granted requests. For each list, added a bitmap
that holds information what lock types a list contains.
Added a helper class MDL_lock::List to manipulate with granted
and waited lists while keeping the bitmaps in sync
with list contents.
- Changed lock-compatibility functions to use bitmaps that
define compatibility.
- Introduced a graph based deadlock detector inspired by
waiting_threads.c from Maria implementation.
- Now that we have a deadlock detector, and no longer have
a global lock to protect individual lock objects, but rather
use an rw lock per object, removed redundant code for upgrade,
and the global read lock. Changed the MDL API to
no longer require the caller to acquire the global
intention exclusive lock by means of a separate method.
Removed a few more methods that became redundant.
- Removed deadlock detection heuristic, it has been made
obsolete by the deadlock detector.
- With operation-type-aware metadata locks, MDL subsystem has
become aware of potential conflicts between DDL and open
transactions. This made it possible to remove calls to
mysql_abort_transactions_with_shared_lock() from acquisition
paths for exclusive lock and lock upgrade. Now we can simply
wait for these transactions to complete without fear of
deadlock. Function mysql_lock_abort() has also become
unnecessary for all conflicting cases except when a DDL
conflicts with a connection that has an open HANDLER.
sql/mdl.h:
Introduced new type-of-operation aware metadata locks.
Introduced a graph based deadlock detector and supporting
methods.
Added comments.
God rid of redundant API calls.
Renamed m_lt_or_ha_sentinel to m_trans_sentinel,
since now it guards the global read lock as well as
LOCK TABLES and HANDLER locks.
sql/mysql_priv.h:
Moved the global read lock functionality into a
class.
Added MYSQL_OPEN_FORCE_SHARED_MDL flag which forces
open_tables() to take MDL_SHARED on tables instead of
metadata locks specified in the parser. We use this to
allow PREPARE run concurrently in presence of
LOCK TABLES ... WRITE.
Added signature for find_table_for_mdl_ugprade().
sql/set_var.cc:
Global read lock functionality has been
moved into a class.
sql/sp_head.cc:
When creating TABLE_LIST elements for prelocking or
system tables set the type of request for metadata
lock according to the operation that will be performed
on the table.
sql/sql_base.cc:
- Updated code to use the new MDL API.
- In order to avoid locks starvation we take upgradable
locks all at once. As result implicitly locked tables no
longer get an upgradable lock. Consequently DDL and FLUSH
TABLES for such tables is prohibited.
find_write_locked_table() was replaced by
find_table_for_mdl_upgrade() function.
open_table() was adjusted to return TABLE instance with
upgradable ticket when necessary.
- We no longer wait for all locks on OT_WAIT back off
action -- only on the lock that caused the wait
conflict. Moreover, now we distinguish cases when we
have to wait due to conflict in MDL and old version
of table in TDC.
- Upate mysql_notify_threads_having_share_locks()
to only abort thr_lock.c waits of threads that
have open HANDLERs, since lock conflicts with only
these threads now can lead to deadlocks not detectable
by the MDL deadlock detector.
- Remove mysql_abort_transactions_with_shared_locks()
which is no longer needed.
sql/sql_class.cc:
Global read lock functionality has been moved into a class.
Re-arranged code in THD::cleanup() to simplify assert.
sql/sql_class.h:
Introduced class to incapsulate global read lock
functionality.
Now sentinel in MDL subsystem guards the global read lock
as well as LOCK TABLES and HANDLER locks. Adjusted code
accordingly.
sql/sql_db.cc:
Global read lock functionality has been moved into a class.
sql/sql_delete.cc:
We no longer acquire upgradable metadata locks on tables
which are locked by LOCK TABLES implicitly. As result
TRUNCATE TABLE is no longer allowed for such tables.
Updated code to use the new MDL API.
sql/sql_handler.cc:
Inform MDL_context about presence of open HANDLERs.
Since HANLDERs break MDL protocol by acquiring table-level
lock while holding only S metadata lock on a table MDL
subsystem should take special care about such contexts (Now
this is the only case when mysql_lock_abort() is used).
sql/sql_parse.cc:
Global read lock functionality has been moved into a class.
Do not take upgradable metadata locks when opening tables
for CREATE TABLE SELECT as it is not necessary and limits
concurrency.
When initializing TABLE_LIST objects before adding them
to the table list set the type of request for metadata lock
according to the operation that will be performed on the
table.
We no longer acquire upgradable metadata locks on tables
which are locked by LOCK TABLES implicitly. As result FLUSH
TABLES is no longer allowed for such tables.
sql/sql_prepare.cc:
Use MYSQL_OPEN_FORCE_SHARED_MDL flag when opening
tables during PREPARE. This allows PREPARE to run
concurrently in presence of LOCK TABLES ... WRITE.
sql/sql_rename.cc:
Global read lock functionality has been moved into a class.
sql/sql_show.cc:
Updated code to use the new MDL API.
sql/sql_table.cc:
Global read lock functionality has been moved into a class.
We no longer acquire upgradable metadata locks on tables
which are locked by LOCK TABLES implicitly. As result DROP
TABLE is no longer allowed for such tables.
Updated code to use the new MDL API.
sql/sql_trigger.cc:
Global read lock functionality has been moved into a class.
We no longer acquire upgradable metadata locks on tables
which are locked by LOCK TABLES implicitly. As result
CREATE/DROP TRIGGER is no longer allowed for such tables.
Updated code to use the new MDL API.
sql/sql_view.cc:
Global read lock functionality has been moved into a class.
Fixed results of wrong merge that led to misuse of GLR API.
CREATE VIEW statement is not a commit statement.
sql/table.cc:
When resetting TABLE_LIST objects for PS or SP re-execution
set the type of request for metadata lock according to the
operation that will be performed on the table. Do the same
in auxiliary function initializing metadata lock requests
in a table list.
sql/table.h:
When initializing TABLE_LIST objects set the type of request
for metadata lock according to the operation that will be
performed on the table.
sql/transaction.cc:
Global read lock functionality has been moved into a class.
The root cause of the crash is that a TranxNode is freed before it is used.
A TranxNode is allocated and inserted into the active list each time
a log event is written and flushed into the binlog file.
The memory for TranxNode is allocated with thd_alloc and will be freed
at the end of the statement. The after_commit/after_rollback callback
was supposed to be called before the end of each statement and remove the node from
the active list. However this assumption is not correct in all cases(e.g. call
'CREATE TEMPORARY TABLE myisam_t SELECT * FROM innodb_t' in a transaction
and delete all temporary tables automatically when a session closed),
and can cause the memory allocated for TranxNode be freed
before it was removed from the active list. So The TranxNode pointer in the active
list would become a wild pointer and cause the crash.
After this patch, We have a class called a TranxNodeAllocate which manages the memory
for allocating and freeing TranxNode. It uses my_malloc to allocate memory.
sql/rpl_handler.cc:
params are not initialized.
REVOKE/GRANT; ALTER EVENT.
The following statements support the CURRENT_USER() where a user is needed.
DROP USER
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENT
but, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave
executes the log event, 'CURRENT_USER()' is expand to the user of slave
SQL thread, but SQL thread's user name always NULL. This breaks the replication.
After this patch, All above statements are rewritten when they are binlogged.
The CURRENT_USER() is expanded to the real user's name and host.
removed in MySQL 6.0
CREATE TABLE... TYPE= returns the warning "The syntax
'TYPE=storage_engine' is deprecated and will be removed in
MySQL 6.0. Please use 'ENGINE=storage_engine' instead"
This syntax is deprecated already from version 5.4.4, so
the message has been changed.
In addition, the deprecation macro was changed to reflect
the ServerPT decision not to include version number in the
warning message.
A number of test result files have been changed as a
consequence of the change in the deprecation macro.
WL#5182 is a follow-up to WL#5154, deprecating a few more options
and system variables.
client/client_priv.h:
The warning message has been changed to not include
a specific version number in the text.
client/mysql.cc:
--no-tee is deprecated
client/mysqldump.c:
--all is deprecated
-a now points to create-options
mysql-test/r/mysqlbinlog.result:
Warning text changed
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Warning text changed
sql/mysql_priv.h:
The warning message has been changed to not include
a specific version number in the text.
sql/mysqld.cc:
--use-symbolic-links is deprecated
-s now points to --symbolic-links
--warnings is deprecated
-W now points to --log-warnings
myisam_max_extra_sort_file_size is deprecated
record_buffer is deprecated
--log-update is deprecated
--sql-bin-update-same is deprecated
--skip-locking is deprecated
--skip-symlink is deprecated
--enable-locking is deprecated
--delay-key-write-for-all-tables is deprecated
The 'rpl_get_master_version_and_clock' test verifies if the slave I/O
thread tries to reconnect to master when it tries to get the values of
the UNIX_TIMESTAMP, SERVER_ID from master under network disconnection.
So the master server is restarted for making the transient network
disconnection, during the period the COM_REGISTER_SLAVE failures are
produced in server log file when the slave I/O thread tries to
register on master.
To fix the problem, suppress COM_REGISTER_SLAVE failures in server log
file by mtr suppression, because they are expected.
mysql-test/suite/rpl/r/rpl_get_master_version_and_clock.result:
Removed mtr.add_suppression("Get master clock failed with error: ")
and mtr.add_suppression("Get master SERVER_ID failed with error: ").
Because they are suppressed globally.
In RBR, DDL statement will change binlog format to non row-based
format before it is binlogged, but the binlog format was not be
restored, and then manipulating a temporary table can not reset binlog
format to row-based format rightly. So that the manipulated statement
is binlogged with statement-based format.
To fix the problem, restore the state of binlog format after the DDL
statement is binlogged.
mysql-test/extra/rpl_tests/rpl_tmp_table_and_DDL.test:
Added the test file to verify if executing DDL statement before
trying to manipulate a temporary table causes row-based replication
to break with error 'table does not exist'.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Correct the test result, all the above binlog event
should be row-based after the bug49132 is fixed IN RBR.
mysql-test/suite/ndb/r/ndb_tmp_table_and_DDL.result:
Test result for bug#49132 base on ndb engine.
mysql-test/suite/ndb/t/ndb_tmp_table_and_DDL.test:
Added the test file to verify if executing DDL statement before
trying to manipulate a temporary table causes row-based replication
to break with error 'table does not exist' base on ndb engine.
mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result:
Test result for bug#49132 base on myisam engine.
mysql-test/suite/rpl/t/rpl_tmp_table_and_DDL.test:
Added the test file to verify if executing DDL statement before
trying to manipulate a temporary table causes row-based replication
to break with error 'table does not exist' base on myisam engine.
sql/event_db_repository.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/events.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/sp.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/sql_acl.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/sql_udf.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
cant find record
Some engines return data for the record. Despite the fact that
the null bit is set for some fields, their old value may still in
the row. This can happen when unpacking an AI from the binlog on
top of a previous record in which a field is set to NULL, which
previously contained a value. Ultimately, this may cause the
comparison of records to fail when the slave is doing an index or
range scan.
We fix this by deploying a call to reset() for each field that is
set to null while unpacking a row from the binary log.
Furthermore, we also add mixed mode test case to cover the
scenario where updating and setting a field to null through a
Query event and later searching it through a rows event will
succeed.
Finally, we also change the reset() method, from Field_bit class,
so that it takes into account bits stored among the null bits and
not only the ones stored in the record.
mysql-test/suite/rpl/t/rpl_set_null_innodb.test:
InnoDB test.
mysql-test/suite/rpl/t/rpl_set_null_myisam.test:
MyISAM test.
mysql-test/suite/rpl_ndb/t/rpl_ndb_set_null.test:
NDB test.
sql/field.h:
Changed reset so that it also clears the bits
among the null_bits for the Field_bit class.
sql/rpl_record.cc:
Resetting field after setting it to null when unpacking
row.
Resetting the master before stopping the slave was generating the message
"[ERROR] Slave I/O: Got fatal error 1236 from master when reading data from
binary log: 'could not find next log', Error_code: 1236". In consequence,
the test case was failing because the message had not been suppressed.
To circumvent the failure, we rewrote the test stopping the slave before
resetting the master. We prefer this alternative rather than suppressing
the message.
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
Besides, it is used to enable the WL#2687 which is disabled by default.
mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache. It also fixes comments.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
The STMT mode is unsafe when mixed-statements are executed thus making slaves to go out of sync. For that reason, it checks consistency if not in STMT mode.
mysql-test/include/default_mysqld.cnf:
Makes binlog-direct-non-transactional-updates "TRUE" by default in the test
cases.
mysql-test/r/mysqld--help-notwin.result:
Updates the result file with the new option.
mysql-test/r/mysqld--help-win.result:
Updates the result file with the new option.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/binlog/r/binlog_switch_inside_trans.result:
Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
within a transaction or a procedure/function/trigger.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam-master.opt:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing
the test case and its result file.
mysql-test/suite/binlog/t/binlog_switch_inside_trans.test:
Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
within a transaction or a procedure/function/trigger.
mysql-test/suite/ndb/r/ndb_binlog_format.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
Updates the result file because non-trx-changes are written ahead of the
transaction
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
Updates the result file because non-trx-changes are written ahead of the
transaction.
mysql-test/suite/rpl/r/rpl_stm_start_stop_slave.result:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_start_stop_slave.test:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_stop_middle_group.test:
Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
sql/log.cc:
Verifies if changes should be written to either the trx-cache or non-trx-cache through the use of the function use_trans_cache(). It also organizes the code.
sql/log.h:
Changes the signature of some functions by adding the modifier "const" to the thd parameter. Specifically, the following functions are changed:
bool trans_has_updated_trans_table(const THD* thd);
bool stmt_has_updated_trans_table(const THD *thd);
bool use_trans_cache(const THD*, bool is_transactional);
sql/share/errmsg-utf8.txt:
Creates error messages to report when an user tries to change the new option
binlog_direct_non_transactional_updates within a transaction or a procedure/
function/trigger.
sql/share/errmsg.txt:
Creates error messages to report when an user tries to change the new option
binlog_direct_non_transactional_updates within a transaction or a procedure/
function/trigger.
sql/sql_class.h:
Adds the new option binlog_direct_non_transactional_updates.
sql/sys_vars.cc:
Adds the new option binlog_direct_non_transactional_updates.
support-files/my-small.cnf.sh:
Adds binlog-direct-non-transactional-updates to the example file. By default
the option is disabled.
enabled binary
The test case injects an error in the server by deleting the
temporary file that it uses during the load data statement
execution. The error consisted of closing, deleting and setting
the file descriptor to -1 right before calling mysql_file_write.
Although, this error injection seems to work OK in Unix like
environments, in Windows, this would cause the server to hit an
assertion in 'my_get_open_flags':
DBUG_ASSERT(fd >= MY_FILE_MIN && fd < (int)my_file_limit)
We fix this by changing the error injection to just call the
macro my_delete_allow_opened, instead of the close + delete + set
fd=-1. The macro deletes the file and is platform
independent. Additionally, this required some changes to how the
assertion is handled in the test case to make it cope with this
change.
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables in statement and mixed modes.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
mysql-test/extra/rpl_tests/rpl_mixing_engines.inc:
Backported this from Celosia to improve the test cases related to the NTM issue.
sql/log.cc:
Checks the --binlog-direct-non-transactional-updates before choosing
to either use the trxn-cache or not.
sql/mysqld.cc:
Introduces the option --binlog-direct-non-transactional-updates.
sql/set_var.cc:
Introduces the option --binlog-direct-non-transactional-updates.
sql/sql_class.h:
Introduces the option --binlog-direct-non-transactional-updates.
The test case was failing because it contained instructions
to close/reopen files, when they were in use. This raises
problems in windows. Example of such instruction:
---exec echo "failure" > $MYSQLD_SLAVE_DATADIR/$file
The test also contains commands that are not platform
agnostic. Example:
--exec cat $MYSQLD_SLAVE_DATADIR/master.backup > \
$MYSQLD_SLAVE_DATADIR/master.info
We fix this by just truncating the necessary file and write
"failure" into it (ie, without closing the file). The
platform specific instruction is removed from the test
case as it seems redundant.
'CREATE TABLE IF NOT EXISTS ... SELECT' statement were causing 'CREATE
TEMPORARY TABLE ...' to be written to the binary log in row-based
mode (a.k.a. RBR), when there was a temporary table with the same name.
Because the 'CREATE TABLE ... SELECT' statement was executed as
'INSERT ... SELECT' into the temporary table. Since in RBR mode no
other statements related to temporary tables are written into binary log,
this sometimes broke replication.
This patch changes behavior of 'CREATE TABLE [IF NOT EXISTS] ... SELECT ...'.
it ignores existence of temporary table with the
same name as table being created and is interpreted
as attempt to create/insert into base table. This makes behavior of
'CREATE TABLE [IF NOT EXISTS] ... SELECT' consistent with
how ordinary 'CREATE TABLE' and 'CREATE TABLE ... LIKE' behave.
variable
The User_var_log_event was not serializing the unsigned
flag. This would cause the slave to always assume signed values.
We fix this by extending the User_var_log_event to also contain
information on the unsigned_flag, meaning that it gets into the
binlog as well, therefore the slave will get this information as
well. Events without information on unsigned flag (old events)
are treated as they were before (always signed: unsigned_flag=
FALSE).
The information on the unsigned_flag, is shipped in an extra byte
appended to the end of the User_var_log_event and added by this
patch. This extra byte holds values for general purpose
User_var_log_event flags which are now packed in the binlog as
well. One of these flags contains information about whether the
value is signed or unsigned (currently this extra byte is only
used to hold data on the unsigned flag, in the future we can use
it to pack extra flags if there is the need to).
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_servers.cc
Text conflict in sql/sql_update.cc
Text conflict in support-files/mysql.spec.sh
BUG#49481: RBR: MyISAM and bit fields may cause slave to stop on delete:
cant find record
BUG#49482: RBR: Replication may break on deletes when MyISAM tables +
char field are used
When using MyISAM tables, despite the fact that the null bit is
set for some fields, their old value is still in the row. This
can cause the comparison of records to fail when the slave is
doing an index or range scan.
We fix this by avoiding memcmp for MyISAM tables when comparing
records. Additionally, when comparing field by field, we first
check if both fields are not null and if so, then we compare
them. If just one field is null we return failure immediately. If
both fields are null, we move on to the next field.
Problem: When RAND() is binlogged in statement mode, the seed is
binlogged too, so the replication slave generates the same
sequence of random numbers. This makes replication work in many
cases, but not in all cases: the order of rows is not guaranteed
for, e.g., UPDATE or INSERT...SELECT statements, so the row data
will be different if master and slave retrieve the rows in
different orders.
Fix: Mark RAND() as unsafe. It will generate a warning if
binlog_format=STATEMENT and switch to row-logging if
binlog_format=ROW.
mysql-test/extra/rpl_tests/rpl_row_func003.test:
updated test case to ignore new warnings
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file
mysql-test/suite/binlog/t/binlog_unsafe.test:
Added test for RAND().
Also clarified some old comments.
mysql-test/suite/rpl/r/rpl_misc_functions.result:
updated result file
mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result:
updated test case to ignore new warnings
mysql-test/suite/rpl/r/rpl_optimize.result:
updated result file
mysql-test/suite/rpl/r/rpl_row_func003.result:
updated result file
mysql-test/suite/rpl/t/rpl_misc_functions.test:
updated test case to ignore new warnings
mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test:
updated test case to ignore new warnings
mysql-test/suite/rpl/t/rpl_optimize.test:
updated test case to ignore new warnings
mysql-test/suite/rpl/t/rpl_trigger.test:
updated test case to ignore new warnings
mysql-test/suite/rpl_ndb/r/rpl_ndb_func003.result:
updated result file
sql/item_create.cc:
Mark RAND() unsafe.
- mysqld--help-win
Updated result so that it contains missing
value for slave-type-conversions
- rpl_idempotency
This seems a bad merge. In BUG#39934, the contents of
this file had been split into rpl_row_idempontency and
rpl_idempotency. The patch was pushed to 5.1-rep+3 which
was later merged in rep+2-delivery1 which in turn was
merged in 5.1-rpl-merge. Now while merging next-mr in
5.1-rpl-merge, the file got back it's old content (which
is in rpl_row_idempotency now because of BUG#39934). This
cset reverts the bad merge:
bzr merge -r revid:dao-gang.qu@sun.com-20100112120709-ioxp11yl9bvquaqd..\
before:revid:dao-gang.qu@sun.com-20100112120709-ioxp11yl9bvquaqd\
suite/rpl/t/rpl_idempotency.test
- sys_vars.all_vars:
Added test case for slave_type_conversions variable
- rpl_row_idempotency
Removed ER_SLAVE_AMBIGOUS_EXEC_MODE (which was removed by WL 4738)
from the test case. Using ER_WRONG_VALUE_FOR_VAR instead.
- mysqld--help-win
Added missing help for --slave-type-conversions from the
result file.
The test case did not start with fresh binlogs, so in some
cases, dependending on the order MTR runs the tests, it would
try to show binlog contents from invalid positions (binary log
would contain unexpected events from previous test).
We fix this by deploying a RESET MASTER at the beginning of the
test case.
Manually deleteing one or more entries from 'master-bin.index', will
cause master infinitely loop to send one binlog file.
When starting a dump session, master opens index file and search the binlog file
which is being requested by the slave. The position of the binlog file in the
index file is recorded. it will be used to find the next binlog file when current
binlog file has dumped completely. As only the position is used, it may
not get the correct file if some entries has been removed manually from the index file.
the master will reopen the current binlog file which has been dump completely
and redump it if it can not get the next binlog file's name from index file.
It obviously is a logical error.
Even though it is allowed to manually change index file,
but it is not recommended. so after this patch, master
sends a fatal error to slave and close the dump session if a new binlog file
has been generated and master can not get it from the index file.
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase. Created directory.
Conflict because storage/innobase is not versioned, but has versioned children. Versioned directory.
Conflict adding file storage/innobase. Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler. Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children. Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
For tables with metadata sizes ranging from 251 to 255 the size
of the event data (m_data_size) was being improperly calculated
in the Table_map_log_event constructor. This was due to the fact
that when writing the Table_map_log_event body (in
Table_map_log_event::write_data_body) a call to net_store_length
is made for packing the m_field_metadata_size. It happens that
net_store_length uses *one* byte for storing
m_field_metadata_size when it is smaller than 251 but *three*
bytes when it exceeds that value. BUG 42749 had already
pinpointed and fix this fact, but the fix was incomplete, as the
calculation in the Table_map_log_event constructor considers 255
instead of 251 as the threshold to increment m_data_size by
three. Thence, the window for having a mismatch between the
number of bytes written and the number of bytes accounted in the
event length (m_data_size) was left open for
m_field_metadata_size values between 251 and 255.
We fix this by changing the condition in the Table_map_log_event
constructor to match the one in the net_store_length, ie,
increment one byte if m_field_metadata_size < 251 and three if it
exceeds this value.
mysql-test/suite/rpl/r/rpl_row_tbl_metadata.result:
Updated result file.
mysql-test/suite/rpl/t/rpl_row_tbl_metadata.test:
Changes to the original test case: added slave and moved
file into the rpl suite.
New test case: replicates two tables one with 250 and
another with 252 metadata sizes. This exercises the usage
of 1 or 3 bytes while packing the m_field_metadata_size.
sql/log_event.cc:
Made the m_data_size calculation for the table map log event
to match the number of bytes used while packing the
m_field_metadata_size value (according to net_store_length
function in pack.c).
In auto-commit mode, updating both trx and non-trx tables (i.e. issuing a mixed
statement) causes the following sequence of events:
1 - "Flush trx changes" (MYSQL_BIN_LOG::write) - T1:
1.1 - mutex_lock (&LOCK_log)
1.2 - mutex_lock (&LOCK_prep_xids)
1.3 - increase prepared_xids
1.4 - mutex_unlock (&LOCK_prep_xids)
1.5 - mutex_unlock (&LOCK_log)
2 - "Flush non-trx changes" (MYSQL_BIN_LOG::write) - T1:
2.1 - mutex_lock (&LOCK_log)
2.2 - mutex_unlock (&LOCK_log)
3. "unlog" - T1
3.1 - mutex_lock (&LOCK_prep_xids)
3.2 - decrease prepared xids
3.3 - pthread_cond_signal(&COND_prep_xids);
3.4 - mutex_unlock (&LOCK_prep_xids)
The "FLUSH logs" command produces the following sequence of events:
1 - "FLUSH logs" command (MYSQL_BIN_LOG::new_file_impl) - user thread:
1.1 - mutex_lock (&LOCK_log)
1.2 - mutex_lock (&LOCK_prep_xids)
1.3 - while (prepared_xids) pthread_cond_wait(..., &LOCK_prep_xids);
1.4 - mutex_unlock (&LOCK_prep_xids)
1.5 - mutex_unlock (&LOCK_log)
A deadlock will arise if T1 flushes the trx changes and thus increases
prepared_xids but before it is able to continue the execution and flush the
non-trx changes, an user thread calls the "FLUSH logs" command and wait that
the prepared_xids is decreased and gets to zero. However, T1 cannot proceed
with the call to "Flush non-trx changes" because it will block in the mutex
"LOCK_log" and by consequence cannot complete the execution and call the
unlog to decrease the prepared_xids.
To fix the problem, we ensure that the non-trx changes are always flushed
before the trx changes.
Note that if you call "Flush non-trx changes" and a concurrent "FLUSH logs" is
issued, the "Flush non-trx changes" may block, but a deadlock will never happen
because the prepared_xids will eventually get to zero. Bottom line, there will
not be any transaction able to increase the prepared_xids because they will
block in the mutex "LOCK_log" (MYSQL_BIN_LOG::write) and those that increased
the prepared_xids will eventually commit and decrease the prepared_xids.
This bug is the same problem as Bug 49836 for 5.1 versions.
mysql-test/suite/rpl/r/rpl_geometry.result:
Test case for bug 48776
mysql-test/suite/rpl/t/rpl_geometry.test:
Test case for bug 48776
sql/rpl_utility.h:
Add missing case MYSQL_TYPE_GEOMETRY
In statement-based or mixed-mode replication, use DROP TEMPORARY TABLE
to drop multiple tables causes different errors on master and slave,
when one or more of these tables do not exist. Because when executed
on slave, it would automatically add IF EXISTS to the query to ignore
all ER_BAD_TABLE_ERROR errors.
To fix the problem, do not add IF EXISTS when executing DROP TEMPORARY
TABLE on the slave, and clear the ER_BAD_TABLE_ERROR error after
execution if the query does not expect any errors.
mysql-test/suite/rpl/r/rpl_drop_temp.result:
Updated for the patch of bug#49137.
mysql-test/suite/rpl/t/rpl_drop_temp.test:
Added the test file to verify if DROP MULTI TEMPORARY TABLE
will cause different errors on master and slave, when one or
more of these tables do not exist.
sql/log_event.cc:
Added code to handle above cases which are
removed from sql_parse.cc
sql/sql_parse.cc:
Remove the code to issue the 'Unknown table' error,
if the temporary table does not exist when dropping
it on slave. The above cases decribed in comments
will be handled later in log_event.cc.
3655 Jon Olav Hauglid 2009-10-19
Bug #30977 Concurrent statement using stored function and DROP FUNCTION
breaks SBR
Bug #48246 assert in close_thread_table
Implement a fix for:
Bug #41804 purge stored procedure cache causes mysterious hang for many
minutes
Bug #49972 Crash in prepared statements
The problem was that concurrent execution of DML statements that
use stored functions and DDL statements that drop/modify the same
function might result in incorrect binary log in statement (and
mixed) mode and therefore break replication.
This patch fixes the problem by introducing metadata locking for
stored procedures and functions. This is similar to what is done
in Bug#25144 for views. Procedures and functions now are
locked using metadata locks until the transaction is either
committed or rolled back. This prevents other statements from
modifying the procedure/function while it is being executed. This
provides commit ordering - guaranteeing serializability across
multiple transactions and thus fixes the reported binlog problem.
Note that we do not take locks for top-level CALLs. This means
that procedures called directly are not protected from changes by
simultaneous DDL operations so they are executed at the state they
had at the time of the CALL. By not taking locks for top-level
CALLs, we still allow transactions to be started inside
procedures.
This patch also changes stored procedure cache invalidation.
Upon a change of cache version, we no longer invalidate the entire
cache, but only those routines which we use, only when a statement
is executed that uses them.
This patch also changes the logic of prepared statement validation.
A stored procedure used by a prepared statement is now validated
only once a metadata lock has been acquired. A version mismatch
causes a flush of the obsolete routine from the cache and
statement reprepare.
Incompatible changes:
1) ER_LOCK_DEADLOCK is reported for a transaction trying to access
a procedure/function that is locked by a DDL operation in
another connection.
2) Procedure/function DDL operations are now prohibited in LOCK
TABLES mode as exclusive locks must be taken all at once and
LOCK TABLES provides no way to specifiy procedures/functions to
be locked.
Test cases have been added to sp-lock.test and rpl_sp.test.
Work on this bug has very much been a team effort and this patch
includes and is based on contributions from Davi Arnaut, Dmitry
Lenev, Magne Mæhre and Konstantin Osipov.
mysql-test/r/ps_ddl.result:
Update results (Bug#30977).
mysql-test/r/ps_ddl1.result:
Update results (Bug#30977).
mysql-test/r/sp-error.result:
Update results (Bug#30977).
mysql-test/r/sp-lock.result:
Update results (Bug#30977).
mysql-test/suite/rpl/r/rpl_sp.result:
Update results (Bug#30977).
mysql-test/suite/rpl/t/rpl_sp.test:
Add a test case for Bug#30977.
mysql-test/t/ps_ddl.test:
Update comments. We no longer re-prepare a prepared statement
when a stored procedure used in top-level CALL is changed.
mysql-test/t/ps_ddl1.test:
Modifying stored procedure p1 no longer invalidates prepared
statement "call p1" -- we can re-use the prepared statement
without invalidation.
mysql-test/t/sp-error.test:
Use a constant for an error value.
mysql-test/t/sp-lock.test:
Add test coverage for Bug#30977.
sql/lock.cc:
Implement lock_routine_name() - a way to acquire an
exclusive metadata lock (ex- name-lock) on
stored procedure/function.
sql/sp.cc:
Change semantics of sp_cache_routine() -- now it has an option
to make sure that the routine that is cached is up to date (has
the latest sp cache version).
Add sp_cache_invalidate() to sp_drop_routine(), where it was
missing (a bug!).
Acquire metadata locks for SP DDL (ALTER/CREATE/DROP). This is
the core of the fix for Bug#30977.
Since caching and cache invalidation scheme was changed, make
sure we don't invalidate the SP cache in the middle of a stored
routine execution. At the same time, make sure we don't access
stale data due to lack of invalidation.
For that, change ALTER FUNCTION/PROCEDURE to not use the cache,
and SHOW PROCEDURE CODE/SHOW CREATE PROCEDURE/FUNCTION to always
read an up to date version of the routine from the cache.
sql/sp.h:
Add a helper wrapper around sp_cache_routine().
sql/sp_cache.cc:
Implement new sp_cache_version() and sp_cache_flush_obsolete().
Now we flush stale routines individually, rather than all at once.
sql/sp_cache.h:
Update signatures of sp_cache_version() and sp_cache_flush_obsolete().
sql/sp_head.cc:
Add a default initialization of sp_head::m_sp_cache_version.
Remove a redundant sp_head::create().
sql/sp_head.h:
Add m_sp_cache_version to sp_head class - we now
keep track of every routine in the stored procedure cache, rather than
of the entire cache.
sql/sql_base.cc:
Implement prelocking for stored routines. Validate stored
routines after they were locked.
Flush obsolete routines upon next access, one by one, not all at once
(Bug#41804).
Style fixes.
sql/sql_class.h:
Rename a Open_table_context method.
sql/sql_parse.cc:
Make sure stored procedures DDL commits the active transaction
(issues an implicit commit before and after).
Remove sp_head::create(), a pure redundancy.
Move the semantical check during alter routine inside sp_update_routine() code in order to:
- avoid using SP cache during update, it may be obsolete.
- speed up and simplify the update procedure.
Remove sp_cache_flush_obsolete() calls, we no longer flush the entire
cache, ever, stale routines are flushed before next use, one at a time.
sql/sql_prepare.cc:
Move routine metadata validation to open_and_process_routine().
Fix Bug#49972 (don't swap flags at reprepare).
Reset Sroutine_hash_entries in reinit_stmt_before_use().
Remove SP cache invalidation, it's now done by open_tables().
sql/sql_show.cc:
Fix a warning: remove an unused label.
sql/sql_table.cc:
Reset mdl_request.ticket for tickets acquired for routines inlined
through a view, in CHECK TABLE statement, to satisfy an MDL assert.
sql/sql_update.cc:
Move the cleanup of "translation items" to close_tables_for_reopen(),
since it's needed in all cases when we back off, not just
the back-off in multi-update. This fixes a bug when the server
would crash on attempt to back off when opening tables
for a statement that uses information_schema tables.
The 'rpl_cross_version' fails on mysql-next-mr-bugfixing as following:
mysqltest: In included file "./include/setup_fake_relay_log.inc": At line 80: query
'select './$_fake_filename-fake.000001\n' into dumpfile '$_fake_relay_index'' failed:
1290: The MySQL server is running with the --secure-file-priv option so it cannot execute
this statement.
To fix the problem by removeing the --secure-file-priv option
for adapting the update of the 'setup_fake_relay_log.inc'.
