DDL workload".
When a RENAME TABLE or LOCK TABLE ... WRITE statement which
mentioned the same table several times were aborted during
the process of acquring metadata locks (due to deadlock
which was discovered or because of KILL statement) server
might have crashed.
When attempt to acquire all locks requested had failed we
went through the list of requests and released locks which
we have managed to acquire by that moment one by one. Since
in the scenario described above list of requests contained
duplicates this led to releasing the same ticket twice and
a crash as result.
This patch solves the problem by employing different approach
to releasing locks in case of failure to acquire all locks
requested.
Now we take a MDL savepoint before starting acquiring locks
and simply rollback to it if things go bad.
mysql-test/r/lock_multi.result:
Updated test results (see lock_multi.test).
mysql-test/t/lock_multi.test:
Added test case for bug #51134 "Crash in MDL_lock::destroy
on a concurrent DDL workload".
sql/mdl.cc:
MDL_context::acquire_locks():
When attempt to acquire all locks requested has failed do
not go through the list of requests and release locks which
we have managed to acquire one by one.
Since list of requests can contain duplicates such approach
may lead to releasing the same ticket twice and a crash as
result.
Instead use the following approach - take a MDL savepoint
before starting acquiring locks and simply rollback to it
if things go bad.
failed in enter_locked_tables_mode".
Server was aborted due to assertion failure when one tried to
execute statement requiring prelocking (i.e. firing triggers
or using stored functions) while having open HANDLERs.
The problem was that THD::enter_locked_tables_mode() method
which was called at the beginning of execution of prelocked
statement assumed there are no open HANDLERs. It had to do
so because corresponding THD::leave_locked_tables_mode()
method was unable to properly restore MDL sentinel when
leaving LOCK TABLES/prelocked mode in the presence of open
HANDLERs.
This patch solves this problem by changing the latter method
to properly restore MDL sentinel and thus removing need for
this assumption. As a side-effect, it lifts unjustified
limitation by allowing to keep HANDLERs open when entering
LOCK TABLES mode.
mysql-test/include/handler.inc:
Adjusted tests after making LOCK TABLES not to close
open HANDLERs. Added coverage for bug #50908
"Assertion `handler_tables_hash.records == 0' failed
in enter_locked_tables_mode".
mysql-test/r/handler_innodb.result:
Updated test results (see include/handler.inc).
mysql-test/r/handler_myisam.result:
Updated test results (see include/handler.inc).
sql/mysql_priv.h:
Introduced mysql_ha_move_tickets_after_trans_sentinel()
routine which allows to move tickets for metadata locks
corresponding to open HANDLERs after transaction sentinel.
sql/sql_class.cc:
Changed THD::leave_locked_tables_mode() to correctly restore
MDL sentinel value in the presence of open HANDLERs.
sql/sql_class.h:
Removed assert from THD::enter_locked_tables_mode() as we
no longer have to close HANDLERs when entering LOCK TABLES
or prelocked modes. Instead we keep them open and correctly
restore MDL sentinel value after leaving them.
Removal of assert also fixes problem from the bug report.
sql/sql_handler.cc:
Introduced mysql_ha_move_tickets_after_trans_sentinel()
routine which allows to move tickets for metadata locks
corresponding to open HANDLERs after transaction sentinel.
sql/sql_parse.cc:
We no longer have to close HANDLERs when entering LOCK TABLES
mode. Instead we keep them open and simply correctly restore
MDL sentinel value after leaving this mode.
The test case for this bug relies on getting a ER_LOCK_WAIT_TIMEOUT
error. However with the introduction of MDL, the test would hang
forever since the metadata locks would not timeout.
MDL timeouts are now introduced in the scope of Bug#45225. This
patch changes the testcase for Bug#34604 to set the new server
variable "lock_wait_timeout" to one second which makes the test
generate the necessary timeout again.
This patch introduces timeouts for metadata locks.
The timeout is specified in seconds using the new dynamic system
variable "lock_wait_timeout" which has both GLOBAL and SESSION
scopes. Allowed values range from 1 to 31536000 seconds (= 1 year).
The default value is 1 year.
The new server parameter "lock-wait-timeout" can be used to set
the default value parameter upon server startup.
"lock_wait_timeout" applies to all statements that use metadata locks.
These include DML and DDL operations on tables, views, stored procedures
and stored functions. They also include LOCK TABLES, FLUSH TABLES WITH
READ LOCK and HANDLER statements.
The patch also changes thr_lock.c code (table data locks used by MyISAM
and other simplistic engines) to use the same system variable.
InnoDB row locks are unaffected.
One exception to the handling of the "lock_wait_timeout" variable
is delayed inserts. All delayed inserts are executed with a timeout
of 1 year regardless of the setting for the global variable. As the
connection issuing the delayed insert gets no notification of
delayed insert timeouts, we want to avoid unnecessary timeouts.
It's important to note that the timeout value is used for each lock
acquired and that one statement can take more than one lock.
A statement can therefore block for longer than the lock_wait_timeout
value before reporting a timeout error. When lock timeout occurs,
ER_LOCK_WAIT_TIMEOUT is reported.
Test case added to lock_multi.test.
include/my_pthread.h:
Added macros for comparing two timespec structs.
include/thr_lock.h:
Introduced timeouts for thr_lock.c locks.
mysql-test/r/mysqld--help-notwin.result:
Updated result file with the new server variable.
mysql-test/r/mysqld--help-win.result:
Updated result file with the new server variable.
mysql-test/suite/sys_vars/r/lock_wait_timeout_basic.result:
Added basic test for the new server variable.
mysql-test/suite/sys_vars/t/lock_wait_timeout_basic.test:
Added basic test for the new server variable.
mysys/thr_lock.c:
Introduced timeouts for thr_lock.c locks.
sql/mdl.cc:
Introduced timeouts for metadata locks.
sql/mdl.h:
Introduced timeouts for metadata locks.
sql/sql_base.cc:
Introduced timeouts in tdc_wait_for_old_versions().
sql/sql_class.h:
Added new server variable lock_wait_timeout.
sql/sys_vars.cc:
Added new server variable lock_wait_timeout.
rqg_mdl_stability".
When start of statement's waiting on a metadata lock
created more than one loop in waiters graph server might
have entered deadlock condition.
The problem was that in the case described above MDL deadlock
detector had to perform several searches for deadlock but
forgot to reset Deadlock_detection_context before performing
new search.
Failure to do so has broken assumption in code resposible for
choosing victim that if Deadlock_detection_context::victim
is set we also have read lock on m_waiting_for_lock for this
context. As result this lock could have been unlocked more
times than it was acquired which corrupted rwlock's state
which led to server deadlock.
This fix ensures that such reset is done before each attempt
to find a deadlock.
mysql-test/r/mdl_sync.result:
Added test for bug #50998 "Deadlock in MDL code during test
rqg_mdl_stability" as well as coverage for the case when
addition of statement waiting for metadata lock adds several
loops in the waiters graph and therefore several searches
for deadlock should be performed by MDL deadlock detector.
mysql-test/t/mdl_sync.test:
Added test for bug #50998 "Deadlock in MDL code during test
rqg_mdl_stability" as well as coverage for the case when
addition of statement waiting for metadata lock adds several
loops in the waiters graph and therefore several searches
for deadlock should be performed by MDL deadlock detector.
sql/mdl.cc:
Ensure that in cases when MDL deadlock detector had to
perform several searches for deadlock because several loops
in waiters graph are possible we reset
Deadlock_detection_context before performing each search.
Failure to do so has broken assumption in code resposible
for choosing victim that if Deadlock_detection_context::victim
is set we also have read lock on m_waiting_for_lock for this
context. As result this lock could have been unlocked more
times than it was acquired which corrupted rwlock's state
(no one was able to acquire write lock on it anymore).
and MDL".
Concurrent execution of a multi-DELETE statement and ALTER
TABLE statement which affected one of the tables used in
the multi-DELETE sometimes led to deadlock.
Similar deadlocks might have occured when one performed
INSERT/UPDATE/DELETE on a view and concurrently executed
ALTER TABLE for the view's underlying table, or when one
concurrently executed TRUNCATE TABLE for InnoDB table and
ALTER TABLE for the same table.
These deadlocks were caused by a discrepancy between types of
metadata and thr_lock.cc locks acquired by those statements.
What happened was that multi-DELETE/TRUNCATE/DML-through-the-
view statement in the first connection acquired SR lock on a
table, then ALTER TABLE would come in in the second connection
and acquire SNW metadata lock and TL_WRITE_ALLOW_READ
thr_lock.c lock and then would start waiting for the first
connection during lock upgrade. After that the statement in
the first connection would try to acquire TL_WRITE lock on
table and would start waiting for the second connection,
creating a deadlock.
This patch solves this problem by ensuring that we acquire
SW metadata lock in all cases in which we acquiring write
thr_lock.c lock. This guarantees that deadlocks like the
one described above won't occur since all lock conflicts
in such situation are resolved within MDL subsystem.
This patch also adds assert which should guarantee that
such situations won't arise in future.
mysql-test/r/lock_multi.result:
Added main test for bug #50913 "Deadlock between
open_and_lock_tables_derived and MDL".
mysql-test/r/mdl_sync.result:
Added additional coverage for bug #50913 "Deadlock
between open_and_lock_tables_derived and MDL".
mysql-test/t/lock_multi.test:
Added main test for bug #50913 "Deadlock between
open_and_lock_tables_derived and MDL".
mysql-test/t/mdl_sync.test:
Added additional coverage for bug #50913 "Deadlock
between open_and_lock_tables_derived and MDL".
sql/lock.cc:
Added assert that enforces that when we are locking
a non-temporary table we have an appropriate type of
metadata lock on this table.
sql/mysql_priv.h:
Added separate flag for open_tables() to be able specify that
SH metadata locks on table to be open should be acquired.
We can no longer use MYSQL_LOCK_IGNORE_FLUSH flag for this
as in addition to use in I_S implementation it is also used
for opening system tables. Since in the latter case we also
acquire thr_lock.c locks using SH metadata lock in it instead
of SR or SW locks may lead to deadlock.
sql/sql_base.cc:
When opening tables don't interpret MYSQL_LOCK_IGNORE_FLUSH
flag as request to acquire SH metadata locks. This flag is
also used for opening system tables for which we also take
thr_lock.c locks and thus proper metadata lock to take in
this case is SR or SW lock (otherwise deadlocks can occur).
In cases when SH lock is really required (e.g. when tables
are open by I_S implementation) we rely on that newly
introduced MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag is
used.
sql/sql_delete.cc:
mysql_truncate_by_delete():
Adjust type of metadata lock to be requested after changing
type of thr_lock.c lock for table list element from one
which was set in parser to TL_WRITE.
This removes discrepancy between types of these locks which
allowed deadlocks to creep in.
sql/sql_handler.cc:
When closing table which was open by HANDLER statement clear
TABLE::open_by_handler flag. This allows to use this flag as
a reliable indication that TABLE instance was open by HANDLER
statement in assert which was added to mysql_lock_tables().
sql/sql_parse.cc:
multi_delete_set_locks_and_link_aux_tables():
Adjust type of metadata lock to be requested after changing
type of thr_lock.c lock for table list element from one
which was set in parser to TL_WRITE.
This removes discrepancy between types of these locks which
allowed deadlocks to creep in.
sql/sql_show.cc:
Use newly introduced MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL
flag in order to acquire SH metadata locks when opening tables
in I_S implementation.
sql/sql_update.cc:
Added comment explaining why in multi-update after deciding
that we need weaker thr_lock.c lock on a table we don't
downgrade metadata lock on it.
sql/sql_view.cc:
When merging view into main statement adjust type of metadata
lock to be requested after changing type of thr_lock.c lock
for table. This removes discrepancy between types of these
locks which allowed deadlocks to creep in.
in message printed at end of configure
New text for the success message of "configure".
configure.in:
The message must be changed to drop the "www.mysql.com" URL.
mysql-test/t/disabled.def:
Restore disabled ssl tests: SSL certificates were updated.
Disable sp_sync.test, the test case can't work in next-4284.
mysql-test/t/partition_innodb.test:
Disable parsing of the test case for Bug#47343,
the test can not work in next-4284.
mysql-test/t/ps_ddl.test:
Update results (CREATE TABLE IF NOT EXISTS takes
into account existence of the temporary table).
failed on HANDLER + I_S
This assert was triggered when an I_S query tried to acquire a
metadata lock on a table which was already locked by a HANDLER
statement in the same connection.
First the HANDLER took a MDL_SHARED lock. Afterwards, the I_S query
requested a MDL_SHARED_HIGH_PRIO lock. The existing MDL_SHARED ticket
is found in find_ticket() since it satisfies
ticket->has_stronger_or_equal_type(mdl_request->type) as MDL_SHARED
and MDL_SHARED_HIGH_PRIO have equal strengths, just different priority.
However, two asserts later check lock type strengths using relational
operators (>= and <=) rather than MDL_ticket::has_stronger_or_equal_type().
These asserts are triggered since MDL_SHARED >= MDL_SHARED_HIGH_PRIORITY
is false (mapped to 1 and 2 respectively).
This patch updates the asserts to use MDL_ticket::has_stronger_or_equal_type()
rather than relational operators to check lock type strength.
Test case added to include/handler.inc.
After merge fixes.
Adjust replication test cases.
mysql-test/suite/rpl/r/rpl_mixed_row_innodb.result:
Update results with a new test.
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
Add a warning, which I believe is an expected one.
mysql-test/suite/rpl/t/rpl_killed_ddl.test:
Sort results to avoid test failurs under load.
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
Update results (next-4284 merge).
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
Adjust test output to the new table opening scheme: decide_logging_format() is now called in CREATE/DROP trigger.
Original revision:
------------------------------------------------------------
revision-id: kent.boortz@sun.com-20100204182709-dw1dwpglkd5qrehb
committer: Kent Boortz <kent.boortz@sun.com>
branch nick: mysql-5.1-bugteam
timestamp: Thu 2010-02-04 19:27:09 +0100
message:
LT_INIT and LT_PREREQ was added in libtool 2.2 2008, a bit too
recent, switched back to the older AC_PROG_LIBTOOL
------------------------------------------------------------
Change the error code for ER_WARN_I_S_SKIPPED_TABLE, to not
upset the tests that rely on ER_SLAVE_CONVERSION_ERROR error
code = 1667.
Fix a merge bug with binlogging of CREATE TABLE (temporary tables).
HANDLER OPEN
The problem was a too restrictive assert in the code for
HANDLER ... OPEN and HANDLER ... READ that checked table->next
to verify that we didn't open views or merge tables.
This pointer is also used to link temporary tables together
(see thd->temporary_tables). In this case TABLE::next can be
set even if we're trying to open a single table.
This patch adjust the two asserts to also check for the presence
of temporary tables.
Test case added to handler_myisam.test.
fix a merge bug when write_bin_log called from mysql_routine_grant()
would chew up the error.
rpl_do_grant test would fail on assert that the diagnostics area is
empty.
Cherry-pick a fix Bug#37148 from next-mr, to preserve
file ids of the added files, and ensure that all the necessary
changes have been pulled.
Since initially Bug#37148 was null-merged into 6.0,
the changeset that is now being cherry-picked was likewise
null merged into next-4284.
Now that Bug#37148 has been reapplied to 6.0, try to make
it work with next-4284. This is also necessary to be able
to pull other changes from 5.1-rep into next-4284.
To resolve the merge issues use this changeset applied
to 6.0:
revid:jperkin@sun.com-20091216103628-ylhqf7s6yegui2t9
revno: 3776.1.1
committer: He Zhenxing <zhenxing.he@sun.com>
branch nick: 6.0-codebase-bugfixing
timestamp: Thu 2009-12-17 17:02:50 +0800
message:
Fix merge problem with Bug#37148
Removed local variables which became unused when we
have switched to new approach for CREATE TABLE LIKE
(i.e. abondoned .FRM file copying) and were causing
warnings during compilation.
This was a deadlock between ALTER TABLE and another DML statement
(or LOCK TABLES ... READ). ALTER TABLE would wait trying to upgrade
its lock to MDL_EXCLUSIVE and the DML statement would wait trying
to acquire a TL_READ_NO_INSERT table level lock.
This could happen if one connection first acquired a MDL_SHARED_READ
lock on a table. In another connection ALTER TABLE is then started.
ALTER TABLE eventually blocks trying to upgrade to MDL_EXCLUSIVE,
but while holding a TL_WRITE_ALLOW_READ table level lock.
If the first connection then tries to acquire TL_READ_NO_INSERT,
it will block and we have a deadlock since neither connection can
proceed.
This patch fixes the problem by allowing TL_READ_NO_INSERT
locks to be granted if another connection holds TL_WRITE_ALLOW_READ
on the same table. This will allow the DML statement to proceed
such that it eventually can release its MDL lock which in turn
makes ALTER TABLE able to proceed.
Note that TL_READ_NO_INSERT was already partially compatible with
TL_WRITE_ALLOW_READ as the latter would be granted if the former
lock was held. This patch just makes the opposite true as well.
Also note that since ALTER TABLE takes an upgradable MDL lock,
there will be no starvation of ALTER TABLE statements by
statements acquiring TL_READ or TL_READ_NO_INSERT.
Test case added to lock_sync.test.
