mariadb/mysql-test/r/mdl_sync.result

2945 lines
83 KiB
Text
Raw Normal View History

Backport of: ------------------------------------------------------------ revno: 2617.23.18 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Mon 2009-03-02 18:18:26 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking This is a prerequisite patch: These changes are intended to split lock requests from granted locks and to allow the memory and lifetime of granted locks to be managed within the MDL subsystem. Furthermore, tickets can now be shared and therefore are used to satisfy multiple lock requests, but only shared locks can be recursive. The problem is that the MDL subsystem morphs lock requests into granted locks locks but does not manage the memory and lifetime of lock requests, and hence, does not manage the memory of granted locks either. This can be problematic because it puts the burden of tracking references on the users of the subsystem and it can't be easily done in transactional contexts where the locks have to be kept around for the duration of a transaction. Another issue is that recursive locks (when the context trying to acquire a lock already holds a lock on the same object) requires that each time the lock is granted, a unique lock request/granted lock structure structure must be kept around until the lock is released. This can lead to memory leaks in transactional contexts as locks taken during the transaction should only be released at the end of the transaction. This also leads to unnecessary wake ups (broadcasts) in the MDL subsystem if the context still holds a equivalent of the lock being released. These issues are exacerbated due to the fact that WL#4284 low-level design says that the implementation should "2) Store metadata locks in transaction memory root, rather than statement memory root" but this is not possible because a memory root, as implemented in mysys, requires all objects allocated from it to be freed all at once. This patch combines review input and significant code contributions from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev).
2009-12-04 02:29:40 +03:00
SET DEBUG_SYNC= 'RESET';
drop table if exists t1,t2,t3;
create table t1 (i int);
create table t2 (i int);
connection: default
lock tables t2 read;
connection: con1
set debug_sync='mdl_upgrade_shared_lock_to_exclusive SIGNAL parked WAIT_FOR go';
alter table t1 rename t3;
connection: default
set debug_sync= 'now WAIT_FOR parked';
connection: con2
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
set debug_sync='mdl_acquire_lock_wait SIGNAL go';
Backport of: ------------------------------------------------------------ revno: 2617.23.18 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Mon 2009-03-02 18:18:26 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking This is a prerequisite patch: These changes are intended to split lock requests from granted locks and to allow the memory and lifetime of granted locks to be managed within the MDL subsystem. Furthermore, tickets can now be shared and therefore are used to satisfy multiple lock requests, but only shared locks can be recursive. The problem is that the MDL subsystem morphs lock requests into granted locks locks but does not manage the memory and lifetime of lock requests, and hence, does not manage the memory of granted locks either. This can be problematic because it puts the burden of tracking references on the users of the subsystem and it can't be easily done in transactional contexts where the locks have to be kept around for the duration of a transaction. Another issue is that recursive locks (when the context trying to acquire a lock already holds a lock on the same object) requires that each time the lock is granted, a unique lock request/granted lock structure structure must be kept around until the lock is released. This can lead to memory leaks in transactional contexts as locks taken during the transaction should only be released at the end of the transaction. This also leads to unnecessary wake ups (broadcasts) in the MDL subsystem if the context still holds a equivalent of the lock being released. These issues are exacerbated due to the fact that WL#4284 low-level design says that the implementation should "2) Store metadata locks in transaction memory root, rather than statement memory root" but this is not possible because a memory root, as implemented in mysys, requires all objects allocated from it to be freed all at once. This patch combines review input and significant code contributions from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev).
2009-12-04 02:29:40 +03:00
drop table t1,t2;
connection: con1
connection: default
unlock tables;
connection: con2
ERROR 42S02: Unknown table 't1'
drop table t3;
SET DEBUG_SYNC= 'RESET';
#
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
# Basic test coverage for type-of-operation aware metadata locks.
#
drop table if exists t1, t2, t3;
set debug_sync= 'RESET';
create table t1 (c1 int);
#
# A) First let us check compatibility rules between differend kinds of
# type-of-operation aware metadata locks.
# Of course, these rules are already covered by the tests scattered
# across the test suite. But it still makes sense to have one place
# which covers all of them.
#
# 1) Acquire S (simple shared) lock on the table (by using HANDLER):
#
handler t1 open;
#
# Switching to connection 'mdl_con1'.
# Check that S, SH, SR and SW locks are compatible with it.
handler t1 open t;
handler t close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
select count(*) from t1;
count(*)
0
insert into t1 values (1), (1);
# Check that SNW lock is compatible with it. To do this use ALTER TABLE
# which will fail after opening the table and thus obtaining SNW metadata
# lock.
alter table t1 add primary key (c1);
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
# Check that SNRW lock is compatible with S lock.
lock table t1 write;
insert into t1 values (1);
unlock tables;
# Check that X lock is incompatible with S lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above RENAME is blocked because of S lock.
#
# Switching to connection 'default'.
# Unblock RENAME TABLE.
handler t1 close;
#
# Switching to connection 'mdl_con1'.
# Reaping RENAME TABLE.
# Restore the original state of the things.
rename table t2 to t1;
#
# Switching to connection 'default'.
handler t1 open;
#
# Switching to connection 'mdl_con1'.
# Check that upgrade from SNW to X is blocked by presence of S lock.
# Sending:
alter table t1 add column c2 int;;
#
# Switching to connection 'mdl_con2'.
# Check that the above ALTER TABLE is blocked because of S lock.
#
# Switching to connection 'default'.
# Unblock ALTER TABLE.
handler t1 close;
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
# Restore the original state of the things.
alter table t1 drop column c2;
#
# Switching to connection 'default'.
handler t1 open;
#
# Switching to connection 'mdl_con1'.
# Check that upgrade from SNRW to X is blocked by presence of S lock.
lock table t1 write;
# Sending:
alter table t1 add column c2 int;;
#
# Switching to connection 'mdl_con2'.
# Check that the above upgrade of SNRW to X in ALTER TABLE is blocked
# because of S lock.
#
# Switching to connection 'default'.
# Unblock ALTER TABLE.
handler t1 close;
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
# Restore the original state of the things.
alter table t1 drop column c2;
unlock tables;
#
# Switching to connection 'default'.
#
# 2) Acquire SH (shared high-priority) lock on the table.
# We have to involve DEBUG_SYNC facility for this as usually
# such kind of locks are short-lived.
#
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
select table_name, table_type, auto_increment, table_comment from information_schema.tables where table_schema='test' and table_name='t1';;
#
# Switching to connection 'mdl_con1'.
set debug_sync= 'now WAIT_FOR locked';
# Check that S, SH, SR and SW locks are compatible with it.
handler t1 open;
handler t1 close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
select count(*) from t1;
count(*)
3
insert into t1 values (1);
# Check that SNW lock is compatible with it. To do this use ALTER TABLE
# which will fail after opening the table and thus obtaining SNW metadata
# lock.
alter table t1 add primary key (c1);
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
# Check that SNRW lock is compatible with SH lock.
lock table t1 write;
delete from t1 limit 1;
unlock tables;
# Check that X lock is incompatible with SH lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above RENAME is blocked because of SH lock.
# Unblock RENAME TABLE.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'default'.
# Reaping SELECT ... FROM I_S.
table_name table_type auto_increment table_comment
t1 BASE TABLE NULL
#
# Switching to connection 'mdl_con1'.
# Reaping RENAME TABLE.
# Restore the original state of the things.
rename table t2 to t1;
#
# Switching to connection 'default'.
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
select table_name, table_type, auto_increment, table_comment from information_schema.tables where table_schema='test' and table_name='t1';;
#
# Switching to connection 'mdl_con1'.
set debug_sync= 'now WAIT_FOR locked';
# Check that upgrade from SNW to X is blocked by presence of SH lock.
# Sending:
alter table t1 add column c2 int;;
#
# Switching to connection 'mdl_con2'.
# Check that the above ALTER TABLE is blocked because of SH lock.
# Unblock RENAME TABLE.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'default'.
# Reaping SELECT ... FROM I_S.
table_name table_type auto_increment table_comment
t1 BASE TABLE NULL
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
# Restore the original state of the things.
alter table t1 drop column c2;
#
# Switching to connection 'default'.
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
select table_name, table_type, auto_increment, table_comment from information_schema.tables where table_schema='test' and table_name='t1';;
#
# Switching to connection 'mdl_con1'.
set debug_sync= 'now WAIT_FOR locked';
# Check that upgrade from SNRW to X is blocked by presence of S lock.
lock table t1 write;
# Sending:
alter table t1 add column c2 int;;
#
# Switching to connection 'mdl_con2'.
# Check that the above upgrade of SNRW to X in ALTER TABLE is blocked
# because of S lock.
# Unblock RENAME TABLE.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'default'.
# Reaping SELECT ... FROM I_S.
table_name table_type auto_increment table_comment
t1 BASE TABLE NULL
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
# Restore the original state of the things.
alter table t1 drop column c2;
unlock tables;
#
# Switching to connection 'default'.
#
#
# 3) Acquire SR lock on the table.
#
#
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Check that S, SH, SR and SW locks are compatible with it.
handler t1 open;
handler t1 close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
select count(*) from t1;
count(*)
3
insert into t1 values (1);
# Check that SNW lock is compatible with it. To do this use ALTER TABLE
# which will fail after opening the table and thus obtaining SNW metadata
# lock.
alter table t1 add primary key (c1);
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
# Check that SNRW lock is not compatible with SR lock.
# Sending:
lock table t1 write;;
#
# Switching to connection 'default'.
# Check that the above LOCK TABLES is blocked because of SR lock.
# Unblock LOCK TABLES.
commit;
#
# Switching to connection 'mdl_con1'.
# Reaping LOCK TABLES.
delete from t1 limit 1;
unlock tables;
#
# Switching to connection 'default'.
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Check that X lock is incompatible with SR lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above RENAME is blocked because of SR lock.
#
# Switching to connection 'default'.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reaping RENAME TABLE.
# Restore the original state of the things.
rename table t2 to t1;
#
# Switching to connection 'default'.
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Check that upgrade from SNW to X is blocked by presence of SR lock.
# Sending:
alter table t1 add column c2 int;;
#
# Switching to connection 'mdl_con2'.
# Check that the above ALTER TABLE is blocked because of SR lock.
#
# Switching to connection 'default'.
# Unblock ALTER TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
# Restore the original state of the things.
alter table t1 drop column c2;
#
# There is no need to check that upgrade from SNRW to X is blocked
# by presence of SR lock because SNRW is incompatible with SR anyway.
#
#
# Switching to connection 'default'.
#
#
# 4) Acquire SW lock on the table.
#
#
begin;
insert into t1 values (1);
#
# Switching to connection 'mdl_con1'.
# Check that S, SH, SR and SW locks are compatible with it.
handler t1 open;
handler t1 close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
# Disable result log to make test robust against
# effects of concurrent insert.
select * from t1;
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
insert into t1 values (1);
# Check that SNW lock is not compatible with SW lock.
# Again we use ALTER TABLE which fails after opening
# the table to avoid upgrade of SNW -> X.
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'default'.
# Check that the above ALTER TABLE is blocked because of SW lock.
# Unblock ALTER TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'default'.
begin;
insert into t1 values (1);
#
# Switching to connection 'mdl_con1'.
# Check that SNRW lock is not compatible with SW lock.
# Sending:
lock table t1 write;;
#
# Switching to connection 'default'.
# Check that the above LOCK TABLES is blocked because of SW lock.
# Unblock LOCK TABLES.
commit;
#
# Switching to connection 'mdl_con1'.
# Reaping LOCK TABLES.
delete from t1 limit 2;
unlock tables;
#
# Switching to connection 'default'.
begin;
insert into t1 values (1);
#
# Switching to connection 'mdl_con1'.
# Check that X lock is incompatible with SW lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above RENAME is blocked because of SW lock.
#
# Switching to connection 'default'.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reaping RENAME TABLE.
# Restore the original state of the things.
rename table t2 to t1;
#
# There is no need to check that upgrade from SNW/SNRW to X is
# blocked by presence of SW lock because SNW/SNRW is incompatible
# with SW anyway.
#
#
# Switching to connection 'default'.
#
#
# 5) Acquire SNW lock on the table. We have to use DEBUG_SYNC for
# this, to prevent SNW from being immediately upgraded to X.
#
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con1'.
set debug_sync= 'now WAIT_FOR locked';
# Check that S, SH and SR locks are compatible with it.
handler t1 open;
handler t1 close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
select count(*) from t1;
count(*)
5
# Check that SW lock is incompatible with SNW lock.
# Sending:
delete from t1 limit 2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above DELETE is blocked because of SNW lock.
# Unblock ALTER and thus DELETE.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'default'.
# Reaping ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'mdl_con1'.
# Reaping DELETE.
#
# Switching to connection 'default'.
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con1'.
set debug_sync= 'now WAIT_FOR locked';
# Check that SNW lock is incompatible with SNW lock.
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con2'.
# Check that the above ALTER is blocked because of SNW lock.
# Unblock ALTERs.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'default'.
# Reaping first ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'mdl_con1'.
# Reaping another ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'default'.
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con1'.
set debug_sync= 'now WAIT_FOR locked';
# Check that SNRW lock is incompatible with SNW lock.
# Sending:
lock table t1 write;;
#
# Switching to connection 'mdl_con2'.
# Check that the above LOCK TABLES is blocked because of SNW lock.
# Unblock ALTER and thus LOCK TABLES.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'default'.
# Reaping ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'mdl_con1'.
# Reaping LOCK TABLES
insert into t1 values (1);
unlock tables;
#
# Switching to connection 'default'.
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con1'.
set debug_sync= 'now WAIT_FOR locked';
# Check that X lock is incompatible with SNW lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above RENAME is blocked because of SNW lock.
# Unblock ALTER and thus RENAME TABLE.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'default'.
# Reaping ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'mdl_con1'.
# Reaping RENAME TABLE
# Revert back to original state of things.
rename table t2 to t1;
#
# There is no need to check that upgrade from SNW/SNRW to X is
# blocked by presence of another SNW lock because SNW/SNRW is
# incompatible with SNW anyway.
#
# Switching to connection 'default'.
#
#
# 6) Acquire SNRW lock on the table.
#
#
lock table t1 write;
#
# Switching to connection 'mdl_con1'.
# Check that S and SH locks are compatible with it.
handler t1 open;
handler t1 close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
# Check that SR lock is incompatible with SNRW lock.
# Sending:
select count(*) from t1;;
#
# Switching to connection 'default'.
# Check that the above SELECT is blocked because of SNRW lock.
# Unblock SELECT.
unlock tables;
#
# Switching to connection 'mdl_con1'.
# Reaping SELECT.
count(*)
4
#
# Switching to connection 'default'.
lock table t1 write;
#
# Switching to connection 'mdl_con1'.
# Check that SW lock is incompatible with SNRW lock.
# Sending:
delete from t1 limit 1;;
#
# Switching to connection 'default'.
# Check that the above DELETE is blocked because of SNRW lock.
# Unblock DELETE.
unlock tables;
#
# Switching to connection 'mdl_con1'.
# Reaping DELETE.
#
# Switching to connection 'default'.
lock table t1 write;
#
# Switching to connection 'mdl_con1'.
# Check that SNW lock is incompatible with SNRW lock.
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'default'.
# Check that the above ALTER is blocked because of UNWR lock.
# Unblock ALTER.
unlock tables;
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'default'.
lock table t1 write;
#
# Switching to connection 'mdl_con1'.
# Check that SNRW lock is incompatible with SNRW lock.
# Sending:
lock table t1 write;;
#
# Switching to connection 'default'.
# Check that the above LOCK TABLES is blocked because of SNRW lock.
# Unblock waiting LOCK TABLES.
unlock tables;
#
# Switching to connection 'mdl_con1'.
# Reaping LOCK TABLES
insert into t1 values (1);
unlock tables;
#
# Switching to connection 'default'.
lock table t1 write;
#
# Switching to connection 'mdl_con1'.
# Check that X lock is incompatible with SNRW lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'default'.
# Check that the above RENAME is blocked because of SNRW lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'mdl_con1'.
# Reaping RENAME TABLE
# Revert back to original state of things.
rename table t2 to t1;
#
# There is no need to check that upgrade from SNW/SNRW to X is
# blocked by presence of another SNRW lock because SNW/SNRW is
# incompatible with SNRW anyway.
#
# Switching to connection 'default'.
#
#
# 7) Now do the same round of tests for X lock. We use additional
# table to get long-lived lock of this type.
#
create table t2 (c1 int);
#
# Switching to connection 'mdl_con2'.
# Take a lock on t2, so RENAME TABLE t1 TO t2 will get blocked
# after acquiring X lock on t1.
lock tables t2 read;
#
# Switching to connection 'default'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME has acquired X lock on t1 and is waiting for t2.
# Check that S lock in incompatible with X lock.
# Sending:
handler t1 open;;
#
# Switching to connection 'mdl_con2'.
# Check that the above HANDLER statement is blocked because of X lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping HANDLER.
handler t1 close;
#
# Switching to connection 'mdl_con2'.
# Prepare for blocking RENAME TABLE.
lock tables t2 read;
#
# Switching to connection 'default'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME has acquired X lock on t1 and is waiting for t2.
# Check that SH lock in incompatible with X lock.
# Sending:
select column_name from information_schema.columns where table_schema='test' and table_name='t1';;
#
# Switching to connection 'mdl_con2'.
# Check that the above SELECT ... FROM I_S ... statement is blocked
# because of X lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping SELECT ... FROM I_S.
column_name
c1
#
# Switching to connection 'mdl_con2'.
# Prepare for blocking RENAME TABLE.
lock tables t2 read;
#
# Switching to connection 'default'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME has acquired X lock on t1 and is waiting for t2.
# Check that SR lock in incompatible with X lock.
# Sending:
select count(*) from t1;;
#
# Switching to connection 'mdl_con2'.
# Check that the above SELECT statement is blocked
# because of X lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping SELECT.
count(*)
4
#
# Switching to connection 'mdl_con2'.
# Prepare for blocking RENAME TABLE.
lock tables t2 read;
#
# Switching to connection 'default'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME has acquired X lock on t1 and is waiting for t2.
# Check that SW lock in incompatible with X lock.
# Sending:
delete from t1 limit 1;;
#
# Switching to connection 'mdl_con2'.
# Check that the above DELETE statement is blocked
# because of X lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping DELETE.
#
# Switching to connection 'mdl_con2'.
# Prepare for blocking RENAME TABLE.
lock tables t2 read;
#
# Switching to connection 'default'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME has acquired X lock on t1 and is waiting for t2.
# Check that SNW lock is incompatible with X lock.
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con2'.
# Check that the above ALTER statement is blocked
# because of X lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'mdl_con2'.
# Prepare for blocking RENAME TABLE.
lock tables t2 read;
#
# Switching to connection 'default'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME has acquired X lock on t1 and is waiting for t2.
# Check that SNRW lock is incompatible with X lock.
# Sending:
lock table t1 write;;
#
# Switching to connection 'mdl_con2'.
# Check that the above LOCK TABLE statement is blocked
# because of X lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping LOCK TABLE.
unlock tables;
#
# Switching to connection 'mdl_con2'.
# Prepare for blocking RENAME TABLE.
lock tables t2 read;
#
# Switching to connection 'default'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME has acquired X lock on t1 and is waiting for t2.
# Check that X lock is incompatible with X lock.
# Sending:
rename table t1 to t3;;
#
# Switching to connection 'mdl_con2'.
# Check that the above RENAME statement is blocked
# because of X lock.
# Unblock RENAME TABLE
unlock tables;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping RENAME.
rename table t3 to t1;
#
# B) Now let us test compatibility in cases when both locks
# are pending. I.e. let us test rules for priorities between
# different types of metadata locks.
#
#
# Switching to connection 'mdl_con2'.
#
# 1) Check compatibility for pending SNW lock.
#
# Acquire SW lock in order to create pending SNW lock later.
begin;
insert into t1 values (1);
#
# Switching to connection 'default'.
# Add pending SNW lock.
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con1'.
# Check that ALTER TABLE is waiting with pending SNW lock.
# Check that S, SH and SR locks are compatible with pending SNW
handler t1 open t;
handler t close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
select count(*) from t1;
count(*)
4
# Check that SW is incompatible with pending SNW
# Sending:
delete from t1 limit 1;;
#
# Switching to connection 'mdl_con2'.
# Check that the above DELETE is blocked because of pending SNW lock.
# Unblock ALTER TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping ALTER.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'mdl_con1'.
# Reaping DELETE.
#
# We can't do similar check for SNW, SNRW and X locks because
# they will also be blocked by active SW lock.
#
#
# Switching to connection 'mdl_con2'.
#
# 2) Check compatibility for pending SNRW lock.
#
# Acquire SR lock in order to create pending SNRW lock.
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'default'.
# Add pending SNRW lock.
# Sending:
lock table t1 write;;
#
# Switching to connection 'mdl_con1'.
# Check that LOCK TABLE is waiting with pending SNRW lock.
# Check that S and SH locks are compatible with pending SNRW
handler t1 open t;
handler t close;
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
# Check that SR is incompatible with pending SNRW
# Sending:
select count(*) from t1;;
#
# Switching to connection 'mdl_con2'.
# Check that the above SELECT is blocked because of pending SNRW lock.
# Unblock LOCK TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping LOCK TABLE.
unlock tables;
#
# Switching to connection 'mdl_con1'.
# Reaping SELECT.
count(*)
3
# Restore pending SNRW lock.
#
# Switching to connection 'mdl_con2'.
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'default'.
# Sending:
lock table t1 write;;
#
# Switching to connection 'mdl_con1'.
# Check that LOCK TABLE is waiting with pending SNRW lock.
# Check that SW is incompatible with pending SNRW
# Sending:
insert into t1 values (1);;
#
# Switching to connection 'mdl_con2'.
# Check that the above INSERT is blocked because of pending SNRW lock.
# Unblock LOCK TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping LOCK TABLE.
unlock tables;
#
# Switching to connection 'mdl_con1'.
# Reaping INSERT.
# Restore pending SNRW lock.
#
# Switching to connection 'mdl_con2'.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'default'.
# Sending:
lock table t1 write;;
#
# Switching to connection 'mdl_con1'.
# Check that LOCK TABLE is waiting with pending SNRW lock.
# Check that SNW is compatible with pending SNRW
# So ALTER TABLE statements are not starved by LOCK TABLEs.
alter table t1 add primary key (c1);
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'mdl_con2'.
# Unblock LOCK TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping LOCK TABLE.
unlock tables;
#
# We can't do similar check for SNRW and X locks because
# they will also be blocked by active SR lock.
#
#
# Switching to connection 'mdl_con2'.
#
# 3) Check compatibility for pending X lock.
#
# Acquire SR lock in order to create pending X lock.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'default'.
# Add pending X lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME TABLE is waiting with pending X lock.
# Check that SH locks are compatible with pending X
select column_name from information_schema.columns where
table_schema='test' and table_name='t1';
column_name
c1
# Check that S is incompatible with pending X
# Sending:
handler t1 open;;
#
# Switching to connection 'mdl_con2'.
# Check that the above HANDLER OPEN is blocked because of pending X lock.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping HANDLER t1 OPEN.
handler t1 close;
# Restore pending X lock.
#
# Switching to connection 'mdl_con2'.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'default'.
# Add pending X lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME TABLE is waiting with pending X lock.
# Check that SR is incompatible with pending X
# Sending:
select count(*) from t1;;
#
# Switching to connection 'mdl_con2'.
# Check that the above SELECT is blocked because of pending X lock.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping SELECT.
count(*)
4
# Restore pending X lock.
#
# Switching to connection 'mdl_con2'.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'default'.
# Add pending X lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME TABLE is waiting with pending X lock.
# Check that SW is incompatible with pending X
# Sending:
delete from t1 limit 1;;
#
# Switching to connection 'mdl_con2'.
# Check that the above DELETE is blocked because of pending X lock.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping DELETE.
# Restore pending X lock.
#
# Switching to connection 'mdl_con2'.
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'default'.
# Add pending X lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME TABLE is waiting with pending X lock.
# Check that SNW is incompatible with pending X
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'mdl_con2'.
# Check that the above ALTER TABLE is blocked because of pending X lock.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
# Restore pending X lock.
#
# Switching to connection 'mdl_con2'.
handler t1 open;
#
# Switching to connection 'default'.
# Add pending X lock.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'mdl_con1'.
# Check that RENAME TABLE is waiting with pending X lock.
# Check that SNRW is incompatible with pending X
# Sending:
lock table t1 write;;
#
# Switching to connection 'mdl_con3'.
# Check that the above LOCK TABLES is blocked because of pending X lock.
#
# Switching to connection 'mdl_con2'.
# Unblock RENAME TABLE.
handler t1 close;
#
# Switching to connection 'default'.
# Reaping RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'mdl_con1'.
# Reaping LOCK TABLES.
unlock tables;
#
# Switching to connection 'default'.
#
#
# C) Now let us test how type-of-operation locks are handled in
# transactional context. Obviously we are mostly interested
# in conflicting types of locks.
#
#
# 1) Let us check how various locks used within transactional
# context interact with active/pending SNW lock.
#
# We start with case when we are acquiring lock on the table
# which was not used in the transaction before.
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Create an active SNW lock on t2.
# We have to use DEBUG_SYNC facility as otherwise SNW lock
# will be immediately released (or upgraded to X lock).
insert into t2 values (1), (1);
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
alter table t2 add primary key (c1);;
#
# Switching to connection 'default'.
set debug_sync= 'now WAIT_FOR locked';
# SR lock should be acquired without any waiting.
select count(*) from t2;
count(*)
2
commit;
# Now let us check that we will wait in case of SW lock.
begin;
select count(*) from t1;
count(*)
3
# Sending:
insert into t2 values (1);;
#
# Switching to connection 'mdl_con2'.
# Check that the above INSERT is blocked.
# Unblock ALTER TABLE and thus INSERT.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'mdl_con1'.
# Reap ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'default'.
# Reap INSERT.
commit;
#
# Now let us see what happens when we are acquiring lock on the table
# which is already used in transaction.
#
# *) First, case when transaction which has SR lock on the table also
# locked in SNW mode acquires yet another SR lock and then tries
# to acquire SW lock.
begin;
select count(*) from t1;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Create an active SNW lock on t1.
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'default'.
set debug_sync= 'now WAIT_FOR locked';
# We should still be able to get SR lock without waiting.
select count(*) from t1;
count(*)
3
# Since the above ALTER TABLE is not upgrading SNW lock to X by waiting
# for SW lock we won't create deadlock.
# So the below INSERT should not end-up with ER_LOCK_DEADLOCK error.
# Sending:
insert into t1 values (1);;
#
# Switching to connection 'mdl_con2'.
# Check that the above INSERT is blocked.
# Unblock ALTER TABLE and thus INSERT.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'mdl_con1'.
# Reap ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'default'.
# Reap INSERT.
commit;
#
# **) Now test in which transaction that has SW lock on the table
# against which there is pending SNW lock acquires SR and SW
# locks on this table.
#
begin;
insert into t1 values (1);
#
# Switching to connection 'mdl_con1'.
# Create pending SNW lock on t1.
# Sending:
alter table t1 add primary key (c1);;
#
# Switching to connection 'default'.
# Wait until ALTER TABLE starts waiting for SNW lock.
# We should still be able to get both SW and SR locks without waiting.
select count(*) from t1;
count(*)
5
delete from t1 limit 1;
# Unblock ALTER TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap ALTER TABLE.
ERROR 23000: Duplicate entry '1' for key 'PRIMARY'
#
# Switching to connection 'default'.
#
# 2) Now similar tests for active SNW lock which is being upgraded
# to X lock.
#
# Again we start with case when we are acquiring lock on the
# table which was not used in the transaction before.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con2'.
# Start transaction which will prevent SNW -> X upgrade from
# completing immediately.
begin;
select count(*) from t2;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Create SNW lock pending upgrade to X on t2.
# Sending:
alter table t2 add column c2 int;;
#
# Switching to connection 'default'.
# Wait until ALTER TABLE starts waiting X lock.
# Check that attempt to acquire SR lock on t2 causes waiting.
# Sending:
select count(*) from t2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above SELECT is blocked.
# Unblock ALTER TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap ALTER TABLE.
#
# Switching to connection 'default'.
# Reap SELECT.
count(*)
3
commit;
# Do similar check for SW lock.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con2'.
# Start transaction which will prevent SNW -> X upgrade from
# completing immediately.
begin;
select count(*) from t2;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Create SNW lock pending upgrade to X on t2.
# Sending:
alter table t2 drop column c2;;
#
# Switching to connection 'default'.
# Wait until ALTER TABLE starts waiting X lock.
# Check that attempt to acquire SW lock on t2 causes waiting.
# Sending:
insert into t2 values (1);;
#
# Switching to connection 'mdl_con2'.
# Check that the above INSERT is blocked.
# Unblock ALTER TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap ALTER TABLE.
#
# Switching to connection 'default'.
# Reap INSERT.
commit;
#
# Test for the case in which we are acquiring lock on the table
# which is already used in transaction.
#
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con1'.
# Create SNW lock pending upgrade to X.
# Sending:
alter table t1 add column c2 int;;
#
# Switching to connection 'default'.
# Wait until ALTER TABLE starts waiting X lock.
# Check that transaction is still able to acquire SR lock.
select count(*) from t1;
count(*)
4
# Waiting trying to acquire SW lock will cause deadlock and
# therefore should cause an error.
delete from t1 limit 1;
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
# Unblock ALTER TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap ALTER TABLE.
#
# Switching to connection 'default'.
#
# 3) Check how various locks used within transactional context
# interact with active/pending SNRW lock.
#
# Once again we start with case when we are acquiring lock on
# the table which was not used in the transaction before.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con1'.
lock table t2 write;
#
# Switching to connection 'default'.
# Attempt to acquire SR should be blocked. It should
# not cause errors as it does not creates deadlock.
# Sending:
select count(*) from t2;;
#
# Switching to connection 'mdl_con1'.
# Check that the above SELECT is blocked
# Unblock SELECT.
unlock tables;
#
# Switching to connection 'default'.
# Reap SELECT.
count(*)
4
commit;
# Repeat the same test for SW lock.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con1'.
lock table t2 write;
#
# Switching to connection 'default'.
# Again attempt to acquire SW should be blocked and should
# not cause any errors.
# Sending:
delete from t2 limit 1;;
#
# Switching to connection 'mdl_con1'.
# Check that the above DELETE is blocked
# Unblock DELETE.
unlock tables;
#
# Switching to connection 'default'.
# Reap DELETE.
commit;
#
# Now coverage for the case in which we are acquiring lock on
# the table which is already used in transaction and against
# which there is a pending SNRW lock request.
#
# *) Let us start with case when transaction has only a SR lock.
#
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con1'.
# Sending:
lock table t1 write;;
#
# Switching to connection 'default'.
# Wait until LOCK TABLE is blocked creating pending request for X lock.
# Check that another instance of SR lock is granted without waiting.
select count(*) from t1;
count(*)
4
# Attempt to wait for SW lock will lead to deadlock, thus
# the below statement should end with ER_LOCK_DEADLOCK error.
delete from t1 limit 1;
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
# Unblock LOCK TABLES.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap LOCK TABLES.
unlock tables;
#
# Switching to connection 'default'.
#
# **) Now case when transaction has a SW lock.
#
begin;
delete from t1 limit 1;
#
# Switching to connection 'mdl_con1'.
# Sending:
lock table t1 write;;
#
# Switching to connection 'default'.
# Wait until LOCK TABLE is blocked creating pending request for X lock.
# Check that both SR and SW locks are granted without waiting
# and errors.
select count(*) from t1;
count(*)
3
insert into t1 values (1, 1);
# Unblock LOCK TABLES.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap LOCK TABLES.
unlock tables;
#
# Switching to connection 'default'.
#
# 4) Check how various locks used within transactional context
# interact with active/pending X lock.
#
# As usual we start with case when we are acquiring lock on
# the table which was not used in the transaction before.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con2'.
# Start transaction which will prevent X lock from going away
# immediately.
begin;
select count(*) from t2;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Create pending X lock on t2.
# Sending:
rename table t2 to t3;;
#
# Switching to connection 'default'.
# Wait until RENAME TABLE starts waiting with pending X lock.
# Check that attempt to acquire SR lock on t2 causes waiting.
# Sending:
select count(*) from t2;;
#
# Switching to connection 'mdl_con2'.
# Check that the above SELECT is blocked.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap RENAME TABLE.
#
# Switching to connection 'default'.
# Reap SELECT.
ERROR 42S02: Table 'test.t2' doesn't exist
commit;
rename table t3 to t2;
# The same test for SW lock.
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con2'.
# Start transaction which will prevent X lock from going away
# immediately.
begin;
select count(*) from t2;
count(*)
3
#
# Switching to connection 'mdl_con1'.
# Create pending X lock on t2.
# Sending:
rename table t2 to t3;;
#
# Switching to connection 'default'.
# Wait until RENAME TABLE starts waiting with pending X lock.
# Check that attempt to acquire SW lock on t2 causes waiting.
# Sending:
delete from t2 limit 1;;
#
# Switching to connection 'mdl_con2'.
# Check that the above DELETE is blocked.
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap RENAME TABLE.
#
# Switching to connection 'default'.
# Reap DELETE.
ERROR 42S02: Table 'test.t2' doesn't exist
commit;
rename table t3 to t2;
#
# Coverage for the case in which we are acquiring lock on
# the table which is already used in transaction and against
# which there is a pending X lock request.
#
# *) The first case is when transaction has only a SR lock.
#
begin;
select count(*) from t1;
count(*)
4
#
# Switching to connection 'mdl_con1'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'default'.
# Wait until RENAME TABLE is blocked creating pending request for X lock.
# Check that another instance of SR lock is granted without waiting.
select count(*) from t1;
count(*)
4
# Attempt to wait for SW lock will lead to deadlock, thus
# the below statement should end with ER_LOCK_DEADLOCK error.
delete from t1 limit 1;
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'default'.
#
# **) The second case is when transaction has a SW lock.
#
begin;
delete from t1 limit 1;
#
# Switching to connection 'mdl_con1'.
# Sending:
rename table t1 to t2;;
#
# Switching to connection 'default'.
# Wait until RENAME TABLE is blocked creating pending request for X lock.
# Check that both SR and SW locks are granted without waiting
# and errors.
select count(*) from t1;
count(*)
3
insert into t1 values (1, 1);
# Unblock RENAME TABLE.
commit;
#
# Switching to connection 'mdl_con1'.
# Reap RENAME TABLE.
ERROR 42S01: Table 't2' already exists
#
# Switching to connection 'default'.
# Clean-up.
set debug_sync= 'RESET';
drop table t1, t2;
#
# Additional coverage for some scenarios in which not quite
# correct use of S metadata locks by HANDLER statement might
# have caused deadlocks.
#
drop table if exists t1, t2;
create table t1 (i int);
create table t2 (j int);
insert into t1 values (1);
#
# First, check scenario in which we upgrade SNRW lock to X lock
# on a table while having HANDLER READ trying to acquire TL_READ
# on the same table.
#
handler t1 open;
#
# Switching to connection 'handler_con1'.
lock table t1 write;
# Upgrade SNRW to X lock.
# Sending:
alter table t1 add column j int;;
#
# Switching to connection 'handler_con2'.
# Wait until ALTER is blocked during upgrade.
#
# Switching to connection 'default'.
# The below statement should not cause deadlock.
handler t1 read first;;
#
# Switching to connection 'handler_con1'.
# Reap ALTER TABLE.
unlock tables;
#
# Switching to connection 'default'.
# Reap HANDLER READ.
i j
1 NULL
handler t1 close;
#
# Now, check scenario in which upgrade of SNRW lock to X lock
# can be blocked by HANDLER which is open in connection currently
# waiting to get table-lock owned by connection doing upgrade.
#
handler t1 open;
#
# Switching to connection 'handler_con1'.
lock table t1 write, t2 read;
#
# Switching to connection 'default'.
# Execute statement which will be blocked on table-level lock
# owned by connection 'handler_con1'.
# Sending:
insert into t2 values (1);;
#
# Switching to connection 'handler_con1'.
# Wait until INSERT is blocked on table-level lock.
# Sending 'alter table t1 drop column j'. It should not cause
# deadlock.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
alter table t1 drop column j;
# Switching to connection 'handler_con2'.
# Wait until ALTER is blocked during upgrade.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
#
# Switching to connection 'default'.
# Reap INSERT.
ERROR HY000: Wait on a lock was aborted due to a pending exclusive lock
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
handler t1 close;
#
# Switching to connection 'handler_con1'.
# Reaping 'alter table t1 drop column j'
unlock tables;
# Switching to connection 'default'.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
# Then, check the scenario in which upgrade of SNRW lock to X
# lock is blocked by HANDLER which is open in connection currently
# waiting to get SW lock on the same table.
#
handler t1 open;
#
# Switching to connection 'handler_con1'.
lock table t1 write;
#
# Switching to connection 'default'.
# The below insert should be blocked because active SNRW lock on 't1'.
# Sending:
insert into t1 values (1);;
#
# Switching to connection 'handler_con1'.
# Wait until INSERT is blocked because of SNRW lock.
# The below ALTER TABLE will be blocked because of presence of HANDLER.
# Sending:
alter table t1 add column j int;;
#
# Switching to connection 'default'.
# INSERT should be chosen as victim for resolving deadlock.
# Reaping INSERT.
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
# Close HANDLER to unblock ALTER TABLE.
handler t1 close;
#
# Switching to connection 'handler_con1'.
# Reaping ALTER TABLE.
unlock tables;
#
# Switching to connection 'default'.
#
# Finally, test in which upgrade of SNRW lock to X lock is blocked
# by HANDLER which is open in connection currently waiting to get
# SR lock on the table on which lock is upgraded.
#
handler t1 open;
#
# Switching to connection 'handler_con1'.
lock table t1 write, t2 write;
#
# Switching to connection 'default'.
# The below insert should be blocked because active SNRW lock on 't1'.
# Sending:
insert into t2 values (1);;
#
# Switching to connection 'handler_con1'.
# Wait until INSERT is blocked because of SNRW lock.
# The below ALTER TABLE will be blocked because of presence of HANDLER.
# Sending:
alter table t1 drop column j;;
#
# Switching to connection 'default'.
# INSERT should be chosen as victim for resolving deadlock.
# Reaping INSERT.
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
# Close HANDLER to unblock ALTER TABLE.
handler t1 close;
#
# Switching to connection 'handler_con1'.
# Reaping ALTER TABLE.
unlock tables;
#
# Switching to connection 'default'.
# Clean-up.
drop tables t1, t2;
#
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
# Test coverage for basic deadlock detection in metadata
# locking subsystem.
#
drop tables if exists t0, t1, t2, t3, t4, t5;
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement.
2010-06-07 11:06:55 +04:00
set debug_sync= 'RESET';
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
create table t1 (i int);
create table t2 (j int);
create table t3 (k int);
create table t4 (k int);
#
# Test for the case in which no deadlock occurs.
#
#
# Switching to connection 'deadlock_con1'.
begin;
insert into t1 values (1);
#
# Switching to connection 'deadlock_con2'.
begin;
insert into t2 values (1);
#
# Switching to connection 'default'.
# Send:
rename table t2 to t0, t3 to t2, t0 to t3;;
#
# Switching to connection 'deadlock_con1'.
# Wait until the above RENAME TABLE is blocked because it has to wait
# for 'deadlock_con2' which holds shared metadata lock on 't2'.
# The below statement should wait for exclusive metadata lock
# on 't2' to go away and should not produce ER_LOCK_DEADLOCK
# as no deadlock is possible in this situation.
# Send:
select * from t2;;
#
# Switching to connection 'deadlock_con2'.
# Wait until the above SELECT * FROM t2 is starts waiting
# for an exclusive metadata lock to go away.
#
# Unblock RENAME TABLE by releasing shared metadata lock on t2.
commit;
#
# Switching to connection 'default'.
# Reap RENAME TABLE.
#
# Switching to connection 'deadlock_con1'.
# Reap SELECT.
k
#
# Switching to connection 'default'.
#
# Let us check that in the process of waiting for conflicting lock
# on table 't2' to go away transaction in connection 'deadlock_con1'
# has not released metadata lock on table 't1'.
# Send:
rename table t1 to t0, t3 to t1, t0 to t3;;
#
# Switching to connection 'deadlock_con1'.
# Wait until the above RENAME TABLE is blocked because it has to wait
# for 'deadlock_con1' which should still hold shared metadata lock on
# table 't1'.
# Commit transaction to unblock RENAME TABLE.
commit;
#
# Switching to connection 'default'.
# Reap RENAME TABLE.
#
# Test for case when deadlock occurs and should be detected immediately.
#
#
# Switching to connection 'deadlock_con1'.
begin;
insert into t2 values (2);
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# Switching to connection 'default'.
# Send:
rename table t2 to t0, t1 to t2, t0 to t1;;
#
# Switching to connection 'deadlock_con1'.
# Wait until the above RENAME TABLE is blocked because it has to wait
# for 'deadlock_con1' which holds shared metadata lock on 't2'.
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# The below statement should not wait as doing so will cause deadlock.
Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK". The problem in the first bug report was that although deadlock involving metadata locks was reported using the same error code and message as InnoDB deadlock it didn't rollback transaction like the latter. This caused confusion to users as in some cases after ER_LOCK_DEADLOCK transaction could have been restarted immediately and in some cases rollback was required. The problem in the second bug report was that although InnoDB deadlock caused transaction rollback in all storage engines it didn't cause release of metadata locks. So concurrent DDL on the tables used in transaction was blocked until implicit or explicit COMMIT or ROLLBACK was issued in the connection which got InnoDB deadlock. The former issue has stemmed from the fact that when support for detection and reporting metadata locks deadlocks was added we erroneously assumed that InnoDB doesn't rollback transaction on deadlock but only last statement (while this is what happens on InnoDB lock timeout actually) and so didn't implement rollback of transactions on MDL deadlocks. The latter issue was caused by the fact that rollback of transaction due to deadlock is carried out by setting THD::transaction_rollback_request flag at the point where deadlock is detected and performing rollback inside of trans_rollback_stmt() call when this flag is set. And trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are released. This patch solves these two problems in the following way: - In case when MDL deadlock is detect transaction rollback is requested by setting THD::transaction_rollback_request flag. - Code performing rollback of transaction if THD::transaction_rollback_request is moved out from trans_rollback_stmt(). Now we handle rollback request on the same level as we call trans_rollback_stmt() and release statement/ transaction MDL locks.
2013-08-20 13:12:34 +04:00
# Instead it should fail and emit ER_LOCK_DEADLOCK statement and
# transaction should be rolled back.
select * from t1;
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
#
# Switching to connection 'default'.
# Reap RENAME TABLE.
#
# Test for the case in which deadlock also occurs but not immediately.
#
#
# Switching to connection 'deadlock_con1'.
begin;
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
insert into t2 values (1);
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# Switching to connection 'default'.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
lock table t1 write;
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# Switching to connection 'deadlock_con1'.
# The below SELECT statement should wait for metadata lock
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
# on table 't1' and should not produce ER_LOCK_DEADLOCK
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
# immediately as no deadlock is possible at the moment.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
select * from t1;;
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
# Switching to connection 'deadlock_con2'.
# Wait until the above SELECT * FROM t1 is starts waiting
# for an UNRW metadata lock to go away.
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
# Send RENAME TABLE statement that will deadlock with the
# SELECT statement and thus should abort the latter.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
rename table t1 to t0, t2 to t1, t0 to t2;;
#
# Switching to connection 'default'.
# Wait till above RENAME TABLE is blocked while holding
# pending X lock on t1.
# Allow the above RENAME TABLE to acquire lock on t1 and
# create pending lock on t2 thus creating deadlock.
unlock tables;
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# Switching to connection 'deadlock_con1'.
# Since the latest RENAME TABLE entered in deadlock with SELECT
# statement the latter should be aborted and emit ER_LOCK_DEADLOCK
Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK". The problem in the first bug report was that although deadlock involving metadata locks was reported using the same error code and message as InnoDB deadlock it didn't rollback transaction like the latter. This caused confusion to users as in some cases after ER_LOCK_DEADLOCK transaction could have been restarted immediately and in some cases rollback was required. The problem in the second bug report was that although InnoDB deadlock caused transaction rollback in all storage engines it didn't cause release of metadata locks. So concurrent DDL on the tables used in transaction was blocked until implicit or explicit COMMIT or ROLLBACK was issued in the connection which got InnoDB deadlock. The former issue has stemmed from the fact that when support for detection and reporting metadata locks deadlocks was added we erroneously assumed that InnoDB doesn't rollback transaction on deadlock but only last statement (while this is what happens on InnoDB lock timeout actually) and so didn't implement rollback of transactions on MDL deadlocks. The latter issue was caused by the fact that rollback of transaction due to deadlock is carried out by setting THD::transaction_rollback_request flag at the point where deadlock is detected and performing rollback inside of trans_rollback_stmt() call when this flag is set. And trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are released. This patch solves these two problems in the following way: - In case when MDL deadlock is detect transaction rollback is requested by setting THD::transaction_rollback_request flag. - Code performing rollback of transaction if THD::transaction_rollback_request is moved out from trans_rollback_stmt(). Now we handle rollback request on the same level as we call trans_rollback_stmt() and release statement/ transaction MDL locks.
2013-08-20 13:12:34 +04:00
# error and transaction should be rolled back.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
# Reap SELECT * FROM t1.
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
#
# Switching to connection 'deadlock_con2'.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
# Reap RENAME TABLE ... .
#
# Switching to connection 'default'.
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
drop tables t1, t2, t3, t4;
#
# Now, test case which shows that deadlock detection empiric
# also takes into account requests for metadata lock upgrade.
#
create table t1 (i int);
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
insert into t1 values (1);
# Avoid race which occurs when SELECT in 'deadlock_con1' connection
# accesses table before the above INSERT unlocks the table and thus
# its result becomes visible to other connections.
select * from t1;
i
1
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# Switching to connection 'deadlock_con1'.
begin;
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
select * from t1;
i
1
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# Switching to connection 'default'.
# Send:
alter table t1 add column j int, rename to t2;;
#
# Switching to connection 'deadlock_con1'.
# Wait until the above ALTER TABLE ... RENAME acquires exclusive
# metadata lock on 't2' and starts waiting for connection
# 'deadlock_con1' which holds shared lock on 't1'.
# The below statement should not wait as it will cause deadlock.
Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK". The problem in the first bug report was that although deadlock involving metadata locks was reported using the same error code and message as InnoDB deadlock it didn't rollback transaction like the latter. This caused confusion to users as in some cases after ER_LOCK_DEADLOCK transaction could have been restarted immediately and in some cases rollback was required. The problem in the second bug report was that although InnoDB deadlock caused transaction rollback in all storage engines it didn't cause release of metadata locks. So concurrent DDL on the tables used in transaction was blocked until implicit or explicit COMMIT or ROLLBACK was issued in the connection which got InnoDB deadlock. The former issue has stemmed from the fact that when support for detection and reporting metadata locks deadlocks was added we erroneously assumed that InnoDB doesn't rollback transaction on deadlock but only last statement (while this is what happens on InnoDB lock timeout actually) and so didn't implement rollback of transactions on MDL deadlocks. The latter issue was caused by the fact that rollback of transaction due to deadlock is carried out by setting THD::transaction_rollback_request flag at the point where deadlock is detected and performing rollback inside of trans_rollback_stmt() call when this flag is set. And trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are released. This patch solves these two problems in the following way: - In case when MDL deadlock is detect transaction rollback is requested by setting THD::transaction_rollback_request flag. - Code performing rollback of transaction if THD::transaction_rollback_request is moved out from trans_rollback_stmt(). Now we handle rollback request on the same level as we call trans_rollback_stmt() and release statement/ transaction MDL locks.
2013-08-20 13:12:34 +04:00
# An appropriate error should be reported instead and transaction
# should be rolled back.
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
select * from t2;
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
#
# Switching to connection 'default'.
# Reap ALTER TABLE ... RENAME.
drop table t2;
#
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement.
2010-06-07 11:06:55 +04:00
# Test that in situation when MDL subsystem detects a deadlock
# but it turns out that it can be resolved by backing-off locks
# acquired by one of participating transactions (which is
# possible when one of transactions consists only of currently
# executed statement, e.g. in autocommit mode) no error is
# reported.
#
create table t1 (i int);
create table t2 (j int);
# Ensure that the below SELECT stops once it has acquired metadata
# lock on table 't2'.
set debug_sync= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
# Sending:
select * from t2, t1;
#
# Switching to connection 'deadlock_con1'.
# Wait till SELECT acquires MDL on 't2' and starts waiting for signal.
set debug_sync= 'now WAIT_FOR locked';
# Sending:
lock tables t1 write, t2 write;
#
# Switching to connection 'deadlock_con2'.
# Wait until LOCK TABLES acquires SNRW lock on 't1' and is blocked
# while trying to acquire SNRW lock on 't1'.
# Resume SELECT execution, this should eventually unblock LOCK TABLES.
set debug_sync= 'now SIGNAL finish';
#
# Switching to connection 'deadlock_con1'.
# Reaping LOCK TABLES.
unlock tables;
#
# Switching to connection 'default'.
# Reaping SELECT. It succeed and not report ER_LOCK_DEADLOCK error.
j i
drop tables t1, t2;
#
# Test coverage for situation in which a race has happened
# during deadlock detection process which led to unwarranted
# ER_LOCK_DEADLOCK error.
#
create table t1 (i int);
# Ensure that ALTER waits once it has acquired SNW lock.
set debug_sync='after_open_table_mdl_shared SIGNAL parked1 WAIT_FOR go1';
# Sending:
alter table t1 add column j int;
#
# Switching to connection 'deadlock_con1'.
# Wait till ALTER acquires SNW lock and stops.
set debug_sync='now WAIT_FOR parked1';
# Ensure that INSERT is paused once it detects that there is
# a conflicting metadata lock so it has to wait, but before
# deadlock detection is run.
set debug_sync='mdl_acquire_lock_wait SIGNAL parked2 WAIT_FOR go2';
# Sending:
insert into t1 values ();
#
# Switching to connection 'deadlock_con2'.
# Wait till INSERT is paused.
set debug_sync='now WAIT_FOR parked2';
# Resume ALTER execution. Eventually it will release its
# metadata lock and INSERT's request for SW lock will be
# satisified.
set debug_sync='now SIGNAL go1';
#
# Switching to connection 'default'.
# Reaping ALTER TABLE.
# Add a new request for SNW lock to waiting graph.
# Sending:
alter table t1 drop column j;
#
# Switching to connection 'deadlock_con2'.
# Wait until ALTER is blocked.
# Resume INSERT so it can start deadlock detection.
#
# At this point there is a discrepancy between the fact that INSERT's
# SW lock is already satisfied, but INSERT's connection is still
# marked as waiting for it. Looking for a loop in waiters graph
# without additional checks has detected a deadlock (INSERT waits
# for SW lock; which is not granted because of pending SNW lock from
# ALTER; which waits for active SW lock from INSERT). Since requests
# for SW and SNW locks have same weight ALTER was selected as a victim
# and ended with ER_LOCK_DEADLOCK error.
set debug_sync='now SIGNAL go2';
#
# Switching to connection 'deadlock_con1'.
# Reaping INSERT.
#
# Switching to connection 'default'.
# Reaping ALTER. It should succeed and not produce ER_LOCK_DEADLOCK.
drop table t1;
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
#
# Now, test for a situation in which deadlock involves waiting not
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
# only in MDL subsystem but also for TDC. Such deadlocks should be
# successfully detected. If possible, they should be resolved without
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
# resorting to ER_LOCK_DEADLOCK error.
#
create table t1(i int);
create table t2(j int);
#
# First, let us check how we handle a simple scenario involving
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
# waits in MDL and TDC.
#
set debug_sync= 'RESET';
# Switching to connection 'deadlock_con1'.
# Start a statement, which will acquire SR metadata lock on t1, open it
# and then stop, before trying to acquire SW lock on t2 and opening it.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
set debug_sync='open_tables_after_open_and_process_table SIGNAL parked WAIT_FOR go';
# Sending:
select * from t1 where i in (select j from t2 for update);
# Switching to connection 'deadlock_con2'.
# Wait till the above SELECT stops.
set debug_sync='now WAIT_FOR parked';
# The below FLUSH TABLES WITH READ LOCK should acquire
# SNW locks on t1 and t2 and wait till SELECT closes t1.
# Sending:
flush tables t1, t2 with read lock;
# Switching to connection 'deadlock_con3'.
# Wait until FLUSH TABLES WITH t1, t2 READ LOCK starts waiting
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
# for SELECT to close t1.
# Resume SELECT, so it tries to acquire SW lock on t1 and blocks,
# creating a deadlock. This deadlock should be detected and resolved
# by backing-off SELECT. As a result FTWRL should be able to finish.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
set debug_sync='now SIGNAL go';
# Switching to connection 'deadlock_con2'.
# Reap FLUSH TABLES WITH READ LOCK.
unlock tables;
# Switching to connection 'deadlock_con1'.
# Reap SELECT.
i
#
# The same scenario with a slightly different order of events
# which emphasizes that setting correct deadlock detector weights
# for flush waits is important.
#
set debug_sync= 'RESET';
# Switching to connection 'deadlock_con2'.
set debug_sync='flush_tables_with_read_lock_after_acquire_locks SIGNAL parked WAIT_FOR go';
# The below FLUSH TABLES WITH READ LOCK should acquire
# SNW locks on t1 and t2 and wait on debug sync point.
# Sending:
flush tables t1, t2 with read lock;
# Switching to connection 'deadlock_con1'.
# Wait till FLUSH TABLE WITH READ LOCK stops.
set debug_sync='now WAIT_FOR parked';
# Start statement which will acquire SR metadata lock on t1, open
# it and then will block while trying to acquire SW lock on t2.
# Sending:
select * from t1 where i in (select j from t2 for update);
# Switching to connection 'deadlock_con3'.
# Wait till the above SELECT blocks.
# Resume FLUSH TABLES, so it tries to flush t1, thus creating
# a deadlock. This deadlock should be detected and resolved by
# backing-off SELECT. As a result FTWRL should be able to finish.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
set debug_sync='now SIGNAL go';
# Switching to connection 'deadlock_con2'.
# Reap FLUSH TABLES WITH READ LOCK.
unlock tables;
# Switching to connection 'deadlock_con1'.
# Reap SELECT.
i
#
# Now a more complex scenario involving two connections
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
# waiting for MDL and one for TDC.
#
set debug_sync= 'RESET';
# Switching to connection 'deadlock_con1'.
# Start a statement which will acquire SR metadata lock on t2, open it
# and then stop, before trying to acquire SR on t1 and opening it.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
set debug_sync='open_tables_after_open_and_process_table SIGNAL parked WAIT_FOR go';
# Sending:
select * from t2, t1;
# Switching to connection 'deadlock_con2'.
# Wait till the above SELECT stops.
set debug_sync='now WAIT_FOR parked';
# The below FLUSH TABLES WITH READ LOCK should acquire
# SNW locks on t2 and wait till SELECT closes t2.
# Sending:
flush tables t2 with read lock;
# Switching to connection 'deadlock_con3'.
# Wait until FLUSH TABLES WITH READ LOCK starts waiting
# for SELECT to close t2.
# The below DROP TABLES should acquire X lock on t1 and start
# waiting for X lock on t2.
# Sending:
drop tables t1, t2;
# Switching to connection 'default'.
# Wait until DROP TABLES starts waiting for X lock on t2.
# Resume SELECT, so it tries to acquire SR lock on t1 and blocks,
# creating a deadlock. This deadlock should be detected and resolved
# by backing-off SELECT. As a result, FTWRL should be able to finish.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
set debug_sync='now SIGNAL go';
# Switching to connection 'deadlock_con2'.
# Reap FLUSH TABLES WITH READ LOCK.
# Unblock DROP TABLES.
unlock tables;
# Switching to connection 'deadlock_con3'.
# Reap DROP TABLES.
# Switching to connection 'deadlock_con1'.
# Reap SELECT. It should emit error about missing table.
ERROR 42S02: Table 'test.t2' doesn't exist
# Switching to connection 'default'.
set debug_sync= 'RESET';
#
# Test for a scenario in which FLUSH TABLES <list> WITH READ LOCK
# used to erroneously release metadata locks.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
#
drop tables if exists t1, t2;
set debug_sync= 'RESET';
create table t1(i int);
create table t2(j int);
# Switching to connection 'con2'.
set debug_sync='open_tables_after_open_and_process_table SIGNAL parked WAIT_FOR go';
# The below FLUSH TABLES <list> WITH READ LOCK should acquire
# SNW locks on t1 and t2, open table t1 and block on the debug
# sync point.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
# Sending:
flush tables t1, t2 with read lock;
# Switching to connection 'con1'.
# Wait till FLUSH TABLES <list> WITH READ LOCK stops.
set debug_sync='now WAIT_FOR parked';
# Start a statement which will flush all tables and thus
# invalidate table t1 open by FLUSH TABLES <list> WITH READ LOCK.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
# Sending:
flush tables;
# Switching to connection 'default'.
# Wait till the above FLUSH TABLES blocks.
# Resume FLUSH TABLES <list> WITH READ LOCK, so it tries to open t2
# discovers that its t1 is obsolete and tries to reopen all tables.
# Such reopen should not cause releasing of SNW metadata locks
# which would result in assertion failures.
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush.
2010-07-27 17:34:58 +04:00
set debug_sync='now SIGNAL go';
# Switching to connection 'con2'.
# Reap FLUSH TABLES <list> WITH READ LOCK.
unlock tables;
# Switching to connection 'con1'.
# Reap FLUSH TABLES.
# Clean-up.
# Switching to connection 'default'.
drop tables t1, t2;
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement.
2010-06-07 11:06:55 +04:00
set debug_sync= 'RESET';
#
# Test for bug #46748 "Assertion in MDL_context::wait_for_locks()
# on INSERT + CREATE TRIGGER".
#
drop tables if exists t1, t2, t3, t4, t5;
# Let us simulate scenario in which we open some tables from extended
# part of prelocking set but then encounter conflicting metadata lock,
# so have to back-off and wait for it to go away.
create table t1 (i int);
create table t2 (j int);
create table t3 (k int);
create table t4 (l int);
create trigger t1_bi before insert on t1 for each row
insert into t2 values (new.i);
create trigger t2_bi before insert on t2 for each row
insert into t3 values (new.j);
#
# Switching to connection 'con1root'.
lock tables t4 read;
#
# Switching to connection 'con2root'.
# Send :
rename table t3 to t5, t4 to t3;;
#
# Switching to connection 'default'.
# Wait until the above RENAME TABLE adds pending requests for exclusive
# metadata lock on its tables and blocks due to 't4' being used by LOCK
# TABLES.
# Send :
insert into t1 values (1);;
#
# Switching to connection 'con1root'.
# Wait until INSERT statement waits due to encountering pending
# exclusive metadata lock on 't3'.
unlock tables;
#
# Switching to connection 'con2root'.
# Reap RENAME TABLE.
#
# Switching to connection 'default'.
# Reap INSERT.
# Clean-up.
drop tables t1, t2, t3, t5;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
#
# Bug#42546 - Backup: RESTORE fails, thinking it finds an existing table
#
DROP TABLE IF EXISTS t1;
set @save_log_output=@@global.log_output;
set global log_output=file;
#
# Test 1: CREATE TABLE
#
# Connection 2
# Start insert on the not-yet existing table
# Wait after taking the MDL lock
SET DEBUG_SYNC= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
INSERT INTO t1 VALUES(1,"def");
# Connection 1
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Now INSERT has a MDL on the non-existent table t1.
#
# Continue the INSERT once CREATE waits for exclusive lock
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
SET DEBUG_SYNC= 'mdl_acquire_lock_wait SIGNAL finish';
# Try to create that table.
CREATE TABLE t1 (c1 INT, c2 VARCHAR(100), KEY(c1));
# Connection 2
# Insert fails
ERROR 42S02: Table 'test.t1' doesn't exist
# Connection 1
SET DEBUG_SYNC= 'RESET';
SHOW TABLES;
Tables_in_test
t1
DROP TABLE IF EXISTS t1;
#
# Test 2: CREATE TABLE LIKE
#
CREATE TABLE t2 (c1 INT, c2 VARCHAR(100), KEY(c1));
# Connection 2
# Start insert on the not-yet existing table
# Wait after taking the MDL
SET DEBUG_SYNC= 'after_open_table_mdl_shared SIGNAL locked WAIT_FOR finish';
INSERT INTO t1 VALUES(1,"def");
# Connection 1
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Now INSERT has a MDL on the non-existent table t1.
#
# Continue the INSERT once CREATE waits for exclusive lock
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
SET DEBUG_SYNC= 'mdl_acquire_lock_wait SIGNAL finish';
# Try to create that table.
CREATE TABLE t1 LIKE t2;
# Connection 2
# Insert fails
ERROR 42S02: Table 'test.t1' doesn't exist
# Connection 1
SET DEBUG_SYNC= 'RESET';
SHOW TABLES;
Tables_in_test
t1
t2
DROP TABLE t2;
DROP TABLE IF EXISTS t1;
set global log_output=@save_log_output;
#
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY
# FOR UPDATE"
#
drop tables if exists t1, t2;
create table t1 (i int);
# Let us check that we won't deadlock if during filling
# of I_S table we encounter conflicting metadata lock
# which owner is in its turn waiting for our connection.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
lock tables t1 read;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'con46044'.
# Sending:
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
create table t2 select * from t1 for update;;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'default'.
# Waiting until CREATE TABLE ... SELECT ... is blocked.
# First let us check that SHOW FIELDS/DESCRIBE doesn't
# gets blocked and emits and error.
show fields from t2;
ERROR HY000: Table 'test'.'t2' was skipped since its definition is being modified by concurrent DDL statement
# Now test for I_S query which reads only .FRMs.
#
# Query below should only emit a warning.
select column_name from information_schema.columns
where table_schema='test' and table_name='t2';
column_name
Warnings:
Warning 1684 Table 'test'.'t2' was skipped since its definition is being modified by concurrent DDL statement
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Finally, test for I_S query which does full-blown table open.
#
# Query below should not be blocked. Warning message should be
# stored in the 'table_comment' column.
select table_name, table_type, auto_increment, table_comment
from information_schema.tables where table_schema='test' and table_name='t2';
table_name table_type auto_increment table_comment
t2 BASE TABLE NULL Table 'test'.'t2' was skipped since its definition is being modified by concurrent DDL statement
Warnings:
Warning 1684 Table 'test'.'t2' was skipped since its definition is being modified by concurrent DDL statement
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'default'.
unlock tables;
# Switching to connection 'con46044'.
# Reaping CREATE TABLE ... SELECT ... .
drop table t2;
#
# Let us also check that queries to I_S wait for conflicting metadata
# locks to go away instead of skipping table with a warning in cases
# when deadlock is not possible. This is a nice thing from compatibility
# and ease of use points of view.
#
# We check same three queries to I_S in this new situation.
# Switching to connection 'con46044_2'.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
lock tables t1 read;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'con46044'.
# Sending:
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
create table t2 select * from t1 for update;;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'default'.
# Waiting until CREATE TABLE ... SELECT ... is blocked.
# Let us check that SHOW FIELDS/DESCRIBE gets blocked.
# Sending:
show fields from t2;;
# Switching to connection 'con46044_2'.
# Wait until SHOW FIELDS gets blocked.
unlock tables;
# Switching to connection 'con46044'.
# Reaping CREATE TABLE ... SELECT ... .
# Switching to connection 'default'.
# Reaping SHOW FIELDS ...
Field Type Null Key Default Extra
i int(11) YES NULL
drop table t2;
# Switching to connection 'con46044_2'.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
lock tables t1 read;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'con46044'.
# Sending:
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
create table t2 select * from t1 for update;;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'default'.
# Waiting until CREATE TABLE ... SELECT ... is blocked.
# Check that I_S query which reads only .FRMs gets blocked.
# Sending:
select column_name from information_schema.columns where table_schema='test' and table_name='t2';;
# Switching to connection 'con46044_2'.
# Wait until SELECT COLUMN_NAME FROM I_S.COLUMNS gets blocked.
unlock tables;
# Switching to connection 'con46044'.
# Reaping CREATE TABLE ... SELECT ... .
# Switching to connection 'default'.
# Reaping SELECT COLUMN_NAME FROM I_S.COLUMNS
column_name
i
drop table t2;
# Switching to connection 'con46044_2'.
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
lock tables t1 read;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'con46044'.
# Sending:
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
create table t2 select * from t1 for update;;
Backport of: ------------------------------------------------------------ revno: 2617.68.7 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46044 timestamp: Thu 2009-08-27 10:22:17 +0400 message: Fix for bug #46044 "MDL deadlock on LOCK TABLE + CREATE TABLE HIGH_PRIORITY FOR UPDATE". Deadlock occured when during execution of query to I_S we tried to open a table or its .FRM in order to get information about it and had to wait because we have encountered exclusive metadata lock on this table held by a DDL operation from another connection which in its turn waited for some resource currently owned by connection executing this I_S query. For example, this might have happened if one under LOCK TABLES executed I_S query targeted to particular table (which was not among locked) and also concurrently tried to create this table using CREATE TABLE SELECT which had to wait for one of tables locked by the first connection. Another situation in which deadlock might have occured is when I_S query, which was executed as part of transaction, tried to get information about table which just has been dropped by concurrent DROP TABLES executed under LOCK TABLES and this DROP TABLES for its completion also had to wait transaction from the first connection. This problem stemmed from the fact that opening of tables/.FRMs for I_S filling is happening outside of connection's main MDL_context so code which tries to detect deadlocks due to conflicting metadata locks doesn't work in this case. Indeed, this led to deadlocks when during I_S filling we tried to wait for conflicting metadata lock to go away, while its owner was waiting for some resource held by connection executing I_S query. This patch solves this problem by avoiding waiting in such situation. Instead we skip this table and produce warning that information about it was omitted from I_S due to concurrent DDL operation. We still wait for conflicting metadata lock to go away when it is known that deadlock is not possible (i.e. when connection executing I_S query does not hold any metadata or table-level locks). Basically, we apply our standard deadlock avoidance technique for metadata locks to the process of filling of I_S tables but replace ER_LOCK_DEADLOCK error with a warning. Note that this change is supposed to be safe for 'mysqldump' since the only its mode which is affected by this change is --single-transaction mode is not safe in the presence of concurrent DDL anyway (and this fact is documented). Other modes are unaffected because they either use SHOW TABLES/SELECT * FROM I_S.TABLE_NAMES which do not take any metadata locks in the process of I_S table filling and thus cannot skip tables or execute I_S queries for tables which were previously locked by LOCK TABLES (or in the presence of global read lock) which excludes possibility of encountering conflicting metadata lock.
2009-12-09 18:48:42 +03:00
# Switching to connection 'default'.
# Waiting until CREATE TABLE ... SELECT ... is blocked.
# Finally, check that I_S query which does full-blown table open
# also gets blocked.
# Sending:
select table_name, table_type, auto_increment, table_comment from information_schema.tables where table_schema='test' and table_name='t2';;
# Switching to connection 'con46044_2'.
# Wait until SELECT ... FROM I_S.TABLES gets blocked.
unlock tables;
# Switching to connection 'con46044'.
# Reaping CREATE TABLE ... SELECT ... .
# Switching to connection 'default'.
# Reaping SELECT ... FROM I_S.TABLES
table_name table_type auto_increment table_comment
t2 BASE TABLE NULL
drop table t2;
# Switching to connection 'default'.
# Clean-up.
drop table t1;
Backport of: ------------------------------------------------------------ revno: 2617.68.10 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46673 timestamp: Tue 2009-09-01 19:57:05 +0400 message: Fix for bug #46673 "Deadlock between FLUSH TABLES WITH READ LOCK and DML". Deadlocks occured when one concurrently executed transactions with several statements modifying data and FLUSH TABLES WITH READ LOCK statement or SET READ_ONLY=1 statement. These deadlocks were introduced by the patch for WL 4284: "Transactional DDL locking"/Bug 989: "If DROP TABLE while there's an active transaction, wrong binlog order" which has changed FLUSH TABLES WITH READ LOCK/SET READ_ONLY=1 to wait for pending transactions. What happened was that FLUSH TABLES WITH READ LOCK blocked all further statements changing tables by setting global_read_lock global variable and has started waiting for all pending transactions to complete. Then one of those transactions tried to executed DML, detected that global_read_lock non-zero and tried to wait until global read lock will be released (i.e. global_read_lock becomes 0), indeed, this led to a deadlock. Proper solution for this problem should probably involve full integration of global read lock with metadata locking subsystem (which will allow to implement waiting for pending transactions without blocking DML in them). But since it requires significant changes another, short-term solution for the problem is implemented in this patch. Basically, this patch restores behavior of FLUSH TABLES WITH READ LOCK/ SET READ_ONLY=1 before the patch for WL 4284/bug 989. By ensuring that extra references to TABLE_SHARE are not stored for active metadata locks it changes these statements not to wait for pending transactions. As result deadlock is eliminated. Note that this does not change the fact that active FLUSH TABLES WITH READ LOCK lock or SET READ_ONLY=1 prevent modifications to tables as they also block transaction commits.
2009-12-09 18:56:34 +03:00
#
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
# Test for bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed
# in case of ALTER".
#
drop table if exists t1;
set debug_sync= 'RESET';
create table t1 (c1 int primary key, c2 int, c3 int);
insert into t1 values (1,1,0),(2,2,0),(3,3,0),(4,4,0),(5,5,0);
begin;
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
select * from t1 where c2 = 3;
c1 c2 c3
3 3 0
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
#
# Switching to connection 'con46273'.
set debug_sync='after_lock_tables_takes_lock SIGNAL alter_table_locked WAIT_FOR alter_go';
alter table t1 add column e int, rename to t2;;
#
# Switching to connection 'default'.
set debug_sync='now WAIT_FOR alter_table_locked';
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement.
2010-06-07 11:06:55 +04:00
set debug_sync='mdl_acquire_lock_wait SIGNAL alter_go';
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
# The below statement should get ER_LOCK_DEADLOCK error
# (i.e. it should not allow ALTER to proceed, and then
# fail due to 't1' changing its name to 't2').
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
2010-02-01 14:43:06 +03:00
update t1 set c3=c3+1 where c2 = 3;
Implementation of simple deadlock detection for metadata locks. This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it.
2009-12-30 20:53:30 +03:00
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
#
# Switching to connection 'con46273'.
# Reap ALTER TABLE.
#
# Switching to connection 'default'.
# Clean-up.
set debug_sync= 'RESET';
drop table t2;
#
Backport of: ------------------------------------------------------------ revno: 2617.68.10 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg46673 timestamp: Tue 2009-09-01 19:57:05 +0400 message: Fix for bug #46673 "Deadlock between FLUSH TABLES WITH READ LOCK and DML". Deadlocks occured when one concurrently executed transactions with several statements modifying data and FLUSH TABLES WITH READ LOCK statement or SET READ_ONLY=1 statement. These deadlocks were introduced by the patch for WL 4284: "Transactional DDL locking"/Bug 989: "If DROP TABLE while there's an active transaction, wrong binlog order" which has changed FLUSH TABLES WITH READ LOCK/SET READ_ONLY=1 to wait for pending transactions. What happened was that FLUSH TABLES WITH READ LOCK blocked all further statements changing tables by setting global_read_lock global variable and has started waiting for all pending transactions to complete. Then one of those transactions tried to executed DML, detected that global_read_lock non-zero and tried to wait until global read lock will be released (i.e. global_read_lock becomes 0), indeed, this led to a deadlock. Proper solution for this problem should probably involve full integration of global read lock with metadata locking subsystem (which will allow to implement waiting for pending transactions without blocking DML in them). But since it requires significant changes another, short-term solution for the problem is implemented in this patch. Basically, this patch restores behavior of FLUSH TABLES WITH READ LOCK/ SET READ_ONLY=1 before the patch for WL 4284/bug 989. By ensuring that extra references to TABLE_SHARE are not stored for active metadata locks it changes these statements not to wait for pending transactions. As result deadlock is eliminated. Note that this does not change the fact that active FLUSH TABLES WITH READ LOCK lock or SET READ_ONLY=1 prevent modifications to tables as they also block transaction commits.
2009-12-09 18:56:34 +03:00
# Test for bug #46673 "Deadlock between FLUSH TABLES WITH READ LOCK
# and DML".
#
drop tables if exists t1;
create table t1 (i int);
# Switching to connection 'con46673'.
begin;
insert into t1 values (1);
# Switching to connection 'default'.
# Statement below should not get blocked. And if after some
# changes to code it is there should not be a deadlock between
# it and transaction from connection 'con46673'.
flush tables with read lock;
unlock tables;
# Switching to connection 'con46673'.
delete from t1 where i = 1;
commit;
# Switching to connection 'default'.
# Clean-up
drop table t1;
Backport of revno: 3685 Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks against concurrent CREATE PROCEDURE This deadlock occured between a) CREATE PROCEDURE (or other commands listed below) b) FLUSH TABLES WITH READ LOCK If the execution of them happened in the following order: - a) opens a table (e.g. mysql.proc) - b) locks the global read lock (or GRL) - a) sleeps inside wait_if_global_read_lock() - b) increases refresh_version and sleeps waiting for old tables to go away Note that a) must start waiting on the GRL before FLUSH increases refresh_version. Otherwise a) won't wait on the GRL and instead close its tables for reopen, allowing FLUSH to complete and thus avoid the deadlock. With this patch the deadlock is avoided by making CREATE PROCEDURE acquire a protection against global read locks before it starts executing. This means that FLUSH TABLES WITH READ LOCK will have to wait until CREATE PROCEDURE completes before acquiring the global read lock, thereby avoiding the deadlock. This is implemented by introducing a new SQL command flag called CF_PROTECT_AGAINST_GRL. Commands marked with this flag will acquire a GRL protection in the beginning of mysql_execute_command(). This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and REVOKE ALL. All these commands either call open_grant_tables() or open_system_table_for_updated() which make them susceptible for this deadlock. The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number of commands that previously acquired GRL protection in their respective SQLCOM case in mysql_execute_command(). Test case that checks for GRL protection for CREATE PROCEDURE and CREATE USER added to mdl_sync.test.
2009-12-10 15:09:00 +01:00
#
# Bug#48210 FLUSH TABLES WITH READ LOCK deadlocks
# against concurrent CREATE PROCEDURE
#
# Test 1: CREATE PROCEDURE
# Connection 1
# Start CREATE PROCEDURE and open mysql.proc
SET DEBUG_SYNC= 'after_open_table_mdl_shared SIGNAL table_opened WAIT_FOR grlwait';
CREATE PROCEDURE p1() SELECT 1;
# Connection 2
SET DEBUG_SYNC= 'now WAIT_FOR table_opened';
# Check that FLUSH must wait to get the GRL
# and let CREATE PROCEDURE continue
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
SET DEBUG_SYNC= 'mdl_acquire_lock_wait SIGNAL grlwait';
Backport of revno: 3685 Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks against concurrent CREATE PROCEDURE This deadlock occured between a) CREATE PROCEDURE (or other commands listed below) b) FLUSH TABLES WITH READ LOCK If the execution of them happened in the following order: - a) opens a table (e.g. mysql.proc) - b) locks the global read lock (or GRL) - a) sleeps inside wait_if_global_read_lock() - b) increases refresh_version and sleeps waiting for old tables to go away Note that a) must start waiting on the GRL before FLUSH increases refresh_version. Otherwise a) won't wait on the GRL and instead close its tables for reopen, allowing FLUSH to complete and thus avoid the deadlock. With this patch the deadlock is avoided by making CREATE PROCEDURE acquire a protection against global read locks before it starts executing. This means that FLUSH TABLES WITH READ LOCK will have to wait until CREATE PROCEDURE completes before acquiring the global read lock, thereby avoiding the deadlock. This is implemented by introducing a new SQL command flag called CF_PROTECT_AGAINST_GRL. Commands marked with this flag will acquire a GRL protection in the beginning of mysql_execute_command(). This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and REVOKE ALL. All these commands either call open_grant_tables() or open_system_table_for_updated() which make them susceptible for this deadlock. The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number of commands that previously acquired GRL protection in their respective SQLCOM case in mysql_execute_command(). Test case that checks for GRL protection for CREATE PROCEDURE and CREATE USER added to mdl_sync.test.
2009-12-10 15:09:00 +01:00
FLUSH TABLES WITH READ LOCK;
# Connection 1
# Connection 2
UNLOCK TABLES;
# Connection 1
SET DEBUG_SYNC= 'RESET';
# Test 2: DROP PROCEDURE
# Start DROP PROCEDURE and open tables
Backport of revno: 3685 Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks against concurrent CREATE PROCEDURE This deadlock occured between a) CREATE PROCEDURE (or other commands listed below) b) FLUSH TABLES WITH READ LOCK If the execution of them happened in the following order: - a) opens a table (e.g. mysql.proc) - b) locks the global read lock (or GRL) - a) sleeps inside wait_if_global_read_lock() - b) increases refresh_version and sleeps waiting for old tables to go away Note that a) must start waiting on the GRL before FLUSH increases refresh_version. Otherwise a) won't wait on the GRL and instead close its tables for reopen, allowing FLUSH to complete and thus avoid the deadlock. With this patch the deadlock is avoided by making CREATE PROCEDURE acquire a protection against global read locks before it starts executing. This means that FLUSH TABLES WITH READ LOCK will have to wait until CREATE PROCEDURE completes before acquiring the global read lock, thereby avoiding the deadlock. This is implemented by introducing a new SQL command flag called CF_PROTECT_AGAINST_GRL. Commands marked with this flag will acquire a GRL protection in the beginning of mysql_execute_command(). This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and REVOKE ALL. All these commands either call open_grant_tables() or open_system_table_for_updated() which make them susceptible for this deadlock. The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number of commands that previously acquired GRL protection in their respective SQLCOM case in mysql_execute_command(). Test case that checks for GRL protection for CREATE PROCEDURE and CREATE USER added to mdl_sync.test.
2009-12-10 15:09:00 +01:00
SET DEBUG_SYNC= 'after_open_table_mdl_shared SIGNAL table_opened WAIT_FOR grlwait';
DROP PROCEDURE p1;
Backport of revno: 3685 Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks against concurrent CREATE PROCEDURE This deadlock occured between a) CREATE PROCEDURE (or other commands listed below) b) FLUSH TABLES WITH READ LOCK If the execution of them happened in the following order: - a) opens a table (e.g. mysql.proc) - b) locks the global read lock (or GRL) - a) sleeps inside wait_if_global_read_lock() - b) increases refresh_version and sleeps waiting for old tables to go away Note that a) must start waiting on the GRL before FLUSH increases refresh_version. Otherwise a) won't wait on the GRL and instead close its tables for reopen, allowing FLUSH to complete and thus avoid the deadlock. With this patch the deadlock is avoided by making CREATE PROCEDURE acquire a protection against global read locks before it starts executing. This means that FLUSH TABLES WITH READ LOCK will have to wait until CREATE PROCEDURE completes before acquiring the global read lock, thereby avoiding the deadlock. This is implemented by introducing a new SQL command flag called CF_PROTECT_AGAINST_GRL. Commands marked with this flag will acquire a GRL protection in the beginning of mysql_execute_command(). This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and REVOKE ALL. All these commands either call open_grant_tables() or open_system_table_for_updated() which make them susceptible for this deadlock. The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number of commands that previously acquired GRL protection in their respective SQLCOM case in mysql_execute_command(). Test case that checks for GRL protection for CREATE PROCEDURE and CREATE USER added to mdl_sync.test.
2009-12-10 15:09:00 +01:00
# Connection 2
SET DEBUG_SYNC= 'now WAIT_FOR table_opened';
# Check that FLUSH must wait to get the GRL
# and let DROP PROCEDURE continue
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
SET DEBUG_SYNC= 'mdl_acquire_lock_wait SIGNAL grlwait';
Backport of revno: 3685 Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks against concurrent CREATE PROCEDURE This deadlock occured between a) CREATE PROCEDURE (or other commands listed below) b) FLUSH TABLES WITH READ LOCK If the execution of them happened in the following order: - a) opens a table (e.g. mysql.proc) - b) locks the global read lock (or GRL) - a) sleeps inside wait_if_global_read_lock() - b) increases refresh_version and sleeps waiting for old tables to go away Note that a) must start waiting on the GRL before FLUSH increases refresh_version. Otherwise a) won't wait on the GRL and instead close its tables for reopen, allowing FLUSH to complete and thus avoid the deadlock. With this patch the deadlock is avoided by making CREATE PROCEDURE acquire a protection against global read locks before it starts executing. This means that FLUSH TABLES WITH READ LOCK will have to wait until CREATE PROCEDURE completes before acquiring the global read lock, thereby avoiding the deadlock. This is implemented by introducing a new SQL command flag called CF_PROTECT_AGAINST_GRL. Commands marked with this flag will acquire a GRL protection in the beginning of mysql_execute_command(). This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and REVOKE ALL. All these commands either call open_grant_tables() or open_system_table_for_updated() which make them susceptible for this deadlock. The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number of commands that previously acquired GRL protection in their respective SQLCOM case in mysql_execute_command(). Test case that checks for GRL protection for CREATE PROCEDURE and CREATE USER added to mdl_sync.test.
2009-12-10 15:09:00 +01:00
FLUSH TABLES WITH READ LOCK;
# Connection 1
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
# Once FLUSH TABLES WITH READ LOCK starts waiting
# DROP PROCEDURE will be waked up and will drop
# procedure. Global read lock will be granted after
# this statement ends.
#
# Reaping DROP PROCEDURE.
Backport of revno: 3685 Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks against concurrent CREATE PROCEDURE This deadlock occured between a) CREATE PROCEDURE (or other commands listed below) b) FLUSH TABLES WITH READ LOCK If the execution of them happened in the following order: - a) opens a table (e.g. mysql.proc) - b) locks the global read lock (or GRL) - a) sleeps inside wait_if_global_read_lock() - b) increases refresh_version and sleeps waiting for old tables to go away Note that a) must start waiting on the GRL before FLUSH increases refresh_version. Otherwise a) won't wait on the GRL and instead close its tables for reopen, allowing FLUSH to complete and thus avoid the deadlock. With this patch the deadlock is avoided by making CREATE PROCEDURE acquire a protection against global read locks before it starts executing. This means that FLUSH TABLES WITH READ LOCK will have to wait until CREATE PROCEDURE completes before acquiring the global read lock, thereby avoiding the deadlock. This is implemented by introducing a new SQL command flag called CF_PROTECT_AGAINST_GRL. Commands marked with this flag will acquire a GRL protection in the beginning of mysql_execute_command(). This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and REVOKE ALL. All these commands either call open_grant_tables() or open_system_table_for_updated() which make them susceptible for this deadlock. The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number of commands that previously acquired GRL protection in their respective SQLCOM case in mysql_execute_command(). Test case that checks for GRL protection for CREATE PROCEDURE and CREATE USER added to mdl_sync.test.
2009-12-10 15:09:00 +01:00
# Connection 2
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
# Reaping FTWRL.
Backport of revno: 3685 Bug #48210 FLUSH TABLES WITH READ LOCK deadlocks against concurrent CREATE PROCEDURE This deadlock occured between a) CREATE PROCEDURE (or other commands listed below) b) FLUSH TABLES WITH READ LOCK If the execution of them happened in the following order: - a) opens a table (e.g. mysql.proc) - b) locks the global read lock (or GRL) - a) sleeps inside wait_if_global_read_lock() - b) increases refresh_version and sleeps waiting for old tables to go away Note that a) must start waiting on the GRL before FLUSH increases refresh_version. Otherwise a) won't wait on the GRL and instead close its tables for reopen, allowing FLUSH to complete and thus avoid the deadlock. With this patch the deadlock is avoided by making CREATE PROCEDURE acquire a protection against global read locks before it starts executing. This means that FLUSH TABLES WITH READ LOCK will have to wait until CREATE PROCEDURE completes before acquiring the global read lock, thereby avoiding the deadlock. This is implemented by introducing a new SQL command flag called CF_PROTECT_AGAINST_GRL. Commands marked with this flag will acquire a GRL protection in the beginning of mysql_execute_command(). This patch adds the flag to CREATE, ALTER and DROP for PROCEDURE and FUNCTION, as well as CREATE USER, DROP USER, RENAME USER and REVOKE ALL. All these commands either call open_grant_tables() or open_system_table_for_updated() which make them susceptible for this deadlock. The patch also adds the CF_PROTECT_AGAINST_GRL flag to a number of commands that previously acquired GRL protection in their respective SQLCOM case in mysql_execute_command(). Test case that checks for GRL protection for CREATE PROCEDURE and CREATE USER added to mdl_sync.test.
2009-12-10 15:09:00 +01:00
UNLOCK TABLES;
# Connection 1
SET DEBUG_SYNC= 'RESET';
#
# Bug#50786 Assertion `thd->mdl_context.trans_sentinel() == __null'
# failed in open_ltable()
#
# Supress warnings written to the log file
call mtr.add_suppression("Wait on a lock was aborted due to a pending exclusive lock");
DROP TABLE IF EXISTS t1, t2;
CREATE TABLE t1 (i INT);
CREATE TABLE t2 (i INT);
SET @old_general_log= @@global.general_log;
SET @@global.general_log= 1;
SET @old_log_output= @@global.log_output;
SET @@global.log_output= 'TABLE';
SET @old_sql_log_off= @@session.sql_log_off;
SET @@session.sql_log_off= 1;
# connection: con1
HANDLER t1 OPEN;
# connection: con3
SET @@session.sql_log_off= 1;
# connection: con2
SET DEBUG_SYNC= 'thr_multi_lock_after_thr_lock SIGNAL parked WAIT_FOR go';
# Sending:
SELECT 1;
# connection: con3
SET DEBUG_SYNC= 'now WAIT_FOR parked';
# connection: con1
# Sending:
SELECT 1;
# connection: con3
ALTER TABLE t1 ADD COLUMN j INT;
# connection: default
SET DEBUG_SYNC= 'now SIGNAL go';
# connection: con1
# Reaping SELECT 1
1
1
HANDLER t1 CLOSE;
# connection: con2
# Reaping SELECT 1
1
1
# connection: default
DROP TABLE t1, t2;
SET DEBUG_SYNC= 'RESET';
SET @@global.general_log= @old_general_log;
SET @@global.log_output= @old_log_output;
SET @@session.sql_log_off= @old_sql_log_off;
2010-02-08 23:19:55 +03:00
#
# Additional coverage for bug #50913 "Deadlock between
# open_and_lock_tables_derived and MDL". The main test
# case is in lock_multi.test
#
drop table if exists t1;
set debug_sync= 'RESET';
create table t1 (i int) engine=InnoDB;
# Switching to connection 'con50913_1'.
set debug_sync= 'thr_multi_lock_after_thr_lock SIGNAL parked WAIT_FOR go';
# Sending:
alter table t1 add column j int;
# Switching to connection 'default'.
# Wait until ALTER TABLE gets blocked on a sync point after
# acquiring thr_lock.c lock.
set debug_sync= 'now WAIT_FOR parked';
# The below statement should wait on MDL lock and not deadlock on
# thr_lock.c lock.
# Sending:
truncate table t1;
# Switching to connection 'con50913_2'.
# Wait until TRUNCATE TABLE is blocked on MDL lock.
# Unblock ALTER TABLE.
set debug_sync= 'now SIGNAL go';
# Switching to connection 'con50913_1'.
# Reaping ALTER TABLE.
# Switching to connection 'default'.
# Reaping TRUNCATE TABLE.
set debug_sync= 'RESET';
drop table t1;
#
# Test for bug #50998 "Deadlock in MDL code during test
# rqg_mdl_stability".
# Also provides coverage for the case when addition of
# waiting statement adds several loops in the waiters
# graph and therefore several searches for deadlock
# should be performed.
drop table if exists t1;
set debug_sync= 'RESET';
create table t1 (i int);
# Switching to connection 'con1'.
begin;
select * from t1;
i
# Switching to connection 'con2'.
begin;
select * from t1;
i
# Switching to connection 'default'.
# Start ALTER TABLE which will acquire SNW lock and
# table lock and get blocked on sync point.
set debug_sync= 'thr_multi_lock_after_thr_lock SIGNAL parked WAIT_FOR go';
# Sending:
alter table t1 add column j int;
# Switching to connection 'con1'.
# Wait until ALTER TABLE gets blocked on a sync point.
set debug_sync= 'now WAIT_FOR parked';
# Sending:
insert into t1 values (1);
# Switching to connection 'con2'.
# Sending:
insert into t1 values (1);
# Switching to connection 'con3'.
# Wait until both 'con1' and 'con2' are blocked trying to acquire
# SW lock on the table.
# Unblock ALTER TABLE. Since it will try to upgrade SNW to X lock
# deadlock with two loops in waiting graph will occur. Both loops
# should be found and DML statements in both 'con1' and 'con2'
# should be aborted with ER_LOCK_DEADLOCK errors.
set debug_sync= 'now SIGNAL go';
# Switching to connection 'con1'.
# Reaping INSERT. It should end with ER_LOCK_DEADLOCK error and
# not wait indefinitely (as it happened before the bugfix).
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
commit;
# Switching to connection 'con2'.
# Reaping INSERT.
ERROR 40001: Deadlock found when trying to get lock; try restarting transaction
commit;
# Switching to connection 'default'.
# Reap ALTER TABLE.
set debug_sync= 'RESET';
drop table t1;
#
# Bug#42643: InnoDB does not support replication of TRUNCATE TABLE
#
# Ensure that a acquired lock is not given up due to a conflict.
#
DROP TABLE IF EXISTS t1;
CREATE TABLE t1 (a INT) ENGINE=InnoDB;
INSERT INTO t1 VALUES (1),(2),(3);
# Connection: con1
Bug#49938: Failing assertion: inode or deadlock in fsp/fsp0fsp.c Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock - Incompatible change: truncate no longer resorts to a row by row delete if the storage engine does not support the truncate method. Consequently, the count of affected rows does not, in any case, reflect the actual number of rows. - Incompatible change: it is no longer possible to truncate a table that participates as a parent in a foreign key constraint, unless it is a self-referencing constraint (both parent and child are in the same table). To work around this incompatible change and still be able to truncate such tables, disable foreign checks with SET foreign_key_checks=0 before truncate. Alternatively, if foreign key checks are necessary, please use a DELETE statement without a WHERE condition. Problem description: The problem was that for storage engines that do not support truncate table via a external drop and recreate, such as InnoDB which implements truncate via a internal drop and recreate, the delete_all_rows method could be invoked with a shared metadata lock, causing problems if the engine needed exclusive access to some internal metadata. This problem originated with the fact that there is no truncate specific handler method, which ended up leading to a abuse of the delete_all_rows method that is primarily used for delete operations without a condition. Solution: The solution is to introduce a truncate handler method that is invoked when the engine does not support truncation via a table drop and recreate. This method is invoked under a exclusive metadata lock, so that there is only a single instance of the table when the method is invoked. Also, the method is not invoked and a error is thrown if the table is a parent in a non-self-referencing foreign key relationship. This was necessary to avoid inconsistency as some integrity checks are bypassed. This is inline with the fact that truncate is primarily a DDL operation that was designed to quickly remove all data from a table.
2010-10-06 11:34:28 -03:00
LOCK TABLES t1 WRITE;
SET debug_sync='upgrade_lock_for_truncate SIGNAL parked_truncate WAIT_FOR go_truncate';
TRUNCATE TABLE t1;
# Connection: default
SET debug_sync='now WAIT_FOR parked_truncate';
# Connection: con2
SET debug_sync='after_open_table_ignore_flush SIGNAL parked_show WAIT_FOR go_show';
SHOW FIELDS FROM t1;
# Connection: default
SET debug_sync='now WAIT_FOR parked_show';
# Connection: con3
SET debug_sync='after_flush_unlock SIGNAL parked_flush WAIT_FOR go_flush';
FLUSH TABLES t1;
# Connection: default
SET debug_sync='now WAIT_FOR parked_flush';
SET debug_sync='now SIGNAL go_truncate';
Bug#49938: Failing assertion: inode or deadlock in fsp/fsp0fsp.c Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock - Incompatible change: truncate no longer resorts to a row by row delete if the storage engine does not support the truncate method. Consequently, the count of affected rows does not, in any case, reflect the actual number of rows. - Incompatible change: it is no longer possible to truncate a table that participates as a parent in a foreign key constraint, unless it is a self-referencing constraint (both parent and child are in the same table). To work around this incompatible change and still be able to truncate such tables, disable foreign checks with SET foreign_key_checks=0 before truncate. Alternatively, if foreign key checks are necessary, please use a DELETE statement without a WHERE condition. Problem description: The problem was that for storage engines that do not support truncate table via a external drop and recreate, such as InnoDB which implements truncate via a internal drop and recreate, the delete_all_rows method could be invoked with a shared metadata lock, causing problems if the engine needed exclusive access to some internal metadata. This problem originated with the fact that there is no truncate specific handler method, which ended up leading to a abuse of the delete_all_rows method that is primarily used for delete operations without a condition. Solution: The solution is to introduce a truncate handler method that is invoked when the engine does not support truncation via a table drop and recreate. This method is invoked under a exclusive metadata lock, so that there is only a single instance of the table when the method is invoked. Also, the method is not invoked and a error is thrown if the table is a parent in a non-self-referencing foreign key relationship. This was necessary to avoid inconsistency as some integrity checks are bypassed. This is inline with the fact that truncate is primarily a DDL operation that was designed to quickly remove all data from a table.
2010-10-06 11:34:28 -03:00
# Ensure that truncate waits for a exclusive lock
SET debug_sync= 'now SIGNAL go_show';
Bug#49938: Failing assertion: inode or deadlock in fsp/fsp0fsp.c Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock - Incompatible change: truncate no longer resorts to a row by row delete if the storage engine does not support the truncate method. Consequently, the count of affected rows does not, in any case, reflect the actual number of rows. - Incompatible change: it is no longer possible to truncate a table that participates as a parent in a foreign key constraint, unless it is a self-referencing constraint (both parent and child are in the same table). To work around this incompatible change and still be able to truncate such tables, disable foreign checks with SET foreign_key_checks=0 before truncate. Alternatively, if foreign key checks are necessary, please use a DELETE statement without a WHERE condition. Problem description: The problem was that for storage engines that do not support truncate table via a external drop and recreate, such as InnoDB which implements truncate via a internal drop and recreate, the delete_all_rows method could be invoked with a shared metadata lock, causing problems if the engine needed exclusive access to some internal metadata. This problem originated with the fact that there is no truncate specific handler method, which ended up leading to a abuse of the delete_all_rows method that is primarily used for delete operations without a condition. Solution: The solution is to introduce a truncate handler method that is invoked when the engine does not support truncation via a table drop and recreate. This method is invoked under a exclusive metadata lock, so that there is only a single instance of the table when the method is invoked. Also, the method is not invoked and a error is thrown if the table is a parent in a non-self-referencing foreign key relationship. This was necessary to avoid inconsistency as some integrity checks are bypassed. This is inline with the fact that truncate is primarily a DDL operation that was designed to quickly remove all data from a table.
2010-10-06 11:34:28 -03:00
# Connection: con1 (TRUNCATE)
# Reaping...
UNLOCK TABLES;
# Connection: con2 (SHOW FIELDS FROM t1)
# Reaping...
Field Type Null Key Default Extra
a int(11) YES NULL
# Connection: default
SET debug_sync= 'now SIGNAL go_flush';
# Connection: con3 (FLUSH TABLES t1)
# Reaping...
# Connection: default
SET debug_sync= 'RESET';
DROP TABLE t1;
#
# Bug#52856 concurrent show columns or show full columns causes a crash!!!
#
CREATE TABLE t1(a CHAR(255));
SET DEBUG_SYNC= "get_schema_column SIGNAL waiting WAIT_FOR completed";
SHOW FULL COLUMNS FROM t1;
SET DEBUG_SYNC= "now WAIT_FOR waiting";
SHOW FULL COLUMNS FROM t1;
Field Type Collation Null Key Default Extra Privileges Comment
2010-05-31 12:38:45 +04:00
a char(255) latin1_swedish_ci YES NULL #
SET DEBUG_SYNC= "now SIGNAL completed";
Field Type Collation Null Key Default Extra Privileges Comment
2010-05-31 12:38:45 +04:00
a char(255) latin1_swedish_ci YES NULL #
DROP TABLE t1;
#
# Tests for schema-scope locks
#
DROP DATABASE IF EXISTS db1;
DROP DATABASE IF EXISTS db2;
# Test 1:
# CREATE DATABASE blocks database DDL on the same database, but
# not database DDL on different databases. Tests X vs X lock.
#
# Connection default
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
CREATE DATABASE db1;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
CREATE DATABASE db1;
# Connection con3
CREATE DATABASE db2;
ALTER DATABASE db2 DEFAULT CHARACTER SET utf8;
DROP DATABASE db2;
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: CREATE DATABASE db1
# Connection con2
# Reaping: CREATE DATABASE db1
ERROR HY000: Can't create database 'db1'; database exists
# Test 2:
# ALTER DATABASE blocks database DDL on the same database, but
# not database DDL on different databases. Tests X vs X lock.
#
# Connection default
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
ALTER DATABASE db1 DEFAULT CHARACTER SET utf8;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
ALTER DATABASE db1 DEFAULT CHARACTER SET utf8;
# Connection con3
CREATE DATABASE db2;
ALTER DATABASE db2 DEFAULT CHARACTER SET utf8;
DROP DATABASE db2;
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: ALTER DATABASE db1 DEFAULT CHARACTER SET utf8
# Connection con2
# Reaping: ALTER DATABASE db1 DEFAULT CHARACTER SET utf8
# Connection default
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
ALTER DATABASE db1 DEFAULT CHARACTER SET utf8;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
DROP DATABASE db1;
# Connection con3
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: ALTER DATABASE db1 DEFAULT CHARACTER SET utf8
# Connection con2
# Reaping: DROP DATABASE db1
CREATE DATABASE db1;
# Test 3:
# Two ALTER..UPGRADE of the same database are mutually exclusive, but
# two ALTER..UPGRADE of different databases are not. Tests X vs X lock.
#
# Connection default
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
ALTER DATABASE `#mysql50#a-b-c` UPGRADE DATA DIRECTORY NAME;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
ALTER DATABASE `#mysql50#a-b-c` UPGRADE DATA DIRECTORY NAME;
# Connection con3
ALTER DATABASE `#mysql50#a-b-c-d` UPGRADE DATA DIRECTORY NAME;
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: ALTER DATABASE '#mysql50#a-b-c' UPGRADE DATA DIRECTORY NAME
# Connection con2
# Reaping: ALTER DATABASE '#mysql50#a-b-c' UPGRADE DATA DIRECTORY NAME
ERROR 42000: Unknown database '#mysql50#a-b-c'
DROP DATABASE `a-b-c`;
DROP DATABASE `a-b-c-d`;
# Test 4:
# DROP DATABASE blocks database DDL on the same database, but
# not database DDL on different databases. Tests X vs X lock.
#
# Connection default
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
DROP DATABASE db1;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
DROP DATABASE db1;
# Connection con3
CREATE DATABASE db2;
ALTER DATABASE db2 DEFAULT CHARACTER SET utf8;
DROP DATABASE db2;
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: DROP DATABASE db1
# Connection con2
# Reaping: DROP DATABASE db1
ERROR HY000: Can't drop database 'db1'; database doesn't exist
# Connection default
CREATE DATABASE db1;
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
DROP DATABASE db1;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
ALTER DATABASE db1 DEFAULT CHARACTER SET utf8;
# Connection con3
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: DROP DATABASE db1
# Connection con2
# Reaping: ALTER DATABASE db1 DEFAULT CHARACTER SET utf8
Got one of the listed errors
# Test 5:
# Locked database name prevents CREATE of tables in that database.
# Tests X vs IX lock.
#
# Connection default
CREATE DATABASE db1;
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
DROP DATABASE db1;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
CREATE TABLE db1.t1 (a INT);
# Connection con3
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: DROP DATABASE db1
# Connection con2
# Reaping: CREATE TABLE db1.t1 (a INT)
ERROR 42000: Unknown database 'db1'
# Test 6:
# Locked database name prevents RENAME of tables to/from that database.
# Tests X vs IX lock.
#
# Connection default
CREATE DATABASE db1;
CREATE TABLE db1.t1 (a INT);
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
DROP DATABASE db1;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
RENAME TABLE db1.t1 TO test.t1;
# Connection con3
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: DROP DATABASE db1
# Connection con2
# Reaping: RENAME TABLE db1.t1 TO test.t1
Got one of the listed errors
# Connection default
CREATE DATABASE db1;
CREATE TABLE test.t2 (a INT);
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
DROP DATABASE db1;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
RENAME TABLE test.t2 TO db1.t2;
# Connection con3
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: DROP DATABASE db1
# Connection con2
# Reaping: RENAME TABLE test.t2 TO db1.t2
Got one of the listed errors
DROP TABLE test.t2;
# Test 7:
# Locked database name prevents DROP of tables in that database.
# Tests X vs IX lock.
#
# Connection default
CREATE DATABASE db1;
CREATE TABLE db1.t1 (a INT);
SET DEBUG_SYNC= 'after_wait_locked_schema_name SIGNAL locked WAIT_FOR blocked';
# Sending:
DROP DATABASE db1;
# Connection con2
SET DEBUG_SYNC= 'now WAIT_FOR locked';
# Sending:
DROP TABLE db1.t1;
# Connection con3
SET DEBUG_SYNC= 'now SIGNAL blocked';
# Connection default
# Reaping: DROP DATABASE db1
# Connection con2
# Reaping: DROP TABLE db1.t1
ERROR 42S02: Unknown table 't1'
# Connection default
SET DEBUG_SYNC= 'RESET';
#
# End of tests for schema-scope locks
#
#
# Tests of granted global S lock (FLUSH TABLE WITH READ LOCK)
#
CREATE DATABASE db1;
CREATE TABLE db1.t1(a INT);
# Connection default
FLUSH TABLE WITH READ LOCK;
# Connection con2
CREATE TABLE db1.t2(a INT);
# Connection default
UNLOCK TABLES;
# Connection con2
# Reaping CREATE TABLE db1.t2(a INT)
# Connection default
FLUSH TABLE WITH READ LOCK;
# Connection con2
ALTER DATABASE db1 DEFAULT CHARACTER SET utf8;
# Connection default
UNLOCK TABLES;
# Connection con2
# Reaping ALTER DATABASE db1 DEFAULT CHARACTER SET utf8
# Connection default
FLUSH TABLE WITH READ LOCK;
# Connection con2
FLUSH TABLE WITH READ LOCK;
UNLOCK TABLES;
# Connection default
UNLOCK TABLES;
DROP DATABASE db1;
Bug #56292 Deadlock with ALTER TABLE and MERGE tables ALTER TABLE on a MERGE table could cause a deadlock with two other connections if we reached a situation where: 1) A connection doing ALTER TABLE can't upgrade to MDL_EXCLUSIVE on the parent table, but holds TL_READ_NO_INSERT on the child tables. 2) A connection doing DELETE on a child table can't get TL_WRITE on it since ALTER TABLE holds TL_READ_NO_INSERT. 3) A connection doing SELECT on the parent table can't get TL_READ on the child tables since TL_WRITE is ahead in the lock queue, but holds MDL_SHARED_READ on the parent table preventing ALTER TABLE from upgrading. For regular tables, this deadlock is avoided by having ALTER TABLE take a MDL_SHARED_NO_WRITE metadata lock on the table. This prevents DELETE from acquiring MDL_SHARED_WRITE on the table before ALTER TABLE tries to upgrade to MDL_EXCLUSIVE. In the example above, SELECT would therefore not be blocked by the pending DELETE as DELETE would not be able to enter TL_WRITE in the table lock queue. This patch fixes the problem for merge tables by using the same metadata lock type for child tables as for the parent table. The child tables will in this case therefore be locked with MDL_SHARED_NO_WRITE, preventing DELETE from acquiring a metadata lock and enter into the table lock queue. Change in behavior: By taking the same metadata lock for child tables as for the parent table, LOCK TABLE on the parent table will now also implicitly lock the child tables. Since LOCK TABLE on the parent table now takes more than one metadata lock, it is possible for LOCK TABLE ... WRITE on the parent table or child tables to give ER_LOCK_DEADLOCK error. Test case added to mdl_sync.test. Merge.test/.result has been updated to reflect the change to LOCK TABLE.
2010-09-08 10:25:37 +02:00
#
# Bug#56292 Deadlock with ALTER TABLE and MERGE tables
#
DROP TABLE IF EXISTS t1, t2, m1;
CREATE TABLE t1(a INT) engine=MyISAM;
CREATE TABLE t2(a INT) engine=MyISAM;
CREATE TABLE m1(a INT) engine=MERGE UNION=(t1, t2);
INSERT INTO t1 VALUES (1), (2);
INSERT INTO t2 VALUES (3), (4);
# Connection con1
SET DEBUG_SYNC= 'mdl_upgrade_shared_lock_to_exclusive SIGNAL upgrade WAIT_FOR continue';
# Sending:
ALTER TABLE m1 engine=MERGE UNION=(t2, t1);
# Connection con2
# Waiting for ALTER TABLE to try lock upgrade
SET DEBUG_SYNC= 'now WAIT_FOR upgrade';
# Sending:
DELETE FROM t2 WHERE a = 3;
# Connection default
# Check that DELETE is waiting on a metadata lock and not a table lock.
# Now that DELETE blocks on a metadata lock, we should be able to do
# SELECT * FROM m1 here. SELECT used to be blocked by a DELETE table
# lock request.
SELECT * FROM m1;
a
1
2
3
4
# Resuming ALTER TABLE
SET DEBUG_SYNC= 'now SIGNAL continue';
# Connection con1
# Reaping: ALTER TABLE m1 engine=MERGE UNION=(t2, t1)
# Connection con2
# Reaping: DELETE FROM t2 WHERE a = 3
# Connection default
DROP TABLE m1, t1, t2;
SET DEBUG_SYNC= 'RESET';