mariadb/mysql-test/main/sp_sync.test

171 lines
4.6 KiB
Text
Raw Normal View History

# This test should work in embedded server after mysqltest is fixed
-- source include/not_embedded.inc
Fixed Bug#11756013 (formerly known as bug#47870): BOGUS "THE TABLE MYSQL.PROC IS MISSING,..." There was a race condition between loading a stored routine (function/procedure/trigger) specified by fully qualified name SCHEMA_NAME.PROC_NAME and dropping the stored routine database. The problem was that there is a window for race condition when one server thread tries to load a stored routine being executed and the other thread tries to drop the stored routine schema. This condition race window exists in implementation of function mysql_change_db() called by db_load_routine() during loading of stored routine to cache. Function mysql_change_db() calls check_db_dir_existence() that might failed because specified database was dropped during concurrent execution of DROP SCHEMA statement. db_load_routine() calls mysql_change_db() with flag 'force_switch' set to 'true' value so when referenced db is not found then my_error() is not called and function mysql_change_db() returns ok. This shadows information about schema opening error in db_load_routine(). Then db_load_routine() makes attempt to parse stored routine that is failed. This makes to return error to sp_cache_routines_and_add_tables_aux() but since during error generation a call to my_error wasn't made and hence THD::main_da wasn't set we set the generic "mysql.proc table corrupt" error when running sp_cache_routines_and_add_tables_aux(). The fix is to install an error handler inside db_load_routine() for the mysql_op_change_db() call, and check later if the ER_BAD_DB_ERROR was caught. sql/sql_db.cc: Added synchronization point "before_db_dir_check" to emulate a race condition during processing of CALL/DROP SCHEMA.
2011-06-23 15:41:04 +02:00
--echo Tests of synchronization of stored procedure execution.
--source include/have_debug_sync.inc
# Save the initial number of concurrent sessions.
--source include/count_sessions.inc
# Clean up resources used in this test case.
--disable_warnings
SET DEBUG_SYNC= 'RESET';
--enable_warnings
--echo #
--echo # Bug #30977 Concurrent statement using stored function and
--echo # DROP FUNCTION breaks SBR
--echo #
--echo # A stored routine could change after dispatch_command()
--echo # but before a MDL lock is taken. This must be noticed and the
--echo # sp cache flushed so the correct version can be loaded.
--echo #
connect (con2, localhost, root);
connection default;
CREATE FUNCTION f1() RETURNS INT RETURN 1;
--echo # Get f1 cached
SELECT f1();
--echo # Then start executing it again...
SET DEBUG_SYNC= 'before_execute_sql_command SIGNAL before WAIT_FOR changed';
--echo # Sending:
--send SELECT f1()
connection con2;
SET DEBUG_SYNC= 'now WAIT_FOR before';
--echo # ... but before f1 is locked, change it.
DROP FUNCTION f1;
CREATE FUNCTION f1() RETURNS INT RETURN 2;
SET DEBUG_SYNC= 'now SIGNAL changed';
--echo # We should now get '2' and not '1'.
connection default;
--echo # Reaping: SELECT f1()
--reap
disconnect con2;
DROP FUNCTION f1;
SET DEBUG_SYNC= 'RESET';
--echo #
--echo # Field translation items must be cleared in case of back-offs
--echo # for queries that use Information Schema tables. Otherwise
--echo # memory allocated in fix_fields() for views may end up referring
--echo # to freed memory.
--echo #
--disable_warnings
DROP FUNCTION IF EXISTS f1;
--enable_warnings
connect (con2, localhost, root);
connect (con3, localhost, root);
connection default;
CREATE FUNCTION f1() RETURNS INT RETURN 0;
connection con2;
SET DEBUG_SYNC= 'after_wait_locked_pname SIGNAL locked WAIT_FOR issued';
--echo # con2 will now have an x-lock on f1
--echo # Sending:
--send ALTER FUNCTION f1 COMMENT 'comment'
connection default;
SET DEBUG_SYNC= 'now WAIT_FOR locked';
--disable_result_log
--echo # This query will block due to the x-lock on f1 and back-off
--send SHOW OPEN TABLES WHERE f1()=0
connection con3;
let $wait_condition= SELECT COUNT(*)= 1 FROM information_schema.processlist
Part of fix for bug#52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible" to be pushed as separate patch. Replaced thread state name "Waiting for table", which was used by threads waiting for a metadata lock or table flush, with a set of names which better reflect types of resources being waited for. Also replaced "Table lock" thread state name, which was used by threads waiting on thr_lock.c table level lock, with more elaborate "Waiting for table level lock", to make it more consistent with other thread state names. Updated test cases and their results according to these changes. Fixed sys_vars.query_cache_wlock_invalidate_func test to not to wait for timeout of wait_condition.inc script. mysql-test/r/query_cache.result: Added test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables. mysql-test/suite/sys_vars/r/query_cache_wlock_invalidate_func.result: Fixed sys_vars.query_cache_wlock_invalidate_func test to not to wait for timeout of wait_condition.inc script. Reverted changes to test which introduced timeout and replaced waiting condition with a more appropriate one. Test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables was added to query_cache.test. mysql-test/suite/sys_vars/t/query_cache_wlock_invalidate_func.test: Fixed sys_vars.query_cache_wlock_invalidate_func test to not to wait for timeout of wait_condition.inc script. Reverted changes to test which introduced timeout and replaced waiting condition with a more appropriate one. Test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables was added to query_cache.test. mysql-test/t/query_cache.test: Added test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables. mysys/thr_lock.c: Replaced "Table lock" thread state name, which was used by threads waiting on thr_lock.c table level lock, with more elaborate "Waiting for table level lock", to make it consistent with thread state names which are used while waiting for metadata locks and table flush. sql/mdl.cc: Replaced thread state name "Waiting for table", which was used by threads waiting for a metadata lock or table flush, with a set of names which better reflect types of resources being waited for. To implement this: - Adjusted MDL_wait::timed_wait() to take thread state name as parameter. - Introduced method of MDL_key class which allows to get thread state name to be used while waiting for resource corresponding to the key and changed code to use it. Added array translating namespaces to thread state names as part of this change. sql/mdl.h: To implement this: - Adjusted MDL_wait::timed_wait() to take thread state name as parameter. - Introduced method of MDL_key class which allows to get thread state name to be used while waiting for resource corresponding to the key and changed code to use it. Added array translating namespaces to thread state names as part of this change. sql/sql_base.cc: Replaced thread state name "Waiting for table", which was used by threads waiting for table flush, with a more elaborate "Waiting for table flush".
2010-08-06 13:29:37 +02:00
WHERE state= 'Waiting for stored function metadata lock'
AND info='SHOW OPEN TABLES WHERE f1()=0';
--source include/wait_condition.inc
--echo # Check that the IS query is blocked before releasing the x-lock
SET DEBUG_SYNC= 'now SIGNAL issued';
connection default;
--echo # Reaping: ALTER FUNCTION f1 COMMENT 'comment'
--reap
--enable_result_log
DROP FUNCTION f1;
SET DEBUG_SYNC= 'RESET';
disconnect con2;
disconnect con3;
--echo #
--echo # Bug #48246 assert in close_thread_table
--echo #
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement. mysql-test/r/mdl_sync.result: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/r/sp_sync.result: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after renaming MDL_context:: m_waiting_for_lock rwlock to m_LOCK_waiting_for. mysql-test/suite/rpl/r/rpl_sp.result: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/suite/rpl/t/rpl_sp.test: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/t/mdl_sync.test: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/t/sp_sync.test: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). sql/mdl.cc: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Made releasing of the lock and granting it to the most appropriate pending request atomic operation. As result it became possible to wake up only those waiters requests from which can be satisfied at the moment as well as wake-up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solved thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in Bug #52289 "performance regression for MyISAM in sysbench OLTP_RW test". To emphasize above changes wake_up_waiters() was renamed to MDL_context::reschedule_waiters(). - Changed code to add tickets for new requests to the back of waiters queue and to select tickets to be satisfied from the head of the queue if possible (this makes scheduling of requests with the same priority fair). To be able to do this efficiently we now use for waiting and granted queues version of I_P_List class which provides fast push_back() method. - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status() (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. As another consequence MDL_wait::timed_wait() method was changed to handle timeout (optionally) and abort due to kill as signals which make signal slot occupied. - Renamed MDL_context::acquire_lock_impl() to acquire_lock(). Changed it to be able correctly process requests for shared locks when there are open HANDLERs, made this method more optimized for acquisition of shared locks. As part of this change moved code common between try_acquire_lock() and acquire_lock() to new try_acquire_lock_impl() method. Also adjusted acquire_lock()'s code to take into account the fact that in cases when lock is granted as result of MDL_context::reschedule_waiters() call (i.e. when it is granted after waiting for lock conflict to go away) updating MDL_lock state is responsibility of the thread calling reschedule_waiters(). - Changed MDL_context::find_deadlock() to send VICTIM signal even if victim is the context which has initiated deadlock detection. This is required in order to avoid races in cases when the same context simultaneously is chosen as a victim and its request for lock is satisfied. As result return value of this method became unnecessary and it was changed to return void. Adjusted MDL_lock::find_deadlock() method to take into account that now there can be a discrepancy between MDL_context::m_waiting_for value being set and real state of the ticket this member points to. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - Finally, removed MDL_context::wait_for_lock() method. sql/mdl.h: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since now in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. Also NORMAL_WAKE_UP signal became GRANTED, and timeouts and aborts due to kill became full blown signals rather than simple return values. - MDL_wait::timed_wait() now takes extra parameter that indicates whether signal should be set if timeout is reached. - Enabled fast push_back() operation in MDL_context::m_tickets list to make move_ticket_after_trans_sentinel() method more efficient. - Removed MDL_context::wait_for_lock() method. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - MDL_context::acquire_lock_impl() became acquire_lock(). - Introduced MDL_context::try_acquire_lock_impl() as a place for code shared by try_acquire_lock and acquire_lock(). - Due to fact that now VICTIM signal is sent even if victim is the context which has initiated deadlock detection find_deadlock() no longer needs a return value. sql/sql_base.cc: Implemented new approach to acquiring metadata locks in open_tables(). We no longer perform back-off when conflicting metadata lock is encountered. Instead we wait for this lock to go away while holding all locks which were acquired so far. Back-off is only used in situation when further waiting will cause a deadlock which could be avoided by performing back-off and restarting open_tables() process. Absence of waiting between back-off and restart of acquiring metadata locks can't lead to livelocks as MDL subsystem was changed to make release of lock and granting it to waiting lock an atomic action, so back-off will automatically give way to other participants of deadlock loop. Accordingly: - open_table_get_mdl_lock() and open_and_process_routine() were changed to wait for conflicting metadata lock to go away without back-off. Only if such wait leads to a deadlock back-off is requested. As part of this change new error handler class was introduced which converts, if possible, ER_LOCK_DEADLOCK error to a request for back-off and re-start of open_tables() process. - Open_table_context::recover_from_failed_open() was changed not to wait in case of metadata lock conflict. Instead we immediately proceed to re-acquiring locks. - Open_table_context::request_backoff_action() now always emits error if back-off is requested in the middle of transaction as we can't be sure that releasing lock which were acquired only by current statement will resolve a deadlock. Before this patch such situations were successfully detected thanks to the fact that we called MDL_context::wait_for_lock() method in recover_from_failed_open(). - In order to avoid deadlocks open_tables() code was adjusted to flush open HANDLERs for which there are pending requests for X locks before restarting the process of acquiring metadata locks. - Changed close_tables_for_reopen() not to reset MDL_request for tables belonging to the tail of prelocking list. It is no longer necessary as these MDL_request objects won't be used for any waiting. - Adjusted comment in tdc_wait_for_old_version() to avoid mentioning removed MDL_context::wait_for_lock() method. sql/sql_base.h: As we no longer wait for conflicting metadata lock away in Open_table_context::recover_from_failed_open() method, Open_table_context::OT_WAIT_MDL_LOCK action was renamed to OT_MDL_CONFLICT. Also Open_table_context::m_failed_mdl_request became unnecessary and was removed. sql/sql_plist.h: Extended I_P_List template to support efficient push_back() operation if it is parameterized with an appropriate policy class. sql/sql_show.cc: Adjusted code after removal of MDL_context::wait_for_lock() method. Now if one needs to acquire metadata lock with waiting one has to use a variant of MDL_context::acquire_lock() method.
2010-06-07 09:06:55 +02:00
CREATE TABLE t0 (b INTEGER);
CREATE TABLE t1 (a INTEGER);
CREATE FUNCTION f1(b INTEGER) RETURNS INTEGER RETURN 1;
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement. mysql-test/r/mdl_sync.result: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/r/sp_sync.result: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after renaming MDL_context:: m_waiting_for_lock rwlock to m_LOCK_waiting_for. mysql-test/suite/rpl/r/rpl_sp.result: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/suite/rpl/t/rpl_sp.test: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/t/mdl_sync.test: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/t/sp_sync.test: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). sql/mdl.cc: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Made releasing of the lock and granting it to the most appropriate pending request atomic operation. As result it became possible to wake up only those waiters requests from which can be satisfied at the moment as well as wake-up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solved thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in Bug #52289 "performance regression for MyISAM in sysbench OLTP_RW test". To emphasize above changes wake_up_waiters() was renamed to MDL_context::reschedule_waiters(). - Changed code to add tickets for new requests to the back of waiters queue and to select tickets to be satisfied from the head of the queue if possible (this makes scheduling of requests with the same priority fair). To be able to do this efficiently we now use for waiting and granted queues version of I_P_List class which provides fast push_back() method. - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status() (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. As another consequence MDL_wait::timed_wait() method was changed to handle timeout (optionally) and abort due to kill as signals which make signal slot occupied. - Renamed MDL_context::acquire_lock_impl() to acquire_lock(). Changed it to be able correctly process requests for shared locks when there are open HANDLERs, made this method more optimized for acquisition of shared locks. As part of this change moved code common between try_acquire_lock() and acquire_lock() to new try_acquire_lock_impl() method. Also adjusted acquire_lock()'s code to take into account the fact that in cases when lock is granted as result of MDL_context::reschedule_waiters() call (i.e. when it is granted after waiting for lock conflict to go away) updating MDL_lock state is responsibility of the thread calling reschedule_waiters(). - Changed MDL_context::find_deadlock() to send VICTIM signal even if victim is the context which has initiated deadlock detection. This is required in order to avoid races in cases when the same context simultaneously is chosen as a victim and its request for lock is satisfied. As result return value of this method became unnecessary and it was changed to return void. Adjusted MDL_lock::find_deadlock() method to take into account that now there can be a discrepancy between MDL_context::m_waiting_for value being set and real state of the ticket this member points to. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - Finally, removed MDL_context::wait_for_lock() method. sql/mdl.h: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since now in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. Also NORMAL_WAKE_UP signal became GRANTED, and timeouts and aborts due to kill became full blown signals rather than simple return values. - MDL_wait::timed_wait() now takes extra parameter that indicates whether signal should be set if timeout is reached. - Enabled fast push_back() operation in MDL_context::m_tickets list to make move_ticket_after_trans_sentinel() method more efficient. - Removed MDL_context::wait_for_lock() method. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - MDL_context::acquire_lock_impl() became acquire_lock(). - Introduced MDL_context::try_acquire_lock_impl() as a place for code shared by try_acquire_lock and acquire_lock(). - Due to fact that now VICTIM signal is sent even if victim is the context which has initiated deadlock detection find_deadlock() no longer needs a return value. sql/sql_base.cc: Implemented new approach to acquiring metadata locks in open_tables(). We no longer perform back-off when conflicting metadata lock is encountered. Instead we wait for this lock to go away while holding all locks which were acquired so far. Back-off is only used in situation when further waiting will cause a deadlock which could be avoided by performing back-off and restarting open_tables() process. Absence of waiting between back-off and restart of acquiring metadata locks can't lead to livelocks as MDL subsystem was changed to make release of lock and granting it to waiting lock an atomic action, so back-off will automatically give way to other participants of deadlock loop. Accordingly: - open_table_get_mdl_lock() and open_and_process_routine() were changed to wait for conflicting metadata lock to go away without back-off. Only if such wait leads to a deadlock back-off is requested. As part of this change new error handler class was introduced which converts, if possible, ER_LOCK_DEADLOCK error to a request for back-off and re-start of open_tables() process. - Open_table_context::recover_from_failed_open() was changed not to wait in case of metadata lock conflict. Instead we immediately proceed to re-acquiring locks. - Open_table_context::request_backoff_action() now always emits error if back-off is requested in the middle of transaction as we can't be sure that releasing lock which were acquired only by current statement will resolve a deadlock. Before this patch such situations were successfully detected thanks to the fact that we called MDL_context::wait_for_lock() method in recover_from_failed_open(). - In order to avoid deadlocks open_tables() code was adjusted to flush open HANDLERs for which there are pending requests for X locks before restarting the process of acquiring metadata locks. - Changed close_tables_for_reopen() not to reset MDL_request for tables belonging to the tail of prelocking list. It is no longer necessary as these MDL_request objects won't be used for any waiting. - Adjusted comment in tdc_wait_for_old_version() to avoid mentioning removed MDL_context::wait_for_lock() method. sql/sql_base.h: As we no longer wait for conflicting metadata lock away in Open_table_context::recover_from_failed_open() method, Open_table_context::OT_WAIT_MDL_LOCK action was renamed to OT_MDL_CONFLICT. Also Open_table_context::m_failed_mdl_request became unnecessary and was removed. sql/sql_plist.h: Extended I_P_List template to support efficient push_back() operation if it is parameterized with an appropriate policy class. sql/sql_show.cc: Adjusted code after removal of MDL_context::wait_for_lock() method. Now if one needs to acquire metadata lock with waiting one has to use a variant of MDL_context::acquire_lock() method.
2010-06-07 09:06:55 +02:00
CREATE PROCEDURE p1() SELECT COUNT(f1(a)) FROM t1, t0;
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement. mysql-test/r/mdl_sync.result: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/r/sp_sync.result: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after renaming MDL_context:: m_waiting_for_lock rwlock to m_LOCK_waiting_for. mysql-test/suite/rpl/r/rpl_sp.result: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/suite/rpl/t/rpl_sp.test: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/t/mdl_sync.test: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/t/sp_sync.test: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). sql/mdl.cc: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Made releasing of the lock and granting it to the most appropriate pending request atomic operation. As result it became possible to wake up only those waiters requests from which can be satisfied at the moment as well as wake-up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solved thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in Bug #52289 "performance regression for MyISAM in sysbench OLTP_RW test". To emphasize above changes wake_up_waiters() was renamed to MDL_context::reschedule_waiters(). - Changed code to add tickets for new requests to the back of waiters queue and to select tickets to be satisfied from the head of the queue if possible (this makes scheduling of requests with the same priority fair). To be able to do this efficiently we now use for waiting and granted queues version of I_P_List class which provides fast push_back() method. - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status() (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. As another consequence MDL_wait::timed_wait() method was changed to handle timeout (optionally) and abort due to kill as signals which make signal slot occupied. - Renamed MDL_context::acquire_lock_impl() to acquire_lock(). Changed it to be able correctly process requests for shared locks when there are open HANDLERs, made this method more optimized for acquisition of shared locks. As part of this change moved code common between try_acquire_lock() and acquire_lock() to new try_acquire_lock_impl() method. Also adjusted acquire_lock()'s code to take into account the fact that in cases when lock is granted as result of MDL_context::reschedule_waiters() call (i.e. when it is granted after waiting for lock conflict to go away) updating MDL_lock state is responsibility of the thread calling reschedule_waiters(). - Changed MDL_context::find_deadlock() to send VICTIM signal even if victim is the context which has initiated deadlock detection. This is required in order to avoid races in cases when the same context simultaneously is chosen as a victim and its request for lock is satisfied. As result return value of this method became unnecessary and it was changed to return void. Adjusted MDL_lock::find_deadlock() method to take into account that now there can be a discrepancy between MDL_context::m_waiting_for value being set and real state of the ticket this member points to. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - Finally, removed MDL_context::wait_for_lock() method. sql/mdl.h: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since now in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. Also NORMAL_WAKE_UP signal became GRANTED, and timeouts and aborts due to kill became full blown signals rather than simple return values. - MDL_wait::timed_wait() now takes extra parameter that indicates whether signal should be set if timeout is reached. - Enabled fast push_back() operation in MDL_context::m_tickets list to make move_ticket_after_trans_sentinel() method more efficient. - Removed MDL_context::wait_for_lock() method. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - MDL_context::acquire_lock_impl() became acquire_lock(). - Introduced MDL_context::try_acquire_lock_impl() as a place for code shared by try_acquire_lock and acquire_lock(). - Due to fact that now VICTIM signal is sent even if victim is the context which has initiated deadlock detection find_deadlock() no longer needs a return value. sql/sql_base.cc: Implemented new approach to acquiring metadata locks in open_tables(). We no longer perform back-off when conflicting metadata lock is encountered. Instead we wait for this lock to go away while holding all locks which were acquired so far. Back-off is only used in situation when further waiting will cause a deadlock which could be avoided by performing back-off and restarting open_tables() process. Absence of waiting between back-off and restart of acquiring metadata locks can't lead to livelocks as MDL subsystem was changed to make release of lock and granting it to waiting lock an atomic action, so back-off will automatically give way to other participants of deadlock loop. Accordingly: - open_table_get_mdl_lock() and open_and_process_routine() were changed to wait for conflicting metadata lock to go away without back-off. Only if such wait leads to a deadlock back-off is requested. As part of this change new error handler class was introduced which converts, if possible, ER_LOCK_DEADLOCK error to a request for back-off and re-start of open_tables() process. - Open_table_context::recover_from_failed_open() was changed not to wait in case of metadata lock conflict. Instead we immediately proceed to re-acquiring locks. - Open_table_context::request_backoff_action() now always emits error if back-off is requested in the middle of transaction as we can't be sure that releasing lock which were acquired only by current statement will resolve a deadlock. Before this patch such situations were successfully detected thanks to the fact that we called MDL_context::wait_for_lock() method in recover_from_failed_open(). - In order to avoid deadlocks open_tables() code was adjusted to flush open HANDLERs for which there are pending requests for X locks before restarting the process of acquiring metadata locks. - Changed close_tables_for_reopen() not to reset MDL_request for tables belonging to the tail of prelocking list. It is no longer necessary as these MDL_request objects won't be used for any waiting. - Adjusted comment in tdc_wait_for_old_version() to avoid mentioning removed MDL_context::wait_for_lock() method. sql/sql_base.h: As we no longer wait for conflicting metadata lock away in Open_table_context::recover_from_failed_open() method, Open_table_context::OT_WAIT_MDL_LOCK action was renamed to OT_MDL_CONFLICT. Also Open_table_context::m_failed_mdl_request became unnecessary and was removed. sql/sql_plist.h: Extended I_P_List template to support efficient push_back() operation if it is parameterized with an appropriate policy class. sql/sql_show.cc: Adjusted code after removal of MDL_context::wait_for_lock() method. Now if one needs to acquire metadata lock with waiting one has to use a variant of MDL_context::acquire_lock() method.
2010-06-07 09:06:55 +02:00
INSERT INTO t0 VALUES(1);
INSERT INTO t1 VALUES(1), (2);
connect (con2, localhost, root);
CALL p1();
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement. mysql-test/r/mdl_sync.result: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/r/sp_sync.result: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after renaming MDL_context:: m_waiting_for_lock rwlock to m_LOCK_waiting_for. mysql-test/suite/rpl/r/rpl_sp.result: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/suite/rpl/t/rpl_sp.test: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/t/mdl_sync.test: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/t/sp_sync.test: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). sql/mdl.cc: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Made releasing of the lock and granting it to the most appropriate pending request atomic operation. As result it became possible to wake up only those waiters requests from which can be satisfied at the moment as well as wake-up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solved thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in Bug #52289 "performance regression for MyISAM in sysbench OLTP_RW test". To emphasize above changes wake_up_waiters() was renamed to MDL_context::reschedule_waiters(). - Changed code to add tickets for new requests to the back of waiters queue and to select tickets to be satisfied from the head of the queue if possible (this makes scheduling of requests with the same priority fair). To be able to do this efficiently we now use for waiting and granted queues version of I_P_List class which provides fast push_back() method. - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status() (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. As another consequence MDL_wait::timed_wait() method was changed to handle timeout (optionally) and abort due to kill as signals which make signal slot occupied. - Renamed MDL_context::acquire_lock_impl() to acquire_lock(). Changed it to be able correctly process requests for shared locks when there are open HANDLERs, made this method more optimized for acquisition of shared locks. As part of this change moved code common between try_acquire_lock() and acquire_lock() to new try_acquire_lock_impl() method. Also adjusted acquire_lock()'s code to take into account the fact that in cases when lock is granted as result of MDL_context::reschedule_waiters() call (i.e. when it is granted after waiting for lock conflict to go away) updating MDL_lock state is responsibility of the thread calling reschedule_waiters(). - Changed MDL_context::find_deadlock() to send VICTIM signal even if victim is the context which has initiated deadlock detection. This is required in order to avoid races in cases when the same context simultaneously is chosen as a victim and its request for lock is satisfied. As result return value of this method became unnecessary and it was changed to return void. Adjusted MDL_lock::find_deadlock() method to take into account that now there can be a discrepancy between MDL_context::m_waiting_for value being set and real state of the ticket this member points to. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - Finally, removed MDL_context::wait_for_lock() method. sql/mdl.h: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since now in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. Also NORMAL_WAKE_UP signal became GRANTED, and timeouts and aborts due to kill became full blown signals rather than simple return values. - MDL_wait::timed_wait() now takes extra parameter that indicates whether signal should be set if timeout is reached. - Enabled fast push_back() operation in MDL_context::m_tickets list to make move_ticket_after_trans_sentinel() method more efficient. - Removed MDL_context::wait_for_lock() method. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - MDL_context::acquire_lock_impl() became acquire_lock(). - Introduced MDL_context::try_acquire_lock_impl() as a place for code shared by try_acquire_lock and acquire_lock(). - Due to fact that now VICTIM signal is sent even if victim is the context which has initiated deadlock detection find_deadlock() no longer needs a return value. sql/sql_base.cc: Implemented new approach to acquiring metadata locks in open_tables(). We no longer perform back-off when conflicting metadata lock is encountered. Instead we wait for this lock to go away while holding all locks which were acquired so far. Back-off is only used in situation when further waiting will cause a deadlock which could be avoided by performing back-off and restarting open_tables() process. Absence of waiting between back-off and restart of acquiring metadata locks can't lead to livelocks as MDL subsystem was changed to make release of lock and granting it to waiting lock an atomic action, so back-off will automatically give way to other participants of deadlock loop. Accordingly: - open_table_get_mdl_lock() and open_and_process_routine() were changed to wait for conflicting metadata lock to go away without back-off. Only if such wait leads to a deadlock back-off is requested. As part of this change new error handler class was introduced which converts, if possible, ER_LOCK_DEADLOCK error to a request for back-off and re-start of open_tables() process. - Open_table_context::recover_from_failed_open() was changed not to wait in case of metadata lock conflict. Instead we immediately proceed to re-acquiring locks. - Open_table_context::request_backoff_action() now always emits error if back-off is requested in the middle of transaction as we can't be sure that releasing lock which were acquired only by current statement will resolve a deadlock. Before this patch such situations were successfully detected thanks to the fact that we called MDL_context::wait_for_lock() method in recover_from_failed_open(). - In order to avoid deadlocks open_tables() code was adjusted to flush open HANDLERs for which there are pending requests for X locks before restarting the process of acquiring metadata locks. - Changed close_tables_for_reopen() not to reset MDL_request for tables belonging to the tail of prelocking list. It is no longer necessary as these MDL_request objects won't be used for any waiting. - Adjusted comment in tdc_wait_for_old_version() to avoid mentioning removed MDL_context::wait_for_lock() method. sql/sql_base.h: As we no longer wait for conflicting metadata lock away in Open_table_context::recover_from_failed_open() method, Open_table_context::OT_WAIT_MDL_LOCK action was renamed to OT_MDL_CONFLICT. Also Open_table_context::m_failed_mdl_request became unnecessary and was removed. sql/sql_plist.h: Extended I_P_List template to support efficient push_back() operation if it is parameterized with an appropriate policy class. sql/sql_show.cc: Adjusted code after removal of MDL_context::wait_for_lock() method. Now if one needs to acquire metadata lock with waiting one has to use a variant of MDL_context::acquire_lock() method.
2010-06-07 09:06:55 +02:00
SET DEBUG_SYNC= 'after_open_table_mdl_shared SIGNAL locked_t1 WAIT_FOR go_for_t0';
--echo # This call used to cause an assertion. MDL deadlock with upcoming
--echo # LOCK TABLES statement will cause back-off and retry.
--echo # A variable indicating if a prelocking list exists, used to be not
--echo # reset properly causing an eventual assert.
--echo # Sending:
--send CALL p1()
connection default;
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement. mysql-test/r/mdl_sync.result: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/r/sp_sync.result: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after renaming MDL_context:: m_waiting_for_lock rwlock to m_LOCK_waiting_for. mysql-test/suite/rpl/r/rpl_sp.result: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/suite/rpl/t/rpl_sp.test: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/t/mdl_sync.test: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/t/sp_sync.test: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). sql/mdl.cc: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Made releasing of the lock and granting it to the most appropriate pending request atomic operation. As result it became possible to wake up only those waiters requests from which can be satisfied at the moment as well as wake-up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solved thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in Bug #52289 "performance regression for MyISAM in sysbench OLTP_RW test". To emphasize above changes wake_up_waiters() was renamed to MDL_context::reschedule_waiters(). - Changed code to add tickets for new requests to the back of waiters queue and to select tickets to be satisfied from the head of the queue if possible (this makes scheduling of requests with the same priority fair). To be able to do this efficiently we now use for waiting and granted queues version of I_P_List class which provides fast push_back() method. - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status() (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. As another consequence MDL_wait::timed_wait() method was changed to handle timeout (optionally) and abort due to kill as signals which make signal slot occupied. - Renamed MDL_context::acquire_lock_impl() to acquire_lock(). Changed it to be able correctly process requests for shared locks when there are open HANDLERs, made this method more optimized for acquisition of shared locks. As part of this change moved code common between try_acquire_lock() and acquire_lock() to new try_acquire_lock_impl() method. Also adjusted acquire_lock()'s code to take into account the fact that in cases when lock is granted as result of MDL_context::reschedule_waiters() call (i.e. when it is granted after waiting for lock conflict to go away) updating MDL_lock state is responsibility of the thread calling reschedule_waiters(). - Changed MDL_context::find_deadlock() to send VICTIM signal even if victim is the context which has initiated deadlock detection. This is required in order to avoid races in cases when the same context simultaneously is chosen as a victim and its request for lock is satisfied. As result return value of this method became unnecessary and it was changed to return void. Adjusted MDL_lock::find_deadlock() method to take into account that now there can be a discrepancy between MDL_context::m_waiting_for value being set and real state of the ticket this member points to. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - Finally, removed MDL_context::wait_for_lock() method. sql/mdl.h: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since now in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. Also NORMAL_WAKE_UP signal became GRANTED, and timeouts and aborts due to kill became full blown signals rather than simple return values. - MDL_wait::timed_wait() now takes extra parameter that indicates whether signal should be set if timeout is reached. - Enabled fast push_back() operation in MDL_context::m_tickets list to make move_ticket_after_trans_sentinel() method more efficient. - Removed MDL_context::wait_for_lock() method. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - MDL_context::acquire_lock_impl() became acquire_lock(). - Introduced MDL_context::try_acquire_lock_impl() as a place for code shared by try_acquire_lock and acquire_lock(). - Due to fact that now VICTIM signal is sent even if victim is the context which has initiated deadlock detection find_deadlock() no longer needs a return value. sql/sql_base.cc: Implemented new approach to acquiring metadata locks in open_tables(). We no longer perform back-off when conflicting metadata lock is encountered. Instead we wait for this lock to go away while holding all locks which were acquired so far. Back-off is only used in situation when further waiting will cause a deadlock which could be avoided by performing back-off and restarting open_tables() process. Absence of waiting between back-off and restart of acquiring metadata locks can't lead to livelocks as MDL subsystem was changed to make release of lock and granting it to waiting lock an atomic action, so back-off will automatically give way to other participants of deadlock loop. Accordingly: - open_table_get_mdl_lock() and open_and_process_routine() were changed to wait for conflicting metadata lock to go away without back-off. Only if such wait leads to a deadlock back-off is requested. As part of this change new error handler class was introduced which converts, if possible, ER_LOCK_DEADLOCK error to a request for back-off and re-start of open_tables() process. - Open_table_context::recover_from_failed_open() was changed not to wait in case of metadata lock conflict. Instead we immediately proceed to re-acquiring locks. - Open_table_context::request_backoff_action() now always emits error if back-off is requested in the middle of transaction as we can't be sure that releasing lock which were acquired only by current statement will resolve a deadlock. Before this patch such situations were successfully detected thanks to the fact that we called MDL_context::wait_for_lock() method in recover_from_failed_open(). - In order to avoid deadlocks open_tables() code was adjusted to flush open HANDLERs for which there are pending requests for X locks before restarting the process of acquiring metadata locks. - Changed close_tables_for_reopen() not to reset MDL_request for tables belonging to the tail of prelocking list. It is no longer necessary as these MDL_request objects won't be used for any waiting. - Adjusted comment in tdc_wait_for_old_version() to avoid mentioning removed MDL_context::wait_for_lock() method. sql/sql_base.h: As we no longer wait for conflicting metadata lock away in Open_table_context::recover_from_failed_open() method, Open_table_context::OT_WAIT_MDL_LOCK action was renamed to OT_MDL_CONFLICT. Also Open_table_context::m_failed_mdl_request became unnecessary and was removed. sql/sql_plist.h: Extended I_P_List template to support efficient push_back() operation if it is parameterized with an appropriate policy class. sql/sql_show.cc: Adjusted code after removal of MDL_context::wait_for_lock() method. Now if one needs to acquire metadata lock with waiting one has to use a variant of MDL_context::acquire_lock() method.
2010-06-07 09:06:55 +02:00
SET DEBUG_SYNC= 'now WAIT_FOR locked_t1';
--echo # Issue LOCK TABLES statement which will enter in MDL deadlock
--echo # with CALL statement and as result will cause it to perform
--echo # back-off and retry.
SET DEBUG_SYNC= 'mdl_acquire_lock_wait SIGNAL go_for_t0';
LOCK TABLES t0 WRITE, t1 WRITE;
UNLOCK TABLES;
connection con2;
--echo # Reaping: CALL p1()
--reap;
connection default;
disconnect con2;
DROP PROCEDURE p1;
DROP FUNCTION f1;
Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement. mysql-test/r/mdl_sync.result: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/r/sp_sync.result: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after renaming MDL_context:: m_waiting_for_lock rwlock to m_LOCK_waiting_for. mysql-test/suite/rpl/r/rpl_sp.result: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/suite/rpl/t/rpl_sp.test: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/t/mdl_sync.test: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/t/sp_sync.test: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). sql/mdl.cc: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Made releasing of the lock and granting it to the most appropriate pending request atomic operation. As result it became possible to wake up only those waiters requests from which can be satisfied at the moment as well as wake-up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solved thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in Bug #52289 "performance regression for MyISAM in sysbench OLTP_RW test". To emphasize above changes wake_up_waiters() was renamed to MDL_context::reschedule_waiters(). - Changed code to add tickets for new requests to the back of waiters queue and to select tickets to be satisfied from the head of the queue if possible (this makes scheduling of requests with the same priority fair). To be able to do this efficiently we now use for waiting and granted queues version of I_P_List class which provides fast push_back() method. - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status() (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. As another consequence MDL_wait::timed_wait() method was changed to handle timeout (optionally) and abort due to kill as signals which make signal slot occupied. - Renamed MDL_context::acquire_lock_impl() to acquire_lock(). Changed it to be able correctly process requests for shared locks when there are open HANDLERs, made this method more optimized for acquisition of shared locks. As part of this change moved code common between try_acquire_lock() and acquire_lock() to new try_acquire_lock_impl() method. Also adjusted acquire_lock()'s code to take into account the fact that in cases when lock is granted as result of MDL_context::reschedule_waiters() call (i.e. when it is granted after waiting for lock conflict to go away) updating MDL_lock state is responsibility of the thread calling reschedule_waiters(). - Changed MDL_context::find_deadlock() to send VICTIM signal even if victim is the context which has initiated deadlock detection. This is required in order to avoid races in cases when the same context simultaneously is chosen as a victim and its request for lock is satisfied. As result return value of this method became unnecessary and it was changed to return void. Adjusted MDL_lock::find_deadlock() method to take into account that now there can be a discrepancy between MDL_context::m_waiting_for value being set and real state of the ticket this member points to. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - Finally, removed MDL_context::wait_for_lock() method. sql/mdl.h: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since now in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. Also NORMAL_WAKE_UP signal became GRANTED, and timeouts and aborts due to kill became full blown signals rather than simple return values. - MDL_wait::timed_wait() now takes extra parameter that indicates whether signal should be set if timeout is reached. - Enabled fast push_back() operation in MDL_context::m_tickets list to make move_ticket_after_trans_sentinel() method more efficient. - Removed MDL_context::wait_for_lock() method. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - MDL_context::acquire_lock_impl() became acquire_lock(). - Introduced MDL_context::try_acquire_lock_impl() as a place for code shared by try_acquire_lock and acquire_lock(). - Due to fact that now VICTIM signal is sent even if victim is the context which has initiated deadlock detection find_deadlock() no longer needs a return value. sql/sql_base.cc: Implemented new approach to acquiring metadata locks in open_tables(). We no longer perform back-off when conflicting metadata lock is encountered. Instead we wait for this lock to go away while holding all locks which were acquired so far. Back-off is only used in situation when further waiting will cause a deadlock which could be avoided by performing back-off and restarting open_tables() process. Absence of waiting between back-off and restart of acquiring metadata locks can't lead to livelocks as MDL subsystem was changed to make release of lock and granting it to waiting lock an atomic action, so back-off will automatically give way to other participants of deadlock loop. Accordingly: - open_table_get_mdl_lock() and open_and_process_routine() were changed to wait for conflicting metadata lock to go away without back-off. Only if such wait leads to a deadlock back-off is requested. As part of this change new error handler class was introduced which converts, if possible, ER_LOCK_DEADLOCK error to a request for back-off and re-start of open_tables() process. - Open_table_context::recover_from_failed_open() was changed not to wait in case of metadata lock conflict. Instead we immediately proceed to re-acquiring locks. - Open_table_context::request_backoff_action() now always emits error if back-off is requested in the middle of transaction as we can't be sure that releasing lock which were acquired only by current statement will resolve a deadlock. Before this patch such situations were successfully detected thanks to the fact that we called MDL_context::wait_for_lock() method in recover_from_failed_open(). - In order to avoid deadlocks open_tables() code was adjusted to flush open HANDLERs for which there are pending requests for X locks before restarting the process of acquiring metadata locks. - Changed close_tables_for_reopen() not to reset MDL_request for tables belonging to the tail of prelocking list. It is no longer necessary as these MDL_request objects won't be used for any waiting. - Adjusted comment in tdc_wait_for_old_version() to avoid mentioning removed MDL_context::wait_for_lock() method. sql/sql_base.h: As we no longer wait for conflicting metadata lock away in Open_table_context::recover_from_failed_open() method, Open_table_context::OT_WAIT_MDL_LOCK action was renamed to OT_MDL_CONFLICT. Also Open_table_context::m_failed_mdl_request became unnecessary and was removed. sql/sql_plist.h: Extended I_P_List template to support efficient push_back() operation if it is parameterized with an appropriate policy class. sql/sql_show.cc: Adjusted code after removal of MDL_context::wait_for_lock() method. Now if one needs to acquire metadata lock with waiting one has to use a variant of MDL_context::acquire_lock() method.
2010-06-07 09:06:55 +02:00
DROP TABLES t0, t1;
Fixed Bug#11756013 (formerly known as bug#47870): BOGUS "THE TABLE MYSQL.PROC IS MISSING,..." There was a race condition between loading a stored routine (function/procedure/trigger) specified by fully qualified name SCHEMA_NAME.PROC_NAME and dropping the stored routine database. The problem was that there is a window for race condition when one server thread tries to load a stored routine being executed and the other thread tries to drop the stored routine schema. This condition race window exists in implementation of function mysql_change_db() called by db_load_routine() during loading of stored routine to cache. Function mysql_change_db() calls check_db_dir_existence() that might failed because specified database was dropped during concurrent execution of DROP SCHEMA statement. db_load_routine() calls mysql_change_db() with flag 'force_switch' set to 'true' value so when referenced db is not found then my_error() is not called and function mysql_change_db() returns ok. This shadows information about schema opening error in db_load_routine(). Then db_load_routine() makes attempt to parse stored routine that is failed. This makes to return error to sp_cache_routines_and_add_tables_aux() but since during error generation a call to my_error wasn't made and hence THD::main_da wasn't set we set the generic "mysql.proc table corrupt" error when running sp_cache_routines_and_add_tables_aux(). The fix is to install an error handler inside db_load_routine() for the mysql_op_change_db() call, and check later if the ER_BAD_DB_ERROR was caught. sql/sql_db.cc: Added synchronization point "before_db_dir_check" to emulate a race condition during processing of CALL/DROP SCHEMA.
2011-06-23 15:41:04 +02:00
--echo #
--echo # test for bug#11756013
--echo #
--disable_warnings
DROP SCHEMA IF EXISTS s1;
--enable_warnings
CREATE SCHEMA s1;
CREATE PROCEDURE s1.p1() BEGIN END;
connect (con3, localhost, root);
SET DEBUG_SYNC='before_db_dir_check SIGNAL check_db WAIT_FOR dropped_schema';
--send CALL s1.p1
connection default;
SET DEBUG_SYNC='now WAIT_FOR check_db';
DROP SCHEMA s1;
SET DEBUG_SYNC='now SIGNAL dropped_schema';
connection con3;
--error ER_BAD_DB_ERROR
--reap
connection default;
disconnect con3;
SET DEBUG_SYNC = 'RESET';
# Check that all connections opened by test cases in this file are really
# gone so execution of other tests won't be affected by their presence.
--source include/wait_until_count_sessions.inc