mirror of
https://github.com/MariaDB/server.git
synced 2025-01-16 03:52:35 +01:00
141 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jon Olav Hauglid
|
ec002090b9 |
Bug #55223 assert in Protocol::end_statement during CREATE DATABASE
The problem was that a statement could cause an assert if it was aborted by KILL QUERY while it waited on a metadata lock. This assert checks that a statement either sends OK or an error to the client. If the bug was triggered on release builds, it caused OK to be sent to the client instead of ER_QUERY_INTERRUPTED. The root cause of the problem was that there are two separate ways to tell if a statement is killed: thd->killed and mysys_var->abort. KILL QUERY causes both to be set, thd->killed before mysys_var->abort. Also, both values are reset at the end of statement execution. This means that it is possible for KILL QUERY to first set thd->killed, then have the killed statement reset both thd->killed and mysys_var->abort and finally have KILL QUERY set mysys_var->abort. This means that the connection with the killed statement will start executing the next statement with the two values out of sync - i.e. thd->killed not set but mysys_var->abort set. Since mysys_var->abort is used to check if a wait for a metadata lock should be aborted, the next statement would immediately abort any such waiting. When waiting is aborted, no OK message is sent and thd->killed is checked to see if ER_QUERY_INTERRUPTED should be sent to the client. But since the->killed had been reset, neither OK nor an error message was sent to the client. This then triggered the assert. This patch fixes the problem by changing the metadata lock waiting code to check thd->killed. No test case added as reproducing the assert is dependent on very exact timing of two (or more) threads. The patch has been checked using RQG and the grammar posted on the bug report. |
||
|
Dmitry Lenev
|
a6a8e10c79 |
A pre-requisite for patch fixing bug #52044 "FLUSH TABLES
WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible", which adds information about waits caused by FLUSH TABLES statement to deadlock detector in MDL subsystem. Remove API supporting caching of pointers to TABLE_SHARE object in MDL subsystem and all code related to it. The problem was that locking requirements of code implementing this API conflicted with locking requirements of code which adds information about waits caused by flushes to deadlock detector in MDL subsystem (the former needed to lock LOCK_open or its future equivalent while having write-lock on MDL_lock's rwlock, and the latter needs to be able to read-lock MDL_lock rwlock while owning LOCK_open or its future equivalent). Since caching of pointers to TABLE_SHARE objects in MDL subsystem didn't bring expected performance benefits we decided to remove caching API rather than try to come up with some complex solution for this problem. |
||
|
Jon Olav Hauglid
|
9ff272fbbd |
A 5.5 version of the fix for Bug #54360 "Deadlock DROP/ALTER/CREATE
DATABASE with open HANDLER" Remove LOCK_create_db, database name locks, and use metadata locks instead. This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based deadlock detector in MDL, and paves the way for a safe, deadlock-free implementation of RENAME DATABASE. Database DDL statements will now take exclusive metadata locks on the database name, while table/view/routine DDL statements take intention exclusive locks on the database name. This prevents race conditions between database DDL and table/view/routine DDL. (e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE) By adding database name locks, this patch implements WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and WL#4985 "DDL locking: namespace/hierarchical locks". The patch also changes code to use init_one_table() where appropriate. The new lock_table_names() function requires TABLE_LIST::db_length to be set correctly, and this is taken care of by init_one_table(). This patch also adds a simple template to help work with the mysys HASH data structure. Most of the patch was written by Konstantin Osipov. |
||
|
Konstantin Osipov
|
429454f76e |
A new implementation for the TABLE_SHARE cache in MDL
subsystem. Fix a number of caveates that the previous implementation suffered from, including unprotected access to shared data and lax resource accounting (share->ref_count) that could lead to deadlocks. The new implementation still suffers from a number of potential deadlocks in some edge cases, and this is still not enabled by default. Especially since performance testing has shown that it gives only marginable (not even exceeding measuring accuracy) improvements. @todo: - Remove calls to close_cached_tables() with REFRESH_FAST, and have_lock, because they break the MDL cache. - rework FLUSH TABLES <list> to not use close_cached_tables() - make sure that whenever we set TABLE_SHARE::version to 0 we free MDL cache references to it. sql/mdl.cc: We may cache references to TABLE_SHARE objects in MDL_lock objects for tables. Create a separate MDL_lock class to represent a table. sql/mdl.h: Adjust the MDL caching API to avoid races. sql/sql_base.cc: Move all caching functionality close together. Implement a solution for deadlocks caused by close_cached_tables() when MDL cache is enabled (incomplete). sql/sql_yacc.yy: Adjust FLUSH rule to do the necessary initialization of TABLE_LIST elements used in for FLUSH TABLES <list>, and thus work OK with flush_mdl_cache() function. |
||
|
Dmitry Lenev
|
9dbd9ce185 |
Patch that changes approach to how we acquire metadata
locks for DML statements and changes the way MDL locks
are acquired/granted in contended case.
Instead of backing-off when a lock conflict is encountered
and waiting for it to go away before restarting open_tables()
process we now wait for lock to be released without releasing
any previously acquired locks. If conflicting lock goes away
we resume opening tables. If waiting leads to a deadlock we
try to resolve it by backing-off and restarting open_tables()
immediately.
As result both waiting for possibility to acquire and
acquiring of a metadata lock now always happen within the
same MDL API call. This has allowed to make release of a lock
and granting it to the most appropriate pending request an
atomic operation.
Thanks to this it became possible to wake up during release
of lock only those waiters which requests can be satisfied
at the moment as well as wake up only one waiter in case
when granting its request would prevent all other requests
from being satisfied. This solves thundering herd problem
which occured in cases when we were releasing some lock and
woke up many waiters for SNRW or X locks (this was the issue
in bug#52289 "performance regression for MyISAM in sysbench
OLTP_RW test".
This also allowed to implement more fair (FIFO) scheduling
among waiters with the same priority.
It also opens the door for introducing new types of requests
for metadata locks such as low-prio SNRW lock which is
necessary in order to support LOCK TABLES LOW_PRIORITY WRITE.
Notice that after this sometimes can report ER_LOCK_DEADLOCK
error in cases in which it has not happened before.
Particularly we will always report this error if waiting for
conflicting lock has happened in the middle of transaction
and resulted in a deadlock. Before this patch the error was
not reported if deadlock could have been resolved by backing
off all metadata locks acquired by the current statement.
mysql-test/r/mdl_sync.result:
Added test coverage for some aspects of deadlock handling in
metadata locking subsystem.
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
mysql-test/r/sp_sync.result:
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
mysql-test/suite/perfschema/r/dml_setup_instruments.result:
Adjusted test results after renaming MDL_context::
m_waiting_for_lock rwlock to m_LOCK_waiting_for.
mysql-test/suite/rpl/r/rpl_sp.result:
Adjusted test case after implementing new approach to
acquiring metadata locks in open_tables(). We no longer
release all MDL locks acquired by statement before waiting
for conflicting lock to go away. As result DROP FUNCTION
statement has to wait for DML statement which managed to
acquire metadata lock on function being dropped and now
waits for other conflicting metadata lock to go away.
mysql-test/suite/rpl/t/rpl_sp.test:
Adjusted test case after implementing new approach to
acquiring metadata locks in open_tables(). We no longer
release all MDL locks acquired by statement before waiting
for conflicting lock to go away. As result DROP FUNCTION
statement has to wait for DML statement which managed to
acquire metadata lock on function being dropped and now
waits for other conflicting metadata lock to go away.
mysql-test/t/mdl_sync.test:
Added test coverage for some aspects of deadlock handling in
metadata locking subsystem.
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
mysql-test/t/sp_sync.test:
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
sql/mdl.cc:
Changed MDL subsystem to support new approach to acquring
metadata locks in open tables and more fair and efficient
scheduling of metadata locks. To implement this:
- Made releasing of the lock and granting it to the most
appropriate pending request atomic operation. As result it
became possible to wake up only those waiters requests from
which can be satisfied at the moment as well as wake-up
only one waiter in case when granting its request would
prevent all other requests from being satisfied.
This solved thundering herd problem which occured in cases
when we were releasing some lock and woke up many waiters
for SNRW or X locks (this was the issue in Bug #52289
"performance regression for MyISAM in sysbench OLTP_RW
test".
To emphasize above changes wake_up_waiters() was renamed
to MDL_context::reschedule_waiters().
- Changed code to add tickets for new requests to the back of
waiters queue and to select tickets to be satisfied from
the head of the queue if possible (this makes scheduling of
requests with the same priority fair). To be able to do
this efficiently we now use for waiting and granted queues
version of I_P_List class which provides fast push_back()
method.
- Members and methods of MDL_context related to sending
and waiting for signal were moved to separate MDL_wait
class.
- Since in order to avoid race conditions we must grant the
lock only to the context which was not chosen as a victim
of deadlock, killed or aborted due to timeout
MDL_wait::set_status() (former awake()) was changed not to
send signal if signal slot is already occupied and to
indicate this fact through its return value. As another
consequence MDL_wait::timed_wait() method was changed to
handle timeout (optionally) and abort due to kill as
signals which make signal slot occupied.
- Renamed MDL_context::acquire_lock_impl() to acquire_lock().
Changed it to be able correctly process requests for shared
locks when there are open HANDLERs, made this method more
optimized for acquisition of shared locks. As part of this
change moved code common between try_acquire_lock() and
acquire_lock() to new try_acquire_lock_impl() method.
Also adjusted acquire_lock()'s code to take into account
the fact that in cases when lock is granted as result of
MDL_context::reschedule_waiters() call (i.e. when it is
granted after waiting for lock conflict to go away)
updating MDL_lock state is responsibility of the thread
calling reschedule_waiters().
- Changed MDL_context::find_deadlock() to send VICTIM
signal even if victim is the context which has initiated
deadlock detection. This is required in order to avoid
races in cases when the same context simultaneously is
chosen as a victim and its request for lock is satisfied.
As result return value of this method became unnecessary
and it was changed to return void.
Adjusted MDL_lock::find_deadlock() method to take into
account that now there can be a discrepancy between
MDL_context::m_waiting_for value being set and real state
of the ticket this member points to.
- Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
and MDL_context::stop_waiting() to done_waiting_for().
- Finally, removed MDL_context::wait_for_lock() method.
sql/mdl.h:
Changed MDL subsystem to support new approach to acquring
metadata locks in open tables and more fair and efficient
scheduling of metadata locks. To implement this:
- Members and methods of MDL_context related to sending
and waiting for signal were moved to separate MDL_wait
class.
- Since now in order to avoid race conditions we must grant
the lock only to the context which was not chosen as a
victim of deadlock, killed or aborted due to timeout
MDL_wait::set_status (former awake()) was changed not to
send signal if signal slot is already occupied and to
indicate this fact through its return value.
Also NORMAL_WAKE_UP signal became GRANTED, and timeouts
and aborts due to kill became full blown signals rather
than simple return values.
- MDL_wait::timed_wait() now takes extra parameter that
indicates whether signal should be set if timeout is
reached.
- Enabled fast push_back() operation in MDL_context::m_tickets
list to make move_ticket_after_trans_sentinel() method more
efficient.
- Removed MDL_context::wait_for_lock() method.
- Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
and MDL_context::stop_waiting() to done_waiting_for().
- MDL_context::acquire_lock_impl() became acquire_lock().
- Introduced MDL_context::try_acquire_lock_impl() as a
place for code shared by try_acquire_lock and
acquire_lock().
- Due to fact that now VICTIM signal is sent even if victim
is the context which has initiated deadlock detection
find_deadlock() no longer needs a return value.
sql/sql_base.cc:
Implemented new approach to acquiring metadata locks in
open_tables(). We no longer perform back-off when conflicting
metadata lock is encountered. Instead we wait for this lock
to go away while holding all locks which were acquired so
far. Back-off is only used in situation when further waiting
will cause a deadlock which could be avoided by performing
back-off and restarting open_tables() process. Absence of
waiting between back-off and restart of acquiring metadata
locks can't lead to livelocks as MDL subsystem was changed
to make release of lock and granting it to waiting lock
an atomic action, so back-off will automatically give way
to other participants of deadlock loop.
Accordingly:
- open_table_get_mdl_lock() and open_and_process_routine()
were changed to wait for conflicting metadata lock to
go away without back-off. Only if such wait leads to a
deadlock back-off is requested. As part of this change
new error handler class was introduced which converts,
if possible, ER_LOCK_DEADLOCK error to a request for
back-off and re-start of open_tables() process.
- Open_table_context::recover_from_failed_open() was changed
not to wait in case of metadata lock conflict. Instead we
immediately proceed to re-acquiring locks.
- Open_table_context::request_backoff_action() now always
emits error if back-off is requested in the middle of
transaction as we can't be sure that releasing lock
which were acquired only by current statement will
resolve a deadlock. Before this patch such situations were
successfully detected thanks to the fact that we called
MDL_context::wait_for_lock() method in
recover_from_failed_open().
- In order to avoid deadlocks open_tables() code was adjusted
to flush open HANDLERs for which there are pending requests
for X locks before restarting the process of acquiring
metadata locks.
- Changed close_tables_for_reopen() not to reset MDL_request
for tables belonging to the tail of prelocking list. It is
no longer necessary as these MDL_request objects won't be
used for any waiting.
- Adjusted comment in tdc_wait_for_old_version() to avoid
mentioning removed MDL_context::wait_for_lock() method.
sql/sql_base.h:
As we no longer wait for conflicting metadata lock away in
Open_table_context::recover_from_failed_open() method,
Open_table_context::OT_WAIT_MDL_LOCK action was renamed to
OT_MDL_CONFLICT.
Also Open_table_context::m_failed_mdl_request became
unnecessary and was removed.
sql/sql_plist.h:
Extended I_P_List template to support efficient push_back()
operation if it is parameterized with an appropriate policy
class.
sql/sql_show.cc:
Adjusted code after removal of MDL_context::wait_for_lock()
method. Now if one needs to acquire metadata lock with waiting
one has to use a variant of MDL_context::acquire_lock() method.
|
||
|
Jon Olav Hauglid
|
142a162c66 |
manual merge from mysql-trunk-bugfixing
Conflicts: Text conflict in mysql-test/r/archive.result Contents conflict in mysql-test/r/innodb_bug38231.result Text conflict in mysql-test/r/mdl_sync.result Text conflict in mysql-test/suite/binlog/t/disabled.def Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result Text conflict in mysql-test/t/archive.test Contents conflict in mysql-test/t/innodb_bug38231.test Text conflict in mysql-test/t/mdl_sync.test Text conflict in sql/sp_head.cc Text conflict in sql/sql_show.cc Text conflict in sql/table.cc Text conflict in sql/table.h |
||
|
Konstantin Osipov
|
e7854c86a7 |
A code review comment for Bug#52289.
Encapsulate the deadlock detection functionality into a visitor class, and separate it from the wait-for graph traversal code. Use "Internal iterator" and "Visitor" patterns to achieve the desired separation of responsibilities. Add comments. sql/mdl.cc: Encapsulate deadlock detection into a class. sql/mdl.h: Adjust for a rename of a class. |
||
|
Konstantin Osipov
|
8867c0a52c |
Add comments to a few MDL deadlock-search related variables
and methods. sql/mdl.cc: Add comments. sql/mdl.h: Add a comment. |
||
|
Davi Arnaut
|
a8c288054e |
Bug#53445: Build with -Wall and fix warnings that it generates
Fix various mismatches between function's language linkage. Any particular function that is declared in C++ but should be callable from C must have C linkage. Note that function types with different linkages are also distinct. Thus, if a function type is declared in C code, it will have C linkage (same if declared in a extern "C" block). client/mysql.cc: Mismatch between prototype and declaration. client/mysqltest.cc: mysqltest used to be C code. Use C linkage where appropriate. cmd-line-utils/readline/input.c: Isolate unreachable code. include/my_alloc.h: Function type must have C linkage. include/my_base.h: Function type must have C linkage. include/my_global.h: Add helper macros to avoid spurious namespace indentation. include/mysql.h.pp: Update ABI file. mysys/my_gethwaddr.c: Remove stray carriage return and fix coding style. plugin/semisync/semisync_master_plugin.cc: Callback function types have C linkage. plugin/semisync/semisync_slave_plugin.cc: Callback function types have C linkage. sql/derror.cc: Expected function type has C linkage. sql/field.cc: Use helper macro and fix indentation. sql/handler.cc: Expected function type has C linkage. sql/item_sum.cc: Correct function linkages. Remove now unnecessary cast. sql/item_sum.h: Add prototypes with the appropriate linkage as otherwise they are distinct. sql/mysqld.cc: Wrap functions in C linkage mode. sql/opt_range.cc: C language linkage is ignored for class member functions. sql/partition_info.cc: Add wrapper functions with C linkage for class member functions. sql/rpl_utility.h: Use helper macro and fix indentation. sql/sql_class.cc: Change type of thd argument -- THD is a class. Use helper macro and fix indentation. sql/sql_class.h: Change type of thd argument -- THD is a class. sql/sql_select.cc: Expected function type has C linkage. sql/sql_select.h: Move prototype to sql_test.h sql/sql_show.cc: Expected function type has C linkage. sql/sql_test.cc: Fix required function prototype and fix coding style. sql/sql_test.h: Removed unnecessary export and add another. storage/myisammrg/ha_myisammrg.cc: Expected function type has C linkage. storage/perfschema/pfs.cc: PSI headers are declared with C language linkage, which also applies to function types. |
||
|
Dmitry Lenev
|
bee0f214fd |
Pre-requisite patch for bug #51263 "Deadlock between
transactional SELECT and ALTER TABLE ... REBUILD PARTITION".
The goal of this patch is to decouple type of metadata
lock acquired for table by open_tables() from type of
table-level lock to be acquired on it.
To achieve this we change approach to how we determine what
type of metadata lock should be acquired on table to be open.
Now instead of inferring it at open_tables() time from flags
and type of table-level lock we rely on that type of metadata
lock is properly set at parsing time and is not changed
further.
sql/ha_ndbcluster.cc:
Now one needs to properly initialize table list element's
MDL_request object before calling mysql_rm_table_part2().
sql/lock.cc:
lock_table_names() no longer initializes table list elements'
MDL_request objects. Now proper initialization of these
requests is a responsibility of the caller.
sql/lock.h:
Removed MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag which became
unnecessary. Thanks to the fact that we don't reset type of
requests for metadata locks between re-executions we now can
figure out that upgradable locks are requested by simply
looking at their type which were set in the parser. As result
this flag became redundant.
sql/mdl.h:
Added version of new operator which simplifies allocation of
MDL_request objects on a MEM_ROOT.
sql/sp_head.cc:
Added comment explaining why it is OK to infer type of
metadata lock to request from type of table-level lock
for prelocking.
Added enum_mdl_type argument to sp_add_to_query_tables()
to simplify its usage in trigger implementation.
sql/sp_head.h:
Added enum_mdl_type argument to sp_add_to_query_tables()
to simplify its usage in trigger implementation.
sql/sql_base.cc:
- open_table_get_mdl_lock():
Preserve type of MDL_request for table list element which
was set in the parser by creating MDL_request objects on
memory root if MYSQL_OPEN_FORCE_SHARED_MDL or
MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag were specified.
Thanks to this and to the fact that we no longer reset
type of requests for metadata locks between re-executions
we no longer need to acquire exclusive metadata lock on
table to be created in a special way. This lock is acquired
by code handling acquiring of upgradable locks.
Also changed signature/calling convention for this function
to simplify its usage.
- Accordingly special lock strategy for table list elements
which was used for such locks became unnecessary and was
removed. Other strategies were renamed.
- Since we no longer have guarantee that MDL_request object
which were not satisfied due to lock conflict belongs to
table list element Open_table_context class and its methods
were extended to remember pointer to MDL_request which has
caused problem at request_backoff_action() time and use it
in recover_from_failed_open(). Similar approach is used
for cases when problem from which we need to recover is
not related to MDL but to the table itself. In this case
we store pointer to the element of table list.
- Changed open_tables()/open_tables_check_upgradable_mdl()/
open_tables_acquire_upgradable_mdl() not to rely on
MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to understand when
upgradable metadata locks should be acquired and not to
infer type of MDL lock from type of table-level lock.
Instead we assume that type of MDL to be acquired was set
in the parser (we can do this as type of MDL_request is
no longer reset between re-executions).
sql/sql_class.h:
Since we no longer have guarantee that MDL_request object
which were not satisfied due to lock conflict belongs to
table list element Open_table_context class and its methods
were extended to remember pointer to MDL_request which has
caused problem at request_backoff_action() time and use it
in recover_from_failed_open(). Similar approach is used
for cases when problem from which we need to recover is
not related to MDL but to the table itself. In this case
we store pointer to the element of table list.
sql/sql_db.cc:
Now one needs to properly initialize table list element's
MDL_request object before calling mysql_rm_table_part2()
or mysql_rename_tables().
sql/sql_lex.cc:
st_select_lex/st_select_lex_node::add_table_to_list() method
now has argument which allows specify type of metadata lock
to be requested for table list element being added.
sql/sql_lex.h:
- st_select_lex/st_select_lex_node::add_table_to_list()
method now has argument which specifies type of metadata
lock to be requested for table list element being added.
This allows to explicitly set type of MDL lock to be
acquired for a DDL statement in parser. It is also more
future-proof than inferring type of MDL request from type
of table-level lock.
- Added Yacc_state::m_mdl_type member which specifies which
type of metadata lock should be requested for tables to be
added to table list by a grammar rule in cases when the same
rule is used in several statements requiring different kinds
of metadata locks.
sql/sql_parse.cc:
- st_select_lex::add_table_to_list() method now has argument
which specifies type of metadata lock to be requested for
table list element being added. This allows to explicitly
set type of MDL lock to be acquired for a DDL statement in
parser. It is also more future-proof than inferring type of
MDL request from type of table-level lock.
- EXCLUSIVE_DOWNGRADABLE_MDL lock strategy has a new name -
OTLS_DOWNGRADE_IF_EXISTS.
- Adjusted LOCK TABLES implementation to the fact that we no
longer infer type of metadata lock to be acquired from table
level lock and that type of MDL request is set at parsing.
And thus MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag became
unnecessary.
sql/sql_prepare.cc:
TABLE_LIST's lock strategy SHARED_MDL was renamed to OTLS_NONE
as now it means that metadata lock should not be changed during
call to open_table() (if it has been already acquired) and is
also used for exclusive metadata lock.
sql/sql_show.cc:
st_select_lex::add_table_to_list() method now has argument
which specifies type of metadata lock to be requested for
table list element being added.
sql/sql_table.cc:
- Adjusted mysql_admin_table()'s code to the fact that
open_tables() no longer determines what kind of metadata
lock should be obtained basing on type of table-level
lock and flags. Instead type of metadata lock for table
to be open should be set before calling open_tables().
- Changed mysql_alter_table() code to the facts:
a) that now it is responsibility of caller to properly
initalize MDL_request in table list elements before calling
lock_table_names()
b) and that MYSQL_OPEN_TAKE_UPGRADABLE_MDL is no longer
necessary since type of metadata lock to be obtained
at open_tables() time is set during parsing.
- Changed code of mysql_recreate_table() to properly set
type of metadata and table-level lock to be obtained
by mysql_alter_table() which it calls.
sql/sql_trigger.cc:
Instead of relying on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to
force open_tables() to take an upgradable lock we now specify
exact type of lock to be taken when constructing table list
element for table to be open for CREATE/DROP TRIGGER.
sql/sql_view.cc:
We no longer use TABLE_LIST::EXCLUSIVE_MDL strategy to force
open_tables() to take an exclusive metadata lock on view to
be created. Instead we rely on parser setting proper type of
metadata lock to request and open_tables() acquiring it.
This became possible thanks to the fact that we no longer
reset type of MDL_request between statement re-executions.
sql/sql_yacc.yy:
Instead of inferring type of MDL_request for table to be
open from type of table-level lock and flags passed to
open_tables() we now explicitly specify them at parsing.
This became possible thanks to the fact that we no longer
reset type of MDL_request between statement re-executions.
In future this should allow to decouple type of metadata
lock from type of table-level lock.
The only exception to this approach is statements implemented
through mysql_admin_table() which re-uses same table list
element several times with different types of table-level
and metadata locks.
We now also properly initialize MDL_request objects for table
list elements which are later passed to lock_table_names()
function.
sql/table.cc:
Do not reset type of MDL_request between statement
re-executions. This became unnecessesary as we no longer
change type of MDL_request residing in table list element.
In its turn this change allows to set type of MDL_request
only once - at parsing time.
sql/table.h:
Got rid of TABLE_LIST::EXCLUSIVE_MDL lock strategy.
Now we can specify that we need to acquire exclusive lock
on table to be processed by open_tables() through setting
an appropriate type of MDL_request at parsing time (this
became possible thanks to the fact that we no longer reset
types of MDL_request's belonging to table list elements
between statement re-execution).
Strategy SHARED_MDL was renamed to OTLS_NONE as now it
means that metadata lock should not be changed during call
to open_table() (if it has been already acquired) and is
also used for exclusive metadata lock.
Strategy EXCLUSIVE_DOWNGRADABLE_MDL was renamed to
OTLS_DOWNGRADE_IF_EXISTS.
|
||
|
Joerg Bruehe
|
056e670542 |
Fix an issue with the IBM C++ compiler
which was detected during the build of 5.5.3-m3. This requires version 9 of IBM C++ to help. More fixes will still be needed, it is very strict (or rather: a bit picky?) about templates. |
||
|
Marc Alff
|
a7c9bf2ccf |
Bug#51295 Build warnings in mdl.cc
Before this fix, the performance schema instrumentation in mdl.h / mdl.cc was incomplete, causing: - build warnings, - no data collection for the performance schema This fix: - added instrumentation helpers for the new preferred reader read write lock, mysql_prlock_* - implemented completely the performance schema instrumentation of mdl.h / mdl.cc |
||
|
Dmitry Lenev
|
6b5c4a9ef6 |
Fix for bug #51105 "MDL deadlock in rqg_mdl_stability test
on Windows". On platforms where read-write lock implementation does not prefer readers by default (Windows, Solaris) server might have deadlocked while detecting MDL deadlock. MDL deadlock detector relies on the fact that read-write locks which are used in its implementation prefer readers (see new comment for MDL_lock::m_rwlock for details). So far MDL code assumed that default implementation of read/write locks for the system has this property. Indeed, this turned out ot be wrong, for example, for Windows or Solaris. Thus MDL deadlock detector might have deadlocked on these systems. This fix simply adds portable implementation of read/write lock which prefer readers and changes MDL code to use this new type of synchronization primitive. No test case is added as existing rqg_mdl_stability test can serve as one. config.h.cmake: Check for presence of pthread_rwlockattr_setkind_np to be able to determine if system natively supports read-write locks for which we can specify if readers or writers should be preferred. configure.cmake: Check for presence of pthread_rwlockattr_setkind_np to be able to determine if system natively supports read-write locks for which we can specify if readers or writers should be preferred. configure.in: Check for presence of pthread_rwlockattr_setkind_np to be able to determine if system natively supports read-write locks for which we can specify if readers or writers should be preferred. include/my_pthread.h: Added support for portable read-write locks which prefer readers. To do so extended existing my_rw_lock_t implementation to support selection of whom to prefer depending on a flag. mysys/thr_rwlock.c: Extended existing my_rw_lock_t implementation to support selection of whom to prefer depending on a flag. Added rw_pr_init() function implementing initialization of read-write locks preferring readers. sql/mdl.cc: Use portable read-write locks which prefer readers instead of relying on that system implementation of read-write locks has this property (this was true for Linux/NPTL but was false, for example, for Windows and Solaris). Added comment explaining why preferring readers is important for MDL deadlock detector (thanks to Serg for example!). sql/mdl.h: Use portable read-write locks which prefer readers instead of relying on that system implementation of read-write locks has this property (this was true for Linux/NPTL but was false, for example, for Windows and Solaris). |
||
|
Dmitry Lenev
|
be3e256d25 |
Fix for bug #51136 "Crash in pthread_rwlock_rdlock on
TEMPORARY + HANDLER + LOCK + SP". Server crashed when one: 1) Opened HANDLER or acquired global read lock 2) Then locked one or several temporary tables with LOCK TABLES statement (but no base tables). 3) Then issued any statement causing commit (explicit or implicit). 4) Issued statement which should have closed HANDLER or released global read lock. The problem was that when entering LOCK TABLES mode in the scenario described above we incorrectly set transactional MDL sentinel to zero. As result during commit all metadata locks were released (including lock for open HANDLER or global metadata shared lock). Indeed, attempt to release metadata lock for the second time which happened during HANLDER CLOSE or during release of GLR caused crash. This patch fixes problem by changing MDL_context's set_trans_sentinel() method to set sentinel to correct value (it should point to the most recent ticket). mysql-test/include/handler.inc: Added test for bug #51136 "Crash in pthread_rwlock_rdlock on TEMPORARY + HANDLER + LOCK + SP". mysql-test/r/flush.result: Updated test results (see flush.test). mysql-test/r/handler_innodb.result: Updated test results (see include/handler.inc). mysql-test/r/handler_myisam.result: Updated test results (see include/handler.inc). mysql-test/t/flush.test: Added additional coverage for bug #51136 "Crash in pthread_rwlock_rdlock on TEMPORARY + HANDLER + LOCK + SP". sql/mdl.h: When setting new value of transactional sentinel use pointer to the most recent ticket instead of value returned by MDL_context::mdl_savepoint(). This allows to handle correctly situation when the new value of sentinel should be the same as its current value (MDL_context::mdl_savepoint() returns NULL in this case). |
||
|
Jon Olav Hauglid
|
3d6a89e792 |
Bug #45225 Locking: hang if drop table with no timeout
This patch introduces timeouts for metadata locks. The timeout is specified in seconds using the new dynamic system variable "lock_wait_timeout" which has both GLOBAL and SESSION scopes. Allowed values range from 1 to 31536000 seconds (= 1 year). The default value is 1 year. The new server parameter "lock-wait-timeout" can be used to set the default value parameter upon server startup. "lock_wait_timeout" applies to all statements that use metadata locks. These include DML and DDL operations on tables, views, stored procedures and stored functions. They also include LOCK TABLES, FLUSH TABLES WITH READ LOCK and HANDLER statements. The patch also changes thr_lock.c code (table data locks used by MyISAM and other simplistic engines) to use the same system variable. InnoDB row locks are unaffected. One exception to the handling of the "lock_wait_timeout" variable is delayed inserts. All delayed inserts are executed with a timeout of 1 year regardless of the setting for the global variable. As the connection issuing the delayed insert gets no notification of delayed insert timeouts, we want to avoid unnecessary timeouts. It's important to note that the timeout value is used for each lock acquired and that one statement can take more than one lock. A statement can therefore block for longer than the lock_wait_timeout value before reporting a timeout error. When lock timeout occurs, ER_LOCK_WAIT_TIMEOUT is reported. Test case added to lock_multi.test. include/my_pthread.h: Added macros for comparing two timespec structs. include/thr_lock.h: Introduced timeouts for thr_lock.c locks. mysql-test/r/mysqld--help-notwin.result: Updated result file with the new server variable. mysql-test/r/mysqld--help-win.result: Updated result file with the new server variable. mysql-test/suite/sys_vars/r/lock_wait_timeout_basic.result: Added basic test for the new server variable. mysql-test/suite/sys_vars/t/lock_wait_timeout_basic.test: Added basic test for the new server variable. mysys/thr_lock.c: Introduced timeouts for thr_lock.c locks. sql/mdl.cc: Introduced timeouts for metadata locks. sql/mdl.h: Introduced timeouts for metadata locks. sql/sql_base.cc: Introduced timeouts in tdc_wait_for_old_versions(). sql/sql_class.h: Added new server variable lock_wait_timeout. sql/sys_vars.cc: Added new server variable lock_wait_timeout. |
||
|
Konstantin Osipov
|
a6f804228d |
A post-merge fix for next-mr -> next-4284 merge:
Make all mutexes and conditions of type mysql_mutex_t, mysql_cond_t, since it's now the expectation of THD::awake(). |
||
|
Dmitry Lenev
|
f677e97746 |
A follow-up for the patch which implemented new
type-of-operation-aware metadata locks and added a wait-for graph based deadlock detector to the MDL subsystem (this patch fixed bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table"). Removed unused and redundant method. |
||
|
Jon Olav Hauglid
|
b417300c9a |
Bug #50786 Assertion `thd->mdl_context.trans_sentinel() == __null'
failed in open_ltable() The problem was too restrictive asserts that enforced that open_ltable() was called without any active HANDLERs, LOCK TABLES or global read locks. However, this can happen in several cases when opening system tables. The assert would, for example, be triggered when drop function was called from a connection with active HANDLERs as this would cause open_ltable() to be called for mysql.proc. The assert could also be triggered when using table-based general log (mysql.general_log). This patch removes the asserts since they will be triggered in several legitimate cases and because the asserts are no longer relevant due to changes in how locks are released. The patch also fixes set_needs_thr_lock_abort() that before ignored its parameter and always set the member variable to TRUE. Test case added to mdl_sync.test. Thanks to Dmitry Lenev for help with this bug! |
||
|
Konstantin Osipov
|
665100b69d | Merge next-mr -> next-4284. | ||
|
Dmitry Lenev
|
e89c08b670 |
Fix for sporadical crashes of lock_multi_bug38499.test
caused by patch which implemented new type-of-operation-aware metadata locks and added a wait-for graph based deadlock detector to the MDL subsystem (this patch fixed bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table"). Crashes were caused by a race in MDL_context::try_acquire_lock(). This method added MDL_ticket to the list of granted tickets and released lock protecting list before setting MDL_ticket::m_lock. Thus some other thread was able to see ticket without properly set m_lock member for some short period of time. If this thread called method involving this member during this period crash happened. This fix ensures that MDL_ticket::m_lock is set in all cases when ticket is added to granted/pending lists in MDL_lock. sql/mdl.cc: We must set MDL_ticket::m_lock member before adding ticket to the list of granted tickets, since such tickets can be accessed by other threads which might call methods using this member. Added assert which ensures that all MDL_tickets which are added to the granted/pending lists have properly set MDL_ticket::m_lock member. sql/mdl.h: Adjusted comment describing MDL_ticket::m_lock member to reflect current reality. Added accessor method for this member. |
||
|
Dmitry Lenev
|
eba5d30e67 |
Implement new type-of-operation-aware metadata locks.
Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list. mysql-test/include/handler.inc: Adjusted test case to trigger an execution path on which bug 41110 "crash with handler command when used concurrently with alter table" and bug 41112 "crash in mysql_ha_close_table/get_lock_data with alter table" were originally discovered. Left old test case which no longer triggers this execution path for the sake of coverage. Added test coverage for HANDLER SQL statements and type-aware metadata locks. Added a test for the global shared lock and HANDLER SQL. Updated tests to take into account that the old simple deadlock detection heuristics was replaced with a graph-based deadlock detector. mysql-test/r/debug_sync.result: Updated results (see debug_sync.test). mysql-test/r/handler_innodb.result: Updated results (see handler.inc test). mysql-test/r/handler_myisam.result: Updated results (see handler.inc test). mysql-test/r/innodb-lock.result: Updated results (see innodb-lock.test). mysql-test/r/innodb_mysql_lock.result: Updated results (see innodb_mysql_lock.test). mysql-test/r/lock.result: Updated results (see lock.test). mysql-test/r/lock_multi.result: Updated results (see lock_multi.test). mysql-test/r/lock_sync.result: Updated results (see lock_sync.test). mysql-test/r/mdl_sync.result: Updated results (see mdl_sync.test). mysql-test/r/sp-threads.result: SHOW PROCESSLIST output has changed due to the fact that waiting for LOCK TABLES WRITE now happens within metadata locking subsystem. mysql-test/r/truncate_coverage.result: Updated results (see truncate_coverage.test). mysql-test/suite/funcs_1/datadict/processlist_val.inc: SELECT FROM I_S.PROCESSLIST output has changed due to fact that waiting for LOCK TABLES WRITE now happens within metadata locking subsystem. mysql-test/suite/funcs_1/r/processlist_val_no_prot.result: SELECT FROM I_S.PROCESSLIST output has changed due to fact that waiting for LOCK TABLES WRITE now happens within metadata locking subsystem. mysql-test/suite/rpl/t/rpl_sp.test: Updated to a new SHOW PROCESSLIST state name. mysql-test/t/debug_sync.test: Use LOCK TABLES READ instead of LOCK TABLES WRITE as the latter no longer allows to trigger execution path involving waiting on thr_lock.c lock and therefore reaching debug sync-point covered by this test. mysql-test/t/innodb-lock.test: Adjusted test case to the fact that innodb_table_locks=0 option is no longer supported, since LOCK TABLES WRITE handles all its conflicts within MDL subsystem. mysql-test/t/innodb_mysql_lock.test: Added test for bug #37346 "innodb does not detect deadlock between update and alter table". mysql-test/t/lock.test: Added test coverage which checks the fact that we no longer support DDL under LOCK TABLES on tables which were locked implicitly. Adjusted existing test cases accordingly. mysql-test/t/lock_multi.test: Added test for bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock". Adjusted other test cases to take into account the fact that waiting for LOCK TABLES ... WRITE now happens within MDL subsystem. mysql-test/t/lock_sync.test: Since LOCK TABLES ... WRITE now takes SNRW metadata lock for tables locked explicitly we have to implicitly lock InnoDB tables (through view) to trigger the table-level lock conflict between TL_WRITE and TL_WRITE_ALLOW_WRITE. mysql-test/t/mdl_sync.test: Added basic test coverage for type-of-operation-aware metadata locks. Also covered with tests some use cases involving HANDLER statements in which a deadlock could arise. Adjusted existing tests to take type-of-operation-aware MDL into account. mysql-test/t/multi_update.test: Update to a new SHOW PROCESSLIST state name. mysql-test/t/truncate_coverage.test: Adjusted test case after making LOCK TABLES WRITE to wait until transactions that use the table to be locked are completed. Updated to the changed name of DEBUG_SYNC point. sql/handler.cc: Global read lock functionality has been moved into a class. sql/lock.cc: Global read lock functionality has been moved into a class. Updated code to use the new MDL API. sql/mdl.cc: Introduced new type-of-operation aware metadata locks. To do this: - Changed MDL_lock to use one list for waiting requests and one list for granted requests. For each list, added a bitmap that holds information what lock types a list contains. Added a helper class MDL_lock::List to manipulate with granted and waited lists while keeping the bitmaps in sync with list contents. - Changed lock-compatibility functions to use bitmaps that define compatibility. - Introduced a graph based deadlock detector inspired by waiting_threads.c from Maria implementation. - Now that we have a deadlock detector, and no longer have a global lock to protect individual lock objects, but rather use an rw lock per object, removed redundant code for upgrade, and the global read lock. Changed the MDL API to no longer require the caller to acquire the global intention exclusive lock by means of a separate method. Removed a few more methods that became redundant. - Removed deadlock detection heuristic, it has been made obsolete by the deadlock detector. - With operation-type-aware metadata locks, MDL subsystem has become aware of potential conflicts between DDL and open transactions. This made it possible to remove calls to mysql_abort_transactions_with_shared_lock() from acquisition paths for exclusive lock and lock upgrade. Now we can simply wait for these transactions to complete without fear of deadlock. Function mysql_lock_abort() has also become unnecessary for all conflicting cases except when a DDL conflicts with a connection that has an open HANDLER. sql/mdl.h: Introduced new type-of-operation aware metadata locks. Introduced a graph based deadlock detector and supporting methods. Added comments. God rid of redundant API calls. Renamed m_lt_or_ha_sentinel to m_trans_sentinel, since now it guards the global read lock as well as LOCK TABLES and HANDLER locks. sql/mysql_priv.h: Moved the global read lock functionality into a class. Added MYSQL_OPEN_FORCE_SHARED_MDL flag which forces open_tables() to take MDL_SHARED on tables instead of metadata locks specified in the parser. We use this to allow PREPARE run concurrently in presence of LOCK TABLES ... WRITE. Added signature for find_table_for_mdl_ugprade(). sql/set_var.cc: Global read lock functionality has been moved into a class. sql/sp_head.cc: When creating TABLE_LIST elements for prelocking or system tables set the type of request for metadata lock according to the operation that will be performed on the table. sql/sql_base.cc: - Updated code to use the new MDL API. - In order to avoid locks starvation we take upgradable locks all at once. As result implicitly locked tables no longer get an upgradable lock. Consequently DDL and FLUSH TABLES for such tables is prohibited. find_write_locked_table() was replaced by find_table_for_mdl_upgrade() function. open_table() was adjusted to return TABLE instance with upgradable ticket when necessary. - We no longer wait for all locks on OT_WAIT back off action -- only on the lock that caused the wait conflict. Moreover, now we distinguish cases when we have to wait due to conflict in MDL and old version of table in TDC. - Upate mysql_notify_threads_having_share_locks() to only abort thr_lock.c waits of threads that have open HANDLERs, since lock conflicts with only these threads now can lead to deadlocks not detectable by the MDL deadlock detector. - Remove mysql_abort_transactions_with_shared_locks() which is no longer needed. sql/sql_class.cc: Global read lock functionality has been moved into a class. Re-arranged code in THD::cleanup() to simplify assert. sql/sql_class.h: Introduced class to incapsulate global read lock functionality. Now sentinel in MDL subsystem guards the global read lock as well as LOCK TABLES and HANDLER locks. Adjusted code accordingly. sql/sql_db.cc: Global read lock functionality has been moved into a class. sql/sql_delete.cc: We no longer acquire upgradable metadata locks on tables which are locked by LOCK TABLES implicitly. As result TRUNCATE TABLE is no longer allowed for such tables. Updated code to use the new MDL API. sql/sql_handler.cc: Inform MDL_context about presence of open HANDLERs. Since HANLDERs break MDL protocol by acquiring table-level lock while holding only S metadata lock on a table MDL subsystem should take special care about such contexts (Now this is the only case when mysql_lock_abort() is used). sql/sql_parse.cc: Global read lock functionality has been moved into a class. Do not take upgradable metadata locks when opening tables for CREATE TABLE SELECT as it is not necessary and limits concurrency. When initializing TABLE_LIST objects before adding them to the table list set the type of request for metadata lock according to the operation that will be performed on the table. We no longer acquire upgradable metadata locks on tables which are locked by LOCK TABLES implicitly. As result FLUSH TABLES is no longer allowed for such tables. sql/sql_prepare.cc: Use MYSQL_OPEN_FORCE_SHARED_MDL flag when opening tables during PREPARE. This allows PREPARE to run concurrently in presence of LOCK TABLES ... WRITE. sql/sql_rename.cc: Global read lock functionality has been moved into a class. sql/sql_show.cc: Updated code to use the new MDL API. sql/sql_table.cc: Global read lock functionality has been moved into a class. We no longer acquire upgradable metadata locks on tables which are locked by LOCK TABLES implicitly. As result DROP TABLE is no longer allowed for such tables. Updated code to use the new MDL API. sql/sql_trigger.cc: Global read lock functionality has been moved into a class. We no longer acquire upgradable metadata locks on tables which are locked by LOCK TABLES implicitly. As result CREATE/DROP TRIGGER is no longer allowed for such tables. Updated code to use the new MDL API. sql/sql_view.cc: Global read lock functionality has been moved into a class. Fixed results of wrong merge that led to misuse of GLR API. CREATE VIEW statement is not a commit statement. sql/table.cc: When resetting TABLE_LIST objects for PS or SP re-execution set the type of request for metadata lock according to the operation that will be performed on the table. Do the same in auxiliary function initializing metadata lock requests in a table list. sql/table.h: When initializing TABLE_LIST objects set the type of request for metadata lock according to the operation that will be performed on the table. sql/transaction.cc: Global read lock functionality has been moved into a class. |
||
|
Dmitry Lenev
|
6ddd01c27a |
Patch that changes metadata locking subsystem to use mutex per lock and
condition variable per context instead of one mutex and one conditional variable for the whole subsystem. This should increase concurrency in this subsystem. It also opens the way for further changes which are necessary to solve such bugs as bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". Two other notable changes done by this patch: - MDL subsystem no longer implicitly acquires global intention exclusive metadata lock when per-object metadata lock is acquired. Now this has to be done by explicit calls outside of MDL subsystem. - Instead of using separate MDL_context for opening system tables/tables for purposes of I_S we now create MDL savepoint in the main context before opening tables and rollback to this savepoint after closing them. This means that it is now possible to get ER_LOCK_DEADLOCK error even not inside a transaction. This might happen in unlikely case when one runs DDL on one of system tables while also running DDL on some other tables. Cases when this ER_LOCK_DEADLOCK error is not justified will be addressed by advanced deadlock detector for MDL subsystem which we plan to implement. mysql-test/include/handler.inc: Adjusted handler_myisam.test and handler_innodb.test to the fact that exclusive metadata locks on tables are now acquired according to alphabetical order of fully qualified table names instead of order in which tables are mentioned in statement. mysql-test/r/handler_innodb.result: Adjusted handler_myisam.test and handler_innodb.test to the fact that exclusive metadata locks on tables are now acquired according to alphabetical order of fully qualified table names instead of order in which tables are mentioned in statement. mysql-test/r/handler_myisam.result: Adjusted handler_myisam.test and handler_innodb.test to the fact that exclusive metadata locks on tables are now acquired according to alphabetical order of fully qualified table names instead of order in which tables are mentioned in statement. mysql-test/r/mdl_sync.result: Adjusted mdl_sync.test to the fact that exclusive metadata locks on tables are now acquired according to alphabetical order of fully qualified table names instead of order in which tables are mentioned in statement. mysql-test/t/mdl_sync.test: Adjusted mdl_sync.test to the fact that exclusive metadata locks on tables are now acquired according to alphabetical order of fully qualified table names instead of order in which tables are mentioned in statement. sql/events.cc: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing system tables was changed to use Open_tables_backup instead of Open_table_state class as well. sql/ha_ndbcluster.cc: Since manipulations with open table state no longer install proxy MDL_context it does not make sense to perform them in order to satisfy assert in mysql_rm_tables_part2(). Removed them per agreement with Cluster team. This has not broken test suite since scenario in which deadlock can occur and assertion fails is not covered by tests. sql/lock.cc: MDL subsystem no longer implicitly acquires global intention exclusive metadata lock when per-object exclusive metadata lock is acquired. Now this has to be done by explicit calls outside of MDL subsystem. sql/log.cc: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing system tables was changed to use Open_tables_backup instead of Open_table_state class as well. sql/mdl.cc: Changed metadata locking subsystem to use mutex per lock and condition variable per context instead of one mutex and one conditional variable for the whole subsystem. Changed approach to handling of global metadata locks. Instead of implicitly acquiring intention exclusive locks when user requests per-object upgradeable or exclusive locks now we require them to be acquired explicitly in the same way as ordinary metadata locks. In fact global lock are now ordinary metadata locks in new GLOBAL namespace. To implement these changes: - Removed LOCK_mdl mutex and COND_mdl condition variable. - Introduced MDL_lock::m_mutex mutexes which protect individual lock objects. - Replaced mdl_locks hash with MDL_map class, which has hash for MDL_lock objects as a member and separate mutex which protects this hash. Methods of this class allow to find(), find_or_create() or remove() MDL_lock objects in concurrency-friendly fashion (i.e. for most common operation, find_or_create(), we don't acquire MDL_lock::m_mutex while holding MDL_map::m_mutex. Thanks to MikaelR for this idea and benchmarks!). Added three auxiliary members to MDL_lock class (m_is_destroyed, m_ref_usage, m_ref_release) to support this concurrency-friendly behavior. - Introduced MDL_context::m_ctx_wakeup_cond condition variable to be used for waiting until this context's pending request can be satisfied or its thread has to perform actions to resolve potential deadlock. Context which want to wait add ticket corresponding to the request to an appropriate queue of waiters in MDL_lock object so they can be noticed when other contexts change state of lock and be awaken by them by signalling on MDL_context::m_ctx_wakeup_cond. As consequence MDL_ticket objects has to be used for any waiting in metadata locking subsystem including one which happens in MDL_context::wait_for_locks() method. Another consequence is that MDL_context is no longer copyable and can't be saved/restored when working with system tables. - Made MDL_lock an abstract class, which delegates specifying exact compatibility matrix to its descendants. Added MDL_global_lock child class for global lock (The old is_lock_type_compatible() method became can_grant_lock() method of this class). Added MDL_object_lock class to represent per-object lock (The old MDL_lock::can_grant_lock() became its method). Choice between two classes happens based on MDL namespace in MDL_lock::create() method. - Got rid of MDL_lock::type member as its meaning became ambigous for global locks. - To simplify waking up of contexts waiting for lock split waiting queue in MDL_lock class in two queues. One for pending requests for exclusive (including intention exclusive) locks and another for requests for shared locks. - Added virtual wake_up_waiters() method to MDL_lock, MDL_global_lock and MDL_object_lock classes which allows to wake up waiting contexts after state of lock changes. Replaced old duplicated code with calls to this method. - Adjusted MDL_context::try_acquire_shared_lock()/exclusive_lock()/ global_shared_lock(), MDL_ticket::upgrade_shared_lock_to_exclusive_lock() and MDL_context::release_ticket() methods to use MDL_map and MDL_lock::m_mutex instead of single LOCK_mdl mutex and wake up waiters according to the approach described above. The latter method also was renamed to MDL_context::release_lock(). - Changed MDL_context::try_acquire_shared_lock()/exclusive_lock() and release_lock() not to handle global locks. They are now supposed to be taken explicitly like ordinary metadata locks. - Added helper MDL_context::try_acquire_global_intention_exclusive_lock() and acquire_global_intention_exclusive_lock() methods. - Moved common code from MDL_context::acquire_global_shared_lock() and acquire_global_intention_exclusive_lock() to new method - MDL_context::acquire_lock_impl(). - Moved common code from MDL_context::try_acquire_shared_lock(), try_acquire_global_intention_exclusive_lock()/exclusive_lock() to MDL_context::try_acquire_lock_impl(). - Since acquiring of several exclusive locks can no longer happen under single LOCK_mdl mutex the approach to it had to be changed. Now we do it in one by one fashion. This is done in alphabetical order to avoid deadlocks. Changed MDL_context::acquire_exclusive_locks() accordingly (as part of this change moved code responsible for acquiring single exclusive lock to new MDL_context::acquire_exclusive_lock_impl() method). - Since we no longer have single LOCK_mdl mutex which protects all MDL_context::m_is_waiting_in_mdl members using these members to determine if we have really awaken context holding conflicting shared lock became inconvinient. Got rid of this member and changed notify_shared_lock() helper function and process of acquiring of/upgrading to exclusive lock not to rely on such information. Now in MDL_context::acquire_exclusive_lock_impl() and MDL_ticket::upgrade_shared_lock_to_exclusive_lock() we simply re-try to wake up threads holding conflicting shared locks after small time out. - Adjusted MDL_context::can_wait_lead_to_deadlock() and MDL_ticket::has_pending_conflicting_lock() to use per-lock mutexes instead of LOCK_mdl. To do this introduced MDL_lock::has_pending_exclusive_lock() method. sql/mdl.h: Changed metadata locking subsystem to use mutex per lock and condition variable per context instead of one mutex and one conditional variable for the whole subsystem. In order to implement this change: - Added MDL_key::cmp() method to be able to sort MDL_key objects alphabetically. Changed length fields in MDL_key class to uint16 as 16-bit is enough for length of any key. - Changed MDL_ticket::get_ctx() to return pointer to non-const object in order to be able to use MDL_context::awake() method for such contexts. - Got rid of unlocked versions of can_wait_lead_to_deadlock()/ has_pending_conflicting_lock() methods in MDL_context and MDL_ticket. We no longer has single mutex which protects all locks. Thus one always has to use versions of these methods which acquire per-lock mutexes. - MDL_request_list type of list now counts its elements. - Added MDL_context::m_ctx_wakeup_cond condition variable to be used for waiting until this context's pending request can be satisfied or its thread has to perform actions to resolve potential deadlock. Added awake() method to wake up context from such wait. Addition of condition variable made MDL_context uncopyable. As result we no longer can save/restore MDL_context when working with system tables. Instead we create MDL savepoint before opening those tables and rollback to it once they are closed. - MDL_context::release_ticket() became release_lock() method. - Added auxiliary MDL_context::acquire_exclusive_lock_impl() method which does all necessary work to acquire exclusive lock on one object but should not be used directly as it does not enforce any asserts ensuring that no deadlocks are possible. - Since we no longer need to know if thread trying to acquire exclusive lock managed to wake up any threads having conflicting shared locks (as, anyway, we will try to wake up such threads again shortly) - MDL_context::m_is_waiting_in_mdl member became unnecessary and notify_shared_lock() no longer needs to be friend of MDL_context. Changed approach to handling of global metadata locks. Instead of implicitly acquiring intention exclusive locks when user requests per-object upgradeable or exclusive locks now we require them to be acquired explicitly in the same way as ordinary metadata locks. - Added new GLOBAL namespace for such locks. - Added new type of lock to be requested MDL_INTENTION_EXCLISIVE. - Added MDL_context::try_acquire_global_intention_exclusive_lock() and acquire_global_intention_exclusive_lock() methods. - Moved common code from MDL_context::acquire_global_shared_lock() and acquire_global_intention_exclusive_lock() to new method - MDL_context::acquire_lock_impl(). - Moved common code from MDL_context::try_acquire_shared_lock(), try_acquire_global_intention_exclusive_lock()/exclusive_lock() to MDL_context::try_acquire_lock_impl(). - Added helper MDL_context::is_global_lock_owner() method to be able easily to find what kind of global lock this context holds. - MDL_context::m_has_global_shared_lock became unnecessary as global read lock is now represented by ordinary ticket. - Removed assert in MDL_context::set_lt_or_ha_sentinel() which became false for cases when we execute LOCK TABLES under global read lock mode. sql/mysql_priv.h: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result calls opening/closing system tables were changed to use Open_tables_backup instead of Open_table_state class as well. sql/sp.cc: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing system tables was changed to use Open_tables_backup instead of Open_table_state class as well. sql/sp.h: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing system tables was changed to use Open_tables_backup instead of Open_table_state class as well. sql/sql_base.cc: close_thread_tables(): Since we no longer use separate MDL_context for opening system tables we need to avoid releasing all transaction locks when closing system table. Releasing metadata lock on system table is now responsibility of THD::restore_backup_open_tables_state(). open_table_get_mdl_lock(), Open_table_context::recover_from_failed_open(): MDL subsystem no longer implicitly acquires global intention exclusive metadata lock when per-object upgradable or exclusive metadata lock is acquired. So this have to be done explicitly from these calls. Changed Open_table_context class to store MDL_request object for global intention exclusive lock acquired when opening tables. open_table(): Do not release metadata lock if we have failed to open table as this lock might have been acquired by one of previous statements in transaction, and therefore should not be released. open_system_tables_for_read()/close_system_tables()/ open_performance_schema_table(): Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing system tables was changed to use Open_tables_backup instead of Open_table_state class as well. close_performance_schema_table(): Got rid of duplicated code. sql/sql_class.cc: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. Also releasing metadata lock on system table is now responsibility of THD::restore_backup_open_tables_state(). Adjusted assert in THD::cleanup() to take into account fact that now we also use MDL sentinel for global read lock. sql/sql_class.h: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. As result: - 'mdl_context' member was moved out of Open_tables_state to THD class. enter_locked_tables_mode()/leave_locked_tables_mode() had to follow. - Methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. Changed Open_table_context class to store MDL_request object for global intention exclusive lock acquired when opening tables. sql/sql_delete.cc: MDL subsystem no longer implicitly acquires global intention exclusive metadata lock when per-object exclusive metadata lock is acquired. Now this has to be done by explicit calls outside of MDL subsystem. sql/sql_help.cc: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing system tables was changed to use Open_tables_backup instead of Open_table_state class as well. sql/sql_parse.cc: Adjusted assert reload_acl_and_cache() to the fact that global read lock now takes full-blown metadata lock. sql/sql_plist.h: Added support for element counting to I_P_List list template. One can use policy classes to specify if such counting is needed or not needed for particular list. sql/sql_show.cc: Instead of using separate MDL_context for opening tables for I_S purposes we now create MDL savepoint in the main context before opening tables and rollback to this savepoint after closing them. To support this and similar change for system tables methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing tables for I_S purposes was changed to use Open_tables_backup instead of Open_table_state class as well. sql/sql_table.cc: mysql_rm_tables_part2(): Since now global intention exclusive metadata lock is ordinary metadata lock we no longer can rely that by releasing MDL locks on all tables we will release all locks acquired by this routine. So in non-LOCK-TABLES mode we have to release all locks acquired explicitly. prepare_for_repair(), mysql_alter_table(): MDL subsystem no longer implicitly acquires global intention exclusive metadata lock when per-object exclusive metadata lock is acquired. Now this has to be done by explicit calls outside of MDL subsystem. sql/tztime.cc: Instead of using separate MDL_context for opening system tables we now create MDL savepoint in the main context before opening such tables and rollback to this savepoint after closing them. To support this change methods of THD responsible for saving/restoring open table state were changed to use Open_tables_backup class which in addition to Open_table_state has a member for this savepoint. As result code opening/closing system tables was changed to use Open_tables_backup instead of Open_table_state class as well. Also changed code not to use special mechanism for open system tables when it is not really necessary. |
||
|
Dmitry Lenev
|
0228c98936 |
Implementation of simple deadlock detection for metadata locks.
This change is supposed to reduce number of ER_LOCK_DEADLOCK errors which occur when multi-statement transaction encounters conflicting metadata lock in cases when waiting is possible. The idea is not to fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting lock go away. To avoid deadlocks we use simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". The bug was that concurrent execution of UPDATE or MULTI-UPDATE statement as a part of multi-statement transaction that already has used table being updated and ALTER TABLE statement might have resulted of loss of isolation between this transaction and ALTER TABLE statement, which manifested itself as changes performed by ALTER TABLE becoming visible in transaction and wrong binary log order as a consequence. This problem occurred when UPDATE or MULTI-UPDATE's wait in mysql_lock_tables() call was aborted due to metadata lock upgrade performed by concurrent ALTER TABLE. After such abort all metadata locks held by transaction were released but transaction silently continued to be executed as if nothing has happened. We solve this problem by changing our code not to release all locks in such case. Instead we release only locks which were acquired by current statement and then try to reacquire them by restarting open/lock tables process. We piggyback on simple deadlock detector implementation since this change has to be done anyway for it. mysql-test/include/handler.inc: After introduction of basic deadlock detector for metadata locks it became necessary to change parts of test for HANDLER statements which covered some of scenarios in which ER_LOCK_DEADLOCK error was detected in absence of real deadlock (with new deadlock detector this no longer happens). Also adjusted test to the fact that HANDLER READ for the table no longer will be blocked by ALTER TABLE for the same table which awaits for metadata lock upgrade (this is due to removal of mysql_lock_abort() from wait_while_table_is_used()). mysql-test/r/handler_innodb.result: After introduction of basic deadlock detector for metadata locks it became necessary to change parts of test for HANDLER statements which covered some of scenarios in which ER_LOCK_DEADLOCK error was detected in absence of real deadlock (with new deadlock detector this no longer happens). Also adjusted test to the fact that HANDLER READ for the table no longer will be blocked by ALTER TABLE for the same table which awaits for metadata lock upgrade (this is due to removal of mysql_lock_abort() from wait_while_table_is_used()). mysql-test/r/handler_myisam.result: After introduction of basic deadlock detector for metadata locks it became necessary to change parts of test for HANDLER statements which covered some of scenarios in which ER_LOCK_DEADLOCK error was detected in absence of real deadlock (with new deadlock detector this no longer happens). Also adjusted test to the fact that HANDLER READ for the table no longer will be blocked by ALTER TABLE for the same table which awaits for metadata lock upgrade (this is due to removal of mysql_lock_abort() from wait_while_table_is_used()). mysql-test/r/mdl_sync.result: Added test coverage for basic deadlock detection in metadata locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". mysql-test/r/sp-lock.result: Adjusted test coverage for metadata locking for stored routines since after introduction of basic deadlock detector for metadata locks number of scenarios in which ER_LOCK_DEADLOCK error in absence of deadlock has decreased. mysql-test/t/mdl_sync.test: Added test coverage for basic deadlock detection in metadata locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". mysql-test/t/sp-lock.test: Adjusted test coverage for metadata locking for stored routines since after introduction of basic deadlock detector for metadata locks number of scenarios in which ER_LOCK_DEADLOCK error in absence of deadlock has decreased. sql/log_event_old.cc: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/mdl.cc: Changed metadata locking subsystem to support basic deadlock detection with a help of the following simple empiric -- we assume that there is a deadlock if there is a connection which has to wait for a metadata lock which is currently acquired by some connection which is itself waiting to be able to acquire some shared metadata lock. To implement this change: - Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods which allow to find out if there is someone waiting for metadata lock which is held by the connection and therefore deadlocks are possible if this connection is going to wait for some metadata lock. To do this added version of MDL_ticket::has_pending_conflicting_lock() method which assumes that its caller already owns LOCK_mdl mutex. - Changed MDL_context::wait_for_locks() to use one of the above methods to check if somebody is waiting for metadata lock owned by this context (and therefore deadlock is possible) and emit ER_LOCK_DEADLOCK error in this case. Also now we mark context of connections waiting inside of this method by setting MDL_context::m_is_waiting_in_mdl member. Thanks to this such connection could be waken up if some other connection starts waiting for one of its metadata locks and so a deadlock can occur. - Adjusted notify_shared_lock() to wake up connections which wait inside MDL_context::wait_for_locks() while holding shared metadata lock. - Changed MDL_ticket::upgrade_shared_lock_to_exclusive() to add temporary ticket for exclusive lock to MDL_lock::waiting queue, so request for metadata lock upgrade can be properly detected by our empiric. Also now this method invokes a callback which forces transactions holding shared metadata lock on the table to call MDL_context:: can_wait_lead_to_deadlock() method even if they don't need any new metadata locks. Thanks to this such transactions can detect deadlocks/ livelocks between MDL and table-level locks. Also reduced timeouts between calls to notify_shared_lock() in MDL_ticket::upgrade_shared_lock_to_exclusive() and MDL_context::acquire_exclusive_locks(). This was necessary to get rid of call to mysql_lock_abort() in wait_while_table_is_used(). (Now we instead rely on notify_shared_lock() timely calling mysql_lock_abort_for_thread() for the table on which lock is being upgraded/acquired). sql/mdl.h: - Added a version of MDL_ticket::has_pending_conflicting_lock() method to be used in situations when caller already has acquired LOCK_mdl mutex. - Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods which allow to find out if there is someone waiting for metadata lock which is held by this connection and thus deadlocks are possible if this connections will start waiting for some metadata lock. - Added MDL_context::m_is_waiting_in_mdl member to mark connections waiting in MDL_context::wait_for_locks() method of metadata locking subsystem. Added getter method for this private member to make it accessible in notify_shared_lock() auxiliary so we can wake-up such connections if they hold shared metadata locks. - Finally, added mysql_abort_transactions_with_shared_lock() callback to be able force transactions which don't need any new metadata locks still call MDL_context::can_wait_lead_to_deadlock() and detect some of deadlocks between metadata locks and table-level locks. sql/mysql_priv.h: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/sql_base.cc: Changed approach to metadata locking for multi-statement transactions. We no longer fail ER_LOCK_DEADLOCK error immediately when we encounter conflicting metadata lock. Instead we release all metadata locks acquired by current statement and start to wait until conflicting locks to go away by calling MDL_context::wait_for_locks() method. To avoid deadlocks the latter implements simple empiric which aborts waiting with ER_LOCK_DEADLOCK error if it turns out that somebody is waiting for metadata locks owned by this transaction. To implement the change described above: - Introduced Open_table_context::m_start_of_statement_svp member to store state of metadata locks at the start of the statement. - Changed Open_table_context::request_backoff_action() not to fail with ER_LOCK_DEADLOCK immediately if back-off is requested due to conflicting metadata lock. - Added new argument for close_tables_for_reopen() procedure which allows to specify subset of metadata locks to be released. - Changed open_tables() not to release all metadata locks acquired by current transaction when metadata lock conflict is discovered. Instead we release only locks acquired by current statement. - Changed open_ltable() and open_and_lock_tables_derived() not to emit ER_LOCK_DEADLOCK error when mysql_lock_tables() is aborted in multi-statement transaction when somebody tries to acquire exclusive metadata lock on the table. Instead we release metadata locks acquired by current statement and try to wait until they can be re-acquired. - Adjusted tdc_wait_for_old_versions() to check if there is someone waiting for one of metadata locks held by this connection and run deadlock detection in order to avoid deadlocks in some situations. - Added mysql_abort_transactions_with_shared_lock() callback which allows to force transactions holding shared metadata lock on the table to call MDL_context::can_wait_lead_to_deadlock() even if they don't need any new metadata locks so they can detect potential deadlocks between metadata locking subsystem and table-level locks. - Adjusted wait_while_table_is_used() not to set TABLE::version to 0 as it is now done only when necessary by the above-mentioned callback. Also removed unnecessary call to mysql_lock_abort(). Instead we rely on code performing metadata lock upgrade aborting waits on the table-level lock for this table by calling mysql_lock_abort_for_thread() (invoked by mysql_notify_thread_having_shared_lock()). In future this should allow to reduce number of scenarios in which we produce ER_LOCK_DEADLOCK error even though no real deadlock exists. sql/sql_class.h: Introduced Open_table_context::m_start_of_statement_svp member to store state of metadata locks at the start of the statement. Replaced Open_table_context::m_can_deadlock member with m_has_locks member to reflect the fact that we no longer unconditionally emit ER_LOCK_DEADLOCK error for transaction having some metadata locks when conflicting metadata lock is discovered. sql/sql_insert.cc: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/sql_plist.h: Made I_P_List_iterator<T, B> usable with const lists. sql/sql_show.cc: close_tables_for_reopen() now takes one more argument which specifies at which point it should stop releasing metadata locks acquired by this connection. sql/sql_update.cc: Changed UPDATE and MULTI-UPDATE code not to release all metadata locks when calls to mysql_lock_tables() are aborted. Instead we release only locks which are acquired by this statement and then try to reacquire them by calling open_tables(). This solves bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in case of ALTER". |
||
|
Konstantin Osipov
|
dfdbc84585 |
A prerequisite patch for the fix for Bug#46224
"HANDLER statements within a transaction might lead to deadlocks". Introduce a notion of a sentinel to MDL_context. A sentinel is a ticket that separates all tickets in the context into two groups: before and after it. Currently we can have (and need) only one designated sentinel -- it separates all locks taken by LOCK TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK and all other locks, which must be released at COMMIT or ROLLBACK. The tricky part is maintaining the sentinel up to date when someone release its corresponding ticket. This can happen, e.g. if someone issues DROP TABLE under LOCK TABLES (generally, see all calls to release_all_locks_for_name()). MDL_context::release_ticket() is modified to take care of it. ****** A fix and a test case for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". An attempt to mix HANDLER SQL statements, which are transaction- agnostic, an open multi-statement transaction, and DDL against the involved tables (in a concurrent connection) could lead to a deadlock. The deadlock would occur when HANDLER OPEN or HANDLER READ would have to wait on a conflicting metadata lock. If the connection that issued HANDLER statement also had other metadata locks (say, acquired in scope of a transaction), a classical deadlock situation of mutual wait could occur. Incompatible change: entering LOCK TABLES mode automatically closes all open HANDLERs in the current connection. Incompatible change: previously an attempt to wait on a lock in a connection that has an open HANDLER statement could wait indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK is produced. The idea of the fix is to merge thd->handler_mdl_context with the main mdl_context of the connection, used for transactional locks. This makes deadlock detection possible, since all waits with locks are "visible" and available to analysis in a single MDL context of the connection. Since HANDLER locks and transactional locks have a different life cycle -- HANDLERs are explicitly open and closed, and so are HANDLER locks, explicitly acquired and released, whereas transactional locks "accumulate" till the end of a transaction and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT, a concept of "sentinel" was introduced to MDL_context. All locks, HANDLER and others, reside in the same linked list. However, a selected element of the list separates locks with different life cycle. HANDLER locks always reside at the end of the list, after the sentinel. Transactional locks are prepended to the beginning of the list, before the sentinel. Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only release those locks that reside before the sentinel. HANDLER locks must be released explicitly as part of HANDLER CLOSE statement, or an implicit close. The same approach with sentinel is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES statement has never worked together, the implementation is made simple and only maintains one sentinel, which is used either for HANDLER locks, or for LOCK TABLES locks. mysql-test/include/handler.inc: Add test coverage for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". Extended HANDLER coverage to cover a mix of HANDLER, transactions and DDL statements. mysql-test/r/handler_innodb.result: Update results (Bug#46224). mysql-test/r/handler_myisam.result: Update results (Bug#46224). sql/lock.cc: Remove thd->some_tables_deleted, it's never used. sql/log_event.cc: No need to check for thd->locked_tables_mode, it's done inside release_transactional_locks(). sql/mdl.cc: Implement the concept of HANDLER and LOCK TABLES "sentinel". Implement a method to clone an acquired ticket. Do not return tickets beyond the sentinel when acquiring locks, create a copy. Remove methods to merge and backup MDL_context, they are now not used (Hurra!). This opens a path to a proper constructor and destructor of class MDL_context (to be done in a separate patch). Modify find_ticket() to provide information about where the ticket position is with regard to the sentinel. sql/mdl.h: Add declarations necessary for the implementation of the concept of "sentinel", a dedicated ticket separating transactional and non-transactional locks. sql/mysql_priv.h: Add mark_tmp_table_for_reuse() declaration, a function to "close" a single session (temporary) table. sql/sql_base.cc: Remove thd->some_tables_deleted. Modify deadlock-prevention asserts and deadlock detection heuristics to take into account that from now on HANDLER locks reside in the same locking context. Add broadcast_refresh() to mysql_notify_thread_having_shared_lock(): this is necessary for the case when a thread having a shared lock is asleep in tdc_wait_for_old_versions(). This situation is only possible with HANDLER t1 OPEN; FLUSH TABLE (since all over code paths that lead to tdc_wait_for_old_versions() always have an empty MDL_context). Previously the server would simply deadlock in this situation. sql/sql_class.cc: Remove now unused member "THD::some_tables_deleted". Move mysql_ha_cleanup() a few lines above in THD::cleanup() to make sure that all handlers are closed when it's time to destroy the MDL_context of this connection. Remove handler_mdl_context and handler_tables. sql/sql_class.h: Remove THD::handler_tables, THD::handler_mdl_context, THD::some_tables_deleted. sql/sql_handler.cc: Remove thd->handler_tables. Remove thd->handler_mdl_context. Rewrite mysql_ha_open() to have no special provision for MERGE tables, now that we don't have to manipulate with thd->handler_tables it's easy to do. Remove dead code. Fix a bug in mysql_ha_flush() when we would always flush a temporary HANDLER when mysql_ha_flush() is called (actually mysql_ha_flush() never needs to flush temporary tables). sql/sql_insert.cc: Update a comment, no more thd->some_tables_deleted. sql/sql_parse.cc: Implement an incompatible change: entering LOCK TABLES closes active HANDLERs, if any. Now that we have a sentinel, we don't need to check for thd->locked_tables_mode when releasing metadata locks in COMMIT/ROLLBACK. sql/sql_plist.h: Add new (now necessary) methods to the list class. sql/sql_prepare.cc: Make sure we don't release HANDLER locks when rollback to a savepoint, set to not keep locks taken at PREPARE. sql/sql_servers.cc: Update to a new signature of MDL_context::release_all_locks(). sql/sql_table.cc: Remove thd->some_tables_deleted. sql/transaction.cc: Add comments. Make sure rollback to (MDL) savepoint works under LOCK TABLES and with HANDLER tables. |
||
|
Konstantin Osipov
|
f26f632b44 |
Backport of:
------------------------------------------------------------ revno: 2617.68.25 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg-pre2-2 timestamp: Wed 2009-09-16 18:26:50 +0400 message: Follow-up for one of pre-requisite patches for fixing bug #30977 "Concurrent statement using stored function and DROP FUNCTION breaks SBR". Made enum_mdl_namespace enum part of MDL_key class and removed MDL_ prefix from the names of enum members. In order to do the latter changed name of PROCEDURE symbol to PROCEDURE_SYM (otherwise macro which was automatically generated for this symbol conflicted with MDL_key::PROCEDURE enum member). |
||
|
Konstantin Osipov
|
634a810942 |
Backport of:
------------------------------------------------------------ revno: 2617.68.24 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-next-bg-pre2-2 timestamp: Wed 2009-09-16 17:25:29 +0400 message: Pre-requisite patch for fixing bug #30977 "Concurrent statement using stored function and DROP FUNCTION breaks SBR". Added MDL_request for stored routine as member to Sroutine_hash_entry in order to be able perform metadata locking for stored routines in future (Sroutine_hash_entry is an equivalent of TABLE_LIST class for stored routines). (WL#4284, follow up fixes). sql/mdl.cc: Introduced version of MDL_request::init() method which initializes lock request using pre-built MDL key. MDL_key::table_name/table_name_length() getters were renamed to reflect the fact that MDL_key objects are now created not only for tables. sql/mdl.h: Extended enum_mdl_namespace enum with values which correspond to namespaces for stored functions and triggers. Renamed MDL_key::table_name/table_name_length() getters to MDL_key::name() and name_length() correspondingly to reflect the fact that MDL_key objects are now created not only for tables. Added MDL_key::mdl_namespace() getter. Also added version of MDL_request::init() method which initializes lock request using pre-built MDL key. sql/sp.cc: Added MDL_request for stored routine as member to Sroutine_hash_entry. Changed code to use MDL_key from this request as a key for LEX::sroutines set. Removed separate "key" member from Sroutine_hash_entry as it became unnecessary. sql/sp.h: Added MDL_request for stored routine as member to Sroutine_hash_entry in order to be able perform metadata locking for stored routines in future (Sroutine_hash_entry is an equivalent of TABLE_LIST class for stored routines). Removed Sroutine_hash_entry::key member as now we can use MDL_key from this request as a key for LEX::sroutines set. sql/sp_head.cc: Removed sp_name::m_sroutines_key member and set_routine_type() method. Since key for routine in LEX::sroutines set has no longer sp_name::m_qname as suffix we won't save anything by creating it at sp_name construction time. Adjusted sp_name constructor used for creating temporary objects for lookups in SP-cache to accept MDL_key as parameter and to avoid any memory allocation. Finally, removed sp_head::m_soutines_key member for reasons similar to why sp_name::m_sroutines_key was removed sql/sp_head.h: Removed sp_name::m_sroutines_key member and set_routine_type() method. Since key for routine in LEX::sroutines set has no longer sp_name::m_qname as suffix we won't save anything by creating it at sp_name construction time. Adjusted sp_name constructor used for creating temporary objects for lookups in SP-cache to accept MDL_key as parameter and to avoid any memory allocation. Finally, removed sp_head::m_soutines_key member for reasons similar to why sp_name::m_sroutines_key was removed. sql/sql_base.cc: Adjusted code to the fact that we now use MDL_key from Sroutine_hash_entry::mdl_request as a key for LEX::sroutines set. MDL_key::table_name/table_name_length() getters were renamed to reflect the fact that MDL_key objects are now created not only for tables. sql/sql_trigger.cc: sp_add_used_routine() now takes MDL_key as parameter as now we use instance of this class as a key for LEX::sroutines set. |
||
|
Jon Olav Hauglid
|
fcae99271a |
Backport of revno: 2617.68.39
Bug #47249 assert in MDL_global_lock::is_lock_type_compatible This assert could be triggered if LOCK TABLES were used to lock both a table and a view that used the same table. The table would have to be first WRITE locked and then READ locked. So "LOCK TABLES v1 WRITE, t1 READ" would eventually trigger the assert, "LOCK TABLES v1 READ, t1 WRITE" would not. The reason is that the ordering of locks in the interal representation made a difference when executing FLUSH TABLE on the table. During FLUSH TABLE, a lock was upgraded to exclusive. If this lock was of type MDL_SHARED and not MDL_SHARED_UPGRADABLE, an internal counter in the MDL subsystem would get out of sync. This would happen if the *last* mention of the table in LOCK TABLES was a READ lock. The counter in question is the number exclusive locks (active or intention). This is used to make sure a global metadata lock is only taken when the counter is zero (= no conflicts). The counter is increased when a MDL_EXCLUSIVE or MDL_SHARED_UPGRADABLE lock is taken, but not when upgrade_shared_lock_to_exclusive() is used to upgrade directly from MDL_SHARED to MDL_EXCLUSIVE. This patch fixes the problem by searching for a TABLE instance locked with MDL_SHARED_UPGRADABLE or MDL_EXCLUSIVE before calling upgrade_shared_lock_to_exclusive(). The patch also adds an assert checking that only MDL_SHARED_UPGRADABLE locks are upgraded to exclusive. Test case added to lock_multi.test. |
||
|
Jon Olav Hauglid
|
546e16ebd8 |
Backport of revno: 2617.69.40
A pre-requisite patch for Bug#30977 "Concurrent statement using stored function and DROP FUNCTION breaks SBR". This patch changes the MDL API by introducing a namespace for lock keys: MDL_TABLE for tables and views and MDL_PROCEDURE for stored procedures and functions. The latter is needed for the fix for Bug#30977. |
||
|
Konstantin Osipov
|
ce5c87a3d3 |
Backport of:
---------------------------------------------------------- revno: 2617.69.20 committer: Konstantin Osipov <kostja@sun.com> branch nick: 5.4-4284-1-assert timestamp: Thu 2009-08-13 18:29:55 +0400 message: WL#4284 "Transactional DDL locking" A review fix. Since WL#4284 implementation separated MDL_request and MDL_ticket, MDL_request becamse a utility object necessary only to get a ticket. Store it by-value in TABLE_LIST with the intent to merge MDL_request::key with table_list->table_name and table_list->db in future. Change the MDL subsystem to not require MDL_requests to stay around till close_thread_tables(). Remove the list of requests from the MDL context. Requests for shared metadata locks acquired in open_tables() are only used as a list in recover_from_failed_open_table_attempt(), which calls mdl_context.wait_for_locks() for this list. To keep such list for recover_from_failed_open_table_attempt(), introduce a context class (Open_table_context), that collects all requests. A lot of minor cleanups and simplications that became possible with this change. sql/event_db_repository.cc: Remove alloc_mdl_requests(). Now MDL_request instance is a member of TABLE_LIST, and init_one_table() initializes it. sql/ha_ndbcluster_binlog.cc: Remove now unnecessary declaration and initialization of binlog_mdl_request. sql/lock.cc: No need to allocate MDL requests in lock_table_names() now. sql/log.cc: Use init_one_table() method, remove alloc_mdl_requests(), which is now unnecessary. sql/log_event.cc: No need to allocate mdl_request separately now. Use init_one_table() method. sql/log_event_old.cc: Update to the new signature of close_tables_for_reopen(). sql/mdl.cc: Update try_acquire_exclusive_lock() to be more easy to use. Function lock_table_name_if_not_cached() has been removed. Make acquire_shared_lock() signature consistent with try_acquire_exclusive_lock() signature. Remove methods that are no longer used. Update comments. sql/mdl.h: Implement an assignment operator that doesn't copy MDL_key (MDL_key::operator= is private and should remain private). This is a hack to work-around assignment of TABLE_LIST by value in several places. Such assignments violate encapsulation, since only perform a shallow copy. In most cases these assignments are a hack on their own. sql/mysql_priv.h: Update signatures of close_thread_tables() and close_tables_for_reopen(). sql/sp.cc: Allocate TABLE_LIST in thd->mem_root. Use init_one_table(). sql/sp_head.cc: Use init_one_table(). Remove thd->locked_tables_root, it's no longer needed. sql/sql_acl.cc: Use init_mdl_requests() and init_one_table(). sql/sql_base.cc: Update to new signatures of try_acquire_shared_lock() and try_acquire_exclusive_lock(). Remove lock_table_name_if_not_cached(). Fix a bug in open_ltable() that would not return ER_LOCK_DEADLOCK in case of a failed lock_tables() and a multi-statement transaction. Fix a bug in open_and_lock_tables_derived() that would not return ER_LOCK_DEADLOCK in case of a multi-statement transaction and a failure of lock_tables(). Move assignment of enum_open_table_action to a method of Open_table_context, a new class that maintains information for backoff actions. Minor rearrangements of the code. Remove alloc_mdl_requests() in functions that work with system tables: instead the patch ensures that callers always initialize TABLE_LIST argument. sql/sql_class.cc: THD::locked_tables_root is no more. sql/sql_class.h: THD::locked_tables_root is no more. Add a declaration for Open_table_context class. sql/sql_delete.cc: Update to use the simplified MDL API. sql/sql_handler.cc: TABLE_LIST::mdl_request is stored by-value now. Ensure that mdl_request.ticket is NULL for every request that is passed into MDL, to satisfy MDL asserts. @ sql/sql_help.cc Function open_system_tables_for_read() no longer initializes mdl_requests. Move TABLE_LIST::mdl_request initialization closer to TABLE_LIST initialization. sql/sql_help.cc: Function open_system_tables_for_read() no longer initializes mdl_requests. Move TABLE_LIST::mdl_request initialization closer to TABLE_LIST initialization. sql/sql_insert.cc: Remove assignment by-value of TABLE_LIST in TABLEOP_HOOKS. We can't carry over a granted MDL ticket from one table list to another. sql/sql_parse.cc: Change alloc_mdl_requests() -> init_mdl_requests(). @todo We can remove init_mdl_requests() altogether in some places: all places that call add_table_to_list() already have mdl requests initialized. sql/sql_plugin.cc: Use init_one_table(). THD::locked_tables_root is no more. sql/sql_servers.cc: Use init_one_table(). sql/sql_show.cc: Update acquire_high_priority_shared_lock() to use TABLE_LIST::mdl_request rather than allocate an own. Fix get_trigger_table_impl() to use init_one_table(), check for out of memory, follow the coding style. sql/sql_table.cc: Update to work with TABLE_LIST::mdl_request by-value. Remove lock_table_name_if_not_cached(). The code that used to delegate to it is quite simple and concise without it now. sql/sql_udf.cc: Use init_one_table(). sql/sql_update.cc: Update to use the new signature of close_tables_for_reopen(). sql/table.cc: Move re-setting of mdl_requests for prepared statements and stored procedures from close_thread_tables() to reinit_stmt_before_use(). Change alloc_mdl_requests() to init_mdl_requests(). init_mdl_requests() is a hack that can't be deleted until we don't have a list-aware TABLE_LIST constructor. Hopefully its use will be minimal sql/table.h: Change alloc_mdl_requests() to init_mdl_requests() TABLE_LIST::mdl_request is stored by value. sql/tztime.cc: We no longer initialize mdl requests in open_system_tables_for*() functions. Move this initialization closer to initialization of the rest of TABLE_LIST members. storage/myisammrg/ha_myisammrg.cc: Simplify mdl_request initialization. |
||
|
Konstantin Osipov
|
191bb81241 |
Backport of:
---------------------------------------------------------- revno: 2617.23.22 committer: Konstantin Osipov <kostja@sun.com> branch nick: mysql-6.0-runtime timestamp: Wed 2009-03-04 23:29:16 +0300 message: WL#4284, "Transactional DDL locking": fix a Windows compilation warning. |
||
|
Konstantin Osipov
|
a9013f8fba |
Backport of:
---------------------------------------------------------- revno: 2617.23.20 committer: Konstantin Osipov <kostja@sun.com> branch nick: mysql-6.0-runtime timestamp: Wed 2009-03-04 16:31:31 +0300 message: WL#4284 "Transactional DDL locking" Review comments: "Objectify" the MDL API. MDL_request and MDL_context still need manual construction and destruction, since they are used in environment that is averse to constructors/destructors. sql/mdl.cc: Improve comments. Add asserts to backup()/restore_from_backup()/merge() methods. Fix an order bug in the error path of mdl_acquire_exclusive_locks(): we used to first free a ticket object, and only then exclude it from the list of tickets. |
||
|
Konstantin Osipov
|
97abce0e7c |
Backport of:
---------------------------------------------------------- revno: 2617.23.19 committer: Konstantin Osipov <kostja@sun.com> branch nick: mysql-6.0-runtime timestamp: Tue 2009-03-03 01:20:44 +0300 message: Metadata locking: realign comments. No semantical changes, only enforce a bit of the coding style. This is a review fix for WL#4284 "Transactional DDL locking". sql/mdl.cc: Realign doxygen comments. sql/mdl.h: Realign doxygen comments. |
||
|
Konstantin Osipov
|
f477e66ec5 |
Backport of:
------------------------------------------------------------ revno: 2617.23.18 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Mon 2009-03-02 18:18:26 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking This is a prerequisite patch: These changes are intended to split lock requests from granted locks and to allow the memory and lifetime of granted locks to be managed within the MDL subsystem. Furthermore, tickets can now be shared and therefore are used to satisfy multiple lock requests, but only shared locks can be recursive. The problem is that the MDL subsystem morphs lock requests into granted locks locks but does not manage the memory and lifetime of lock requests, and hence, does not manage the memory of granted locks either. This can be problematic because it puts the burden of tracking references on the users of the subsystem and it can't be easily done in transactional contexts where the locks have to be kept around for the duration of a transaction. Another issue is that recursive locks (when the context trying to acquire a lock already holds a lock on the same object) requires that each time the lock is granted, a unique lock request/granted lock structure structure must be kept around until the lock is released. This can lead to memory leaks in transactional contexts as locks taken during the transaction should only be released at the end of the transaction. This also leads to unnecessary wake ups (broadcasts) in the MDL subsystem if the context still holds a equivalent of the lock being released. These issues are exacerbated due to the fact that WL#4284 low-level design says that the implementation should "2) Store metadata locks in transaction memory root, rather than statement memory root" but this is not possible because a memory root, as implemented in mysys, requires all objects allocated from it to be freed all at once. This patch combines review input and significant code contributions from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev). mysql-test/r/mdl_sync.result: Add test case result. mysql-test/t/mdl_sync.test: Add test case for shared lock upgrade case. sql/event_db_repository.cc: Rename mdl_alloc_lock to mdl_request_alloc. sql/ha_ndbcluster_binlog.cc: Use new function names to initialize MDL lock requests. sql/lock.cc: Rename MDL functions. sql/log_event.cc: The MDL request now holds the table and database name data (MDL_KEY). sql/mdl.cc: Move the MDL key to the MDL_LOCK structure in order to make the object suitable for allocation from a fixed-size allocator. This allows the simplification of the lists in the MDL_LOCK object, which now are just two, one for granted tickets and other for waiting (upgraders) tickets. Recursive requests for a shared lock on the same object can now be granted using the same lock ticket. This schema is only used for shared locks because that the only case that matters. This is used to avoid waste of resources in case a context (connection) already holds a shared lock on a object. sql/mdl.h: Introduce a metadata lock object key which is used to uniquely identify lock objects. Separate the structure used to represent pending lock requests from the structure used to represent granted metadata locks. Rename functions used to manipulate locks requests in order to have a more consistent function naming schema. sql/sp_head.cc: Rename mdl_alloc_lock to mdl_request_alloc. sql/sql_acl.cc: Rename alloc_mdl_locks to alloc_mdl_requests. sql/sql_base.cc: Various changes to accommodate that lock requests are separated from lock tickets (granted locks). sql/sql_class.h: Last acquired lock before the savepoint was set. sql/sql_delete.cc: Various changes to accommodate that lock requests are separated from lock tickets (granted locks). sql/sql_handler.cc: Various changes to accommodate that lock requests are separated from lock tickets (granted locks). sql/sql_insert.cc: Rename alloc_mdl_locks to alloc_mdl_requests. sql/sql_parse.cc: Rename alloc_mdl_locks to alloc_mdl_requests. sql/sql_plist.h: Typedef for iterator type. sql/sql_plugin.cc: Rename alloc_mdl_locks to alloc_mdl_requests. sql/sql_servers.cc: Rename alloc_mdl_locks to alloc_mdl_requests. sql/sql_show.cc: Various changes to accommodate that lock requests are separated from lock tickets (granted locks). sql/sql_table.cc: Various changes to accommodate that lock requests are separated from lock tickets (granted locks). sql/sql_trigger.cc: Save reference to the lock ticket so it can be downgraded later. sql/sql_udf.cc: Rename alloc_mdl_locks to alloc_mdl_requests. sql/table.cc: Rename mdl_alloc_lock to mdl_request_alloc. sql/table.h: Separate MDL lock requests from lock tickets (granted locks). storage/myisammrg/ha_myisammrg.cc: Rename alloc_mdl_locks to alloc_mdl_requests. |
||
|
Konstantin Osipov
|
124cda8a0a |
Backport of:
------------------------------------------------------------ revno: 2630.4.33 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-6.0-3726-w2 timestamp: Fri 2008-06-20 17:11:20 +0400 message: WL#3726 "DDL locking for all metadata objects". After-review fixes in progress. Minimized dependency of mdl.cc on other modules (particularly made it independant of mysql_priv.h) in order to be able write unit tests for metadata locking subsystem. sql/ha_ndbcluster_binlog.cc: Use newly introduced MAX_MDLKEY_LENGTH constant for allocating buffer for object key for metadata locking subsystem. sql/log_event.cc: Use newly introduced MAX_MDLKEY_LENGTH constant for allocating buffer for object key for metadata locking subsystem. sql/mdl.cc: Removed dependency on THD class (and thus on mysql_priv.h) by using direct access to members of st_my_thread_var instead of accessing THD::killed/enter_cond()/exit_cond(). sql/mdl.h: Added MAX_MDLKEY_LENGTH constant to be used for allocating buffers for key for metadata locking subsystem. Added declarations of server kernel functions used by metadata locking subsystem to mdl.h in order to decrease dependency of mdl.cc on other files. sql/mysql_priv.h: Moved declaration of notify_thread_having_shared_lock() to the mdl.h (also renamed it to make clear in metadata locking code that it is a callback to SQL-layer). sql/sql_base.cc: Renamed notify_thread_having_shared_lock() to make it clear in metadata locking subsystem code that it is a callback to SQL layer. sql/sql_handler.cc: Use newly introduced MAX_MDLKEY_LENGTH constant for allocating buffer for object key for metadata locking subsystem. sql/sql_show.cc: Use newly introduced MAX_MDLKEY_LENGTH constant for allocating buffer for object key for metadata locking subsystem. |
||
|
Konstantin Osipov
|
e3b3907c4f |
Backport of:
------------------------------------------------------------ revno: 2630.4.32 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-6.0-3726-w2 timestamp: Thu 2008-06-19 16:39:58 +0400 message: WL#3726 "DDL locking for all metadata objects". After-review fixes in progress. Ensure that metadata locking subsystem properly handles out-of-memory conditions. Clarified MDL interface by separating release of locks and removal of lock requests from the context. sql/lock.cc: mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and mdl_remove_lock() to do this. sql/mdl.cc: Ensured that metadata locking subsystem properly handles out-of-memory conditions. Introduced new MDL_INITIALIZED state for metadata lock request which is used in all cases when lock is not acquired and we have not associated request with object respesenting lock. MDL_PENDING is now only used for requests for exclusive locks which are added to the MDL_LOCK::waiting_exclusive queue. mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and newly introduced mdl_remove_lock() to do this. Also renamed mdl_release_all_locks_for_name() to emphasize that it also actually removes lock requests from the context. Finally mdl_try_acquire_exclusive_lock() is now returs information about encountered lock conflict in separate out parameter since its return value is used for distinguishing between error (e.g. due to OOM) and success. sql/mdl.h: Introduced new MDL_INITIALIZED state for metadata lock request which is used in all cases when lock is not acquired and we have not associated request with object respesenting lock. MDL_PENDING is now only used for requests for exclusive locks which are added to the MDL_LOCK::waiting_exclusive queue. mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and newly introduced mdl_remove_lock() to do this. Also renamed mdl_release_all_locks_for_name() to emphasize that it also actually removes lock requests from the context. Finally mdl_try_acquire_exclusive_lock() is now returs information about encountered lock conflict in separate out parameter since its return value is used for distinguishing between error (e.g. due to OOM) and success. sql/sql_base.cc: mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and mdl_remove_lock() to do this. Also adjusted open_table() to ensure that it releases/removes metadata locks in case of error after adding/acquiring them (unless keeping these lock requests is required for recovering action). sql/sql_delete.cc: mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and mdl_remove_lock() to do this. sql/sql_handler.cc: mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and mdl_remove_lock() to do this. sql/sql_show.cc: mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and mdl_remove_lock() to do this. sql/sql_table.cc: Renamed mdl_release_all_locks_for_name() to emphasize that it also actually removes lock requests from the context. mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible for removal of metadata lock requests from the context. One should explicitly call mdl_remove_all_locks() and mdl_remove_lock() to do this. Finally mdl_try_acquire_exclusive_lock() is now returs information about encountered lock conflict in separate out parameter since its return value is used for distinguishing between error (e.g. due to OOM) and success. |
||
|
Konstantin Osipov
|
f7ba9dafd5 |
Backport of:
------------------------------------------------------------ revno: 2630.9.2 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-6.0-3726-w3 timestamp: Tue 2008-06-10 18:01:56 +0400 message: WL#3726 "DDL locking for all metadata objects". After review fixes in progress. sql/mdl.cc: Changed mdl_acquire_shared_lock() signature to accept MDL_CONTEXT as one of arguments as described in specification. sql/mdl.h: Changed mdl_acquire_shared_lock() signature to accept MDL_CONTEXT as one of arguments as described in specification. sql/sql_base.cc: Changed mdl_acquire_shared_lock() signature to accept MDL_CONTEXT as one of arguments as described in specification. Renamed handle_failed_open_table_attempt() to recover_from_failed_open_table_attempt() as suggested by review. Added comment clarifying why we need to check TABLE::db_stat while looking at TABLE instances open by other threads. sql/sql_show.cc: Changed mdl_acquire_shared_lock() signature to accept MDL_CONTEXT as one of arguments as described in specification. |
||
|
Konstantin Osipov
|
d0a1f640db |
Backport of:
------------------------------------------------------------ revno: 2630.4.24 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-6.0-3726-w2 timestamp: Fri 2008-06-06 14:28:58 +0400 message: WL#3726 "DDL locking for all metadata objects". After review fixes in progress. Get rid of upgradability and priority attributes of metadata lock requests by replacing them with two new types of lock requests MDL_SHARED_UPGRADABLE and MDL_SHARED_HIGH_PRIO correspondingly. |
||
|
Konstantin Osipov
|
b05303c132 |
Backport of:
------------------------------------------------------------
revno: 2630.4.18
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w2
timestamp: Tue 2008-06-03 21:07:58 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
Now during upgrading/downgrading metadata locks we deal with
individual metadata lock requests rather than with all requests
for this object in the context. This makes API a bit more clear
and makes adjust_mdl_locks_upgradability() much nicer.
sql/lock.cc:
lock_table_names():
Set TABLE_LIST::mdl_lock_data when allocating new metadata
lock request object for table list element.
sql/mdl.cc:
Now during upgrading/downgrading metadata locks we deal with
individual metadata lock requests rather than with all
requests for this object in the context. Adjusted upgrade/
downgrade functions accordingly.
We also got rid of mdl_release_exclusive_locks() and
now release locks individually. To simplify this process
mdl_release_all_locks_for_name() was introduced.
sql/mdl.h:
Now during upgrading/downgrading metadata locks we deal with
individual metadata lock requests rather than with all
requests for this object in the context. Adjusted upgrade/
downgrade functions accordingly.
We also got rid of mdl_release_exclusive_locks() and
now release locks individually. To simplify this process
mdl_release_all_locks_for_name() was introduced.
sql/sql_base.cc:
Now during upgrading/downgrading metadata locks we deal with
individual metadata lock requests rather than with all
requests for this object in the context.
We also got rid of mdl_release_exclusive_locks() and
now release locks individually.
sql/sql_parse.cc:
adjust_mdl_locks_upgradability() is much simplier now due to the
fact that now during upgrading/downgrading metadata locks we
deal with individual metadata lock requests rather than with
all requests for this object in the context.
sql/sql_table.cc:
Now during upgrading/downgrading metadata locks we deal with
individual metadata lock requests rather than with all
requests for this object in the context. Adjusted upgrade/
downgrade functions accordingly.
We also got rid of mdl_release_exclusive_locks() and
now release locks individually. To simplify this process
mdl_release_all_locks_for_name() was introduced.
sql/sql_trigger.cc:
ow during upgrading/downgrading metadata locks we deal with
individual metadata lock requests rather than with all
requests for this object in the context.
|
||
|
Konstantin Osipov
|
e23046d1bc |
Backport of:
------------------------------------------------------------ revno: 2630.4.17 committer: Dmitry Lenev <dlenev@mysql.com> branch nick: mysql-6.0-3726-w2 timestamp: Thu 2008-05-29 16:52:56 +0400 message: WL#3726 "DDL locking for all metadata objects". After review fixes in progress. "The great correction of names". Renamed MDL_LOCK and MDL_LOCK_DATA classes to make usage of these names in metadata locking subsystem consistent with other parts of server (i.e. thr_lock.cc). Now we MDL_LOCK_DATA corresponds to request for a lock and MDL_LOCK to the lock itself. Adjusted code in MDL subsystem and other places using these classes accordingly. Did similar thing for GLOBAL_MDL_LOCK_DATA class and also changed name of its members to correspond to names of MDL_LOCK_DATA members. Finally got rid of usage of one letter variables in MDL code since it makes code harder to search in (according to reviewer). |
||
|
Konstantin Osipov
|
de1979d3d6 |
Backport of:
------------------------------------------------------------
revno: 2630.4.10
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Mon 2008-05-26 15:11:26 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review changes in progress.
Implemented some renames suggested by reviewer.
sql/mdl.cc:
Renamed:
MDL_LOCK_DATA::users -> lock_count
MDL_LOCK_DATA::has_no_other_users() -> has_one_lock()
MDL_LOCK::upgradable -> is_upgradable
Moved variables used in global metadata lock implementation
to separate strucuture.
sql/mdl.h:
Renamed MDL_LOCK::upgradable to is_upgradable.
|
||
|
Konstantin Osipov
|
eff3780dd8 |
Initial import of WL#3726 "DDL locking for all metadata objects".
Backport of:
------------------------------------------------------------
revno: 2630.4.1
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Fri 2008-05-23 17:54:03 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
------------------------------------------------------------
This is the first patch in series. It transforms the metadata
locking subsystem to use a dedicated module (mdl.h,cc). No
significant changes in the locking protocol.
The import passes the test suite with the exception of
deprecated/removed 6.0 features, and MERGE tables. The latter
are subject to a fix by WL#4144.
Unfortunately, the original changeset comments got lost in a merge,
thus this import has its own (largely insufficient) comments.
This patch fixes Bug#25144 "replication / binlog with view breaks".
Warning: this patch introduces an incompatible change:
Under LOCK TABLES, it's no longer possible to FLUSH a table that
was not locked for WRITE.
Under LOCK TABLES, it's no longer possible to DROP a table or
VIEW that was not locked for WRITE.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.2
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:03:45 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.3
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:08:51 +0400
message:
WL#3726 "DDL locking for all metadata objects"
Fixed failing Windows builds by adding mdl.cc to the lists
of files needed to build server/libmysqld on Windows.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.4
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 21:57:58 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Fix for assert failures in kill.test which occured when one
tried to kill ALTER TABLE statement on merge table while it
was waiting in wait_while_table_is_used() for other connections
to close this table.
These assert failures stemmed from the fact that cleanup code
in this case assumed that temporary table representing new
version of table was open with adding to THD::temporary_tables
list while code which were opening this temporary table wasn't
always fulfilling this.
This patch changes code that opens new version of table to
always do this linking in. It also streamlines cleanup process
for cases when error occurs while we have new version of table
open.
******
WL#3726 "DDL locking for all metadata objects"
Add libmysqld/mdl.cc to .bzrignore.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.6
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sun 2008-05-25 00:33:22 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Addition to the fix of assert failures in kill.test caused by
changes for this worklog.
Make sure we close the new table only once.
.bzrignore:
Add libmysqld/mdl.cc
libmysqld/CMakeLists.txt:
Added mdl.cc to the list of files needed for building of libmysqld.
libmysqld/Makefile.am:
Added files implementing new meta-data locking subsystem to the server.
mysql-test/include/handler.inc:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/create.result:
Adjusted test case after change in implementation of CREATE TABLE
... SELECT. We no longer have special check in open_table() which
catches the case when we select from the table created. Instead we
rely on unique_table() call which happens after opening and
locking all tables.
mysql-test/r/flush.result:
FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
TABLES. Updated test accordingly.
mysql-test/r/flush_table.result:
Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
locked for read. Updated test accordingly.
mysql-test/r/handler_innodb.result:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/handler_myisam.result:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/information_schema.result:
Additional test for WL#3726 "DDL locking for all metadata
objects". Check that we use high-priority metadata lock requests
when filling I_S tables.
Rearrange tests to match 6.0 better (fewer merge conflicts).
mysql-test/r/kill.result:
Added tests checking that DDL and DML statements waiting for
metadata locks can be interrupted by KILL command.
mysql-test/r/lock.result:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation write locks on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/r/partition_column_prune.result:
Update results (same results in 6.0), WL#3726
mysql-test/r/partition_pruning.result:
Update results (same results in 6.0), WL#3726
mysql-test/r/ps_ddl.result:
We no longer invalidate prepared CREATE TABLE ... SELECT statement
if target table changes. This is OK since it is not strictly
necessary.
The first change is wrong, is caused by FLUSH TABLE
now flushing all unused tables. This is a regression that
Dmitri fixed in 6.0 in a follow up patch.
mysql-test/r/sp.result:
Under LOCK TABLES we no longer allow accessing views which were
not explicitly locked. To access view we need to obtain metadata
lock on it and doing this under LOCK TABLES may lead to deadlocks.
mysql-test/r/view.result:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation even "write locks" on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock
mysql-test/r/view_grant.result:
ALTER VIEW implementation was changed to open a view only after
checking that user which does alter has appropriate privileges on
it. This means that in case when user's privileges are
insufficient for this we won't check that new view definer is the
same as original one or user performing alter has SUPER privilege.
Adjusted test case accordingly.
mysql-test/r/view_multi.result:
Added test case for bug#25144 "replication / binlog with view
breaks".
mysql-test/suite/rpl/t/disabled.def:
Disable test for deprecated features (they don't work with new MDL).
mysql-test/t/create.test:
Adjusted test case after change in implementation of CREATE TABLE
... SELECT. We no longer have special check in open_table() which
catches the case when we select from the table created. Instead we
rely on unique_table() call which happens after opening and
locking all tables.
mysql-test/t/disabled.def:
Disable merge.test, subject of WL#4144
mysql-test/t/flush.test:
FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
TABLES. Updated test accordingly.
mysql-test/t/flush_table.test:
Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
locked for read. Updated test accordingly.
mysql-test/t/information_schema.test:
Additional test for WL#3726 "DDL locking for all metadata
objects". Check that we use high-priority metadata lock requests
when filling I_S tables.
Rearrange the results for easier merges with 6.0.
mysql-test/t/kill.test:
Added tests checking that DDL and DML statements waiting for
metadata locks can be interrupted by KILL command.
mysql-test/t/lock.test:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation write locks on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/t/lock_multi.test:
Adjusted test case to the changes of status in various places
caused by change in implementation FLUSH TABLES WITH READ LOCK,
which is now takes global metadata lock before flushing tables and
therefore waits on at these places.
mysql-test/t/ps_ddl.test:
We no longer invalidate prepared CREATE TABLE ... SELECT statement
if target table changes. This is OK since it is not strictly
necessary.
The first change is wrong, is caused by FLUSH TABLE
now flushing all unused tables. This is a regression that
Dmitri fixed in 6.0 in a follow up patch.
mysql-test/t/sp.test:
Under LOCK TABLES we no longer allow accessing views which were
not explicitly locked. To access view we need to obtain metadata
lock on it and doing this under LOCK TABLES may lead to deadlocks.
mysql-test/t/trigger_notembedded.test:
Adjusted test case to the changes of status in various places
caused by change in implementation FLUSH TABLES WITH READ LOCK,
which is now takes global metadata lock before flushing tables and
therefore waits on at these places.
mysql-test/t/view.test:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation even "write locks" on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/t/view_grant.test:
ALTER VIEW implementation was changed to open a view only after
checking that user which does alter has appropriate privileges on
it. This means that in case when user's privileges are
insufficient for this we won't check that new view definer is the
same as original one or user performing alter has SUPER privilege.
Adjusted test case accordingly.
mysql-test/t/view_multi.test:
Added test case for bug#25144 "replication / binlog with view
breaks".
sql/CMakeLists.txt:
Added mdl.cc to the list of files needed for building of server.
sql/Makefile.am:
Added files implementing new meta-data locking subsystem to the
server.
sql/event_db_repository.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when TABLE_LIST objects is also allocated
there or on stack.
sql/ha_ndbcluster.cc:
Adjusted code to work nicely with new metadata locking subsystem.
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/ha_ndbcluster_binlog.cc:
Adjusted code to work with new metadata locking subsystem.
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/handler.cc:
update_frm_version():
Directly update TABLE_SHARE::mysql_version member instead of
going through all TABLE instances for this table (old code was a
legacy from pre-table-definition-cache days).
sql/lock.cc:
Use new metadata locking subsystem. Threw away most of functions
related to name locking as now one is supposed to use metadata
locking API instead. In lock_global_read_lock() and
unlock_global_read_lock() in order to avoid problems with global
read lock sneaking in at the moment when we perform FLUSH TABLES
or ALTER TABLE under LOCK TABLES and when tables being reopened
are protected only by metadata locks we also have to take global
shared meta data lock.
sql/log_event.cc:
Adjusted code to work with new metadata locking subsystem. For
tables open by slave thread for applying RBR events allocate
memory for lock request object in the same chunk of memory as
TABLE_LIST objects for them. In order to ensure that we keep these
objects around until tables are open always close tables before
calling Relay_log_info::clear_tables_to_lock(). Use new auxiliary
Relay_log_info::slave_close_thread_tables() method to enforce
this.
sql/log_event_old.cc:
Adjusted code to work with new metadata locking subsystem. Since
for tables open by slave thread for applying RBR events memory for
lock request object is allocated in the same chunk of memory as
TABLE_LIST objects for them we have to ensure that we keep these
objects around until tables are open. To ensure this we always
close tables before calling
Relay_log_info::clear_tables_to_lock(). To enfore this we use
new auxiliary Relay_log_info::slave_close_thread_tables()
method.
sql/mdl.cc:
Implemented new metadata locking subsystem and API described in
WL3726 "DDL locking for all metadata objects".
sql/mdl.h:
Implemented new metadata locking subsystem and API described in
WL3726 "DDL locking for all metadata objects".
sql/mysql_priv.h:
- close_thread_tables()/close_tables_for_reopen() now has one more
argument which indicates that metadata locks should be released
but not removed from the context in order to be used later in
mdl_wait_for_locks() and tdc_wait_for_old_version().
- close_cached_table() routine is no longer public.
- Thread waiting in wait_while_table_is_used() can be now killed
so this function returns boolean to make caller aware of such
situation.
- We no longer have table cache as separate entity instead used
and unused TABLE instances are linked to TABLE_SHARE objects in
table definition cache.
- Now third argument of open_table() is also used for requesting
table repair or auto-discovery of table's new definition. So its
type was changed from bool to enum.
- Added tdc_open_view() function for opening view by getting its
definition from disk (and table cache in future).
- reopen_name_locked_table() no longer needs "link_in" argument as
now we have exclusive metadata locks instead of dummy TABLE
instances when this function is called.
- find_locked_table() now takes head of list of TABLE instances
instead of always scanning through THD::open_tables list. Also
added find_write_locked_table() auxiliary.
- reopen_tables(), close_cached_tables() no longer have
mark_share_as_old and wait_for_placeholder arguments. Instead of
relying on this parameters and related behavior FLUSH TABLES
WITH READ LOCK now takes global shared metadata lock.
- We no longer need drop_locked_tables() and
abort_locked_tables().
- mysql_ha_rm_tables() now always assume that LOCK_open is not
acquired by caller.
- Added notify_thread_having_shared_lock() callback invoked by
metadata locking subsystem when acquiring an exclusive lock, for
each thread that has a conflicting shared metadata lock.
- Introduced expel_table_from_cache() as replacement for
remove_table_from_cache() (the main difference is that this new
function assumes that caller follows metadata locking protocol
and never waits).
- Threw away most of functions related to name locking. One should
use new metadata locking subsystem and API instead.
sql/mysqld.cc:
Got rid of call initializing/deinitializing table cache since now
it is embedded into table definition cache. Added calls for
initializing/ deinitializing metadata locking subsystem.
sql/rpl_rli.cc:
Introduced auxiliary Relay_log_info::slave_close_thread_tables()
method which is used for enforcing that we always close tables
open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
objects for them.
sql/rpl_rli.h:
Introduced auxiliary Relay_log_info::slave_close_thread_tables()
method which is used for enforcing that we always close tables
open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
objects for them.
sql/set_var.cc:
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/sp_head.cc:
For tables added to the statement's table list by prelocking
algorithm we allocate these objects either on the same memory as
corresponding table list elements or on THD::locked_tables_root
(if we are building table list for LOCK TABLES).
sql/sql_acl.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables. Got rid of redundant code by using unlock_locked_tables()
function.
sql/sql_base.cc:
Changed code to use new MDL subsystem. Got rid of separate table
cache. Now used and unused TABLE instances are linked to the
TABLE_SHAREs in table definition cache.
check_unused():
Adjusted code to the fact that we no longer have separate table
cache. Removed dead code.
table_def_free():
Free TABLE instances referenced from TABLE_SHARE objects before
destroying table definition cache.
get_table_share():
Added assert which ensures that noone will be able to access
table (and its share) without acquiring some kind of metadata
lock first.
close_handle_and_leave_table_as_lock():
Adjusted code to the fact that TABLE instances now are linked to
list in TABLE_SHARE.
list_open_tables():
Changed this function to use table definition cache instead of
table cache.
free_cache_entry():
Unlink freed TABLE elements from the list of all TABLE instances
for the table in TABLE_SHARE.
kill_delayed_thread_for_table():
Added auxiliary for killing delayed insert threads for
particular table.
close_cached_tables():
Got rid of wait_for_refresh argument as we now rely on global
shared metadata lock to prevent FLUSH WITH READ LOCK sneaking in
when we are reopening tables. Heavily reworked this function to
use new MDL code and not to rely on separate table cache entity.
close_open_tables():
We no longer have separate table cache.
close_thread_tables():
Release metadata locks after closing all tables. Added skip_mdl
argument which allows us not to remove metadata lock requests
from the context in case when we are going to use this requests
later in mdl_wait_for_locks() and tdc_wait_for_old_versions().
close_thread_table()/close_table_for_reopen():
Since we no longer have separate table cache and all TABLE
instances are linked to TABLE_SHARE objects in table definition
cache we have to link/unlink TABLE object to/from appropriate
lists in the share.
name_lock_locked_table():
Moved redundant code to find_write_locked_table() function and
adjusted code to the fact that wait_while_table_is_used() can
now return with an error if our thread is killed.
reopen_table_entry():
We no longer need "link_in" argument as with MDL we no longer
call this function with dummy TABLE object pre-allocated and
added to the THD::open_tables. Also now we add newly-open TABLE
instance to the list of share's used TABLE instances.
table_cache_insert_placeholder():
Got rid of name-locking legacy.
lock_table_name_if_not_cached():
Moved to sql_table.cc the only place where it is used. It was
also reimplemented using new MDL API.
open_table():
- Reworked this function to use new MDL subsystem.
- Changed code to deal with table definition cache directly
instead of going through separate table cache.
- Now third argument is also used for requesting table repair
or auto-discovery of table's new definition. So its type was
changed from bool to enum.
find_locked_table()/find_write_locked_table():
Accept head of list of TABLE objects as first argument and use
this list instead of always searching in THD::open_tables list.
Also added auxiliary for finding write-locked locked tables.
reopen_table():
Adjusted function to work with new MDL subsystem and to properly
manuipulate with lists of used/unused TABLE instaces in
TABLE_SHARE.
reopen_tables():
Removed mark_share_as_old parameter. Instead of relying on it
and related behavior FLUSH TABLES WITH READ LOCK now takes
global shared metadata lock. Changed code after removing
separate table cache.
drop_locked_tables()/abort_locked_tables():
Got rid of functions which are no longer needed.
unlock_locked_tables():
Moved this function from sql_parse.cc and changed it to release
memory which was used for allocating metadata lock requests for
tables open and locked by LOCK TABLES.
tdc_open_view():
Intoduced function for opening a view by getting its definition
from disk (and table cache in future).
reopen_table_entry():
Introduced function for opening table definitions while holding
exclusive metatadata lock on it.
open_unireg_entry():
Got rid of this function. Most of its functionality is relocated
to open_table() and open_table_fini() functions, and some of it
to reopen_table_entry() and tdc_open_view(). Also code
resposible for auto-repair and auto-discovery of tables was
moved to separate function.
open_table_entry_fini():
Introduced function which contains common actions which finalize
process of TABLE object creation.
auto_repair_table():
Moved code responsible for auto-repair of table being opened
here.
handle_failed_open_table_attempt()
Moved code responsible for handling failing attempt to open
table to one place (retry due to lock conflict/old version,
auto-discovery and repair).
open_tables():
- Flush open HANDLER tables if they have old version of if there
is conflicting metadata lock against them (before this moment
we had this code in open_table()).
- When we open view which should be processed via derived table
on the second execution of prepared statement or stored
routine we still should call open_table() for it in order to
obtain metadata lock on it and prepare its security context.
- In cases when we discover that some special handling of
failure to open table is needed call
handle_failed_open_table_attempt() which handles all such
scenarios.
open_ltable():
Handling of various special scenarios of failure to open a table
was moved to separate handle_failed_open_table_attempt()
function.
remove_db_from_cache():
Removed this function as it is no longer used.
notify_thread_having_shared_lock():
Added callback which is invoked by MDL subsystem when acquiring
an exclusive lock, for each thread that has a conflicting shared
metadata lock.
expel_table_from_cache():
Introduced function for removing unused TABLE instances. Unlike
remove_table_from_cache() it relies on caller following MDL
protocol and having appropriate locks when calling it and thus
does not do any waiting if table is still in use.
tdc_wait_for_old_version():
Added function which allows open_tables() to wait in cases when
we discover that we should back-off due to presence of old
version of table.
abort_and_upgrade_lock():
Use new MDL calls.
mysql_wait_completed_table():
Got rid of unused function.
open_system_tables_for_read/for_update()/performance_schema_table():
Allocate MDL_LOCK objects on execution memory root in cases when
TABLE_LIST objects for corresponding tables is allocated on
stack.
close_performance_schema_table():
Release metadata locks after closing tables.
******
Use I_P_List for free/used tables list in the table share.
sql/sql_binlog.cc:
Use Relay_log_info::slave_close_thread_tables() method to enforce
that we always close tables open for RBR before deallocating
TABLE_LIST elements and MDL_LOCK objects for them.
sql/sql_class.cc:
Added meta-data locking contexts as part of Open_tables_state
context. Also introduced THD::locked_tables_root memory root
which is to be used for allocating MDL_LOCK objects for tables in
LOCK TABLES statement (end of lifetime for such objects is UNLOCK
TABLES so we can't use statement or execution root for them).
sql/sql_class.h:
Added meta-data locking contexts as part of Open_tables_state
context. Also introduced THD::locked_tables_root memory root
which is to be used for allocating MDL_LOCK objects for tables in
LOCK TABLES statement (end of lifetime for such objects is UNLOCK
TABLES so we can't use statement or execution root for them).
Note: handler_mdl_context and locked_tables_root and
mdl_el_root will be removed by subsequent patches.
sql/sql_db.cc:
mysql_rm_db() does not really need to call remove_db_from_cache()
as it drops each table in the database using
mysql_rm_table_part2(), which performs all necessary operations on
table (definition) cache.
sql/sql_delete.cc:
Use the new metadata locking API for TRUNCATE.
sql/sql_handler.cc:
Changed HANDLER implementation to use new metadata locking
subsystem. Note that MDL_LOCK objects for HANDLER tables are
allocated in the same chunk of heap memory as TABLE_LIST object
for those tables.
sql/sql_insert.cc:
mysql_insert():
find_locked_table() now takes head of list of TABLE object as
its argument instead of always scanning through THD::open_tables
list.
handle_delayed_insert():
Allocate metadata lock request object for table open by delayed
insert thread on execution memroot. create_table_from_items():
We no longer allocate dummy TABLE objects for tables being
created if they don't exist. As consequence
reopen_name_locked_table() no longer has link_in argument.
open_table() now has one more argument which is not relevant for
temporary tables.
sql/sql_parse.cc:
- Moved unlock_locked_tables() routine to sql_base.cc and made
available it in other files. Got rid of some redundant code by
using this function.
- Replaced boolean TABLE_LIST::create member with enum
open_table_type member.
- Use special memory root for allocating MDL_LOCK objects for
tables open and locked by LOCK TABLES (these object should live
till UNLOCK TABLES so we can't allocate them on statement nor
execution memory root). Also properly set metadata lock
upgradability attribure for those tables.
- Under LOCK TABLES it is no longer allowed to flush tables which
are not write-locked as this breaks metadata locking protocol
and thus potentially might lead to deadlock.
- Added auxiliary adjust_mdl_locks_upgradability() function.
sql/sql_partition.cc:
Adjusted code to the fact that reopen_tables() no longer has
"mark_share_as_old" argument. Got rid of comments which are no
longer true.
sql/sql_plist.h:
Added I_P_List template class for parametrized intrusive doubly
linked lists and I_P_List_iterator for corresponding iterator.
Unlike for I_List<> list elements of such list can participate in
several lists. Unlike List<> such lists are doubly-linked and
intrusive.
sql/sql_plugin.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables.
sql/sql_prepare.cc:
Replaced boolean TABLE_LIST::create member with enum
open_table_type member. This allows easily handle situation in
which instead of opening the table we want only to take exclusive
metadata lock on it.
sql/sql_rename.cc:
Use new metadata locking subsystem in implementation of RENAME
TABLE.
sql/sql_servers.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables. Got rid of redundant code by using unlock_locked_tables()
function.
sql/sql_show.cc:
Acquire shared metadata lock when we are getting information for
I_S table directly from TABLE_SHARE without doing full-blown table
open. We use high priority lock request in this situation in
order to avoid deadlocks.
Also allocate metadata lock requests objects (MDL_LOCK) on
execution memory root in cases when TABLE_LIST objects are also
allocated there
sql/sql_table.cc:
mysql_rm_table():
Removed comment which is no longer relevant.
mysql_rm_table_part2():
Now caller of mysql_ha_rm_tables() should not own LOCK_open.
Adjusted code to use new metadata locking subsystem instead of
name-locks.
lock_table_name_if_not_cached():
Moved this function from sql_base.cc to this file and
reimplemented it using metadata locking API.
mysql_create_table():
Adjusted code to use new MDL API.
wait_while_table_is_used():
Changed function to use new MDL subsystem. Made thread waiting
in it killable (this also led to introduction of return value so
caller can distinguish successful executions from situations
when waiting was aborted).
close_cached_tables():
Thread waiting in this function is killable now. As result it
has return value for distinguishing between succes and failure.
Got rid of redundant boradcast_refresh() call.
prepare_for_repair():
Use MDL subsystem instead of name-locks.
mysql_admin_table():
mysql_ha_rm_tables() now always assumes that caller doesn't own
LOCK_open.
mysql_repair_table():
We should mark all elements of table list as requiring
upgradable metadata locks.
mysql_create_table_like():
Use new MDL subsystem instead of name-locks.
create_temporary_tables():
We don't need to obtain metadata locks when creating temporary
table.
mysql_fast_or_online_alter_table():
Thread waiting in wait_while_table_is_used() is now killable.
mysql_alter_table():
Adjusted code to work with new MDL subsystem and to the fact
that threads waiting in what_while_table_is_used() and
close_cached_table() are now killable.
sql/sql_test.cc:
We no longer have separate table cache. TABLE instances are now
associated with/linked to TABLE_SHARE objects in table definition
cache.
sql/sql_trigger.cc:
Adjusted code to work with new metadata locking subsystem. Also
reopen_tables() no longer has mark_share_as_old argument (Instead
of relying on this parameter and related behavior FLUSH TABLES
WITH READ LOCK now takes global shared metadata lock).
sql/sql_udf.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables.
sql/sql_update.cc:
Adjusted code to work with new meta-data locking subsystem.
sql/sql_view.cc:
Added proper meta-data locking to implementations of
CREATE/ALTER/DROP VIEW statements. Now we obtain exclusive
meta-data lock on a view before creating/ changing/dropping it.
This ensures that all concurrent statements that use this view
will finish before our statement will proceed and therefore we
will get correct order of statements in the binary log.
Also ensure that TABLE_LIST::mdl_upgradable attribute is properly
propagated for underlying tables of view.
sql/table.cc:
Added auxiliary alloc_mdl_locks() function for allocating metadata
lock request objects for all elements of table list.
sql/table.h:
TABLE_SHARE:
Got rid of unused members. Introduced members for storing lists
of used and unused TABLE objects for this share.
TABLE:
Added members for linking TABLE objects into per-share lists of
used and unused TABLE instances. Added member for holding
pointer to metadata lock for this table.
TABLE_LIST:
Replaced boolean TABLE_LIST::create member with enum
open_table_type member. This allows easily handle situation in
which instead of opening the table we want only to take
exclusive meta-data lock on it (we need this in order to handle
ALTER VIEW and CREATE VIEW statements).
Introduced new mdl_upgradable member for marking elements of
table list for which we need to take upgradable shared metadata
lock instead of plain shared metadata lock. Added pointer for
holding pointer to MDL_LOCK for the table.
Added auxiliary alloc_mdl_locks() function for allocating metadata
lock requests objects for all elements of table list. Added
auxiliary set_all_mdl_upgradable() function for marking all
elements in table list as requiring upgradable metadata locks.
storage/myisammrg/ha_myisammrg.cc:
Allocate MDL_LOCK objects for underlying tables of MERGE table.
To be reworked once Ingo pushes his patch for WL4144.
|