mariadb/sql/transaction.cc

836 lines
22 KiB
C++
Raw Normal View History

Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK". The problem in the first bug report was that although deadlock involving metadata locks was reported using the same error code and message as InnoDB deadlock it didn't rollback transaction like the latter. This caused confusion to users as in some cases after ER_LOCK_DEADLOCK transaction could have been restarted immediately and in some cases rollback was required. The problem in the second bug report was that although InnoDB deadlock caused transaction rollback in all storage engines it didn't cause release of metadata locks. So concurrent DDL on the tables used in transaction was blocked until implicit or explicit COMMIT or ROLLBACK was issued in the connection which got InnoDB deadlock. The former issue has stemmed from the fact that when support for detection and reporting metadata locks deadlocks was added we erroneously assumed that InnoDB doesn't rollback transaction on deadlock but only last statement (while this is what happens on InnoDB lock timeout actually) and so didn't implement rollback of transactions on MDL deadlocks. The latter issue was caused by the fact that rollback of transaction due to deadlock is carried out by setting THD::transaction_rollback_request flag at the point where deadlock is detected and performing rollback inside of trans_rollback_stmt() call when this flag is set. And trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are released. This patch solves these two problems in the following way: - In case when MDL deadlock is detect transaction rollback is requested by setting THD::transaction_rollback_request flag. - Code performing rollback of transaction if THD::transaction_rollback_request is moved out from trans_rollback_stmt(). Now we handle rollback request on the same level as we call trans_rollback_stmt() and release statement/ transaction MDL locks.
2013-08-20 13:12:34 +04:00
/* Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA */
#ifdef USE_PRAGMA_IMPLEMENTATION
#pragma implementation // gcc: Class implementation
#endif
#include "sql_priv.h"
#include "transaction.h"
#include "rpl_handler.h"
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
#include "debug_sync.h" // DEBUG_SYNC
/* Conditions under which the transaction state must not change. */
static bool trans_check(THD *thd)
{
enum xa_states xa_state= thd->transaction.xid_state.xa_state;
DBUG_ENTER("trans_check");
/*
Always commit statement transaction before manipulating with
the normal one.
*/
DBUG_ASSERT(thd->transaction.stmt.is_empty());
if (unlikely(thd->in_sub_stmt))
my_error(ER_COMMIT_NOT_ALLOWED_IN_SF_OR_TRG, MYF(0));
if (xa_state != XA_NOTR)
my_error(ER_XAER_RMFAIL, MYF(0), xa_state_names[xa_state]);
else
DBUG_RETURN(FALSE);
DBUG_RETURN(TRUE);
}
/**
Mark a XA transaction as rollback-only if the RM unilaterally
rolled back the transaction branch.
@note If a rollback was requested by the RM, this function sets
the appropriate rollback error code and transits the state
to XA_ROLLBACK_ONLY.
@return TRUE if transaction was rolled back or if the transaction
state is XA_ROLLBACK_ONLY. FALSE otherwise.
*/
static bool xa_trans_rolled_back(XID_STATE *xid_state)
{
if (xid_state->rm_error)
{
switch (xid_state->rm_error) {
case ER_LOCK_WAIT_TIMEOUT:
my_error(ER_XA_RBTIMEOUT, MYF(0));
break;
case ER_LOCK_DEADLOCK:
my_error(ER_XA_RBDEADLOCK, MYF(0));
break;
default:
my_error(ER_XA_RBROLLBACK, MYF(0));
}
xid_state->xa_state= XA_ROLLBACK_ONLY;
}
return (xid_state->xa_state == XA_ROLLBACK_ONLY);
}
/**
Rollback the active XA transaction.
@note Resets rm_error before calling ha_rollback(), so
the thd->transaction.xid structure gets reset
by ha_rollback() / THD::transaction::cleanup().
@return TRUE if the rollback failed, FALSE otherwise.
*/
static bool xa_trans_force_rollback(THD *thd)
{
/*
We must reset rm_error before calling ha_rollback(),
so thd->transaction.xid structure gets reset
by ha_rollback()/THD::transaction::cleanup().
*/
thd->transaction.xid_state.rm_error= 0;
if (ha_rollback_trans(thd, true))
{
my_error(ER_XAER_RMERR, MYF(0));
return true;
}
return false;
}
/**
Begin a new transaction.
@note Beginning a transaction implicitly commits any current
transaction and releases existing locks.
@param thd Current thread
@param flags Transaction flags
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_begin(THD *thd, uint flags)
{
int res= FALSE;
DBUG_ENTER("trans_begin");
if (trans_check(thd))
DBUG_RETURN(TRUE);
thd->locked_tables_list.unlock_locked_tables(thd);
DBUG_ASSERT(!thd->locked_tables_mode);
if (thd->in_multi_stmt_transaction_mode() ||
(thd->variables.option_bits & OPTION_TABLE_LOCK))
{
thd->variables.option_bits&= ~OPTION_TABLE_LOCK;
thd->server_status&= ~SERVER_STATUS_IN_TRANS;
res= test(ha_commit_trans(thd, TRUE));
}
thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG);
thd->transaction.all.modified_non_trans_table= FALSE;
if (res)
DBUG_RETURN(TRUE);
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
2009-12-05 02:02:48 +03:00
/*
Release transactional metadata locks only after the
transaction has been committed.
*/
A prerequisite patch for the fix for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". Introduce a notion of a sentinel to MDL_context. A sentinel is a ticket that separates all tickets in the context into two groups: before and after it. Currently we can have (and need) only one designated sentinel -- it separates all locks taken by LOCK TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK and all other locks, which must be released at COMMIT or ROLLBACK. The tricky part is maintaining the sentinel up to date when someone release its corresponding ticket. This can happen, e.g. if someone issues DROP TABLE under LOCK TABLES (generally, see all calls to release_all_locks_for_name()). MDL_context::release_ticket() is modified to take care of it. ****** A fix and a test case for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". An attempt to mix HANDLER SQL statements, which are transaction- agnostic, an open multi-statement transaction, and DDL against the involved tables (in a concurrent connection) could lead to a deadlock. The deadlock would occur when HANDLER OPEN or HANDLER READ would have to wait on a conflicting metadata lock. If the connection that issued HANDLER statement also had other metadata locks (say, acquired in scope of a transaction), a classical deadlock situation of mutual wait could occur. Incompatible change: entering LOCK TABLES mode automatically closes all open HANDLERs in the current connection. Incompatible change: previously an attempt to wait on a lock in a connection that has an open HANDLER statement could wait indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK is produced. The idea of the fix is to merge thd->handler_mdl_context with the main mdl_context of the connection, used for transactional locks. This makes deadlock detection possible, since all waits with locks are "visible" and available to analysis in a single MDL context of the connection. Since HANDLER locks and transactional locks have a different life cycle -- HANDLERs are explicitly open and closed, and so are HANDLER locks, explicitly acquired and released, whereas transactional locks "accumulate" till the end of a transaction and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT, a concept of "sentinel" was introduced to MDL_context. All locks, HANDLER and others, reside in the same linked list. However, a selected element of the list separates locks with different life cycle. HANDLER locks always reside at the end of the list, after the sentinel. Transactional locks are prepended to the beginning of the list, before the sentinel. Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only release those locks that reside before the sentinel. HANDLER locks must be released explicitly as part of HANDLER CLOSE statement, or an implicit close. The same approach with sentinel is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES statement has never worked together, the implementation is made simple and only maintains one sentinel, which is used either for HANDLER locks, or for LOCK TABLES locks.
2009-12-22 19:09:15 +03:00
thd->mdl_context.release_transactional_locks();
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
2009-12-05 02:02:48 +03:00
2010-02-03 03:06:42 +03:00
thd->variables.option_bits|= OPTION_BEGIN;
thd->server_status|= SERVER_STATUS_IN_TRANS;
if (flags & MYSQL_START_TRANS_OPT_WITH_CONS_SNAPSHOT)
res= ha_start_consistent_snapshot(thd);
DBUG_RETURN(test(res));
}
/**
Commit the current transaction, making its changes permanent.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_commit(THD *thd)
{
int res;
DBUG_ENTER("trans_commit");
if (trans_check(thd))
DBUG_RETURN(TRUE);
thd->server_status&= ~SERVER_STATUS_IN_TRANS;
res= ha_commit_trans(thd, TRUE);
if (res)
/*
if res is non-zero, then ha_commit_trans has rolled back the
transaction, so the hooks for rollback will be called.
*/
RUN_HOOK(transaction, after_rollback, (thd, FALSE));
else
RUN_HOOK(transaction, after_commit, (thd, FALSE));
2010-02-03 03:06:42 +03:00
thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG);
thd->transaction.all.modified_non_trans_table= FALSE;
thd->lex->start_transaction_opt= 0;
DBUG_RETURN(test(res));
}
/**
Implicitly commit the current transaction.
@note A implicit commit does not releases existing table locks.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_commit_implicit(THD *thd)
{
bool res= FALSE;
DBUG_ENTER("trans_commit_implicit");
if (trans_check(thd))
DBUG_RETURN(TRUE);
if (thd->in_multi_stmt_transaction_mode() ||
2010-02-03 03:06:42 +03:00
(thd->variables.option_bits & OPTION_TABLE_LOCK))
{
/* Safety if one did "drop table" on locked tables */
if (!thd->locked_tables_mode)
2010-02-03 03:06:42 +03:00
thd->variables.option_bits&= ~OPTION_TABLE_LOCK;
thd->server_status&= ~SERVER_STATUS_IN_TRANS;
res= test(ha_commit_trans(thd, TRUE));
}
2010-02-03 03:06:42 +03:00
thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG);
thd->transaction.all.modified_non_trans_table= FALSE;
/*
Upon implicit commit, reset the current transaction
isolation level. We do not care about
@@session.completion_type since it's documented
to not have any effect on implicit commit.
*/
thd->tx_isolation= (enum_tx_isolation) thd->variables.tx_isolation;
DBUG_RETURN(res);
}
/**
Rollback the current transaction, canceling its changes.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_rollback(THD *thd)
{
int res;
DBUG_ENTER("trans_rollback");
if (trans_check(thd))
DBUG_RETURN(TRUE);
thd->server_status&= ~SERVER_STATUS_IN_TRANS;
res= ha_rollback_trans(thd, TRUE);
RUN_HOOK(transaction, after_rollback, (thd, FALSE));
2010-02-03 03:06:42 +03:00
thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG);
thd->transaction.all.modified_non_trans_table= FALSE;
thd->lex->start_transaction_opt= 0;
DBUG_RETURN(test(res));
}
Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK". The problem in the first bug report was that although deadlock involving metadata locks was reported using the same error code and message as InnoDB deadlock it didn't rollback transaction like the latter. This caused confusion to users as in some cases after ER_LOCK_DEADLOCK transaction could have been restarted immediately and in some cases rollback was required. The problem in the second bug report was that although InnoDB deadlock caused transaction rollback in all storage engines it didn't cause release of metadata locks. So concurrent DDL on the tables used in transaction was blocked until implicit or explicit COMMIT or ROLLBACK was issued in the connection which got InnoDB deadlock. The former issue has stemmed from the fact that when support for detection and reporting metadata locks deadlocks was added we erroneously assumed that InnoDB doesn't rollback transaction on deadlock but only last statement (while this is what happens on InnoDB lock timeout actually) and so didn't implement rollback of transactions on MDL deadlocks. The latter issue was caused by the fact that rollback of transaction due to deadlock is carried out by setting THD::transaction_rollback_request flag at the point where deadlock is detected and performing rollback inside of trans_rollback_stmt() call when this flag is set. And trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are released. This patch solves these two problems in the following way: - In case when MDL deadlock is detect transaction rollback is requested by setting THD::transaction_rollback_request flag. - Code performing rollback of transaction if THD::transaction_rollback_request is moved out from trans_rollback_stmt(). Now we handle rollback request on the same level as we call trans_rollback_stmt() and release statement/ transaction MDL locks.
2013-08-20 13:12:34 +04:00
/**
Implicitly rollback the current transaction, typically
after deadlock was discovered.
@param thd Current thread
@retval False Success
@retval True Failure
@note ha_rollback_low() which is indirectly called by this
function will mark XA transaction for rollback by
setting appropriate RM error status if there was
transaction rollback request.
*/
bool trans_rollback_implicit(THD *thd)
{
int res;
DBUG_ENTER("trans_rollback_implict");
/*
Always commit/rollback statement transaction before manipulating
with the normal one.
Don't perform rollback in the middle of sub-statement, wait till
its end.
*/
DBUG_ASSERT(thd->transaction.stmt.is_empty() && !thd->in_sub_stmt);
thd->server_status&= ~SERVER_STATUS_IN_TRANS;
DBUG_PRINT("info", ("clearing SERVER_STATUS_IN_TRANS"));
res= ha_rollback_trans(thd, true);
/*
We don't reset OPTION_BEGIN flag below to simulate implicit start
of new transacton in @@autocommit=1 mode. This is necessary to
preserve backward compatibility.
*/
thd->variables.option_bits&= ~(OPTION_KEEP_LOG);
Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK". The problem in the first bug report was that although deadlock involving metadata locks was reported using the same error code and message as InnoDB deadlock it didn't rollback transaction like the latter. This caused confusion to users as in some cases after ER_LOCK_DEADLOCK transaction could have been restarted immediately and in some cases rollback was required. The problem in the second bug report was that although InnoDB deadlock caused transaction rollback in all storage engines it didn't cause release of metadata locks. So concurrent DDL on the tables used in transaction was blocked until implicit or explicit COMMIT or ROLLBACK was issued in the connection which got InnoDB deadlock. The former issue has stemmed from the fact that when support for detection and reporting metadata locks deadlocks was added we erroneously assumed that InnoDB doesn't rollback transaction on deadlock but only last statement (while this is what happens on InnoDB lock timeout actually) and so didn't implement rollback of transactions on MDL deadlocks. The latter issue was caused by the fact that rollback of transaction due to deadlock is carried out by setting THD::transaction_rollback_request flag at the point where deadlock is detected and performing rollback inside of trans_rollback_stmt() call when this flag is set. And trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are released. This patch solves these two problems in the following way: - In case when MDL deadlock is detect transaction rollback is requested by setting THD::transaction_rollback_request flag. - Code performing rollback of transaction if THD::transaction_rollback_request is moved out from trans_rollback_stmt(). Now we handle rollback request on the same level as we call trans_rollback_stmt() and release statement/ transaction MDL locks.
2013-08-20 13:12:34 +04:00
thd->transaction.all.modified_non_trans_table= false;
/* Rollback should clear transaction_rollback_request flag. */
DBUG_ASSERT(! thd->transaction_rollback_request);
DBUG_RETURN(test(res));
}
/**
Commit the single statement transaction.
@note Note that if the autocommit is on, then the following call
inside InnoDB will commit or rollback the whole transaction
(= the statement). The autocommit mechanism built into InnoDB
is based on counting locks, but if the user has used LOCK
TABLES then that mechanism does not know to do the commit.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_commit_stmt(THD *thd)
{
DBUG_ENTER("trans_commit_stmt");
int res= FALSE;
/*
We currently don't invoke commit/rollback at end of
a sub-statement. In future, we perhaps should take
a savepoint for each nested statement, and release the
savepoint when statement has succeeded.
*/
DBUG_ASSERT(! thd->in_sub_stmt);
if (thd->transaction.stmt.ha_list)
{
res= ha_commit_trans(thd, FALSE);
if (! thd->in_active_multi_stmt_transaction())
thd->tx_isolation= (enum_tx_isolation) thd->variables.tx_isolation;
}
if (res)
/*
if res is non-zero, then ha_commit_trans has rolled back the
transaction, so the hooks for rollback will be called.
*/
RUN_HOOK(transaction, after_rollback, (thd, FALSE));
else
RUN_HOOK(transaction, after_commit, (thd, FALSE));
thd->transaction.stmt.reset();
DBUG_RETURN(test(res));
}
/**
Rollback the single statement transaction.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_rollback_stmt(THD *thd)
{
DBUG_ENTER("trans_rollback_stmt");
/*
We currently don't invoke commit/rollback at end of
a sub-statement. In future, we perhaps should take
a savepoint for each nested statement, and release the
savepoint when statement has succeeded.
*/
DBUG_ASSERT(! thd->in_sub_stmt);
if (thd->transaction.stmt.ha_list)
{
ha_rollback_trans(thd, FALSE);
if (! thd->in_active_multi_stmt_transaction())
thd->tx_isolation= (enum_tx_isolation) thd->variables.tx_isolation;
}
RUN_HOOK(transaction, after_rollback, (thd, FALSE));
thd->transaction.stmt.reset();
DBUG_RETURN(FALSE);
}
/* Find a named savepoint in the current transaction. */
static SAVEPOINT **
find_savepoint(THD *thd, LEX_STRING name)
{
SAVEPOINT **sv= &thd->transaction.savepoints;
while (*sv)
{
if (my_strnncoll(system_charset_info, (uchar *) name.str, name.length,
(uchar *) (*sv)->name, (*sv)->length) == 0)
break;
sv= &(*sv)->prev;
}
return sv;
}
/**
Set a named transaction savepoint.
@param thd Current thread
@param name Savepoint name
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_savepoint(THD *thd, LEX_STRING name)
{
SAVEPOINT **sv, *newsv;
DBUG_ENTER("trans_savepoint");
if (!(thd->in_multi_stmt_transaction_mode() || thd->in_sub_stmt) ||
!opt_using_transactions)
DBUG_RETURN(FALSE);
enum xa_states xa_state= thd->transaction.xid_state.xa_state;
if (xa_state != XA_NOTR && xa_state != XA_ACTIVE)
{
my_error(ER_XAER_RMFAIL, MYF(0), xa_state_names[xa_state]);
DBUG_RETURN(TRUE);
}
sv= find_savepoint(thd, name);
if (*sv) /* old savepoint of the same name exists */
{
newsv= *sv;
ha_release_savepoint(thd, *sv);
*sv= (*sv)->prev;
}
else if ((newsv= (SAVEPOINT *) alloc_root(&thd->transaction.mem_root,
savepoint_alloc_size)) == NULL)
{
my_error(ER_OUT_OF_RESOURCES, MYF(0));
DBUG_RETURN(TRUE);
}
newsv->name= strmake_root(&thd->transaction.mem_root, name.str, name.length);
newsv->length= name.length;
/*
if we'll get an error here, don't add new savepoint to the list.
we'll lose a little bit of memory in transaction mem_root, but it'll
be free'd when transaction ends anyway
*/
if (ha_savepoint(thd, newsv))
DBUG_RETURN(TRUE);
newsv->prev= thd->transaction.savepoints;
thd->transaction.savepoints= newsv;
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
2009-12-05 02:02:48 +03:00
/*
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
Remember locks acquired before the savepoint was set.
They are used as a marker to only release locks acquired after
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
2009-12-05 02:02:48 +03:00
the setting of this savepoint.
A prerequisite patch for the fix for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". Introduce a notion of a sentinel to MDL_context. A sentinel is a ticket that separates all tickets in the context into two groups: before and after it. Currently we can have (and need) only one designated sentinel -- it separates all locks taken by LOCK TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK and all other locks, which must be released at COMMIT or ROLLBACK. The tricky part is maintaining the sentinel up to date when someone release its corresponding ticket. This can happen, e.g. if someone issues DROP TABLE under LOCK TABLES (generally, see all calls to release_all_locks_for_name()). MDL_context::release_ticket() is modified to take care of it. ****** A fix and a test case for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". An attempt to mix HANDLER SQL statements, which are transaction- agnostic, an open multi-statement transaction, and DDL against the involved tables (in a concurrent connection) could lead to a deadlock. The deadlock would occur when HANDLER OPEN or HANDLER READ would have to wait on a conflicting metadata lock. If the connection that issued HANDLER statement also had other metadata locks (say, acquired in scope of a transaction), a classical deadlock situation of mutual wait could occur. Incompatible change: entering LOCK TABLES mode automatically closes all open HANDLERs in the current connection. Incompatible change: previously an attempt to wait on a lock in a connection that has an open HANDLER statement could wait indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK is produced. The idea of the fix is to merge thd->handler_mdl_context with the main mdl_context of the connection, used for transactional locks. This makes deadlock detection possible, since all waits with locks are "visible" and available to analysis in a single MDL context of the connection. Since HANDLER locks and transactional locks have a different life cycle -- HANDLERs are explicitly open and closed, and so are HANDLER locks, explicitly acquired and released, whereas transactional locks "accumulate" till the end of a transaction and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT, a concept of "sentinel" was introduced to MDL_context. All locks, HANDLER and others, reside in the same linked list. However, a selected element of the list separates locks with different life cycle. HANDLER locks always reside at the end of the list, after the sentinel. Transactional locks are prepended to the beginning of the list, before the sentinel. Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only release those locks that reside before the sentinel. HANDLER locks must be released explicitly as part of HANDLER CLOSE statement, or an implicit close. The same approach with sentinel is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES statement has never worked together, the implementation is made simple and only maintains one sentinel, which is used either for HANDLER locks, or for LOCK TABLES locks.
2009-12-22 19:09:15 +03:00
Note: this works just fine if we're under LOCK TABLES,
since mdl_savepoint() is guaranteed to be beyond
the last locked table. This allows to release some
locks acquired during LOCK TABLES.
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
2009-12-05 02:02:48 +03:00
*/
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
newsv->mdl_savepoint= thd->mdl_context.mdl_savepoint();
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
2009-12-05 02:02:48 +03:00
DBUG_RETURN(FALSE);
}
/**
Rollback a transaction to the named savepoint.
@note Modifications that the current transaction made to
rows after the savepoint was set are undone in the
rollback.
@note Savepoints that were set at a later time than the
named savepoint are deleted.
@param thd Current thread
@param name Savepoint name
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_rollback_to_savepoint(THD *thd, LEX_STRING name)
{
int res= FALSE;
SAVEPOINT *sv= *find_savepoint(thd, name);
DBUG_ENTER("trans_rollback_to_savepoint");
if (sv == NULL)
{
my_error(ER_SP_DOES_NOT_EXIST, MYF(0), "SAVEPOINT", name.str);
DBUG_RETURN(TRUE);
}
enum xa_states xa_state= thd->transaction.xid_state.xa_state;
if (xa_state != XA_NOTR)
{
my_error(ER_XAER_RMFAIL, MYF(0), xa_state_names[xa_state]);
DBUG_RETURN(TRUE);
}
if (ha_rollback_to_savepoint(thd, sv))
res= TRUE;
2010-02-03 03:06:42 +03:00
else if (((thd->variables.option_bits & OPTION_KEEP_LOG) ||
thd->transaction.all.modified_non_trans_table) &&
!thd->slave_thread)
push_warning(thd, MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARNING_NOT_COMPLETE_ROLLBACK,
ER(ER_WARNING_NOT_COMPLETE_ROLLBACK));
thd->transaction.savepoints= sv;
A prerequisite patch for the fix for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". Introduce a notion of a sentinel to MDL_context. A sentinel is a ticket that separates all tickets in the context into two groups: before and after it. Currently we can have (and need) only one designated sentinel -- it separates all locks taken by LOCK TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK and all other locks, which must be released at COMMIT or ROLLBACK. The tricky part is maintaining the sentinel up to date when someone release its corresponding ticket. This can happen, e.g. if someone issues DROP TABLE under LOCK TABLES (generally, see all calls to release_all_locks_for_name()). MDL_context::release_ticket() is modified to take care of it. ****** A fix and a test case for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". An attempt to mix HANDLER SQL statements, which are transaction- agnostic, an open multi-statement transaction, and DDL against the involved tables (in a concurrent connection) could lead to a deadlock. The deadlock would occur when HANDLER OPEN or HANDLER READ would have to wait on a conflicting metadata lock. If the connection that issued HANDLER statement also had other metadata locks (say, acquired in scope of a transaction), a classical deadlock situation of mutual wait could occur. Incompatible change: entering LOCK TABLES mode automatically closes all open HANDLERs in the current connection. Incompatible change: previously an attempt to wait on a lock in a connection that has an open HANDLER statement could wait indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK is produced. The idea of the fix is to merge thd->handler_mdl_context with the main mdl_context of the connection, used for transactional locks. This makes deadlock detection possible, since all waits with locks are "visible" and available to analysis in a single MDL context of the connection. Since HANDLER locks and transactional locks have a different life cycle -- HANDLERs are explicitly open and closed, and so are HANDLER locks, explicitly acquired and released, whereas transactional locks "accumulate" till the end of a transaction and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT, a concept of "sentinel" was introduced to MDL_context. All locks, HANDLER and others, reside in the same linked list. However, a selected element of the list separates locks with different life cycle. HANDLER locks always reside at the end of the list, after the sentinel. Transactional locks are prepended to the beginning of the list, before the sentinel. Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only release those locks that reside before the sentinel. HANDLER locks must be released explicitly as part of HANDLER CLOSE statement, or an implicit close. The same approach with sentinel is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES statement has never worked together, the implementation is made simple and only maintains one sentinel, which is used either for HANDLER locks, or for LOCK TABLES locks.
2009-12-22 19:09:15 +03:00
/*
Release metadata locks that were acquired during this savepoint unit
unless binlogging is on. Releasing locks with binlogging on can break
replication as it allows other connections to drop these tables before
rollback to savepoint is written to the binlog.
A prerequisite patch for the fix for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". Introduce a notion of a sentinel to MDL_context. A sentinel is a ticket that separates all tickets in the context into two groups: before and after it. Currently we can have (and need) only one designated sentinel -- it separates all locks taken by LOCK TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK and all other locks, which must be released at COMMIT or ROLLBACK. The tricky part is maintaining the sentinel up to date when someone release its corresponding ticket. This can happen, e.g. if someone issues DROP TABLE under LOCK TABLES (generally, see all calls to release_all_locks_for_name()). MDL_context::release_ticket() is modified to take care of it. ****** A fix and a test case for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". An attempt to mix HANDLER SQL statements, which are transaction- agnostic, an open multi-statement transaction, and DDL against the involved tables (in a concurrent connection) could lead to a deadlock. The deadlock would occur when HANDLER OPEN or HANDLER READ would have to wait on a conflicting metadata lock. If the connection that issued HANDLER statement also had other metadata locks (say, acquired in scope of a transaction), a classical deadlock situation of mutual wait could occur. Incompatible change: entering LOCK TABLES mode automatically closes all open HANDLERs in the current connection. Incompatible change: previously an attempt to wait on a lock in a connection that has an open HANDLER statement could wait indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK is produced. The idea of the fix is to merge thd->handler_mdl_context with the main mdl_context of the connection, used for transactional locks. This makes deadlock detection possible, since all waits with locks are "visible" and available to analysis in a single MDL context of the connection. Since HANDLER locks and transactional locks have a different life cycle -- HANDLERs are explicitly open and closed, and so are HANDLER locks, explicitly acquired and released, whereas transactional locks "accumulate" till the end of a transaction and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT, a concept of "sentinel" was introduced to MDL_context. All locks, HANDLER and others, reside in the same linked list. However, a selected element of the list separates locks with different life cycle. HANDLER locks always reside at the end of the list, after the sentinel. Transactional locks are prepended to the beginning of the list, before the sentinel. Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only release those locks that reside before the sentinel. HANDLER locks must be released explicitly as part of HANDLER CLOSE statement, or an implicit close. The same approach with sentinel is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES statement has never worked together, the implementation is made simple and only maintains one sentinel, which is used either for HANDLER locks, or for LOCK TABLES locks.
2009-12-22 19:09:15 +03:00
*/
bool binlog_on= mysql_bin_log.is_open() && thd->variables.sql_log_bin;
if (!res && !binlog_on)
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
2009-12-05 02:02:48 +03:00
thd->mdl_context.rollback_to_savepoint(sv->mdl_savepoint);
DBUG_RETURN(test(res));
}
/**
Remove the named savepoint from the set of savepoints of
the current transaction.
@note No commit or rollback occurs. It is an error if the
savepoint does not exist.
@param thd Current thread
@param name Savepoint name
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_release_savepoint(THD *thd, LEX_STRING name)
{
int res= FALSE;
SAVEPOINT *sv= *find_savepoint(thd, name);
DBUG_ENTER("trans_release_savepoint");
if (sv == NULL)
{
my_error(ER_SP_DOES_NOT_EXIST, MYF(0), "SAVEPOINT", name.str);
DBUG_RETURN(TRUE);
}
if (ha_release_savepoint(thd, sv))
res= TRUE;
thd->transaction.savepoints= sv->prev;
DBUG_RETURN(test(res));
}
/**
Starts an XA transaction with the given xid value.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_xa_start(THD *thd)
{
enum xa_states xa_state= thd->transaction.xid_state.xa_state;
DBUG_ENTER("trans_xa_start");
if (xa_state == XA_IDLE && thd->lex->xa_opt == XA_RESUME)
{
bool not_equal= !thd->transaction.xid_state.xid.eq(thd->lex->xid);
if (not_equal)
my_error(ER_XAER_NOTA, MYF(0));
else
thd->transaction.xid_state.xa_state= XA_ACTIVE;
DBUG_RETURN(not_equal);
}
/* TODO: JOIN is not supported yet. */
if (thd->lex->xa_opt != XA_NONE)
my_error(ER_XAER_INVAL, MYF(0));
else if (xa_state != XA_NOTR)
my_error(ER_XAER_RMFAIL, MYF(0), xa_state_names[xa_state]);
else if (thd->locked_tables_mode || thd->in_active_multi_stmt_transaction())
my_error(ER_XAER_OUTSIDE, MYF(0));
else if (!trans_begin(thd))
{
DBUG_ASSERT(thd->transaction.xid_state.xid.is_null());
thd->transaction.xid_state.xa_state= XA_ACTIVE;
thd->transaction.xid_state.rm_error= 0;
thd->transaction.xid_state.xid.set(thd->lex->xid);
if (xid_cache_insert(&thd->transaction.xid_state))
{
thd->transaction.xid_state.xa_state= XA_NOTR;
thd->transaction.xid_state.xid.null();
trans_rollback(thd);
DBUG_RETURN(true);
}
DBUG_RETURN(FALSE);
}
DBUG_RETURN(TRUE);
}
/**
Put a XA transaction in the IDLE state.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_xa_end(THD *thd)
{
DBUG_ENTER("trans_xa_end");
/* TODO: SUSPEND and FOR MIGRATE are not supported yet. */
if (thd->lex->xa_opt != XA_NONE)
my_error(ER_XAER_INVAL, MYF(0));
else if (thd->transaction.xid_state.xa_state != XA_ACTIVE)
my_error(ER_XAER_RMFAIL, MYF(0),
xa_state_names[thd->transaction.xid_state.xa_state]);
else if (!thd->transaction.xid_state.xid.eq(thd->lex->xid))
my_error(ER_XAER_NOTA, MYF(0));
else if (!xa_trans_rolled_back(&thd->transaction.xid_state))
thd->transaction.xid_state.xa_state= XA_IDLE;
DBUG_RETURN(thd->is_error() ||
thd->transaction.xid_state.xa_state != XA_IDLE);
}
/**
Put a XA transaction in the PREPARED state.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_xa_prepare(THD *thd)
{
DBUG_ENTER("trans_xa_prepare");
if (thd->transaction.xid_state.xa_state != XA_IDLE)
my_error(ER_XAER_RMFAIL, MYF(0),
xa_state_names[thd->transaction.xid_state.xa_state]);
else if (!thd->transaction.xid_state.xid.eq(thd->lex->xid))
my_error(ER_XAER_NOTA, MYF(0));
else if (ha_prepare(thd))
{
xid_cache_delete(&thd->transaction.xid_state);
thd->transaction.xid_state.xa_state= XA_NOTR;
my_error(ER_XA_RBROLLBACK, MYF(0));
}
else
thd->transaction.xid_state.xa_state= XA_PREPARED;
DBUG_RETURN(thd->is_error() ||
thd->transaction.xid_state.xa_state != XA_PREPARED);
}
/**
Commit and terminate the a XA transaction.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_xa_commit(THD *thd)
{
bool res= TRUE;
enum xa_states xa_state= thd->transaction.xid_state.xa_state;
DBUG_ENTER("trans_xa_commit");
if (!thd->transaction.xid_state.xid.eq(thd->lex->xid))
{
/*
xid_state.in_thd is always true beside of xa recovery procedure.
Note, that there is no race condition here between xid_cache_search
and xid_cache_delete, since we always delete our own XID
(thd->lex->xid == thd->transaction.xid_state.xid).
The only case when thd->lex->xid != thd->transaction.xid_state.xid
and xid_state->in_thd == 0 is in the function
xa_cache_insert(XID, xa_states), which is called before starting
client connections, and thus is always single-threaded.
*/
XID_STATE *xs= xid_cache_search(thd->lex->xid);
res= !xs || xs->in_thd;
if (res)
my_error(ER_XAER_NOTA, MYF(0));
else
{
res= xa_trans_rolled_back(xs);
ha_commit_or_rollback_by_xid(thd->lex->xid, !res);
xid_cache_delete(xs);
}
DBUG_RETURN(res);
}
if (xa_trans_rolled_back(&thd->transaction.xid_state))
{
xa_trans_force_rollback(thd);
res= thd->is_error();
}
else if (xa_state == XA_IDLE && thd->lex->xa_opt == XA_ONE_PHASE)
{
int r= ha_commit_trans(thd, TRUE);
if ((res= test(r)))
my_error(r == 1 ? ER_XA_RBROLLBACK : ER_XAER_RMERR, MYF(0));
}
else if (xa_state == XA_PREPARED && thd->lex->xa_opt == XA_NONE)
{
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
MDL_request mdl_request;
/*
Acquire metadata lock which will ensure that COMMIT is blocked
by active FLUSH TABLES WITH READ LOCK (and vice versa COMMIT in
progress blocks FTWRL).
We allow FLUSHer to COMMIT; we assume FLUSHer knows what it does.
*/
mdl_request.init(MDL_key::COMMIT, "", "", MDL_INTENTION_EXCLUSIVE,
MDL_TRANSACTION);
if (thd->mdl_context.acquire_lock(&mdl_request,
thd->variables.lock_wait_timeout))
{
ha_rollback_trans(thd, TRUE);
my_error(ER_XAER_RMERR, MYF(0));
}
else
{
Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations.
2010-11-11 20:11:05 +03:00
DEBUG_SYNC(thd, "trans_xa_commit_after_acquire_commit_lock");
res= test(ha_commit_one_phase(thd, 1));
if (res)
my_error(ER_XAER_RMERR, MYF(0));
}
}
else
{
my_error(ER_XAER_RMFAIL, MYF(0), xa_state_names[xa_state]);
DBUG_RETURN(TRUE);
}
2010-02-03 03:06:42 +03:00
thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG);
thd->transaction.all.modified_non_trans_table= FALSE;
thd->server_status&= ~SERVER_STATUS_IN_TRANS;
xid_cache_delete(&thd->transaction.xid_state);
thd->transaction.xid_state.xa_state= XA_NOTR;
DBUG_RETURN(res);
}
/**
Roll back and terminate a XA transaction.
@param thd Current thread
@retval FALSE Success
@retval TRUE Failure
*/
bool trans_xa_rollback(THD *thd)
{
bool res= TRUE;
enum xa_states xa_state= thd->transaction.xid_state.xa_state;
DBUG_ENTER("trans_xa_rollback");
if (!thd->transaction.xid_state.xid.eq(thd->lex->xid))
{
XID_STATE *xs= xid_cache_search(thd->lex->xid);
if (!xs || xs->in_thd)
my_error(ER_XAER_NOTA, MYF(0));
else
{
xa_trans_rolled_back(xs);
ha_commit_or_rollback_by_xid(thd->lex->xid, 0);
xid_cache_delete(xs);
}
DBUG_RETURN(thd->stmt_da->is_error());
}
if (xa_state != XA_IDLE && xa_state != XA_PREPARED && xa_state != XA_ROLLBACK_ONLY)
{
my_error(ER_XAER_RMFAIL, MYF(0), xa_state_names[xa_state]);
DBUG_RETURN(TRUE);
}
res= xa_trans_force_rollback(thd);
2010-02-03 03:06:42 +03:00
thd->variables.option_bits&= ~(OPTION_BEGIN | OPTION_KEEP_LOG);
thd->transaction.all.modified_non_trans_table= FALSE;
thd->server_status&= ~SERVER_STATUS_IN_TRANS;
xid_cache_delete(&thd->transaction.xid_state);
thd->transaction.xid_state.xa_state= XA_NOTR;
DBUG_RETURN(res);
}