REORGANIZE PARTITION
There were several problems which lead to this this,
all related to bad error handling.
1) There was several bugs preventing the ddl-log to be used for
cleaning up created files on error.
2) The error handling after the copy partition rows did not close
and unlock the tables, resulting in deletion of partitions
which were in use, which lead InnoDB to put the partition to
drop in a background queue.
sql/ha_partition.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Better error handling, if partition has been created/opened/locked
then make sure it is unlocked and closed before returning error.
The delete of the newly created partition is handled by the ddl-log.
sql/sql_parse.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Fix a bug found when experimenting, thd could really be NULL here,
as mentioned in the function header.
sql/sql_partition.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Used the correct .frm shadow name to put into the ddl-log.
Really use the ddl-log to handle errors.
sql/sql_table.cc:
Bug#47343: InnoDB fails to clean-up after lock wait timeout on
REORGANIZE PARTITION
Fixes of the ddl-log when used as error recovery (no crash).
When executing an entry from memory (not read from disk)
the name_len was not set correctly.
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase. Created directory.
Conflict because storage/innobase is not versioned, but has versioned children. Versioned directory.
Conflict adding file storage/innobase. Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler. Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children. Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
3655 Jon Olav Hauglid 2009-10-19
Bug #30977 Concurrent statement using stored function and DROP FUNCTION
breaks SBR
Bug #48246 assert in close_thread_table
Implement a fix for:
Bug #41804 purge stored procedure cache causes mysterious hang for many
minutes
Bug #49972 Crash in prepared statements
The problem was that concurrent execution of DML statements that
use stored functions and DDL statements that drop/modify the same
function might result in incorrect binary log in statement (and
mixed) mode and therefore break replication.
This patch fixes the problem by introducing metadata locking for
stored procedures and functions. This is similar to what is done
in Bug#25144 for views. Procedures and functions now are
locked using metadata locks until the transaction is either
committed or rolled back. This prevents other statements from
modifying the procedure/function while it is being executed. This
provides commit ordering - guaranteeing serializability across
multiple transactions and thus fixes the reported binlog problem.
Note that we do not take locks for top-level CALLs. This means
that procedures called directly are not protected from changes by
simultaneous DDL operations so they are executed at the state they
had at the time of the CALL. By not taking locks for top-level
CALLs, we still allow transactions to be started inside
procedures.
This patch also changes stored procedure cache invalidation.
Upon a change of cache version, we no longer invalidate the entire
cache, but only those routines which we use, only when a statement
is executed that uses them.
This patch also changes the logic of prepared statement validation.
A stored procedure used by a prepared statement is now validated
only once a metadata lock has been acquired. A version mismatch
causes a flush of the obsolete routine from the cache and
statement reprepare.
Incompatible changes:
1) ER_LOCK_DEADLOCK is reported for a transaction trying to access
a procedure/function that is locked by a DDL operation in
another connection.
2) Procedure/function DDL operations are now prohibited in LOCK
TABLES mode as exclusive locks must be taken all at once and
LOCK TABLES provides no way to specifiy procedures/functions to
be locked.
Test cases have been added to sp-lock.test and rpl_sp.test.
Work on this bug has very much been a team effort and this patch
includes and is based on contributions from Davi Arnaut, Dmitry
Lenev, Magne Mæhre and Konstantin Osipov.
mysql-test/r/ps_ddl.result:
Update results (Bug#30977).
mysql-test/r/ps_ddl1.result:
Update results (Bug#30977).
mysql-test/r/sp-error.result:
Update results (Bug#30977).
mysql-test/r/sp-lock.result:
Update results (Bug#30977).
mysql-test/suite/rpl/r/rpl_sp.result:
Update results (Bug#30977).
mysql-test/suite/rpl/t/rpl_sp.test:
Add a test case for Bug#30977.
mysql-test/t/ps_ddl.test:
Update comments. We no longer re-prepare a prepared statement
when a stored procedure used in top-level CALL is changed.
mysql-test/t/ps_ddl1.test:
Modifying stored procedure p1 no longer invalidates prepared
statement "call p1" -- we can re-use the prepared statement
without invalidation.
mysql-test/t/sp-error.test:
Use a constant for an error value.
mysql-test/t/sp-lock.test:
Add test coverage for Bug#30977.
sql/lock.cc:
Implement lock_routine_name() - a way to acquire an
exclusive metadata lock (ex- name-lock) on
stored procedure/function.
sql/sp.cc:
Change semantics of sp_cache_routine() -- now it has an option
to make sure that the routine that is cached is up to date (has
the latest sp cache version).
Add sp_cache_invalidate() to sp_drop_routine(), where it was
missing (a bug!).
Acquire metadata locks for SP DDL (ALTER/CREATE/DROP). This is
the core of the fix for Bug#30977.
Since caching and cache invalidation scheme was changed, make
sure we don't invalidate the SP cache in the middle of a stored
routine execution. At the same time, make sure we don't access
stale data due to lack of invalidation.
For that, change ALTER FUNCTION/PROCEDURE to not use the cache,
and SHOW PROCEDURE CODE/SHOW CREATE PROCEDURE/FUNCTION to always
read an up to date version of the routine from the cache.
sql/sp.h:
Add a helper wrapper around sp_cache_routine().
sql/sp_cache.cc:
Implement new sp_cache_version() and sp_cache_flush_obsolete().
Now we flush stale routines individually, rather than all at once.
sql/sp_cache.h:
Update signatures of sp_cache_version() and sp_cache_flush_obsolete().
sql/sp_head.cc:
Add a default initialization of sp_head::m_sp_cache_version.
Remove a redundant sp_head::create().
sql/sp_head.h:
Add m_sp_cache_version to sp_head class - we now
keep track of every routine in the stored procedure cache, rather than
of the entire cache.
sql/sql_base.cc:
Implement prelocking for stored routines. Validate stored
routines after they were locked.
Flush obsolete routines upon next access, one by one, not all at once
(Bug#41804).
Style fixes.
sql/sql_class.h:
Rename a Open_table_context method.
sql/sql_parse.cc:
Make sure stored procedures DDL commits the active transaction
(issues an implicit commit before and after).
Remove sp_head::create(), a pure redundancy.
Move the semantical check during alter routine inside sp_update_routine() code in order to:
- avoid using SP cache during update, it may be obsolete.
- speed up and simplify the update procedure.
Remove sp_cache_flush_obsolete() calls, we no longer flush the entire
cache, ever, stale routines are flushed before next use, one at a time.
sql/sql_prepare.cc:
Move routine metadata validation to open_and_process_routine().
Fix Bug#49972 (don't swap flags at reprepare).
Reset Sroutine_hash_entries in reinit_stmt_before_use().
Remove SP cache invalidation, it's now done by open_tables().
sql/sql_show.cc:
Fix a warning: remove an unused label.
sql/sql_table.cc:
Reset mdl_request.ticket for tickets acquired for routines inlined
through a view, in CHECK TABLE statement, to satisfy an MDL assert.
sql/sql_update.cc:
Move the cleanup of "translation items" to close_tables_for_reopen(),
since it's needed in all cases when we back off, not just
the back-off in multi-update. This fixes a bug when the server
would crash on attempt to back off when opening tables
for a statement that uses information_schema tables.
Conflicts:
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/instance_map.cc
Contents conflict in server-tools/instance-manager/listener.cc
Contents conflict in server-tools/instance-manager/options.cc
Contents conflict in server-tools/instance-manager/user_map.cc
"HANDLER statements within a transaction might lead to deadlocks".
Introduce a notion of a sentinel to MDL_context. A sentinel
is a ticket that separates all tickets in the context into two
groups: before and after it. Currently we can have (and need) only
one designated sentinel -- it separates all locks taken by LOCK
TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK
and all other locks, which must be released at COMMIT or ROLLBACK.
The tricky part is maintaining the sentinel up to date when
someone release its corresponding ticket. This can happen, e.g.
if someone issues DROP TABLE under LOCK TABLES (generally,
see all calls to release_all_locks_for_name()).
MDL_context::release_ticket() is modified to take care of it.
******
A fix and a test case for Bug#46224 "HANDLER statements within a
transaction might lead to deadlocks".
An attempt to mix HANDLER SQL statements, which are transaction-
agnostic, an open multi-statement transaction,
and DDL against the involved tables (in a concurrent connection)
could lead to a deadlock. The deadlock would occur when
HANDLER OPEN or HANDLER READ would have to wait on a conflicting
metadata lock. If the connection that issued HANDLER statement
also had other metadata locks (say, acquired in scope of a
transaction), a classical deadlock situation of mutual wait
could occur.
Incompatible change: entering LOCK TABLES mode automatically
closes all open HANDLERs in the current connection.
Incompatible change: previously an attempt to wait on a lock
in a connection that has an open HANDLER statement could wait
indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK
is produced.
The idea of the fix is to merge thd->handler_mdl_context
with the main mdl_context of the connection, used for transactional
locks. This makes deadlock detection possible, since all waits
with locks are "visible" and available to analysis in a single
MDL context of the connection.
Since HANDLER locks and transactional locks have a different life
cycle -- HANDLERs are explicitly open and closed, and so
are HANDLER locks, explicitly acquired and released, whereas
transactional locks "accumulate" till the end of a transaction
and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT,
a concept of "sentinel" was introduced to MDL_context.
All locks, HANDLER and others, reside in the same linked list.
However, a selected element of the list separates locks with
different life cycle. HANDLER locks always reside at the
end of the list, after the sentinel. Transactional locks are
prepended to the beginning of the list, before the sentinel.
Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only
release those locks that reside before the sentinel. HANDLER locks
must be released explicitly as part of HANDLER CLOSE statement,
or an implicit close.
The same approach with sentinel
is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES
statement has never worked together, the implementation is
made simple and only maintains one sentinel, which is used either
for HANDLER locks, or for LOCK TABLES locks.
mysql-test/include/handler.inc:
Add test coverage for Bug#46224 "HANDLER statements within a
transaction might lead to deadlocks".
Extended HANDLER coverage to cover a mix of HANDLER, transactions
and DDL statements.
mysql-test/r/handler_innodb.result:
Update results (Bug#46224).
mysql-test/r/handler_myisam.result:
Update results (Bug#46224).
sql/lock.cc:
Remove thd->some_tables_deleted, it's never used.
sql/log_event.cc:
No need to check for thd->locked_tables_mode,
it's done inside release_transactional_locks().
sql/mdl.cc:
Implement the concept of HANDLER and LOCK TABLES "sentinel".
Implement a method to clone an acquired ticket.
Do not return tickets beyond the sentinel when acquiring
locks, create a copy.
Remove methods to merge and backup MDL_context, they are now
not used (Hurra!). This opens a path to a proper constructor
and destructor of class MDL_context (to be done in a separate
patch).
Modify find_ticket() to provide information about where
the ticket position is with regard to the sentinel.
sql/mdl.h:
Add declarations necessary for the implementation of the concept
of "sentinel", a dedicated ticket separating transactional and
non-transactional locks.
sql/mysql_priv.h:
Add mark_tmp_table_for_reuse() declaration,
a function to "close" a single session (temporary) table.
sql/sql_base.cc:
Remove thd->some_tables_deleted.
Modify deadlock-prevention asserts and deadlock detection
heuristics to take into account that from now on HANDLER locks
reside in the same locking context.
Add broadcast_refresh() to mysql_notify_thread_having_shared_lock():
this is necessary for the case when a thread having a shared lock
is asleep in tdc_wait_for_old_versions(). This situation is only
possible with HANDLER t1 OPEN; FLUSH TABLE (since all over code paths
that lead to tdc_wait_for_old_versions() always have an
empty MDL_context). Previously the server would simply deadlock
in this situation.
sql/sql_class.cc:
Remove now unused member "THD::some_tables_deleted".
Move mysql_ha_cleanup() a few lines above in THD::cleanup()
to make sure that all handlers are closed when it's time to
destroy the MDL_context of this connection.
Remove handler_mdl_context and handler_tables.
sql/sql_class.h:
Remove THD::handler_tables, THD::handler_mdl_context,
THD::some_tables_deleted.
sql/sql_handler.cc:
Remove thd->handler_tables.
Remove thd->handler_mdl_context.
Rewrite mysql_ha_open() to have no special provision for MERGE
tables, now that we don't have to manipulate with thd->handler_tables
it's easy to do.
Remove dead code.
Fix a bug in mysql_ha_flush() when we would always flush
a temporary HANDLER when mysql_ha_flush() is called (actually
mysql_ha_flush() never needs to flush temporary tables).
sql/sql_insert.cc:
Update a comment, no more thd->some_tables_deleted.
sql/sql_parse.cc:
Implement an incompatible change: entering LOCK TABLES closes
active HANDLERs, if any.
Now that we have a sentinel, we don't need to check
for thd->locked_tables_mode when releasing metadata locks in
COMMIT/ROLLBACK.
sql/sql_plist.h:
Add new (now necessary) methods to the list class.
sql/sql_prepare.cc:
Make sure we don't release HANDLER locks when rollback to a
savepoint, set to not keep locks taken at PREPARE.
sql/sql_servers.cc:
Update to a new signature of MDL_context::release_all_locks().
sql/sql_table.cc:
Remove thd->some_tables_deleted.
sql/transaction.cc:
Add comments.
Make sure rollback to (MDL) savepoint works under LOCK TABLES and
with HANDLER tables.
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies
CMakeLists.txt:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
client/mysql.cc:
don't crash with --show-warnings when mysqld dies
config/ac-macros/plugins.m4:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
include/my_getopt.h:
comments
include/my_pthread.h:
fix double #define
mysql-test/mysql-test-run.pl:
run sys_vars suite by default
properly recognize envirinment variables (e.g. MTR_MAX_SAVE_CORE) set to 0
escape gdb command line arguments
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
init_slave+utf8 bug
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
init_slave+utf8 bug
mysys/my_getopt.c:
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
mysys/typelib.c:
support for flagset
sql/ha_ndbcluster.cc:
backport from telco tree
sql/item_func.cc:
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
sql/sql_builtin.cc.in:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
sql/sql_plugin.cc:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#32902 plugin variables don't know their names
Bug#44797 plugins w/o command-line options have no disabling option in --help
sql/sys_vars.cc:
all server variables are defined here
storage/myisam/ft_parser.c:
remove unnecessary updates of param->quot
storage/myisam/ha_myisam.cc:
myisam_* variables belong here
strings/my_vsnprintf.c:
%o and %llx
unittest/mysys/my_vsnprintf-t.c:
%o and %llx tests
vio/viosocket.c:
bugfix: fix @@wait_timeout to work with socket timeouts (vs. alarm thread)
The problem is a somewhat common misusage of the strmake function.
The strmake(dst, src, len) function writes at most /len/ bytes to
the string pointed to by src, not including the trailing null byte.
Hence, if /len/ is the exact length of the destination buffer, a
one byte buffer overflow can occur if the length of the source
string is equal to or greater than /len/.
client/mysqldump.c:
Make room for the trailing null byte.
libmysql/libmysql.c:
Add comment, there is enough room in the buffer.
Increase buffer length, two strings are concatenated.
libmysqld/lib_sql.cc:
Make room for the trailing null byte.
mysys/default.c:
Make room for the trailing null bytes.
mysys/mf_pack.c:
Make room for the trailing null byte.
server-tools/instance-manager/commands.cc:
Copy only if overflow isn't possible in both cases.
server-tools/instance-manager/listener.cc:
Make room for the trailing null byte.
sql/log.cc:
Make room for the trailing null byte.
sql/sp_pcontext.h:
Cosmetic fix.
sql/sql_acl.cc:
MAX_HOSTNAME already specifies space for the trailing null byte.
sql/sql_parse.cc:
Make room for the trailing null byte.
sql/sql_table.cc:
Make room for the trailing null byte.
with temporary table and partitions
It was possible to create temporary partitioned tables
via create table ... like ... (which is not allowed with
create temporary table). This lead to a new HA_EXTRA flag
(HA_EXTRA_MMAP) was sent to the partitioning handler,
which was caught on an assert in debug builds.
Solution was to check for partitioned tables when
doing create table ... like ... and disallow it.
mysql-test/r/partition_error.result:
Bug#49477: Assertion `0' failed in ha_partition.cc:5530
with temporary table and partitions
Added result
mysql-test/t/partition_error.test:
Bug#49477: Assertion `0' failed in ha_partition.cc:5530
with temporary table and partitions
Added test
sql/sql_table.cc:
Bug#49477: Assertion `0' failed in ha_partition.cc:5530
with temporary table and partitions
Added check to prevent creation of partitioned temporary
tables.
Only copy .par file for partitioned tables.
-----------------------------------------------------------
2497.392.1 Michael Widenius 2008-08-19
Fixes for Bug #38016 Maria: trying to access freed memory when
committing a transaction.
Don't write out states if they haven't changed.
sql/sql_table.cc:
Call extra(HA_EXTRA_PREPARE_FOR_RENAME) before renaming a table.
Bug #46654 False deadlock on concurrent DML/DDL with partitions,
inconsistent behavior
The problem was that if one connection is running a multi-statement
transaction which involves a single partitioned table, and another
connection attempts to alter the table, the first connection gets
ER_LOCK_DEADLOCK and cannot proceed anymore, even when the ALTER TABLE
statement in another connection has timed out or failed.
The reason for this was that the prepare phase for ALTER TABLE for
partitioned tables removed all instances of the table from the table
definition cache before it started waiting on the lock. The transaction
running in the first connection would notice this and report ER_LOCK_DEADLOCK.
This patch changes the prep_alter_part_table() ALTER TABLE code so that
tdc_remove_table() is no longer called. Instead, only the TABLE instance
changed by prep_alter_part_table() is marked as needing reopen.
The patch also removes an unnecessary call to tdc_remove_table() from
mysql_unpack_partition() as the changed TABLE object is destroyed by the
caller at a later point.
Test case added in partition_sync.test.
Bug#42546 Backup: RESTORE fails, thinking it finds an existing table
The problem occured when a MDL locking conflict happened for a non-existent
table between a CREATE and a INSERT statement. The code for CREATE
interpreted this lock conflict to mean that the table existed,
which meant that the statement failed when it should not have.
The problem could occur for CREATE TABLE, CREATE TABLE LIKE and
ALTER TABLE RENAME.
This patch fixes the problem for CREATE TABLE and CREATE TABLE LIKE.
It is based on code backported from the mysql-6.1-fk tree written
by Dmitry Lenev. CREATE now uses normal open_and_lock_tables() code
to acquire exclusive locks. This means that for the test case in the bug
description, CREATE will wait until INSERT completes so that it can
get the exclusive lock. This resolves the reported bug.
The patch also prohibits CREATE TABLE and CREATE TABLE LIKE under
LOCK TABLES. Note that this is an incompatible change and must
be reflected in the documentation. Affected test cases have been
updated.
mdl_sync.test contains tests for CREATE TABLE and CREATE TABLE LIKE.
Fixing the issue for ALTER TABLE RENAME is beyond the scope of this
patch. ALTER TABLE cannot be prohibited from working under LOCK TABLES
as this could seriously impact customers and a proper fix would require
a significant rewrite.
------------------------------------------------------------
revno: 2617.68.25
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-next-bg-pre2-2
timestamp: Wed 2009-09-16 18:26:50 +0400
message:
Follow-up for one of pre-requisite patches for fixing bug #30977
"Concurrent statement using stored function and DROP FUNCTION
breaks SBR".
Made enum_mdl_namespace enum part of MDL_key class and removed MDL_
prefix from the names of enum members. In order to do the latter
changed name of PROCEDURE symbol to PROCEDURE_SYM (otherwise macro
which was automatically generated for this symbol conflicted with
MDL_key::PROCEDURE enum member).
Introduce a counter for protection against global read lock on thread level.
The functions for protection against global read lock sometimes need a local
variable to signal when the protection is set, and hence need to be released.
It would be better to control this behaviour via a counter on the THD struct,
telling how many times the protection has been claimed by the current thread.
A side-effect of the fix is that if protection is claimed twice for a thread,
only a simple increment is required for the second claim, instead of a
mutex-protected increment of the global variable protect_against_global_read_lock.
sql/lock.cc:
Count how many times that we have claimed protection against global read lock.
Assert that we really have the protection when releasing it.
Added comments to all functions operating on global_read_lock.
sql/sql_class.cc:
Added the counter variable global_read_lock_protection.
sql/sql_class.h:
Added the counter variable global_read_lock_protection.
sql/sql_parse.cc:
Replaced test on local variable need_start_waiting with test on
thd->global_read_lock_protection.
sql/sql_table.cc:
Replaced test on local variable need_start_waiting with test on
thd->global_read_lock_protection.
sql/sql_trigger.cc:
Inserted test on thd->global_read_lock_protection.
Bug #48248 assert in MDL_ticket::upgrade_shared_lock_to_exclusive
The assert would happen if REPAIR TABLE was used on a table already
locked by LOCK TABLES READ. REPAIR mistakenly tried to upgrade the
read-lock to exclusive, thereby triggering the assert.
The cause of the problem was that REPAIR TABLE ignored errors
from opening and locking tables. This is by design, as REPAIR
can be used to broken tables that cannot be opened. However,
repair also ignored logical errors such as the inability to
exclusivly lock a table due to conflicting LOCK TABLES.
This patch fixes the problem by not ignoring errors from
opening and locking tables if inside LOCK TABLES mode.
In LOCK TABLES we already know that the table can be opened,
so that the failure to open must be a logical error.
Test added to repair.test.
A pre-requisite patch for Bug#30977 "Concurrent statement using
stored function and DROP FUNCTION breaks SBR".
This patch changes the MDL API by introducing a namespace for
lock keys: MDL_TABLE for tables and views and MDL_PROCEDURE
for stored procedures and functions. The latter is needed for
the fix for Bug#30977.
Bug #42074 concurrent optimize table and
alter table = Assertion failed: thd->is_error()
This assertion could occur if OPTIMIZE TABLE was started on a InnoDB table
and the table was altered to different storage engine after OPTIMIZE
had started. This allowed OPTIMIZE to pass the initial checks for
storage engine support, but fail once it reached "recreate+analyze"
if this operation was not supported by the new storage engine.
The bug had no consequences for non-debug builds of the server.
In detail, the assertion was triggered when ha_analyze() returned
HA_ADMIN_NOT_IMPLEMENTED. This led to a code path which included an
assert checking for diagnostics area contents. Since this area had
not been filled, the assertion was triggered. The diagnostics area
is in this case only used to provide more detailed information about
why optimize failed. The triggered code path sends this information
to the client and clears the diagnostic area.
This patch fixed the problem by adding an error message to the diagnostic
area if ha_analyze() fails. This error message contains the error code
returned by ha_analyze().
Test case added to innodb_mysql_sync.test.
----------------------------------------------------------
revno: 2617.69.24
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-42546
timestamp: Fri 2009-08-14 19:22:05 +0400
message:
A pre-requisite for a fix for Bug#42546 "Backup: RESTORE fails, thinking it
finds an existing table"
Back-port from WL 148 "Foreign keys" feature tree a patch
that introduced Prelocking_strategy class -- a way to parameterize
open_tables() behaviour, implemented by Dmitry Lenev.
(Part of WL#4284).
sql/sql_base.cc:
Implement different prelocking strategies. Use an instance of
prelocking_strategy in open_tables().
sql/sql_class.h:
Add declarations for class Prelocking_strategy.
sql/sql_lex.h:
Add a helper method to access last table of the global table list
(lex->query_tables).
sql/sql_parse.cc:
Use a special prelocking strategy when locking tables for LOCK TABLES.
sql/sql_table.cc:
Use normal open_and_lock_tables_derived() in ALTER TABLE.
sql/sql_yacc.yy:
Modify the grammar to not pollute the global table list with tables
that should not be opened.
----------------------------------------------------------
revno: 2617.69.20
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-4284-1-assert
timestamp: Thu 2009-08-13 18:29:55 +0400
message:
WL#4284 "Transactional DDL locking"
A review fix.
Since WL#4284 implementation separated MDL_request and MDL_ticket,
MDL_request becamse a utility object necessary only to get a ticket.
Store it by-value in TABLE_LIST with the intent to merge
MDL_request::key with table_list->table_name and table_list->db
in future.
Change the MDL subsystem to not require MDL_requests to
stay around till close_thread_tables().
Remove the list of requests from the MDL context.
Requests for shared metadata locks acquired in open_tables()
are only used as a list in recover_from_failed_open_table_attempt(),
which calls mdl_context.wait_for_locks() for this list.
To keep such list for recover_from_failed_open_table_attempt(),
introduce a context class (Open_table_context), that collects
all requests.
A lot of minor cleanups and simplications that became possible
with this change.
sql/event_db_repository.cc:
Remove alloc_mdl_requests(). Now MDL_request instance is a member
of TABLE_LIST, and init_one_table() initializes it.
sql/ha_ndbcluster_binlog.cc:
Remove now unnecessary declaration and initialization
of binlog_mdl_request.
sql/lock.cc:
No need to allocate MDL requests in lock_table_names() now.
sql/log.cc:
Use init_one_table() method, remove alloc_mdl_requests(),
which is now unnecessary.
sql/log_event.cc:
No need to allocate mdl_request separately now.
Use init_one_table() method.
sql/log_event_old.cc:
Update to the new signature of close_tables_for_reopen().
sql/mdl.cc:
Update try_acquire_exclusive_lock() to be more easy to use.
Function lock_table_name_if_not_cached() has been removed.
Make acquire_shared_lock() signature consistent with
try_acquire_exclusive_lock() signature.
Remove methods that are no longer used.
Update comments.
sql/mdl.h:
Implement an assignment operator that doesn't
copy MDL_key (MDL_key::operator= is private and
should remain private).
This is a hack to work-around assignment of TABLE_LIST
by value in several places. Such assignments violate
encapsulation, since only perform a shallow copy.
In most cases these assignments are a hack on their own.
sql/mysql_priv.h:
Update signatures of close_thread_tables() and close_tables_for_reopen().
sql/sp.cc:
Allocate TABLE_LIST in thd->mem_root.
Use init_one_table().
sql/sp_head.cc:
Use init_one_table().
Remove thd->locked_tables_root, it's no longer needed.
sql/sql_acl.cc:
Use init_mdl_requests() and init_one_table().
sql/sql_base.cc:
Update to new signatures of try_acquire_shared_lock() and
try_acquire_exclusive_lock().
Remove lock_table_name_if_not_cached().
Fix a bug in open_ltable() that would not return ER_LOCK_DEADLOCK
in case of a failed lock_tables() and a multi-statement
transaction.
Fix a bug in open_and_lock_tables_derived() that would
not return ER_LOCK_DEADLOCK in case of a multi-statement
transaction and a failure of lock_tables().
Move assignment of enum_open_table_action to a method of Open_table_context, a new class that maintains information
for backoff actions.
Minor rearrangements of the code.
Remove alloc_mdl_requests() in functions that work with system
tables: instead the patch ensures that callers always initialize
TABLE_LIST argument.
sql/sql_class.cc:
THD::locked_tables_root is no more.
sql/sql_class.h:
THD::locked_tables_root is no more.
Add a declaration for Open_table_context class.
sql/sql_delete.cc:
Update to use the simplified MDL API.
sql/sql_handler.cc:
TABLE_LIST::mdl_request is stored by-value now.
Ensure that mdl_request.ticket is NULL for every request
that is passed into MDL, to satisfy MDL asserts.
@ sql/sql_help.cc
Function open_system_tables_for_read() no longer initializes
mdl_requests.
Move TABLE_LIST::mdl_request initialization closer to
TABLE_LIST initialization.
sql/sql_help.cc:
Function open_system_tables_for_read() no longer initializes
mdl_requests.
Move TABLE_LIST::mdl_request initialization closer to
TABLE_LIST initialization.
sql/sql_insert.cc:
Remove assignment by-value of TABLE_LIST in
TABLEOP_HOOKS. We can't carry over a granted
MDL ticket from one table list to another.
sql/sql_parse.cc:
Change alloc_mdl_requests() -> init_mdl_requests().
@todo We can remove init_mdl_requests() altogether
in some places: all places that call add_table_to_list()
already have mdl requests initialized.
sql/sql_plugin.cc:
Use init_one_table().
THD::locked_tables_root is no more.
sql/sql_servers.cc:
Use init_one_table().
sql/sql_show.cc:
Update acquire_high_priority_shared_lock() to use
TABLE_LIST::mdl_request rather than allocate an own.
Fix get_trigger_table_impl() to use init_one_table(),
check for out of memory, follow the coding style.
sql/sql_table.cc:
Update to work with TABLE_LIST::mdl_request by-value.
Remove lock_table_name_if_not_cached().
The code that used to delegate to it is quite simple and
concise without it now.
sql/sql_udf.cc:
Use init_one_table().
sql/sql_update.cc:
Update to use the new signature of close_tables_for_reopen().
sql/table.cc:
Move re-setting of mdl_requests for prepared statements
and stored procedures from close_thread_tables() to
reinit_stmt_before_use().
Change alloc_mdl_requests() to init_mdl_requests().
init_mdl_requests() is a hack that can't be deleted
until we don't have a list-aware TABLE_LIST constructor.
Hopefully its use will be minimal
sql/table.h:
Change alloc_mdl_requests() to init_mdl_requests()
TABLE_LIST::mdl_request is stored by value.
sql/tztime.cc:
We no longer initialize mdl requests in open_system_tables_for*()
functions. Move this initialization closer to initialization
of the rest of TABLE_LIST members.
storage/myisammrg/ha_myisammrg.cc:
Simplify mdl_request initialization.
----------------------------------------------------------
revno: 2617.69.2
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-azalea-bugfixing
timestamp: Mon 2009-08-03 19:26:04 +0400
message:
A fix and a test case for Bug#45035 "Altering table under LOCK TABLES
results in "Error 1213 Deadlock found...".
If a user had a table locked with LOCK TABLES
for READ and for WRITE in the same connection, ALTER TABLE
could fail.
Root cause analysis:
If a connection issues
LOCK TABLE t1 write, t1 a read, t1 b read;
the new LOCK TABLES code in 6.0 (part of WL 3726) will create
the following list of TABLE_LIST objects
(thd->locked_tables_list->m_locked_tables):
{"t1" "b" tl_read_no_insert}, {"t1" "a" tl_read_no_insert},
{"t1" "t1" tl_write }
Later on, when we try to ALTER table t1, mysql_alter_table()
closes all TABLE instances and releases its thr_lock locks,
keeping only an exclusive metadata lock on t1.
But when ALTER is finished, Locked_table_list::reopen_tables()
tries to restore the original list of open and locked tables.
Before this patch, it used to do so one by one:
Open t1 b, get TL_READ_NO_INSERT lock,
Open t1 a, get TL_READ_NO_INSERT lock
Open t1, try to get TL_WRITE lock, deadlock.
The cause of the deadlock is that thr_lock.c doesn't
resolve the situation when the read list only consists
of locks taken by the same thread, followed by this very
thread trying to take a WRITE lock. Indeed, since
thr_lock_multi always gets a sorted list of locks,
WRITE locks always precede READ locks in the list
to lock.
Don't try to fix thr_lock.c deficiency, keep this
code simple.
Instead, try to take all thr_lock locks at once
in ::reopen_tables().
mysql-test/r/lock.result:
Update results: test case for Bug#45035.
mysql-test/t/lock.test:
Add a test case for Bug#45035.
sql/sql_base.cc:
Take all thr_lock locks at once in Locked_tables_list::reopen_tables().
sql/sql_class.h:
Add a helper array to store tables for mysql_lock_tables()
in reopen_tables().
sql/sql_table.cc:
Update unlink_all_closed_tables() to the new signature.
2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1
- initial changeset that introduced the fix for
Bug#989 and follow up fixes for all test suite failures
introduced in the initial changeset.
------------------------------------------------------------
revno: 2617.31.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Fri 2009-03-06 19:17:00 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
Currently the MySQL server does not keep metadata locks on
schema objects for the duration of a transaction, thus failing
to guarantee the integrity of the schema objects being used
during the transaction and to protect then from concurrent
DDL operations. This also poses a problem for replication as
a DDL operation might be replicated even thought there are
active transactions using the object being modified.
The solution is to defer the release of metadata locks until
a active transaction is either committed or rolled back. This
prevents other statements from modifying the table for the
entire duration of the transaction. This provides commitment
ordering for guaranteeing serializability across multiple
transactions.
- Incompatible change:
If MySQL's metadata locking system encounters a lock conflict,
the usual schema is to use the try and back-off technique to
avoid deadlocks -- this schema consists in releasing all locks
and trying to acquire them all in one go.
But in a transactional context this algorithm can't be utilized
as its not possible to release locks acquired during the course
of the transaction without breaking the transaction commitments.
To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be
returned if a lock conflict is encountered during a transaction.
Let's consider an example:
A transaction has two statements that modify table t1, then table
t2, and then commits. The first statement of the transaction will
acquire a shared metadata lock on table t1, and it will be kept
utill COMMIT to ensure serializability.
At the moment when the second statement attempts to acquire a
shared metadata lock on t2, a concurrent ALTER or DROP statement
might have locked t2 exclusively. The prescription of the current
locking protocol is that the acquirer of the shared lock backs off
-- gives up all his current locks and retries. This implies that
the entire multi-statement transaction has to be rolled back.
- Incompatible change:
FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ
LOCK won't cause locked tables to be implicitly unlocked anymore.
mysql-test/extra/binlog_tests/drop_table.test:
Add test case for Bug#989.
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/include/mix1.inc:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/include/mix2.inc:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/r/flush_block_commit.result:
Update test case result (WL#4284).
mysql-test/r/flush_block_commit_notembedded.result:
Update test case result (WL#4284).
mysql-test/r/innodb.result:
Update test case result (WL#4284).
mysql-test/r/innodb_mysql.result:
Update test case result (WL#4284).
mysql-test/r/lock.result:
Add test case result for an effect of WL#4284/Bug#989
(all locks should be released when a connection terminates).
mysql-test/r/mix2_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/not_embedded_server.result:
Update test case result (effects of WL#4284/Bug#989).
Add a test case for interaction of WL#4284 and FLUSH PRIVILEGES.
mysql-test/r/partition_innodb_semi_consistent.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/partition_sync.result:
Temporarily disable the test case for Bug#43867,
which will be fixed by a subsequent backport.
mysql-test/r/ps.result:
Add a test case for effect of PREPARE on transactional
locks: we take a savepoint at beginning of PREAPRE
and release it at the end. Thus PREPARE does not
accumulate metadata locks (Bug#989/WL#4284).
mysql-test/r/read_only_innodb.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_drop_tbl.result:
Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_drop_tbl.result:
Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_unsafe.result:
A side effect of Bug#989 -- slightly different table map ids.
mysql-test/suite/binlog/t/binlog_row_drop_tbl.test:
Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_drop_tbl.test:
Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_row.test:
Update to the new state name. This
is actually a follow up to another patch for WL#4284,
that changes Locked thread state to Table lock.
mysql-test/suite/ndb/r/ndb_index_ordered.result:
Remove result for disabled part of the test case.
mysql-test/suite/ndb/t/disabled.def:
Temporarily disable a test case (Bug#45621).
mysql-test/suite/ndb/t/ndb_index_ordered.test:
Disable a part of a test case (needs update to
reflect semantics of Bug#989).
mysql-test/suite/rpl/t/disabled.def:
Disable tests made meaningless by transactional metadata
locking.
mysql-test/suite/sys_vars/r/autocommit_func.result:
Add a commit (Bug#989).
mysql-test/suite/sys_vars/t/autocommit_func.test:
Add a commit (Bug#989).
mysql-test/t/flush_block_commit.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/t/flush_block_commit_notembedded.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
Add a test case for transaction-scope locks and the global
read lock (Bug#989/WL#4284).
mysql-test/t/innodb.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(effects of Bug#989/WL#4284).
mysql-test/t/lock.test:
Add a test case for Bug#989/WL#4284.
mysql-test/t/not_embedded_server.test:
Add a test case for Bug#989/WL#4284.
mysql-test/t/partition_innodb_semi_consistent.test:
Replace TRUNCATE with DELETE, to not issue
an implicit commit of a transaction, and not depend
on metadata locks.
mysql-test/t/partition_sync.test:
Temporarily disable the test case for Bug#43867,
which needs a fix to be backported from 6.0.
mysql-test/t/ps.test:
Add a test case for semantics of PREPARE and transaction-scope
locks: metadata locks on tables used in PREPARE are enclosed into a
temporary savepoint, taken at the beginning of PREPARE,
and released at the end. Thus PREPARE does not effect
what locks a transaction owns.
mysql-test/t/read_only_innodb.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(Bug#989/WL#4284).
Wait for the read_only statement to actually flush tables before
sending other concurrent statements that depend on its state.
mysql-test/t/xa.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(Bug#989/WL#4284).
sql/ha_ndbcluster_binlog.cc:
Backport bits of changes of ha_ndbcluster_binlog.cc
from 6.0, to fix the failing binlog test suite with
WL#4284. WL#4284 implementation does not work
with 5.1 implementation of ndbcluster binlog index.
sql/log_event.cc:
Release metadata locks after issuing a commit.
sql/mdl.cc:
Style changes (WL#4284).
sql/mysql_priv.h:
Rename parameter to match the name used in the definition (WL#4284).
sql/rpl_injector.cc:
Release metadata locks on commit (WL#4284).
sql/rpl_rli.cc:
Remove assert made meaningless, metadata locks are released
at the end of the transaction.
sql/set_var.cc:
Close tables and release locks if autocommit mode is set.
sql/slave.cc:
Release metadata locks after a rollback.
sql/sql_acl.cc:
Don't implicitly unlock locked tables. Issue a implicit commit
at the end and unlock tables.
sql/sql_base.cc:
Defer the release of metadata locks when closing tables
if not required to.
Issue a deadlock error if the locking protocol requires
that a transaction re-acquire its locks.
Release metadata locks when closing tables for reopen.
sql/sql_class.cc:
Release metadata locks if the thread is killed.
sql/sql_parse.cc:
Release metadata locks after implicitly committing a active
transaction, or after explicit commits or rollbacks.
sql/sql_plugin.cc:
Allocate MDL request on the stack as the use of the table
is contained within the function. It will be removed from
the context once close_thread_tables is called at the end
of the function.
sql/sql_prepare.cc:
The problem is that the prepare phase of the CREATE TABLE
statement takes a exclusive metadata lock lock and this can
cause a self-deadlock the thread already holds a shared lock
on the table being that should be created.
The solution is to make the prepare phase take a shared
metadata lock when preparing a CREATE TABLE statement. The
execution of the statement will still acquire a exclusive
lock, but won't cause any problem as it issues a implicit
commit.
After some discussions with stakeholders it has been decided that
metadata locks acquired during a PREPARE statement must be released
once the statement is prepared even if it is prepared within a multi
statement transaction.
sql/sql_servers.cc:
Don't implicitly unlock locked tables. Issue a implicit commit
at the end and unlock tables.
sql/sql_table.cc:
Close table and release metadata locks after a admin operation.
sql/table.h:
The problem is that the prepare phase of the CREATE TABLE
statement takes a exclusive metadata lock lock and this can
cause a self-deadlock the thread already holds a shared lock
on the table being that should be created.
The solution is to make the prepare phase take a shared
metadata lock when preparing a CREATE TABLE statement. The
execution of the statement will still acquire a exclusive
lock, but won't cause any problem as it issues a implicit
commit.
sql/transaction.cc:
Release metadata locks after the implicitly committed due
to a new transaction being started. Also, release metadata
locks acquired after a savepoint if the transaction is rolled
back to the save point.
The problem is that in some cases transaction-long metadata locks
could be released before the transaction was committed. This could
happen when a active transaction was ended by a "START TRANSACTION"
or "BEGIN" statement, in which case the metadata locks would be
released before the actual commit of the active transaction.
The solution is to defer the release of metadata locks to after the
transaction has been implicitly committed. No test case is provided
as the effort to provide one is too disproportional to the size of
the fix.
----------------------------------------------------------
revno: 2617.23.20
committer: Konstantin Osipov <kostja@sun.com>
branch nick: mysql-6.0-runtime
timestamp: Wed 2009-03-04 16:31:31 +0300
message:
WL#4284 "Transactional DDL locking"
Review comments: "Objectify" the MDL API.
MDL_request and MDL_context still need manual construction and
destruction, since they are used in environment that is averse
to constructors/destructors.
sql/mdl.cc:
Improve comments.
Add asserts to backup()/restore_from_backup()/merge() methods.
Fix an order bug in the error path of mdl_acquire_exclusive_locks():
we used to first free a ticket object, and only then exclude
it from the list of tickets.
------------------------------------------------------------
revno: 2617.23.18
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Mon 2009-03-02 18:18:26 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
This is a prerequisite patch:
These changes are intended to split lock requests from granted
locks and to allow the memory and lifetime of granted locks to
be managed within the MDL subsystem. Furthermore, tickets can
now be shared and therefore are used to satisfy multiple lock
requests, but only shared locks can be recursive.
The problem is that the MDL subsystem morphs lock requests into
granted locks locks but does not manage the memory and lifetime
of lock requests, and hence, does not manage the memory of
granted locks either. This can be problematic because it puts the
burden of tracking references on the users of the subsystem and
it can't be easily done in transactional contexts where the locks
have to be kept around for the duration of a transaction.
Another issue is that recursive locks (when the context trying to
acquire a lock already holds a lock on the same object) requires
that each time the lock is granted, a unique lock request/granted
lock structure structure must be kept around until the lock is
released. This can lead to memory leaks in transactional contexts
as locks taken during the transaction should only be released at
the end of the transaction. This also leads to unnecessary wake
ups (broadcasts) in the MDL subsystem if the context still holds
a equivalent of the lock being released.
These issues are exacerbated due to the fact that WL#4284 low-level
design says that the implementation should "2) Store metadata locks
in transaction memory root, rather than statement memory root" but
this is not possible because a memory root, as implemented in mysys,
requires all objects allocated from it to be freed all at once.
This patch combines review input and significant code contributions
from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev).
mysql-test/r/mdl_sync.result:
Add test case result.
mysql-test/t/mdl_sync.test:
Add test case for shared lock upgrade case.
sql/event_db_repository.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/ha_ndbcluster_binlog.cc:
Use new function names to initialize MDL lock requests.
sql/lock.cc:
Rename MDL functions.
sql/log_event.cc:
The MDL request now holds the table and database name data (MDL_KEY).
sql/mdl.cc:
Move the MDL key to the MDL_LOCK structure in order to make the
object suitable for allocation from a fixed-size allocator. This
allows the simplification of the lists in the MDL_LOCK object,
which now are just two, one for granted tickets and other for
waiting (upgraders) tickets.
Recursive requests for a shared lock on the same object can now
be granted using the same lock ticket. This schema is only used
for shared locks because that the only case that matters. This
is used to avoid waste of resources in case a context (connection)
already holds a shared lock on a object.
sql/mdl.h:
Introduce a metadata lock object key which is used to uniquely
identify lock objects.
Separate the structure used to represent pending lock requests
from the structure used to represent granted metadata locks.
Rename functions used to manipulate locks requests in order to
have a more consistent function naming schema.
sql/sp_head.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/sql_acl.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_base.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_class.h:
Last acquired lock before the savepoint was set.
sql/sql_delete.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_handler.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_insert.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_parse.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_plist.h:
Typedef for iterator type.
sql/sql_plugin.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_servers.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_show.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_table.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_trigger.cc:
Save reference to the lock ticket so it can be downgraded later.
sql/sql_udf.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/table.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/table.h:
Separate MDL lock requests from lock tickets (granted locks).
storage/myisammrg/ha_myisammrg.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
------------------------------------------------------------
revno: 2630.22.3
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Thu 2008-08-07 22:33:43 -0300
message:
WL#4284: Transactional DDL locking
Make transaction management more modular through a new interface.
The overall objective of this change is to provide groundwork
for the design of transactional DDL locking by cleaning up the
transaction high level API to better distinguish operations implicit
and explicit, and single statement transaction from operations on
the normal transaction.
Having a a high-level interface for transaction management provides
a better base for implementing transactional concepts that are not
always tied to storage engines and also makes it easier to interect
with other higher level modules of the server.
client/Makefile.am:
Add new file to the build.
libmysqld/CMakeLists.txt:
Add new file to the build.
libmysqld/Makefile.am:
Add new file to the build.
sql/CMakeLists.txt:
Add new file to the build.
sql/Makefile.am:
Add new file to the build.
sql/handler.cc:
Remove multiplexer commit or rollback function. Most callers already
have enough information to decided whether to rollback or commit.
Having plain and well named functions makes it easier to read
and understand code.
sql/handler.h:
Remove wrapper function as the low level transaction functions
shouldn't be called directly anymore.
sql/log_event.cc:
Rename transaction management functions to the new names.
sql/log_event_old.cc:
Rename transaction management functions to the new names.
sql/mysql_priv.h:
Remove obsolete functions for implicit and explicit commit.
sql/rpl_injector.cc:
Rename transaction management functions to the new names.
sql/rpl_rli.cc:
Rename transaction management functions to the new names.
sql/set_var.cc:
Rename transaction management functions to the new names.
sql/slave.cc:
Rename transaction management functions to the new names.
sql/sql_base.cc:
Rename transaction management functions to the new names.
sql/sql_class.cc:
Rename transaction management functions to the new names.
sql/sql_delete.cc:
Rename transaction management functions to the new names.
sql/sql_do.cc:
Rename transaction management functions to the new names.
sql/sql_insert.cc:
Rename transaction management functions to the new names.
sql/sql_parse.cc:
Rename transaction management functions to the new names.
sql/sql_partition.cc:
Rename transaction management functions to the new names.
sql/sql_table.cc:
Rename transaction management functions to the new names.
sql/transaction.cc:
Implement wrapper functions to differentiate operations on
the single statement transaction from the ones operating
on the normal transaction.
sql/transaction.h:
Export new functions for dealing with transaction commands.
------------------------------------------------------------
revno: 2630.13.16
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: WL#4284
timestamp: Sat 2008-07-26 13:38:20 -0300
message:
WL#4284: Transactional DDL locking
SQL statements' effect on transactions.
Currently the MySQL server and its storage engines are not
capable of rolling back operations that define or modify data
structures (also known as DDL statements) or operations that
alter any of the system tables (the mysql database). Allowing
these group of statements to participate in transactions
is unfeasible at this time (since rollback has no effect
whatsoever on them) and goes against the design of our metadata
locking subsystem.
The solution is to issue implicit commits before and after
those statements execution. This effectively confines each of
those statements to its own special transaction and ensures
that metadata locks taken during this special transaction
are not leaked into posterior statements/transactions.
mysql-test/include/commit.inc:
Alter table rename was not committing the normal transaction at the
end of its execution, and as a consequence, the commit was being
issued in the next DDL command (rename table) that happened to end
the active transaction. Other changes are to take into account the
implicit commits issued before and after the DDL command execution.
mysql-test/include/implicit_commit_helper.inc:
Add auxiliary test that shows if a statement issued a
implicit commit.
mysql-test/r/commit_1innodb.result:
Update test case result.
mysql-test/r/implicit_commit.result:
Test implicit commit behavior of some SQL commands.
mysql-test/t/implicit_commit.test:
Test implicit commit behavior of some SQL commands.
sql/events.cc:
Transaction is now ended before the command execution.
sql/mysql_priv.h:
Add flags array for server commands and remove historical
left over.
sql/sql_class.h:
Add flags to control when to issue implicit commits before and
after a command execution.
sql/sql_delete.cc:
A implicit commit is issued at the end of truncate
statements.
sql/sql_parse.cc:
Mark commands that need implicit commits before and
after their executions. The implicit commits of the
statement and the normal transaction are now issued
regardless of the user access privileges.
sql/sql_table.cc:
A implicit commit is now issued before admin commands.
tests/mysql_client_test.c:
Test that COM_REFRESH issues a implicit commit.
----------------------------------------------------------
revno: 2630.4.38
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-4144
timestamp: Wed 2008-06-25 22:07:06 +0400
message:
WL#4144 - Lock MERGE engine children.
Committing a version of the patch merged with WL#3726
on behalf of Ingo.
Step #1: Move locking from parent to children.
MERGE children are now left in the query list of tables
after inserted there in open_tables(). So they are locked
by lock_tables() as all other tables are.
The MERGE parent does not store locks any more. It appears
in a MYSQL_LOCK with zero lock data. This is kind of a "dummy"
lock.
All other lock handling is also done directly on the children.
To protect against parent or child modifications during LOCK
TABLES, the children are detached after every statement and
attached before every statement, even under LOCK TABLES.
The children table list is removed from the query list of tables
on every detach and on close of the parent.
Step #2: Move MERGE specific functionality from SQL layer
into table handler.
Functionality moved from SQL layer (mainly sql_base.cc)
to the table handler (ha_myisammrg.cc).
Unnecessary code is removed from the SQL layer.
Step #3: Moved all MERGE specific members from TABLE
to ha_myisammrg.
Moved members from TABLE to ha_myisammrg.
Renamed some mebers.
Fixed comments.
Step #4: Valgrind and coverage testing
Valgrind did not uncover new problems.
Added purecov comments.
Added a new test for DATA/INDEX DIRECTORY options.
Changed handling of ::reset() for non-attached children.
Fixed the merge-big test.
Step #5: Fixed crashes detected during review
Changed detection when to attach/detach.
Added new tests.
Backport also the fix for Bug#44040 "MySQL allows creating a
MERGE table upon VIEWs but crashes when using it"
include/my_base.h:
WL#4144 - Lock MERGE engine children
Added HA_EXTRA_ADD_CHILDREN_LIST and HA_EXTRA_IS_ATTACHED_CHILDREN
for MERGE table support
mysql-test/r/merge.result:
WL#4144 - Lock MERGE engine children
Fixed test result.
mysql-test/t/disabled.def:
Enable merge.test, which now is working again (WL#4144).
mysql-test/t/merge-big.test:
Fix the messages for wait_condition (merge with WL#3726).
mysql-test/t/merge.test:
WL#4144 - Lock MERGE engine children
Fixed one test to meet coding standards for tests
(upper case keywords, engine names as in SHOW ENGINES).
Fixed error codes.
Added a test for DATA/INDEX DIRECTORY.
mysys/thr_lock.c:
WL#4144 - Lock MERGE engine children
Added purecov comments.
sql/ha_partition.cc:
WL#4144 - Lock MERGE engine children
Added MERGE specific extra operations to ha_partition::extra().
Extended comments.
Changed function comment to doxygen style.
Fixed nomenclature: 'parameter' -> 'operation'.
sql/mysql_priv.h:
WL#4144 - Lock MERGE engine children
Removed declarations for removed functions.
sql/sql_base.cc:
WL#4144 - Lock MERGE engine children
Leave the children in the query list of tables after open_tables().
Set proper back links (prev_global).
Attach MERGE children before and detach them after every
statement. Even under LOCK TABLES.
Remove children from the query list when they are detached.
Remove lock forwarding from children to parent.
Moved MERGE specific functions to ha_myisammrg.cc.
Added purecov comments.
Backport the fix for Bug#44040 "MySQL allows creating a MERGE table upon VIEWs but crashes when using it"
sql/sql_table.cc:
WL#4144 - Lock MERGE engine children
Changed detection of MERGE tables.
sql/table.cc:
WL#4144 - Lock MERGE engine children
Moved is_children_attached() method from TABLE to ha_myisammrg.
sql/table.h:
WL#4144 - Lock MERGE engine children
Moved all MERGE specific members from TABLE to ha_myisammrg.
storage/myisammrg/ha_myisammrg.cc:
WL#4144 - Lock MERGE engine children
Set proper back links in the child list (prev_global).
Added a function for removal of the child list from the query list.
Remove children from the query list when the parent is closed.
Make parent lock handling a dummy (zero locks).
Moved MERGE specific functionality from SQL layer to here.
Moved all MERGE specific members from TABLE to ha_myisammrg.
Renamed children list pointers.
Added initialization and free for the children list mem_root.
Fixed comments.
Added purecov comments.
storage/myisammrg/ha_myisammrg.h:
WL#4144 - Lock MERGE engine children
Added method add_children_list().
Moved all MERGE specific members from TABLE to ha_myisammrg.
Renamed children list pointers.
Added a mem_root for the children list.
storage/myisammrg/myrg_extra.c:
WL#4144 - Lock MERGE engine children
Changed handling of ::reset() for non-attached children.
------------------------------------------------------------
revno: 2630.4.32
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w2
timestamp: Thu 2008-06-19 16:39:58 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After-review fixes in progress.
Ensure that metadata locking subsystem properly handles
out-of-memory conditions. Clarified MDL interface by
separating release of locks and removal of lock requests
from the context.
sql/lock.cc:
mdl_release_lock(), mdl_acquire_exclusive_locks() and
mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
mdl_remove_lock() to do this.
sql/mdl.cc:
Ensured that metadata locking subsystem properly handles
out-of-memory conditions.
Introduced new MDL_INITIALIZED state for metadata lock
request which is used in all cases when lock is not acquired
and we have not associated request with object respesenting
lock.
MDL_PENDING is now only used for requests for exclusive locks
which are added to the MDL_LOCK::waiting_exclusive queue.
mdl_release_lock(), mdl_acquire_exclusive_locks() and
mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
newly introduced mdl_remove_lock() to do this.
Also renamed mdl_release_all_locks_for_name() to
emphasize that it also actually removes lock requests
from the context.
Finally mdl_try_acquire_exclusive_lock() is now returs
information about encountered lock conflict in separate
out parameter since its return value is used for distinguishing
between error (e.g. due to OOM) and success.
sql/mdl.h:
Introduced new MDL_INITIALIZED state for metadata lock
request which is used in all cases when lock is not acquired
and we have not associated request with object respesenting
lock.
MDL_PENDING is now only used for requests for exclusive locks
which are added to the MDL_LOCK::waiting_exclusive queue.
mdl_release_lock(), mdl_acquire_exclusive_locks() and
mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
newly introduced mdl_remove_lock() to do this.
Also renamed mdl_release_all_locks_for_name() to
emphasize that it also actually removes lock requests
from the context.
Finally mdl_try_acquire_exclusive_lock() is now returs
information about encountered lock conflict in separate
out parameter since its return value is used for distinguishing
between error (e.g. due to OOM) and success.
sql/sql_base.cc:
mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
mdl_remove_lock() to do this.
Also adjusted open_table() to ensure that it
releases/removes metadata locks in case of error
after adding/acquiring them (unless keeping these
lock requests is required for recovering action).
sql/sql_delete.cc:
mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
mdl_remove_lock() to do this.
sql/sql_handler.cc:
mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
mdl_remove_lock() to do this.
sql/sql_show.cc:
mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
mdl_remove_lock() to do this.
sql/sql_table.cc:
Renamed mdl_release_all_locks_for_name() to emphasize
that it also actually removes lock requests from the context.
mdl_release_lock(), mdl_acquire_exclusive_locks() and mdl_try_acquire_exclusive_lock() are no longer responsible
for removal of metadata lock requests from the context.
One should explicitly call mdl_remove_all_locks() and
mdl_remove_lock() to do this.
Finally mdl_try_acquire_exclusive_lock() is now returs
information about encountered lock conflict in separate
out parameter since its return value is used for distinguishing
between error (e.g. due to OOM) and success.
----------------------------------------------------------
revno: 2630.10.1
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-lock-tables-tidyup
timestamp: Wed 2008-06-11 15:49:58 +0400
message:
WL#3726, review fixes.
Now that we have metadata locks, we don't need to keep a crippled
TABLE instance in the table cache to indicate that a table is locked.
Remove all code that used this technique. Instead, rely on metadata
locks and use the standard open_table() and close_thread_table()
to manipulate with the table cache tables.
Removes a list of functions that have become unused (see the comment
for sql_base.cc for details).
Under LOCK TABLES, keep a TABLE_LIST instance for each table
that may be temporarily closed. For that, implement an own class for
LOCK TABLES mode, Locked_tables_list.
This is a pre-requisite patch for WL#4144.
This is not exactly a backport: there is no new
online ALTER table in Celosia, so the old alter table
code was changed to work with the new table cache API.
mysql-test/r/lock.result:
Update results (WL#3726 post-review patch).
mysql-test/r/trigger-compat.result:
We take the table from the table cache now, thus no warning.
mysql-test/suite/rpl/r/rpl_trigger.result:
We take the table from the table cache now, thus no warning.
mysql-test/t/lock.test:
Additional tests for LOCK TABLES mode (previously not covered by
the test suite (WL#3726).
sql/field.h:
Remove reopen_table().
sql/lock.cc:
Remove an obsolete parameter of mysql_lock_remove().
It's not used anywhere now either.
sql/mysql_priv.h:
Add 4 new open_table() flags.
Remove declarations of removed functions.
sql/sp_head.cc:
Rename thd->mdl_el_root to thd->locked_tables_root.
sql/sql_acl.cc:
Use the new implementation of unlock_locked_tables().
sql/sql_base.cc:
Implement class Locked_tables_list.
Implement close_all_tables_for_name().
Rewrite close_cached_tables() to use the new reopen_tables().
Remove reopen_table(), reopen_tables(), reopen_table_entry()
(ex. open_unireg_entry()), close_data_files_and_leave_as_placeholders(),
close_handle_and_leave_table_as_placeholder(), close_cached_table(),
table_def_change_share(), reattach_merge(), reopen_name_locked_table(),
unlink_open_table().
Move acquisition of a metadata lock into an own function
- open_table_get_mdl_lock().
sql/sql_class.cc:
Deploy class Locked_tables_list.
sql/sql_class.h:
Declare class Locked_tables_list.
Keep one instance of this class in class THD.
Rename mdl_el_root to locked_tables_root.
sql/sql_db.cc:
Update a comment.
sql/sql_insert.cc:
Use the plain open_table() to open a just created table in
CREATE TABLE .. SELECT.
sql/sql_parse.cc:
Use thd->locked_tables_list to enter and leave LTM_LOCK_TABLES mode.
sql/sql_partition.cc:
Deploy the new method of working with partitioned table locks.
sql/sql_servers.cc:
Update to the new signature of unlock_locked_tables().
sql/sql_table.cc:
In mysql_rm_table_part2(), the branch that removes a table under
LOCK TABLES, make sure that the table being dropped
is also removed from THD::locked_tables_list.
Update ALTER TABLE and CREATE TABLE LIKE implementation to use
open_table() and close_all_tables_for_name() instead of
reopen_tables().
sql/sql_trigger.cc:
Use the new locking way.
sql/table.h:
Add TABLE::pos_in_locked_tables, which is used only under
LOCK TABLES.
------------------------------------------------------------
revno: 2630.9.3
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w3
timestamp: Wed 2008-06-11 08:33:36 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
Changed close_cached_tables() not to flush all unused TABLE
instances when flushing individual table.
Renamed expel_table_from_cache() to tdc_remove_table() and
added enum parameter to be able more explicitly specify type
of removal, rewrote its code to be more efficient.
******
Backport of:
------------------------------------------------------------
revno: 2630.9.4
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w3
timestamp: Wed 2008-06-11 15:53:53 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After-review fixes in progress.
Minor changes in order to improve code readability
and simplify debugging.
mysql-test/r/ps_ddl.result:
Restore the original (correct) behaviour,
now that the patch that fixes the regression
introduced by the original patch for WL#3726 is fixed.
mysql-test/t/ps_ddl.test:
Restore the original (correct) behaviour,
now that the patch that fixes the regression
introduced by the original patch for WL#3726 is fixed
sql/mysql_priv.h:
Renamed expel_table_from_cache() to tdc_remove_table()
and added enum parameter to be able more explicitly specify
type of removal.
sql/sql_base.cc:
Changed close_cached_tables() not to flush all unused TABLE
instances when flushing individual table.
Renamed expel_table_from_cache() to tdc_remove_table() and added
enum parameter to be able more explicitly specify type of
removal, rewrote its code to be more efficient. As result removed
relink_unused() function which is no longer used.
******
Improved code in close_cached_tables() according to reviewer's
comments.
sql/sql_delete.cc:
Renamed expel_table_from_cache() to tdc_remove_table()
and added enum parameter to be able more explicitly
specify type of removal.
sql/sql_rename.cc:
Renamed expel_table_from_cache() to tdc_remove_table()
and added enum parameter to be able more explicitly
specify type of removal.
sql/sql_show.cc:
Moved acquisition of high-prio shared metadata lock in which
happens in fill_schema_table_from_frm() to separate function.
sql/sql_table.cc:
Renamed expel_table_from_cache() to tdc_remove_table()
and added enum parameter to be able more explicitly
specify type of removal.
sql/sql_trigger.cc:
Renamed expel_table_from_cache() to tdc_remove_table()
and added enum parameter to be able more explicitly
specify type of removal.
