mirror of
https://github.com/MariaDB/server.git
synced 2025-01-17 12:32:27 +01:00
328 commits
Author | SHA1 | Message | Date | |
---|---|---|---|---|
Sergey Petrunya
|
50dda403b7 |
Fix test errors like:
-Note 1031 Table storage engine for 't1' doesn't have this option +Note 1031 Table storage engine for 'InnoDB' doesn't have this option They were caused by a change in MariaDB which changed ER_ILLEGAL_HA message text to be like: "Storage engine InnoDB of the table `test`.`t1` doesn't have this option" Some the error calls were changed to pass new parameters, but some were left to be old. Also the error text in errmsg-ut8.txt was not changed. |
||
Michael Widenius
|
94d722b6a4 |
ha_partition.cc and ha_partition.h are now completely merged
Added sql_mode_t to simplify merges |
||
Michael Widenius
|
2534521f9a |
Fixed some merge issues:
- temporary tables now works - mysql-system_tables updated to not use temporary tables - PASSWORD() function fixed - Support for STATS_AUTO_RECALC, STATS_PERSISTENT and STATS_SAMPLE_PAGES table options |
||
Sergei Golubchik
|
474fe6d9d9 |
fixes for test failures
and small collateral changes mysql-test/lib/My/Test.pm: somehow with "print" we get truncated writes sometimes mysql-test/suite/perfschema/r/digest_table_full.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/perfschema/r/dml_handler.result: host table is not ported over yet mysql-test/suite/perfschema/r/information_schema.result: host table is not ported over yet mysql-test/suite/perfschema/r/nesting.result: this differs, because we don't rewrite general log queries, and multi-statement packets are logged as a one entry. this result file is identical to what mysql-5.6.5 produces with the --log-raw option. mysql-test/suite/perfschema/r/relaylog.result: MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB. mysql-test/suite/perfschema/r/server_init.result: MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup. mysql-test/suite/perfschema/r/stage_mdl_global.result: this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not do that, and this causes useless mutex locks and waits. mysql-test/suite/perfschema/r/statement_digest.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/perfschema/r/statement_digest_consumers.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/perfschema/r/statement_digest_long_query.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result: will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result: will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged |
||
unknown
|
ddd3e261b2 |
MDEV-254: Server hang with FLUSH TABLES WITH READ LOCK AND DISABLE CHECKPOINT
The code to re-enable checkpointing after UNLOCK TABLES was lost in the 5.5 merge, so re-add it back in. |
||
Michael Widenius
|
a579adea66 |
Cleanups:
- Don't use SAFEMALLOC on valgrind builds (slows things down) - Added back lost option from 5.3: debug-mutex-deadlock-detector - Flush pages before taking lock mutex (speeds up closing of Aria tables). BUILD/SETUP.sh: - Don't use SAFEMALLOC on valgrind builds (slows things down) sql/lock.cc: Make default argument explicit (improves readability) sql/mysqld.cc: Removed compiler warnings Sorted debug options alphabetically Added back lost option from 5.3: debug-mutex-deadlock-detector storage/maria/ma_close.c: Flush pages before taking lock mutex (speeds up closing of Aria tables). storage/maria/ma_open.c: More DBUG_PRINT storage/maria/maria_def.h: Better DBUG_PRINT storage/maria/trnman.c: Better DBUG_PRINT |
||
Sergei Golubchik
|
4f435bddfd | 5.3 merge | ||
Michael Widenius
|
24e452a208 | Merge with 5.1 & fixes to IGNORE handling | ||
Michael Widenius
|
6d4224a31c |
Merge with 5.2.
no_error handling for select (used by INSERT ... SELECT) still needs to be fixed, but I will do that in a separate commit |
||
Sergey Petrunya
|
8e25dcfcd7 |
BUG#868908: Crash in check_simple_equality() with semijoin + materialization + prepared statement
- Part 1 of the fix: for semi-join merged subqueries, calling child_join->optimize() until we're done with all PS-lifetime optimizations in the parent. |
||
unknown
|
5e7b949e61 | Fix crash due to wrong my_error() call (5.5 after-merge fix). | ||
Michael Widenius
|
6920457142 | Merge with MariaDB 5.1 | ||
Michael Widenius
|
a8d03ab235 |
Initail merge with MySQL 5.1 (XtraDB still needs to be merged)
Fixed up copyright messages. |
||
Sergei Golubchik
|
0e007344ea | mysql-5.5.18 merge | ||
Sergei Golubchik
|
76f0b94bb0 |
merge with 5.3
sql/sql_insert.cc: CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. ****** CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. sql/sql_table.cc: small cleanup ****** small cleanup |
||
Sergei Golubchik
|
9809f05199 | 5.5-merge | ||
Kent Boortz
|
68f00a5686 | Updated/added copyright headers | ||
Kent Boortz
|
44135d4725 | Updated/added copyright headers | ||
Kent Boortz
|
02e07e3b51 | Updated/added copyright headers | ||
Michael Widenius
|
6da8ac5f71 |
Added option "AND DISABLE CHECKPOINT" to "FLUSH TABLES WITH READ LOCK"
This makes it possible to do safe multi volume snapshots as long as one snapshots the volume with the transaction logs last. include/mysql_com.h: Added REFRESH_CHECKPOINT mysql-test/r/flush.result: Added test of new FLUSH TABLES syntax + calls to checkpoint_status handler calls mysql-test/t/flush.test: Added test of new FLUSH TABLES syntax + calls to checkpoint_status handler calls sql/handler.cc: Added code to call checkpoint_state for all handlertons that supports it sql/handler.h: Added new checkpoint_state() handlerton call to temporarly disable checkpoints. sql/lex.h: Added CHECKPOINT keyword sql/sql_yacc.yy: Added support for FLUSH TABLES WITH READ LOCK AND DISABLE CHECKPOINT storage/maria/ha_maria.cc: Added handlerton call to disable checkpoints. storage/maria/ma_checkpoint.c: Don't do checkpoint if checkpoints are disabled. storage/maria/ma_static.c: Added maria_checkpoint_disabled storage/maria/maria_def.h: Added maria_checkpoint_disabled storage/xtradb/handler/ha_innodb.cc: Added handlerton call to disable checkpoints. storage/xtradb/include/log0log.h: Added option to log_checkpoint() to allow one to ignore not critical checkpoints during the time checkpoints are disabled. storage/xtradb/log/log0log.c: Added code to allow one to disable checkpoints during a FLUSH TABLES ... DISABLE CHECKPOINT This was done by adding a new argument to log_checkpoint() which tells us when the checkpoint is called by srv_master_thread (which are safe to ignore) storage/xtradb/srv/srv0srv.c: Tell log_checkpoint() that checkpoints from srv_master_thread() are safe to ignore (will just delay recovery time a bit). |
||
Sergei Golubchik
|
0accbd0364 | lots of post-merge changes | ||
Michael Widenius
|
139a2b64bf | Merge with 5.2 | ||
Michael Widenius
|
3358cdd504 | Merge with 5.1 to get in changes from MySQL 5.1.55 | ||
Michael Widenius
|
869f5d0e81 | Merge with alias as String | ||
Michael Widenius
|
ff3da0f963 |
Change TABLE->alias to String for less memory reallocation
Changed some String.ptr() -> String.c_ptr() for String that are not guaranteed to end with \0 Removed some c_ptr() usage from parameters to functions that takes ptr & length Use preallocate buffers to avoid calling malloc() for most operations. sql/event_db_repository.cc: alias is now a String sql/event_scheduler.cc: c_ptr -> c_ptr_safe() to avoid warnings from valgrind. sql/events.cc: c_ptr -> c_ptr_safe() to avoid warnings from valgrind. c_ptr -> ptr() as function takes ptr & length sql/field.cc: alias is now a String sql/field.h: alias is now a String sql/ha_partition.cc: alias is now a String sql/handler.cc: alias is now a String ptr() -> c_ptr() as string is not guaranteed to be \0 terminated sql/item.cc: Store error parameter in separarte buffer to ensure correct error message sql/item_func.cc: ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated sql/item_sum.h: Use my_strtod() instead of my_atof() to not have to make string \0 terminated sql/lock.cc: alias is now a String sql/log.cc: c_ptr() -> ptr() as function takes ptr & length sql/log_event.cc: c_ptr_quick() -> ptr() as we only want to get the pointer to String buffer sql/opt_range.cc: ptr() -> c_ptr() as string is not guaranteed to be \0 terminated sql/opt_table_elimination.cc: alias is now a String sql/set_var.cc: ptr() -> c_ptr() as string is not guaranteed to be \0 terminated c_ptr() -> c_ptr_safe() to avoid warnings from valgrind. c_ptr() -> ptr() as function takes ptr & length Simplify some code. sql/sp.cc: c_ptr() -> ptr() as function takes ptr & length sql/sp_rcontext.cc: alias is now a String sql/sql_base.cc: alias is now a String. Here we win a realloc() for most alias usage. sql/sql_class.cc: Use size descriptor for printf() to avoid accessing bytes outside of buffer sql/sql_insert.cc: Change allocation of TABLE as it's now contains a String _ptr() -> ptr() as function takes ptr & length sql/sql_load.cc: Use preallocate buffers to avoid calling malloc() for most operations. sql/sql_parse.cc: Use c_ptr_safe() to ensure string is \0 terminated. sql/sql_plugin.cc: c_ptr_quick() -> ptr() as function takes ptr & length sql/sql_select.cc: alias is now a String sql/sql_show.cc: alias is now a String sql/sql_string.h: Added move() function to change who owns the string (owner does the free) sql/sql_table.cc: alias is now a String c_ptr() -> c_ptr_safe() to avoid warnings from valgrind. sql/sql_test.cc: c_ptr() -> c_ptr_safe() to avoid warnings from valgrind. alias is now a String sql/sql_trigger.cc: c_ptr() -> c_ptr_safe() to avoid warnings from valgrind. Use field->init() to setup pointers to alias. sql/sql_update.cc: alias is now a String sql/sql_view.cc: ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated sql/sql_yacc.yy: r() -> c_ptr() as string is not guaranteed to be \0 terminated sql/table.cc: alias is now a String sql/table.h: alias is now a String storage/federatedx/ha_federatedx.cc: Remove extra 1 byte alloc that is automaticly done by strmake() Ensure that error message ends with \0 storage/maria/ha_maria.cc: alias is now a String storage/myisam/ha_myisam.cc: alias is now a String |
||
Michael Widenius
|
69fe020f01 |
Fixed bugs found by buildbot:
- Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables. - Fixed compiler warnings - Added a name for each thr_lock to get better error messages (This is needed to find out why 'archive.test' sometimes fails) BUILD/SETUP.sh: Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables. BUILD/build_mccge.sh: Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables. client/mysqltest.cc: Fixed bug in remove_files_wildcards (the orignal code never removed anything) extra/libevent/devpoll.c: Fixed compiler warning include/thr_lock.h: Added a name for each thr_lock to get better error messages. mysql-test/suite/maria/t/maria3.test: Speed up test. mysys/thr_lock.c: Added a name for each thr_lock to get better error messages. Added a second 'check_locks' to find if something goes wrong in 'wake_up_waiters'. sql/lock.cc: Added a name for each thr_lock to get better error messages. storage/xtradb/fil/fil0fil.c: Fixed compiler warning |
||
Michael Widenius
|
050c004f5e |
Merge with 5.1
Fixes to get Aria handler tests to work. Fixes LP#697597 "HANDLER + Aria asserts in maria-5.3-handler" |
||
Michael Widenius
|
e63b5546c5 |
Implementation of MWL#172: Add support for prepared statements to HANDLER READ
It includes speed optimizations for HANDLER READ by caching as much as possible in HANDLER OPEN Other things: - Added mysqld option --disable-thr-alarm to be able to benchmark things without thr_alarm - Changed 'Locked' state to 'System lock' and 'Table lock' (these where used in the code but never shown to end user) - Better error message if mysql_install_db.sh fails - Moved handler function prototypes to sql_handler.h - Remove not anymore used 'thd->locked' member include/thr_alarm.h: Added my_disable_thr_alarm include/thr_lock.h: Add new member to THR_LOCK_DATA to remember original lock type state. This is needed as thr_unlock() resets type to TL_UNLOCK. mysql-test/include/check_no_concurrent_insert.inc: Locked -> Table lock mysql-test/include/handler.inc: Locked -> Table lock mysql-test/r/handler_innodb.result: Updated results for new tests mysql-test/r/handler_myisam.result: Updated results for new tests mysql-test/r/sp-threads.result: Locked -> Table lock mysql-test/suite/binlog/t/binlog_stm_row.test: Locked -> Table lock mysql-test/suite/funcs_1/datadict/processlist_val.inc: Locked -> Table lock mysql-test/suite/pbxt/t/lock_multi.test: Locked -> Table lock mysql-test/suite/sys_vars/r/concurrent_insert_func.result: Locked -> Table lock mysql-test/suite/sys_vars/t/concurrent_insert_func.test: Locked -> Table lock mysql-test/suite/sys_vars/t/delayed_insert_limit_func.test: Locked -> Table lock mysql-test/suite/sys_vars/t/query_cache_wlock_invalidate_func.test: Locked -> Table lock mysql-test/suite/sys_vars/t/sql_low_priority_updates_func.test: Locked -> Table lock mysql-test/t/insert_notembedded.test: Locked -> Table lock mysql-test/t/lock_multi.test: Locked -> Table lock mysql-test/t/merge-big.test: Locked -> Table lock mysql-test/t/multi_update.test: Locked -> Table lock mysql-test/t/query_cache_28249.test: Locked -> Table lock mysql-test/t/sp_notembedded.test: Locked -> Table lock mysql-test/t/sp_sync.test: Locked -> Table lock mysql-test/t/status.test: Locked -> Table lock mysql-test/t/trigger_notembedded.test: Locked -> Table lock mysys/thr_alarm.c: Added option to disable thr_alarm mysys/thr_lock.c: Detect loops scripts/mysql_install_db.sh: Give better error message if something goes wrong sql/Makefile.am: Added sql_handler.h sql/lock.cc: Split functions to allow one to cache value if store_lock() (for HANDLER functions). - Split mysql_lock_tables() into two functions, where first one allocates MYSQL_LOCK and other other one uses it. - Made get_lock_data() an external function. - Added argument to mysql_unlock_tables() to not free sql_lock. - Added argument to reset_lock_data() to reset lock structure to initial state (as after get_lock_data()) sql/mysql_priv.h: Moved handler function prototypes to sql_handler.h Added new lock functions. sql/mysqld.cc: Added --thread-alarm startup option sql/net_serv.cc: Don't call vio_blocking() if not needed sql/sql_base.cc: include sql_handler.h sql/sql_class.cc: include sql_handler.h Remove not anymore used 'thd->locked' member sql/sql_class.h: Remove not anymore used 'thd->locked' member sql/sql_db.cc: include sql_handler.h sql/sql_delete.cc: include sql_handler.h sql/sql_handler.cc: Rewrote all code to use SQL_HANDLER instead of TABLE_LIST (original interface) Rewrote mysql_ha_open() to cache all things from TABLE_LIST and items for field list, where etc. In mysql_ha_open() also cache MYSQL_LOCK structure from get_lock_data(). Split functions into smaller sub functions (needed to be able to implement mysql_ha_read_prepare()) Added mysql_ha_read_prepare() to allow one to prepare HANDLER READ. sql/sql_handler.h: Interface to sql_handler.cc sql/sql_parse.cc: include sql_handler.h sql/sql_prepare.cc: Added mysql_test_handler_read(), prepare for HANDLER READ sql/sql_rename.cc: include sql_handler.h sql/sql_show.cc: Removed usage of thd->locked sql/sql_table.cc: include sql_handler.h sql/sql_trigger.cc: include sql_handler.h |
||
Michael Widenius
|
b7b25dc666 |
Merge with 5.1-release.
- Fixed problem with oqgraph and 'make dist' Note that after this merge we have a problem show in join_outer where we examine too many rows in one specific case (related to BUG#57024). This will be fixed when mwl#128 is merged into 5.3. |
||
Sergei Golubchik
|
65ca700def |
merge.
checkpoint. does not compile. |
||
Michael Widenius
|
1a6373e8e2 |
Merge with MySQL 5.1.53
Open issues: - A better fix for #57688; Igor is working on this - Test failure in index_merge_innodb.test ; Igor promised to look at this - Some Innodb tests fails (need to merge with latest xtradb) ; Kristian promised to look at this. - Failing tests: innodb_plugin.innodb_bug56143 innodb_plugin.innodb_bug56632 innodb_plugin.innodb_bug56680 innodb_plugin.innodb_bug57255 - Werror is disabled; Should be enabled after merge with xtradb. |
||
Michael Widenius
|
52090a4434 |
Code cleanup to get fewer reallocs() during execution.
- Changed TABLE->alias to String to get fewer reallocs when alias are used. - Preallocate some buffers Changed some String->c_ptr() -> String->ptr() when \0 is not needed. Fixed wrong usage of String->ptr() when we need a \0 terminated string. Use my_strtod() instead of my_atof() to avoid having to add \0 to string. c_ptr() -> c_ptr_safe() to avoid warnings from valgrind. zr sql/event_db_repository.cc: Update usage of TABLE->alias sql/event_scheduler.cc: c_ptr() -> c_ptr_safe() sql/events.cc: c_ptr() -> ptr() as \0 was not needed sql/field.cc: Update usage of TABLE->alias sql/field.h: Update usage of TABLE->alias sql/ha_partition.cc: Update usage of TABLE->alias sql/handler.cc: Update usage of TABLE->alias Fixed wrong usage of str.ptr() sql/item.cc: Fixed error where code wrongly assumed string was \0 terminated. sql/item_func.cc: c_ptr() -> c_ptr_safe() Update usage of TABLE->alias sql/item_sum.h: Use my_strtod() instead of my_atof() to avoid having to add \0 to string sql/lock.cc: Update usage of TABLE->alias sql/log.cc: c_ptr() -> ptr() as \0 was not needed sql/log_event.cc: c_ptr_quick() -> ptr() as \0 was not needed sql/opt_range.cc: ptr() -> c_ptr() as \0 is needed sql/opt_subselect.cc: Update usage of TABLE->alias sql/opt_table_elimination.cc: Update usage of TABLE->alias sql/set_var.cc: ptr() -> c_ptr() as \0 is needed c_ptr() -> c_ptr_safe() sql/sp.cc: c_ptr() -> ptr() as \0 was not needed sql/sp_rcontext.cc: Update usage of TABLE->alias sql/sql_base.cc: Preallocate buffers Update usage of TABLE->alias sql/sql_class.cc: Fix arguments to sprintf() to work even if string is not \0 terminated sql/sql_insert.cc: Update usage of TABLE->alias c_ptr() -> ptr() as \0 was not needed sql/sql_load.cc: Preallocate buffers Trivial optimizations sql/sql_parse.cc: Trivial optimization sql/sql_plugin.cc: c_ptr() -> ptr() as \0 was not needed sql/sql_select.cc: Update usage of TABLE->alias sql/sql_show.cc: Update usage of TABLE->alias sql/sql_string.h: Added move() function to move allocated memory from one object to another. sql/sql_table.cc: Update usage of TABLE->alias c_ptr() -> c_ptr_safe() sql/sql_test.cc: ptr() -> c_ptr_safe() sql/sql_trigger.cc: Update usage of TABLE->alias c_ptr() -> c_ptr_safe() sql/sql_update.cc: Update usage of TABLE->alias sql/sql_view.cc: ptr() -> c_ptr_safe() sql/sql_yacc.yy: ptr() -> c_ptr() sql/table.cc: Update usage of TABLE->alias sql/table.h: Changed TABLE->alias to String to get fewer reallocs when alias are used. storage/federatedx/ha_federatedx.cc: Use c_ptr_safe() to ensure strings are \0 terminated. storage/maria/ha_maria.cc: Update usage of TABLE->alias storage/myisam/ha_myisam.cc: Update usage of TABLE->alias storage/xtradb/row/row0sel.c: Ensure that null bits in record are properly reset. (Old code didn't work as row_search_for_mysql() can be called twice while reading fields from one row. |
||
Dmitry Lenev
|
6bf6272fda |
Patch that refactors global read lock implementation and fixes
bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations. mysql-test/include/check_ftwrl_compatible.inc: Added helper script which allows to check that a statement is compatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/check_ftwrl_incompatible.inc: Added helper script which allows to check that a statement is incompatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/handler.inc: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/include/wait_show_condition.inc: Fixed small error in the timeout message. The correct name of variable used as parameter for this script is "$condition" and not "$wait_condition". mysql-test/r/delayed.result: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/r/events_2.result: Updated test results after prohibiting event DDL operations under LOCK TABLES. mysql-test/r/flush.result: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/r/flush_read_lock.result: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/r/flush_read_lock_kill.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/r/handler_innodb.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/handler_myisam.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/mdl_sync.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Updated test results after removing global COND_global_read_lock condition variable. mysql-test/suite/perfschema/r/func_file_io.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/func_mutex.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/global_read_lock.result: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/r/server_init.result: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/perfschema/t/func_file_io.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/func_mutex.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/global_read_lock.test: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/t/server_init.test: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result: Updated test results after prohibiting event DDL under LOCK TABLES. mysql-test/t/delayed.test: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/t/events_2.test: Updated test case after prohibiting event DDL operations under LOCK TABLES. mysql-test/t/flush.test: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/t/flush_block_commit.test: Adjusted test case after changing thread state name which is used when COMMIT waits for FLUSH TABLES WITH READ LOCK from "Waiting for release of readlock" to "Waiting for commit lock". mysql-test/t/flush_block_commit_notembedded.test: Adjusted test case after changing thread state name which is used when DML waits for FLUSH TABLES WITH READ LOCK. Now we use "Waiting for global read lock" in this case. mysql-test/t/flush_read_lock.test: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/t/flush_read_lock_kill-master.opt: We no longer need to use make_global_read_lock_block_commit_loop debug tag in this test. Instead we rely on an appropriate debug_sync point in MDL code. mysql-test/t/flush_read_lock_kill.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/t/lock_multi.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". mysql-test/t/mdl_sync.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. Updated thread state names which are used when DDL waits for FTWRL. mysql-test/t/trigger_notembedded.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". sql/event_data_objects.cc: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_data_objects.h: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_db_repository.cc: - Changed Event_db_repository methods to not release all metadata locks once they are done updating mysql.events table. This allows to keep metadata lock protecting against GRL and lock protecting particular event around until corresponding DDL statement is written to the binary log. - Removed logic for conditional update of "status" and "last_executed" fields from update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" is too much hassle. sql/event_db_repository.h: Removed logic for conditional update of "status" and "last_executed" fields from Event_db_repository:: update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" field is too much hassle. sql/event_queue.cc: Changed event scheduler code not to update mysql.events table while holding Event_queue::LOCK_event_queue mutex. Doing so led to a deadlock with a new GRL implementation. This deadlock didn't occur with old implementation due to fact that code acquiring protection against GRL ignored pending GRL requests (which lead to GRL starvation). One of goals of new implementation is to disallow GRL starvation and so we have to solve problem with this deadlock in a different way. sql/events.cc: Changed methods of Events class to acquire protection against GRL while perfoming DDL statement and keep it until statement is written to the binary log. Unfortunately this step together with new GRL implementation exposed deadlock involving Events::LOCK_event_metadata and GRL. To solve it Events::LOCK_event_metadata mutex was replaced with a metadata lock on event. As a side-effect events DDL has to be prohibited under LOCK TABLES even in cases when mysql.events table was explicitly locked for write. sql/events.h: Replaced Events::LOCK_event_metadata mutex with a metadata lock on event. sql/ha_ndbcluster.cc: Updated code after replacing custom global read lock implementation with one based on MDL. Since MDL subsystem should now be able to detect deadlocks involving metadata locks and GRL there is no need for special handling of active GRL. sql/handler.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. sql/lock.cc: Replaced custom implementation of global read lock with one based on metadata locks. This step allows to expose wait for GRL to deadlock detector of MDL subsystem and thus succesfully resolve deadlocks similar to one behind bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". It also solves problem with GRL starvation described in bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'" since metadata locks used by GRL give preference to FTWRL statement instead of DML statements (if needed in future this can be changed to fair scheduling). Similar to old implementation of acquisition of GRL is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. To support this change: - Global_read_lock::lock/unlock_global_read_lock and make_global_read_lock_block_commit methods were changed accordingly. - Global_read_lock::wait_if_global_read_lock() and start_waiting_global_read_lock() methods were dropped. It is now responsibility of code acquiring metadata locks opening tables to acquire protection against GRL by explicitly taking global IX lock with statement duration. - Global variables, mutex and condition variable used by old implementation was removed. - lock_routine_name() was changed to use statement duration for its global IX lock. It was also renamed to lock_object_name() as it now also used to take metadata locks on events. - Global_read_lock::set_explicit_lock_duration() was added which allows not to release locks used for GRL when leaving prelocked mode. sql/lock.h: - Renamed lock_routine_name() to lock_object_name() and changed its signature to allow its usage for events. - Removed broadcast_refresh() function. It is no longer needed with new GRL implementation. sql/log_event.cc: Release metadata locks with statement duration at the end of processing legacy event for LOAD DATA. This ensures that replication thread processing such event properly releases its protection against global read lock. sql/mdl.cc: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Changed thread state name for GLOBAL namespace to "Waiting for global read lock". Optimized MDL_map::find_or_insert() method to avoid taking m_mutex mutex when looking up MDL_lock objects for GLOBAL or COMMIT namespaces. We keep pre-created MDL_lock objects for these namespaces around and simply return pointers to these global objects when needed. Changed MDL_lock/MDL_scoped_lock to properly handle notification of insert delayed handler threads when FTWRL takes global S lock. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mdl.h: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mysqld.cc: Removed global mutex and condition variables which were used by old implementation of GRL. Also we no longer need to initialize Events::LOCK_event_metadata mutex as it was replaced with metadata locks on events. sql/mysqld.h: Removed global variable, mutex and condition variables which were used by old implementation of GRL. sql/rpl_rli.cc: When slave thread closes tables which were open for handling of RBR events ensure that it releases global IX lock which was acquired as protection against GRL. sql/sp.cc: Adjusted code to the new signature of lock_object/routine_name(), to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sp_head.cc: Ensure that statements in stored procedures release statement metadata locks and thus release their protectiong against GRL in proper moment in time. Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_admin.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_base.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. Code doing this also responsible for checking that current connection has no active GRL by calling an Global_read_lock::can_acquire_protection() method. Changed code in open_table() and lock_table_names() accordingly. Note that as result of this change DDL and DML on temporary tables is always compatible with GRL (before it was incompatible in some cases and compatible in other cases). - To speed-up code acquiring protection against GRL introduced m_has_protection_against_grl member in Open_table_context class. It indicates that protection was already acquired sometime during open_tables() execution and new attempts can be skipped. - Thanks to new GRL implementation calls to broadcast_refresh() became unnecessary and were removed. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_base.h: Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. Also introduced Open_table_context::m_has_protection_against_grl member which allows to avoid acquiring protection against GRL while opening tables if such protection was already acquired. sql/sql_class.cc: Changed THD::leave_locked_tables_mode() after transactional sentinel for metadata locks was obsoleted by introduction of locks with explicit duration. sql/sql_class.h: - Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. - Changed Global_read_lock class according to changes in global read lock implementation: * wait_if_global_read_lock and start_waiting_global_read_lock are now gone. Instead code needing protection against GRL has to acquire global IX metadata lock with statement duration itself. To help it new can_acquire_protection() was introduced. Also as result of the above change m_protection_count member is gone too. * Added m_mdl_blocks_commits_lock member to store metadata lock blocking commits. * Adjusted code to the fact that concept of transactional sentinel was obsoleted by concept of lock duration. - Removed CF_PROTECT_AGAINST_GRL flag as it is no longer necessary. New GRL implementation acquires protection against global read lock automagically when statement acquires metadata locks on tables or other objects it is going to change. sql/sql_db.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_handler.cc: Removed call to broadcast_refresh() function. It is no longer needed with new GRL implementation. Adjusted code after introducing duration concept for metadata locks. Particularly to the fact transactional sentinel was replaced with explicit duration. sql/sql_handler.h: Renamed mysql_ha_move_tickets_after_trans_sentinel() to mysql_ha_set_explicit_lock_duration() after transactional sentinel was obsoleted by locks with explicit duration. sql/sql_insert.cc: Adjusted code handling delaying inserts after switching to new GRL implementation. Now connection thread initiating delayed insert has to acquire global IX lock in addition to metadata lock on table being inserted into. This IX lock protects against GRL and similarly to SW lock on table being inserted into has to be passed to handler thread in order to avoid deadlocks. sql/sql_lex.cc: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_lex.h: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_parse.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. This lock is automatically released at the end of statement execution. - Changed implementation of CREATE/DROP PROCEDURE/FUNCTION not to release metadata locks and thus protection against of GRL in the middle of statement execution. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_prepare.cc: Adjusted code to the to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_rename.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before renaming tables. This happens automatically in code which acquires metadata locks on tables being renamed. sql/sql_show.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_table.cc: - With new GRL implementation there is no need to explicitly acquire protection against GRL before dropping tables. This happens automatically in code which acquires metadata locks on tables being dropped. - Changed mysql_alter_table() not to release lock on new table name explicitly and to rely on automatic release of locks at the end of statement instead. This was necessary since now MDL_context::release_lock() is supported only for locks for explicit duration. sql/sql_trigger.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before changing table triggers. This happens automatically in code which acquires metadata locks on tables which triggers are to be changed. sql/sql_update.cc: Fix bug exposed by GRL testing. During prepare phase acquire only S metadata locks instead of SW locks to keep prepare of multi-UPDATE compatible with concurrent LOCK TABLES WRITE and global read lock. sql/sql_view.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before creating view. This happens automatically in code which acquires metadata lock on view to be created. sql/sql_yacc.yy: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/table.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/table.h: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/transaction.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. Also adjusted code to the fact that MDL savepoint is now represented by MDL_savepoint class. |
||
Michael Widenius
|
20acfbf30d |
Fix for: LP #634955: Assert in _ma_update_at_original_place()
Added locking of lock mutex when updating status in external_unlock() for Aria and MyISAM tables. Fixed that 'source' command doesn't cause mysql command line tool to exit on error. DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards. (Allows one to run --debug with mysql-test-run scripts that uses @debug) Fixed several core dump, deadlock and crashed table bugs in handling of LOCK TABLE with MERGE tables: - Added priority of locks to avoid crashes with MERGE tables. - Added thr_lock_merge() to allow one to merge two results of thr_lock(). Fixed 'not found row' bug in REPLACE with Aria tables. Mark MyISAM tables that are part of MERGE with HA_OPEN_MERGE_TABLE and set the locks to have priority THR_LOCK_MERGE_PRIV. - By sorting MERGE tables last in thr_multi_unlock() it's safer to release and relock them many times (can happen when TRIGGERS are created) Avoid printing (null) in debug file (to easier find out wrong NULL pointer usage with %s). client/mysql.cc: Fixed that 'source' command doesn't cause mysql command line tool to exit on error. client/mysqltest.cc: Don't send NULL to fn_format(). (Can cause crash on Solaris when using --debug) dbug/dbug.c: DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards. include/my_base.h: Added flag to signal if one opens a MERGE table. Added extra() command to signal that one is not part of a MERGE table anymore. include/thr_lock.h: Added priority for locks (needed to fix bug in thr_lock when using MERGE tables) Added option to thr_unlock() if get_status() should be called. Added prototype for thr_merge_locks(). mysql-test/mysql-test-run.pl: Ignore crashed table warnings for tables named 'crashed'. mysql-test/r/merge.result: Renamed triggers to make debugging easier. Added some CHECK TABLES to catch errors earlier. Additional tests. mysql-test/r/merge_debug.result: Test of error handling when reopening MERGE tables. mysql-test/r/udf_query_cache.result: Added missing flush status mysql-test/suite/parts/r/partition_repair_myisam.result: Update results mysql-test/t/merge.test: Renamed triggers to make debugging easier. Added some CHECK TABLES to catch errors earlier. Additional tests. mysql-test/t/merge_debug.test: Test of error handling when reopening MERGE tables. mysql-test/t/udf_query_cache.test: Added missing flush status mysys/my_getopt.c: Removed not used variable mysys/my_symlink2.c: Changed (null) to (NULL) to make it easier to find NULL arguments to DBUG_PRINT() functions. (On linux, NULL to sprintf is printed 'null') mysys/thr_lock.c: Added priority of locks to avoid crashes with MERGE tables. Added thr_lock_merge() to allow one to merge two results of thr_lock(). - This is needed for MyISAM as all locked table must share the same status. If not, you will not see newly inserted rows in other instances of the table. If calling thr_unlock() with THR_UNLOCK_UPDATE_STATUS, call update_status() and restore_status() for the locks. This is needed in some rare cases where we call thr_unlock() followed by thr_lock() without calling external_unlock/external_lock in between. Simplify loop in thr_multi_lock(). Added 'start_trans', which is called at end of thr_multi_lock() when all locks are taken. - This was needed by Aria to ensure that transaction is started when we got all locks, not at get_status(). Without this, some rows could not be visible when we lock two tables at the same time, causing REPLACE using two tables to fail unexpectedly. sql/handler.cc: Add an assert() in handler::print_error() for "impossible errors" (like table is crashed) when --debug-assert-if-crashed-table is used. sql/lock.cc: Simplify mysql_lock_tables() code if get_lock_data() returns 0 locks. Added new parameter to thr_multi_unlock() In mysql_unlock_read_tables(), call first externa_unlock(), then thr_multi_unlock(); This is same order as we do in mysql_unlock_tables(). Don't abort locks in mysql_lock_abort() for merged tables when a MERGE table is deleted; Would cause a spin lock. Added call to thr_merge_locks() in mysql_lock_merge() to ensure consistency in thr_locks(). - New locks of same type and table is stored after the old lock to ensure that we get the status from the original lock. sql/mysql_priv.h: Added debug_assert_if_crashed_table sql/mysqld.cc: Added --debug-assert-if-crashed-table sql/parse_file.cc: Don't print '(null)' in DBUG_PRINT of no dir given sql/set_var.cc: Increase default size of buffer for @debug variable. sql/sql_base.cc: In case of error from reopen_table() in reopen_tables(), call unlock_open_table() and restart loop. - This fixed bug when we twice deleted same table from open_cache. Don't take name lock for already name locked table in open_unireg_entry(). - Fixed bug when doing repair in reopen_table(). - In detach_merge_children(), always detach if 'clear_refs' is given. We can't trust parent->children_attached as this function can be called twice, first time with clear_refs set to 0. sql/sql_class.cc: Changed printing of (null) to "" in set_thd_proc_info() sql/sql_parse.cc: Added DBUG sql/sql_trigger.cc: Don't call unlink_open_table() if reopen_table() fails as the table may already be freed. storage/maria/ma_bitmap.c: Fixed DBUG_ASSERT() in allocate_tail() storage/maria/ma_blockrec.c: Fixed wrong calculation of row length for very small rows in undo_row_update(). - Fixes ASSERT() when doing undo. storage/maria/ma_blockrec.h: Added _ma_block_start_trans() and _ma_block_start_trans_no_versioning() storage/maria/ma_locking.c: Call _ma_update_status_with_lock() when releasing write locks. - Fixes potential problem with updating status without the proper lock. storage/maria/ma_open.c: Changed to use start_trans() instead of get_status() to ensure that we see all rows in all locked tables when we got the locks. - Fixed 'not found row' bug in REPLACE with Aria tables. storage/maria/ma_state.c: Added _ma_update_status_with_lock() and _ma_block_start_trans(). This is to ensure that we see all rows in all locked tables when we got the locks. storage/maria/ma_state.h: Added _ma_update_status_with_lock() storage/maria/ma_write.c: More DBUG_PRINT storage/myisam/mi_check.c: Fixed error message storage/myisam/mi_extra.c: Added HA_EXTRA_DETACH_CHILD: - Detach MyISAM table to not be part of MERGE table (remove flag & lock priority). storage/myisam/mi_locking.c: Call mi_update_status_with_lock() when releasing write locks. - Fixes potential problem with updating status without the proper lock. Change to use new HA_OPEN_MERGE_TABLE flag to test if MERGE table. Added mi_fix_status(), called by thr_merge(). storage/myisam/mi_open.c: Added marker if part of MERGE table. Call mi_fix_status() in thr_lock() for transactional tables. storage/myisam/myisamdef.h: Change my_once_flag to uint, as it stored different values than just 0/1 Added 'open_flag' to store state given to mi_open() storage/myisammrg/ha_myisammrg.cc: Add THR_LOCK_MERGE_PRIV to THR_LOCK_DATA to get MERGE locks sorted after other types of locks. storage/myisammrg/myrg_locking.c: Remove windows specific code. storage/myisammrg/myrg_open.c: Use HA_OPEN_MERGE_TABLE to mi_open(). Set HA_OPEN_MERGE_TABLE for linked MyISAM tables. storage/xtradb/buf/buf0buf.c: Fixed compiler warning storage/xtradb/buf/buf0lru.c: Initialize variable that could be used not initialized. |
||
Davi Arnaut
|
2881b8014c |
Bug#37780: Make KILL reliable (main.kill fails randomly)
- A prerequisite cleanup patch for making KILL reliable. The test case main.kill did not work reliably. The following problems have been identified: 1. A kill signal could go lost if it came in, short before a thread went reading on the client connection. 2. A kill signal could go lost if it came in, short before a thread went waiting on a condition variable. These problems have been solved as follows. Please see also added code comments for more details. 1. There is no safe way to detect, when a thread enters the blocking state of a read(2) or recv(2) system call, where it can be interrupted by a signal. Hence it is not possible to wait for the right moment to send a kill signal. It has been decided, not to fix it in the code. Instead, the test case repeats the KILL statement until the connection terminates. 2. Before waiting on a condition variable, we register it together with a synchronizating mutex in THD::mysys_var. After this, we need to test THD::killed again. At some places we did only test it in a loop condition before the registration. When THD::killed had been set between this test and the registration, we entered waiting without noticing the killed flag. Additional checks ahve been introduced where required. In addition to the above, a re-write of the main.kill test case has been done. All sleeps have been replaced by Debug Sync Facility synchronization. A couple of sync points have been added to the server code. To avoid further problems, if the test case fails in spite of the fixes, the test case has been added to the "experimental" list for now. - Most of the work on this patch is authored by Ingo Struewing mysql-test/t/kill.test: Re-wrote test case to use Debug Sync points instead of sleeps sql/event_queue.cc: Fixed kill detection in Event_queue::cond_wait() by adding a check after enter_cond(). sql/lock.cc: Moved Debug Sync points behind enter_cond(). Fixed comments. sql/slave.cc: Fixed kill detection in start_slave_thread() by adding a check after enter_cond(). sql/sql_class.cc: Swapped order of kill and close in THD::awake(). Added comments. sql/sql_class.h: Added a comment to THD::killed. sql/sql_parse.cc: Added a sync point in do_command(). sql/sql_select.cc: Added a sync point in JOIN::optimize(). |
||
Davi Arnaut
|
3e9c52250a |
Bug#45288: pb2 returns a lot of compilation warnings
Fix assorted warnings that are generated in optimized builds. Most of it is silencing variables that are set but unused. This patch also introduces the MY_ASSERT_UNREACHABLE macro which helps the compiler to deduce that a certain piece of code is unreachable. include/my_compiler.h: Use GCC's __builtin_unreachable if available. It allows GCC to deduce the unreachability of certain code paths, thus avoiding warnings that, for example, accused that a variable could be used without being initialized (due to unreachable code paths). |
||
Konstantin Osipov
|
29c4873ad5 |
Commit on behalf of Dmitry Lenev.
Merge his patch for Bug#52044 into 5.5, and apply review comments. |
||
Dmitry Lenev
|
00496b7acd |
Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH
TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush. mysql-test/include/handler.inc: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/r/flush.result: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/r/mdl_sync.result: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/r/server_init.result: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/t/server_init.test: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/t/flush.test: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/t/kill.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/lock_multi.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/mdl_sync.test: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. sql/ha_ndbcluster.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/ha_ndbcluster_binlog.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/lock.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mdl.cc: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. We no longer require global IX metadata lock when acquiring SNW or SNRW locks. Such locks are needed only when metadata locks of these types are upgraded to X locks. This allows to use SNW locks in FLUSH TABLES <list> WITH READ LOCK implementation and keep the latter compatible with global read lock. sql/mdl.h: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. - Deadlock_detection_visitor now has m_table_shares_visited member which allows to support recursive locking for LOCK_open. This is required when deadlock detector inspects waiters graph which contains several edges representing waits for flushes or needs to come through the such edge more than once. sql/mysqld.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mysqld.h: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/sql_base.cc: Changed approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. Thanks to this change MDL deadlock detector can take such waits into account. To implement this/as result of this change: - tdc_wait_for_old_versions() was replaced with tdc_wait_for_old_version() which waits for individual old share to go away and which is called by open_table() after finding out that share is outdated. We don't need to perform back-off before such waiting thanks to the fact that deadlock detector now sees such waits. - As result Open_table_ctx::m_mdl_requests became unnecessary and was removed. We no longer allocate copies of MDL_request objects on MEM_ROOT when MYSQL_OPEN_FORCE_SHARED/SHARED_HIGH_PRIO flags are in effect. - close_cached_tables() and tdc_wait_for_old_version() share code which implements waiting for share to be flushed - the both use TABLE_SHARE::wait_until_flush() method. Thanks to this close_cached_tables() supports timeouts and has extra parameter for this. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. - In cases when we discover that current connection tries to open tables from different generation we now simply back-off and restart process of opening tables. To support this Open_table_context::OT_REOPEN_TABLES enum element was added. - COND_refresh condition variable became unnecessary and was removed. - mysql_notify_thread_having_shared_lock() no longer wakes up connections waiting for flush as all such connections can be waken up by deadlock detector if necessary. sql/sql_base.h: - close_cached_tables() now has one more parameter - timeout for waiting for table to be flushed. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. Added new OT_REOPEN_TABLES enum element to be used in cases when we need to restart open tables process even in the middle of transaction. - Open_table_ctx::m_mdl_requests became unnecessary and was removed. sql/sql_class.h: Added assert ensuring that we won't use LOCK_open mutex with THD::enter_cond(). Otherwise deadlocks can arise in MDL deadlock detector. sql/sql_parse.cc: Changed FLUSH TABLES <list> WITH READ LOCK to take SNW metadata locks instead of X locks on tables to be flushed. Since we no longer require global IX lock to be taken when SNW locks are taken this makes this statement compatible with FLUSH TABLES WITH READ LOCK statement. Since SNW locks allow other connections to have table opened FLUSH TABLES <list> WITH READ LOCK now has to wait during open_tables() for old version to go away. Such waits can lead to deadlocks which will be detected by MDL deadlock detector which now takes waits for table to be flushed into account. Also adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/sql_yacc.yy: FLUSH TABLES <list> WITH READ LOCK now needs only SNW metadata locks on tables. sql/sys_vars.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/table.cc: Implemented new approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. This change allows to make such waits visible inside of MDL deadlock detector. To do it: - Added list of waiters/Flush_tickets to TABLE_SHARE class. - Changed free_table_share() to postpone freeing of share memory until last waiter goes away and to wake up subscribed waiters. - Added TABLE_SHARE::wait_until_flushed() method which implements subscription to the list of waiters for table to be flushed and waiting for this event. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket). sql/table.h: In order to support new strategy of waiting for table flush (see comment for table.cc for details) added list of waiters/Flush_tickets to TABLE_SHARE class. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket). |
||
Konstantin Osipov
|
740c0d3a87 |
Implement WL#5502 Remove dead 5.0 class Sensitive_cursor.
Remove dead and unused code. Update to reflect the code review requests. include/thr_lock.h: Remove declarations for THR_LOCK_OWNER, added along with the patch for sensitive cursors. mysys/thr_lock.c: Remove support for multiple thr_lock requestors per THD. sql/lock.cc: Revert the patch that added support for sensitive cursors. sql/sp_rcontext.cc: Updated the use of mysql_open_cursor(). sql/sql_class.cc: Move the instance of Server_side_cursor from class Prepared_statement to class Statement. sql/sql_class.h: Move the isntance of Server_side_cursor from class Prepared_statement to class Statement. Remove multiple lock_ids of thr_lock. sql/sql_cursor.cc: Remove Sensitive_cursor implementation. sql/sql_cursor.h: Remove declarations for sensitive cursors. sql/sql_prepare.cc: Move the declaration of instance of Server_side_cursor from class Statement to class Prepared_statement, where it's used. sql/sql_select.cc: Remove sensitive cursor support. sql/sql_select.h: Remove sensitive cursor support. sql/sql_union.cc: Remove sensitive cursor support. |
||
Jon Olav Hauglid
|
f45523188e | merge from mysql-trunk-bugfixing | ||
Davi Arnaut
|
f56dd32bf7 |
Bug#34043: Server loops excessively in _checkchunk() when safemalloc is enabled
Essentially, the problem is that safemalloc is excruciatingly slow as it checks all allocated blocks for overrun at each memory management primitive, yielding a almost exponential slowdown for the memory management functions (malloc, realloc, free). The overrun check basically consists of verifying some bytes of a block for certain magic keys, which catches some simple forms of overrun. Another minor problem is violation of aliasing rules and that its own internal list of blocks is prone to corruption. Another issue with safemalloc is rather the maintenance cost as the tool has a significant impact on the server code. Given the magnitude of memory debuggers available nowadays, especially those that are provided with the platform malloc implementation, maintenance of a in-house and largely obsolete memory debugger becomes a burden that is not worth the effort due to its slowness and lack of support for detecting more common forms of heap corruption. Since there are third-party tools that can provide the same functionality at a lower or comparable performance cost, the solution is to simply remove safemalloc. Third-party tools can provide the same functionality at a lower or comparable performance cost. The removal of safemalloc also allows a simplification of the malloc wrappers, removing quite a bit of kludge: redefinition of my_malloc, my_free and the removal of the unused second argument of my_free. Since free() always check whether the supplied pointer is null, redudant checks are also removed. Also, this patch adds unit testing for my_malloc and moves my_realloc implementation into the same file as the other memory allocation primitives. client/mysqldump.c: Pass my_free directly as its signature is compatible with the callback type -- which wasn't the case for free_table_ent. |
||
Jon Olav Hauglid
|
9ff272fbbd |
A 5.5 version of the fix for Bug #54360 "Deadlock DROP/ALTER/CREATE
DATABASE with open HANDLER" Remove LOCK_create_db, database name locks, and use metadata locks instead. This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based deadlock detector in MDL, and paves the way for a safe, deadlock-free implementation of RENAME DATABASE. Database DDL statements will now take exclusive metadata locks on the database name, while table/view/routine DDL statements take intention exclusive locks on the database name. This prevents race conditions between database DDL and table/view/routine DDL. (e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE) By adding database name locks, this patch implements WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and WL#4985 "DDL locking: namespace/hierarchical locks". The patch also changes code to use init_one_table() where appropriate. The new lock_table_names() function requires TABLE_LIST::db_length to be set correctly, and this is taken care of by init_one_table(). This patch also adds a simple template to help work with the mysys HASH data structure. Most of the patch was written by Konstantin Osipov. |
||
Konstantin Osipov
|
52f361ecb8 |
A fix for Bug#54811 "Assert in mysql_lock_have_duplicate()".
Remove mysql_lock_have_duplicate(), since now we always have TABLE_LIST objects for MyISAMMRG children in lex->query_tables and keep it till the end of the statement (sub-statement). mysql-test/r/merge.result: Update results (Bug#54811). mysql-test/t/merge-big.test: Update to new wait state. mysql-test/t/merge.test: Add a test case for Bug#54811. sql/lock.cc: Remove a function that is now unused. sql/lock.h: Remove a function that is now unused. sql/sql_base.cc: Don't try to search for duplicate table among THR_LOCK objects, TABLE_LIST list contains all used tables. |
||
Konstantin Osipov
|
33c57e2f9e |
WL#5419 "LOCK_open scalability: make tdc_refresh_version
an atomic counter" Split the large LOCK_open section in open_table(). Do not call open_table_from_share() under LOCK_open. Remove thd->version. This fixes Bug#50589 "Server hang on a query evaluated using a temporary table" Bug#51557 "LOCK_open and kernel_mutex are not happy together" Bug#49463 "LOCK_table and innodb are not nice when handler instances are created". This patch has effect on storage engines that rely on ha_open() PSEA method being called under LOCK_open. In particular: 1) NDB is broken and left unfixed. NDB relies on LOCK_open being kept as part of ha_open(), since it uses auto-discovery. While previously the NDB open code was race-prone, now it simply fails on asserts. 2) HEAP engine had a race in ha_heap::open() when a share for the same table could be added twice to the list of shares, or a dangling reference to a share stored in HEAP handler. This patch aims to address this problem by 'pinning' the newly created share in the internal HEAP engine share list until at least one handler instance is created using that share. include/heap.h: Add members to HP_CREATE_INFO. Declare heap_release_share(). sql/lock.cc: Remove thd->version, use thd->open_tables->s->version instead. sql/repl_failsafe.cc: Remove thd->version. sql/sql_base.cc: - close_thread_table(): move handler cleanup code outside the critical section protected by LOCK_open. - remove thd->version - split the large critical section in open_table() that opens a new table from share and is protected by LOCK_open into 2 critical sections, thus reducing the critical path. - make check_if_table_exists() acquire LOCK_open internally. - use thd->open_tables->s->version instead of thd->refresh_version to make sure that all tables in thd->open_tables are in the same refresh series. sql/sql_base.h: Add declaration for check_if_table_exists(). sql/sql_class.cc: Remove init_open_tables_state(), it's now equal to reset_open_tables_state(). sql/sql_class.h: Remove thd->version, THD::init_open_tables_state(). sql/sql_plugin.cc: Use table->m_needs_reopen to mark the table as stale rather than manipulate with thd->version, which is no more. sql/sql_udf.cc: Use table->m_needs_reopen to mark the table as stale rather than manipulate with thd->version, which is no more. sql/table.h: Remove an unused variable. sql/tztime.cc: Use table->m_needs_reopen to mark the table as stale rather than manipulate with thd->version, which is no more. storage/heap/CMakeLists.txt: Add heap tests to cmake build files. storage/heap/ha_heap.cc: Fix a race when ha_heap::ha_open() could insert two HP_SHARE objects into the internal share list or store a dangling reference to a share in ha_heap instance, or wrongly set implicit_emptied. storage/heap/hp_create.c: Optionally pin a newly created share in the list of shares by increasing its open_count. This is necessary to make sure that a newly created share doesn't disappear while a HP_INFO object is being created to reference it. storage/heap/hp_open.c: When adding a new HP_INFO object to the list of objects in the heap share, make sure the open_count is not increased twice. storage/heap/hp_test1.c: Adjust the test to new function signatures. storage/heap/hp_test2.c: Adjust the test to new function signatures. |
||
Jon Olav Hauglid
|
142a162c66 |
manual merge from mysql-trunk-bugfixing
Conflicts: Text conflict in mysql-test/r/archive.result Contents conflict in mysql-test/r/innodb_bug38231.result Text conflict in mysql-test/r/mdl_sync.result Text conflict in mysql-test/suite/binlog/t/disabled.def Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result Text conflict in mysql-test/t/archive.test Contents conflict in mysql-test/t/innodb_bug38231.test Text conflict in mysql-test/t/mdl_sync.test Text conflict in sql/sp_head.cc Text conflict in sql/sql_show.cc Text conflict in sql/table.cc Text conflict in sql/table.h |
||
Davi Arnaut
|
1164841167 |
Backport: Remove unused and ancient files, functions, and facilities.
client/mysql.cc: Remove unused functions. client/sql_string.cc: Remove unused functions. include/my_pthread.h: Remove unused prototype. mysys/my_pthread.c: Remove unused function. sql/lock.cc: Remove unused function. sql/lock.h: Remove unused and duplicated prototypes. sql/sql_class.h: Removed unused variables. sql/sql_const.h: Remove unused defines. sql/sql_priv.h: Remove unused defines. sql/sql_string.cc: Remove unused prototype. sql/thr_malloc.cc: Remove unused function. sql/thr_malloc.h: Remove unused prototype. storage/myisam/CMakeLists.txt: Remove obsolete fulltext file. storage/myisam/Makefile.am: Remove obsolete files (were already commented out). storage/myisam/ft_eval.c: Remove obsolete fulltext file. storage/myisam/ft_eval.h: Remove obsolete fulltext file. storage/myisam/ft_stem.c: Remove obsolete fulltext file. storage/myisam/ft_test1.c: Remove obsolete fulltext file. storage/myisam/ft_test1.h: Remove obsolete fulltext file. |
||
Dmitry Lenev
|
ae1ae4bd63 |
Fix for bug #51263 "Deadlock between transactional
SELECT and ALTER TABLE ... REBUILD PARTITION". ALTER TABLE on InnoDB table (including partitioned tables) acquired exclusive locks on rows of table being altered. In cases when there was concurrent transaction which did locking reads from this table this sometimes led to a deadlock which was not detected by MDL subsystem nor by InnoDB engine (and was reported only after exceeding innodb_lock_wait_timeout). This problem stemmed from the fact that ALTER TABLE acquired TL_WRITE_ALLOW_READ lock on table being altered. This lock was interpreted as a write lock and thus for table being altered handler::external_lock() method was called with F_WRLCK as an argument. As result InnoDB engine treated ALTER TABLE as an operation which is going to change data and acquired LOCK_X locks on rows being read from old version of table. In case when there was a transaction which already acquired SR metadata lock on table and some LOCK_S locks on its rows (e.g. by using it in subquery of DML statement) concurrent ALTER TABLE was blocked at the moment when it tried to acquire LOCK_X lock before reading one of these rows. The transaction's attempt to acquire SW metadata lock on table being altered led to deadlock, since it had to wait for ALTER TABLE to release SNW lock. This deadlock was not detected and got resolved only after timeout expiring because waiting were happening in two different subsystems. Similar deadlocks could have occured in other situations. This patch tries to solve the problem by changing ALTER TABLE implementation to use TL_READ_NO_INSERT lock instead of TL_WRITE_ALLOW_READ. After this step handler::external_lock() is called with F_RDLCK as an argument and InnoDB engine correctly interprets ALTER TABLE as operation which only reads data from original version of table. Thanks to this ALTER TABLE acquires only LOCK_S locks on rows it reads. This, in its turn, causes inter-subsystem deadlocks to go away, as all potential lock conflicts and thus deadlocks will be limited to metadata locking subsystem: - When ALTER TABLE reads rows from table being altered it can't encounter any locks which conflict with LOCK_S row locks. There should be no concurrent transactions holding LOCK_X row locks. Such a transaction should have been acquired SW metadata lock on table first which would have conflicted with ALTER's SNW lock. - Vice versa, when DML which runs concurrently with ALTER TABLE tries to lock row it should be requesting only LOCK_S lock which is compatible with locks acquired by ALTER, as otherwise such DML must own an SW metadata lock on table which would be incompatible with ALTER's SNW lock. mysql-test/r/innodb_mysql_lock2.result: Added test for bug #51263 "Deadlock between transactional SELECT and ALTER TABLE ... REBUILD PARTITION". mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result: Since CREATE TRIGGER no longer acquires write lock on table it is no longer interpreted as an operation which modifies table data and therefore no longer fails if invoked for SBR-only engine in ROW mode. mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test: Since CREATE TRIGGER no longer acquires write lock on table it is no longer interpreted as an operation which modifies table data and therefore no longer fails if invoked for SBR-only engine in ROW mode. mysql-test/t/innodb_mysql_lock2.test: Added test for bug #51263 "Deadlock between transactional SELECT and ALTER TABLE ... REBUILD PARTITION". sql/ha_partition.cc: When ALTER TABLE creates a new partition to be filled from other partition lock it in F_WRLCK mode instead of using mode which was used for locking the whole table (it is F_RDLCK now). sql/lock.cc: Replaced conditions which used TL_WRITE_ALLOW_READ lock type with equivalent conditions using TL_WRITE_ALLOW_WRITE. This should allow to get rid of TL_WRITE_ALLOW_READ lock type eventually. sql/mdl.cc: Updated outdated comment to reflect current situation. sql/sql_base.cc: Replaced conditions which used TL_WRITE_ALLOW_READ lock type with equivalent conditions using TL_WRITE_ALLOW_WRITE. This should allow to get rid of TL_WRITE_ALLOW_READ lock type eventually. sql/sql_table.cc: mysql_admin_table(): Use TL_WRITE_ALLOW_WRITE lock type instead of TL_WRITE_ALLOW_READ to determine that we need to acquire upgradable metadata lock. This should allow to completely get rid of TL_WRITE_ALLOW_READ in long term. mysql_recreate_table(): ALTER TABLE now requires TL_READ_NO_INSERT thr_lock.c lock instead of TL_WRITE_ALLOW_READ. sql/sql_trigger.cc: Changed CREATE/DROP TRIGGER implementation to use TL_READ_NO_INSERT lock instead of TL_WRITE_ALLOW_READ lock. The latter is no longer necessary since: a) We now can rely on metadata locks to achieve proper isolation between two DDL statements or DDL and DML statements. b) This statement does not change any data in table so there is no need to inform storage engine about it. sql/sql_yacc.yy: Changed implementation of ALTER TABLE (and CREATE/DROP INDEX as a consequence) to use TL_READ_NO_INSERT lock instead of TL_WRITE_ALLOW_READ lock. This is possible since: a) We now can rely on metadata locks to achieve proper isolation between two DDL statements or DDL and DML statements. b) This statement only reads data in table being open. We write data only to the new version of table and then replace with it old version of table under X metadata lock. Thanks to this change InnoDB will no longer acquire LOCK_X locks on rows being read by ALTER TABLE (instead LOCK_S locks will be acquired) and thus cause of bug #51263 "Deadlock between transactional SELECT and ALTER TABLE ... REBUILD PARTITION" is removed. Did the similar change for CREATE TRIGGER (see comments for sql_trigger.cc for details). |
||
Dmitry Lenev
|
bee0f214fd |
Pre-requisite patch for bug #51263 "Deadlock between
transactional SELECT and ALTER TABLE ... REBUILD PARTITION". The goal of this patch is to decouple type of metadata lock acquired for table by open_tables() from type of table-level lock to be acquired on it. To achieve this we change approach to how we determine what type of metadata lock should be acquired on table to be open. Now instead of inferring it at open_tables() time from flags and type of table-level lock we rely on that type of metadata lock is properly set at parsing time and is not changed further. sql/ha_ndbcluster.cc: Now one needs to properly initialize table list element's MDL_request object before calling mysql_rm_table_part2(). sql/lock.cc: lock_table_names() no longer initializes table list elements' MDL_request objects. Now proper initialization of these requests is a responsibility of the caller. sql/lock.h: Removed MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag which became unnecessary. Thanks to the fact that we don't reset type of requests for metadata locks between re-executions we now can figure out that upgradable locks are requested by simply looking at their type which were set in the parser. As result this flag became redundant. sql/mdl.h: Added version of new operator which simplifies allocation of MDL_request objects on a MEM_ROOT. sql/sp_head.cc: Added comment explaining why it is OK to infer type of metadata lock to request from type of table-level lock for prelocking. Added enum_mdl_type argument to sp_add_to_query_tables() to simplify its usage in trigger implementation. sql/sp_head.h: Added enum_mdl_type argument to sp_add_to_query_tables() to simplify its usage in trigger implementation. sql/sql_base.cc: - open_table_get_mdl_lock(): Preserve type of MDL_request for table list element which was set in the parser by creating MDL_request objects on memory root if MYSQL_OPEN_FORCE_SHARED_MDL or MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag were specified. Thanks to this and to the fact that we no longer reset type of requests for metadata locks between re-executions we no longer need to acquire exclusive metadata lock on table to be created in a special way. This lock is acquired by code handling acquiring of upgradable locks. Also changed signature/calling convention for this function to simplify its usage. - Accordingly special lock strategy for table list elements which was used for such locks became unnecessary and was removed. Other strategies were renamed. - Since we no longer have guarantee that MDL_request object which were not satisfied due to lock conflict belongs to table list element Open_table_context class and its methods were extended to remember pointer to MDL_request which has caused problem at request_backoff_action() time and use it in recover_from_failed_open(). Similar approach is used for cases when problem from which we need to recover is not related to MDL but to the table itself. In this case we store pointer to the element of table list. - Changed open_tables()/open_tables_check_upgradable_mdl()/ open_tables_acquire_upgradable_mdl() not to rely on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to understand when upgradable metadata locks should be acquired and not to infer type of MDL lock from type of table-level lock. Instead we assume that type of MDL to be acquired was set in the parser (we can do this as type of MDL_request is no longer reset between re-executions). sql/sql_class.h: Since we no longer have guarantee that MDL_request object which were not satisfied due to lock conflict belongs to table list element Open_table_context class and its methods were extended to remember pointer to MDL_request which has caused problem at request_backoff_action() time and use it in recover_from_failed_open(). Similar approach is used for cases when problem from which we need to recover is not related to MDL but to the table itself. In this case we store pointer to the element of table list. sql/sql_db.cc: Now one needs to properly initialize table list element's MDL_request object before calling mysql_rm_table_part2() or mysql_rename_tables(). sql/sql_lex.cc: st_select_lex/st_select_lex_node::add_table_to_list() method now has argument which allows specify type of metadata lock to be requested for table list element being added. sql/sql_lex.h: - st_select_lex/st_select_lex_node::add_table_to_list() method now has argument which specifies type of metadata lock to be requested for table list element being added. This allows to explicitly set type of MDL lock to be acquired for a DDL statement in parser. It is also more future-proof than inferring type of MDL request from type of table-level lock. - Added Yacc_state::m_mdl_type member which specifies which type of metadata lock should be requested for tables to be added to table list by a grammar rule in cases when the same rule is used in several statements requiring different kinds of metadata locks. sql/sql_parse.cc: - st_select_lex::add_table_to_list() method now has argument which specifies type of metadata lock to be requested for table list element being added. This allows to explicitly set type of MDL lock to be acquired for a DDL statement in parser. It is also more future-proof than inferring type of MDL request from type of table-level lock. - EXCLUSIVE_DOWNGRADABLE_MDL lock strategy has a new name - OTLS_DOWNGRADE_IF_EXISTS. - Adjusted LOCK TABLES implementation to the fact that we no longer infer type of metadata lock to be acquired from table level lock and that type of MDL request is set at parsing. And thus MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag became unnecessary. sql/sql_prepare.cc: TABLE_LIST's lock strategy SHARED_MDL was renamed to OTLS_NONE as now it means that metadata lock should not be changed during call to open_table() (if it has been already acquired) and is also used for exclusive metadata lock. sql/sql_show.cc: st_select_lex::add_table_to_list() method now has argument which specifies type of metadata lock to be requested for table list element being added. sql/sql_table.cc: - Adjusted mysql_admin_table()'s code to the fact that open_tables() no longer determines what kind of metadata lock should be obtained basing on type of table-level lock and flags. Instead type of metadata lock for table to be open should be set before calling open_tables(). - Changed mysql_alter_table() code to the facts: a) that now it is responsibility of caller to properly initalize MDL_request in table list elements before calling lock_table_names() b) and that MYSQL_OPEN_TAKE_UPGRADABLE_MDL is no longer necessary since type of metadata lock to be obtained at open_tables() time is set during parsing. - Changed code of mysql_recreate_table() to properly set type of metadata and table-level lock to be obtained by mysql_alter_table() which it calls. sql/sql_trigger.cc: Instead of relying on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to force open_tables() to take an upgradable lock we now specify exact type of lock to be taken when constructing table list element for table to be open for CREATE/DROP TRIGGER. sql/sql_view.cc: We no longer use TABLE_LIST::EXCLUSIVE_MDL strategy to force open_tables() to take an exclusive metadata lock on view to be created. Instead we rely on parser setting proper type of metadata lock to request and open_tables() acquiring it. This became possible thanks to the fact that we no longer reset type of MDL_request between statement re-executions. sql/sql_yacc.yy: Instead of inferring type of MDL_request for table to be open from type of table-level lock and flags passed to open_tables() we now explicitly specify them at parsing. This became possible thanks to the fact that we no longer reset type of MDL_request between statement re-executions. In future this should allow to decouple type of metadata lock from type of table-level lock. The only exception to this approach is statements implemented through mysql_admin_table() which re-uses same table list element several times with different types of table-level and metadata locks. We now also properly initialize MDL_request objects for table list elements which are later passed to lock_table_names() function. sql/table.cc: Do not reset type of MDL_request between statement re-executions. This became unnecessesary as we no longer change type of MDL_request residing in table list element. In its turn this change allows to set type of MDL_request only once - at parsing time. sql/table.h: Got rid of TABLE_LIST::EXCLUSIVE_MDL lock strategy. Now we can specify that we need to acquire exclusive lock on table to be processed by open_tables() through setting an appropriate type of MDL_request at parsing time (this became possible thanks to the fact that we no longer reset types of MDL_request's belonging to table list elements between statement re-execution). Strategy SHARED_MDL was renamed to OTLS_NONE as now it means that metadata lock should not be changed during call to open_table() (if it has been already acquired) and is also used for exclusive metadata lock. Strategy EXCLUSIVE_DOWNGRADABLE_MDL was renamed to OTLS_DOWNGRADE_IF_EXISTS. |
||
Mats Kindahl
|
23d8586dbf |
WL#5030: Split and remove mysql_priv.h
This patch: - Moves all definitions from the mysql_priv.h file into header files for the component where the variable is defined - Creates header files if the component lacks one - Eliminates all include directives from mysql_priv.h - Eliminates all circular include cycles - Rename time.cc to sql_time.cc - Rename mysql_priv.h to sql_priv.h |
||
Konstantin Osipov
|
09b7a0d115 |
A post-review fix for type-aware metadata locks.
DDL no longer aborts mysql_lock_tables(), and hence we no longer need to support need_reopen flag of this call. Remove the flag, and all the code in the server that was responsible for handling the case when it was set. This allowed to simplify: open_and_lock_tables_derived(), the delayed thread, multi-update. Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH, since we now only support this flag in open_table(). Rename MYSQL_LOCK_PERF_SCHEMA to MYSQL_LOCK_LOG_TABLE, to avoid confusion. Move the wait for the global read lock for cases when we do updates in SELECT f1() or DO (UPDATE) to open_table() from mysql_lock_tables(). When waiting for the read lock, we could raise need_reopen flag, which is no longer present in mysql_lock_tables(). Since the block responsible for waiting for GRL was moved, MYSQL_LOCK_IGNORE_GLOBAL_READ_LOCK was renamed to MYSQL_OPEN_IGNORE_GLOBAL_READ_LOCK. mysql-test/r/mdl_sync.result: Update test results (see comments for mdl_sync.test). mysql-test/t/mdl_sync.test: Update tests: an abort mysql_lock_tables() called for an INSERT no longer auto-closes SQL HANDLERS, since it no longer leads to back-off and retry. sql/ha_ndbcluster_binlog.cc: Remove unused variables. sql/lock.cc: Remove support for need_reopen parameter of mysql_lock_tables(). Update comments. sql/log_event_old.cc: Remove the loop responsible for handling need_reopen out parameter of mysql_lock_tables(). sql/mysql_priv.h: Update open and lock tables flag names. sql/share/errmsg-utf8.txt: Add a new error message to report when thr_multi_lock() is aborted. sql/sql_base.cc: Update comments. Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH. sql/sql_class.h: Remove unused code. sql/sql_db.cc: Remove an unused bit of code. sql/sql_handler.cc: For backward compatibility, we still want to back off and retry when a call to mysql_lock_tables() is aborted from within an SQL HANDLER. Write an internal error handler to support the case. sql/sql_insert.cc: Call mysql_lock_tables() no longer has need_reopen out parameter. Simplify the code by removing the crud that took care of it. MYSQL_LOCK_IGNORE_FLUSH is now only supported by open_tables(). sql/sql_show.cc: Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH sql/sql_table.cc: Remove an unused parameter. sql/sql_update.cc: Remove the need_reopen loop from multi-update. We no also longer need to cleanup the parse tree in case when mysql_lock_tables() is aborted and thus an infinite source of multi-update bugs is gone. sql/tztime.cc: Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH, since from now on this flag is only supported by open_table(). |