mirror of
https://github.com/MariaDB/server.git
synced 2025-01-16 12:02:42 +01:00
00496b7acd
TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush. mysql-test/include/handler.inc: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/r/flush.result: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/r/mdl_sync.result: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/r/server_init.result: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/t/server_init.test: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/t/flush.test: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/t/kill.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/lock_multi.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/mdl_sync.test: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. sql/ha_ndbcluster.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/ha_ndbcluster_binlog.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/lock.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mdl.cc: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. We no longer require global IX metadata lock when acquiring SNW or SNRW locks. Such locks are needed only when metadata locks of these types are upgraded to X locks. This allows to use SNW locks in FLUSH TABLES <list> WITH READ LOCK implementation and keep the latter compatible with global read lock. sql/mdl.h: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. - Deadlock_detection_visitor now has m_table_shares_visited member which allows to support recursive locking for LOCK_open. This is required when deadlock detector inspects waiters graph which contains several edges representing waits for flushes or needs to come through the such edge more than once. sql/mysqld.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mysqld.h: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/sql_base.cc: Changed approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. Thanks to this change MDL deadlock detector can take such waits into account. To implement this/as result of this change: - tdc_wait_for_old_versions() was replaced with tdc_wait_for_old_version() which waits for individual old share to go away and which is called by open_table() after finding out that share is outdated. We don't need to perform back-off before such waiting thanks to the fact that deadlock detector now sees such waits. - As result Open_table_ctx::m_mdl_requests became unnecessary and was removed. We no longer allocate copies of MDL_request objects on MEM_ROOT when MYSQL_OPEN_FORCE_SHARED/SHARED_HIGH_PRIO flags are in effect. - close_cached_tables() and tdc_wait_for_old_version() share code which implements waiting for share to be flushed - the both use TABLE_SHARE::wait_until_flush() method. Thanks to this close_cached_tables() supports timeouts and has extra parameter for this. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. - In cases when we discover that current connection tries to open tables from different generation we now simply back-off and restart process of opening tables. To support this Open_table_context::OT_REOPEN_TABLES enum element was added. - COND_refresh condition variable became unnecessary and was removed. - mysql_notify_thread_having_shared_lock() no longer wakes up connections waiting for flush as all such connections can be waken up by deadlock detector if necessary. sql/sql_base.h: - close_cached_tables() now has one more parameter - timeout for waiting for table to be flushed. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. Added new OT_REOPEN_TABLES enum element to be used in cases when we need to restart open tables process even in the middle of transaction. - Open_table_ctx::m_mdl_requests became unnecessary and was removed. sql/sql_class.h: Added assert ensuring that we won't use LOCK_open mutex with THD::enter_cond(). Otherwise deadlocks can arise in MDL deadlock detector. sql/sql_parse.cc: Changed FLUSH TABLES <list> WITH READ LOCK to take SNW metadata locks instead of X locks on tables to be flushed. Since we no longer require global IX lock to be taken when SNW locks are taken this makes this statement compatible with FLUSH TABLES WITH READ LOCK statement. Since SNW locks allow other connections to have table opened FLUSH TABLES <list> WITH READ LOCK now has to wait during open_tables() for old version to go away. Such waits can lead to deadlocks which will be detected by MDL deadlock detector which now takes waits for table to be flushed into account. Also adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/sql_yacc.yy: FLUSH TABLES <list> WITH READ LOCK now needs only SNW metadata locks on tables. sql/sys_vars.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/table.cc: Implemented new approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. This change allows to make such waits visible inside of MDL deadlock detector. To do it: - Added list of waiters/Flush_tickets to TABLE_SHARE class. - Changed free_table_share() to postpone freeing of share memory until last waiter goes away and to wake up subscribed waiters. - Added TABLE_SHARE::wait_until_flushed() method which implements subscription to the list of waiters for table to be flushed and waiting for this event. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket). sql/table.h: In order to support new strategy of waiting for table flush (see comment for table.cc for details) added list of waiters/Flush_tickets to TABLE_SHARE class. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket). |
||
|---|---|---|
| .. | ||
| add_anonymous_users.inc | ||
| analyze-sync_with_master.test | ||
| analyze-timeout.test | ||
| big_test.inc | ||
| binlog_inject_error.inc | ||
| bug38347.inc | ||
| check-testcase.test | ||
| check-warnings.test | ||
| check_concurrent_insert.inc | ||
| check_events_off.inc | ||
| check_ipv6.inc | ||
| check_key_reads.inc | ||
| check_key_req.inc | ||
| check_no_concurrent_insert.inc | ||
| check_no_row_lock.inc | ||
| check_shared_row_lock.inc | ||
| check_slave_is_running.inc | ||
| check_slave_no_error.inc | ||
| check_slave_param.inc | ||
| check_var_limit.inc | ||
| circular_rpl_for_4_hosts_init.inc | ||
| circular_rpl_for_4_hosts_sync.inc | ||
| cleanup_fake_relay_log.inc | ||
| commit.inc | ||
| common-tests.inc | ||
| concurrent.inc | ||
| connect2.inc | ||
| count_sessions.inc | ||
| ctype_common.inc | ||
| ctype_datetime.inc | ||
| ctype_filesort.inc | ||
| ctype_german.inc | ||
| ctype_innodb_like.inc | ||
| ctype_like.inc | ||
| ctype_like_escape.inc | ||
| ctype_like_range_f1f2.inc | ||
| ctype_numconv.inc | ||
| ctype_regex.inc | ||
| ctype_utf8_table.inc | ||
| ctype_utf8mb4.inc | ||
| ddl_i18n.check_events.inc | ||
| ddl_i18n.check_sp.inc | ||
| ddl_i18n.check_triggers.inc | ||
| ddl_i18n.check_views.inc | ||
| deadlock.inc | ||
| default_client.cnf | ||
| default_my.cnf | ||
| default_mysqld.cnf | ||
| default_ndbd.cnf | ||
| delete_anonymous_users.inc | ||
| diff_master_slave.inc | ||
| diff_tables.inc | ||
| endspace.inc | ||
| get_binlog_dump_thread_id.inc | ||
| get_relay_log_pos.inc | ||
| gis_generic.inc | ||
| gis_keys.inc | ||
| grant_cache.inc | ||
| handler.inc | ||
| have_32bit.inc | ||
| have_64bit.inc | ||
| have_archive.inc | ||
| have_archive_plugin.inc | ||
| have_big5.inc | ||
| have_binlog_format_mixed.inc | ||
| have_binlog_format_mixed_or_row.inc | ||
| have_binlog_format_mixed_or_statement.inc | ||
| have_binlog_format_row.inc | ||
| have_binlog_format_row_or_statement.inc | ||
| have_binlog_format_statement.inc | ||
| have_blackhole.inc | ||
| have_blackhole_plugin.inc | ||
| have_case_insensitive_file_system.inc | ||
| have_case_insensitive_fs.inc | ||
| have_case_sensitive_file_system.inc | ||
| have_compress.inc | ||
| have_cp866.inc | ||
| have_cp932.inc | ||
| have_cp1250_ch.inc | ||
| have_cp1251.inc | ||
| have_crypt.inc | ||
| have_csv.inc | ||
| have_debug.inc | ||
| have_debug_sync.inc | ||
| have_dynamic_loading.inc | ||
| have_eucjpms.inc | ||
| have_euckr.inc | ||
| have_example_plugin.inc | ||
| have_exampledb.inc | ||
| have_federated_plugin.inc | ||
| have_gb2312.inc | ||
| have_gbk.inc | ||
| have_geometry.inc | ||
| have_innodb.inc | ||
| have_ipv4_mapped.inc | ||
| have_koi8r.inc | ||
| have_latin2_ch.inc | ||
| have_local_infile.inc | ||
| have_log_bin.inc | ||
| have_lowercase0.inc | ||
| have_lowercase1.inc | ||
| have_lowercase2.inc | ||
| have_multi_ndb.inc | ||
| have_mysql_upgrade.inc | ||
| have_ndb.inc | ||
| have_ndb_extra.inc | ||
| have_ndbapi_examples.inc | ||
| have_nodebug.inc | ||
| have_not_innodb_plugin.inc | ||
| have_outfile.inc | ||
| have_partition.inc | ||
| have_perfschema.inc | ||
| have_profiling.inc | ||
| have_query_cache.inc | ||
| have_semisync_plugin.inc | ||
| have_simple_parser.inc | ||
| have_sjis.inc | ||
| have_ssl.inc | ||
| have_ssl_communication.inc | ||
| have_ssl_crypto_functs.inc | ||
| have_symlink.inc | ||
| have_thread_concurrency.inc | ||
| have_tis620.inc | ||
| have_ucs2.inc | ||
| have_udf.inc | ||
| have_ujis.inc | ||
| have_utf8.inc | ||
| have_utf8mb4.inc | ||
| have_utf16.inc | ||
| have_utf32.inc | ||
| implicit_commit_helper.inc | ||
| index_merge1.inc | ||
| index_merge2.inc | ||
| index_merge_2sweeps.inc | ||
| index_merge_ror.inc | ||
| index_merge_ror_cpk.inc | ||
| innodb-index.inc | ||
| innodb_rollback_on_timeout.inc | ||
| innodb_trx_weight.inc | ||
| ipv6.inc | ||
| ipv6_clients.inc | ||
| is_embedded.inc | ||
| kill_query.inc | ||
| kill_query_and_diff_master_slave.inc | ||
| linux_sys_vars.inc | ||
| load_sysvars.inc | ||
| loaddata_autocom.inc | ||
| master-slave-end.inc | ||
| master-slave-reset.inc | ||
| master-slave.inc | ||
| min_null_cond.inc | ||
| mix1.inc | ||
| mix2.inc | ||
| mix2_ucs2.inc | ||
| mtr_check.sql | ||
| mtr_warnings.sql | ||
| mysqlbinlog_row_engine.inc | ||
| mysqld--help.inc | ||
| mysqldump.inc | ||
| mysqlhotcopy.inc | ||
| mysqltest-x.inc | ||
| ndb_backup.inc | ||
| ndb_backup_print.inc | ||
| ndb_default_cluster.inc | ||
| ndb_master-slave.inc | ||
| ndb_master-slave_2ch.inc | ||
| ndb_not_readonly.inc | ||
| ndb_restore_master.inc | ||
| ndb_restore_slave_eoption.inc | ||
| ndb_setup_slave.inc | ||
| ndb_wait_connected.inc | ||
| no_running_event_scheduler.inc | ||
| no_running_events.inc | ||
| no_valgrind_without_big.inc | ||
| not_as_root.inc | ||
| not_binlog_format_row.inc | ||
| not_embedded.inc | ||
| not_ndb.inc | ||
| not_ndb_default.inc | ||
| not_openssl.inc | ||
| not_parallel.inc | ||
| not_valgrind.inc | ||
| not_var_link.inc | ||
| not_windows.inc | ||
| not_windows_embedded.inc | ||
| one_thread_per_connection.inc | ||
| parser_bug21114.inc | ||
| partition_date_range.inc | ||
| ps_conv.inc | ||
| ps_create.inc | ||
| ps_ddl_1.inc | ||
| ps_modify.inc | ||
| ps_modify1.inc | ||
| ps_query.inc | ||
| ps_renew.inc | ||
| query_cache.inc | ||
| query_cache_sql_prepare.inc | ||
| read_many_rows.inc | ||
| report-features.test | ||
| reset_master_and_slave.inc | ||
| restart_mysqld.inc | ||
| rowid_order.inc | ||
| rpl_diff_tables.inc | ||
| rpl_events.inc | ||
| rpl_ip_mix.inc | ||
| rpl_ip_mix2.inc | ||
| rpl_ipv6.inc | ||
| rpl_loaddata_charset.inc | ||
| rpl_multi_engine.inc | ||
| rpl_multi_engine2.inc | ||
| rpl_multi_engine3.inc | ||
| rpl_stmt_seq.inc | ||
| rpl_udf.inc | ||
| running_event_scheduler.inc | ||
| safe_set_to_maybe_ro_var.inc | ||
| select_ndb_apply_status.inc | ||
| set_binlog_format_mixed.sql | ||
| set_binlog_format_row.sql | ||
| set_binlog_format_statement.sql | ||
| setup_fake_relay_log.inc | ||
| show_binary_logs.inc | ||
| show_binlog_events.inc | ||
| show_binlog_events2.inc | ||
| show_binlog_using_logname.inc | ||
| show_events.inc | ||
| show_master_logs.inc | ||
| show_master_status.inc | ||
| show_msg.inc | ||
| show_msg80.inc | ||
| show_relaylog_events.inc | ||
| show_rpl_debug_info.inc | ||
| show_slave_status.inc | ||
| sp-vars.inc | ||
| start_slave.inc | ||
| stop_slave.inc | ||
| strict_autoinc.inc | ||
| sync_slave_io_with_master.inc | ||
| system_db_struct.inc | ||
| test_fieldsize.inc | ||
| test_outfile.inc | ||
| testdb_only.inc | ||
| tpcb.inc | ||
| tpcb_disk_data.inc | ||
| truncate_file.inc | ||
| unsafe_binlog.inc | ||
| uses_vardir.inc | ||
| varchar.inc | ||
| view_alias.inc | ||
| wait_condition.inc | ||
| wait_condition_sp.inc | ||
| wait_for_binlog_event.inc | ||
| wait_for_ndb_to_binlog.inc | ||
| wait_for_query_to_succeed.inc | ||
| wait_for_slave_io_error.inc | ||
| wait_for_slave_io_to_start.inc | ||
| wait_for_slave_io_to_stop.inc | ||
| wait_for_slave_param.inc | ||
| wait_for_slave_sql_error.inc | ||
| wait_for_slave_sql_error_and_skip.inc | ||
| wait_for_slave_sql_to_start.inc | ||
| wait_for_slave_sql_to_stop.inc | ||
| wait_for_slave_to_start.inc | ||
| wait_for_slave_to_stop.inc | ||
| wait_for_status_var.inc | ||
| wait_show_condition.inc | ||
| wait_until_connected_again.inc | ||
| wait_until_count_sessions.inc | ||
| wait_until_disconnected.inc | ||
| wait_until_rows_count.inc | ||
| windows.inc | ||
| windows_sys_vars.inc | ||