mariadb/sql/sql_plugin_services.ic

259 lines
7 KiB
Text
Raw Normal View History

/* Copyright (c) 2009, 2010, Oracle and/or its affiliates.
2020-09-23 11:32:43 +03:00
Copyright (c) 2012, 2020, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
2019-05-11 22:19:05 +03:00
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */
/* support for Services */
#include <service_versions.h>
#include <mysql/service_wsrep.h>
MDEV-16678 Prefer MDL to dict_sys.latch for innodb background tasks This is joint work with Thirunarayanan Balathandayuthapani. The MDL interface between InnoDB and the rest of the server (in storage/innobase/dict/dict0dict.cc and in include/) is my work, while most everything else is Thiru's. The collection of InnoDB persistent statistics and the defragmentation were not refactored to use MDL. They will keep relying on lower-level interlocking with fil_check_pending_operations(). The purge of transaction history and the background operations on fulltext indexes will use MDL. We will revert commit 2c4844c9e76427525e8c39a2d72686085efe89c3 (MDEV-17813) because thanks to MDL, purge cannot conflict with DDL operations anymore. For a similar reason, we will remove the MDEV-16222 test case from gcol.innodb_virtual_debug_purge. Purge is essentially replacing all use of the global dict_sys.latch with MDL. Purge will skip the undo log records for tables whose names start with #sql-ib or #sql2. Theoretically, such tables might be renamed back to visible table names if TRUNCATE fails to create a new table, or the final rename in ALTER TABLE...ALGORITHM=COPY fails. In that case, purge could permanently leave some garbage in the table. Such garbage will be tolerated; the table would not be considered corrupted. To avoid repeated MDL releases and acquisitions, trx_purge_attach_undo_recs() will sort undo log records by table_id, and purge_node_t will keep the MDL and table handle open for multiple successive undo log records. get_purge_table(): A new accessor, used during the purge of history for indexed virtual columns. This interface should ideally not exist at all. thd_mdl_context(): Accessor of THD::mdl_context. Wrapped in a new thd_mdl_service. dict_get_db_name_len(): Define inline. dict_acquire_mdl_shared(): Acquire explicit shared MDL on a table name if needed. dict_table_open_on_id(): Return MDL_ticket, if requested. dict_table_close(): Release MDL ticket, if requested. dict_fts_index_syncing(), dict_index_t::index_fts_syncing: Remove. row_drop_table_for_mysql() no longer needs to check these, because MDL guarantees that a fulltext index sync will not be in progress while MDL_EXCLUSIVE is protecting a DDL operation. dict_table_t::parse_name(): Parse the table name for acquiring MDL. purge_node_t::undo_recs: Change the type to std::list<trx_purge_rec_t*> (different container, and storing also roll_ptr). purge_node_t: Add mdl_ticket, last_table_id, purge_thd, mdl_hold_recs for acquiring MDL and for keeping the table open across multiple undo log records. purge_vcol_info_t, row_purge_store_vsec_cur(), row_purge_restore_vsec_cur(): Remove. We will acquire the MDL earlier. purge_sys_t::heap: Added, for reading undo log records. fts_sync_during_ddl(): Invoked during ALGORITHM=INPLACE operations to ensure that fts_sync_table() will not conflict with MDL_EXCLUSIVE. Uses fts_t::sync_message for bookkeeping.
2019-12-10 15:42:50 +02:00
#include <mysql/service_thd_mdl.h>
struct st_service_ref {
const char *name;
uint version;
void *service;
};
static struct my_snprintf_service_st my_snprintf_handler = {
my_snprintf,
my_vsnprintf
};
static struct thd_alloc_service_st thd_alloc_handler= {
thd_alloc,
thd_calloc,
thd_strdup,
thd_strmake,
thd_memdup,
thd_make_lex_string
};
static struct thd_wait_service_st thd_wait_handler= {
thd_wait_begin,
thd_wait_end
};
Added progress reporting for alter table, LOAD DATA INFILE and for aria tables: check table, repair table, analyze table. - The client gets a progress report message that triggers a callback function if requested with mysql_options(MYSQL_PROGRESS_CALLBACK, function) - Added Progress field last to 'show processlist' - Stage, Max_stage and Progress field added to information_schema.progresslist - The 'mysql' client by defaults enables progress reports when the output is a tty. - Added progress_report_time time variable to configure how often progress reports is sent to client Added read only system variable 'in_transaction' which is 1 if we have executed a BEGIN statement. client/client_priv.h: Added OPT_REPORT_PROGRESS client/mysql.cc: Added option --progress-reports (on by default if not batch mode) Progress reports is written to stdout for long running commands include/Makefile.am: Added mysql/service_progress_report.h include/myisamchk.h: Added variables to be able to do progress reporting in Aria and later in MyISAM include/mysql.h: Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK include/mysql.h.pp: Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK include/mysql/plugin.h: Added functions for reporting progress. include/mysql/plugin_auth.h.pp: Added functions for reporting progress. include/mysql_com.h: Added CLIENT_PROGRESS mysql_real_connect() flag. include/sql_common.h: Added callback function for reporting progress mysql-test/r/old-mode.result: Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode. mysql-test/suite/funcs_1/datadict/datadict_priv.inc: Added new column mysql-test/suite/funcs_1/datadict/processlist_priv.inc: Test all new PROCESSLIST columns mysql-test/suite/funcs_1/r/is_columns_is.result: Updated results mysql-test/suite/funcs_1/r/is_columns_is_embedded.result: Updated results mysql-test/suite/funcs_1/r/is_columns_mysql_embedded.result: Updated results mysql-test/suite/funcs_1/r/is_tables_is_embedded.result: Updated results mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result: Updated results mysql-test/suite/funcs_1/r/processlist_priv_ps.result: Updated results mysql-test/suite/funcs_1/r/processlist_val_no_prot.result: Updated results mysql-test/suite/funcs_1/r/processlist_val_ps.result: Updated results mysql-test/suite/pbxt/r/pbxt_locking.result: Updated results mysql-test/suite/pbxt/r/skip_name_resolve.result: Updated results mysql-test/t/old-mode.test: Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode. plugin/handler_socket/handlersocket/Makefile.am: Added -lmysqlservices scripts/mytop.sh: Made 'State' field width dynamic. Added 'Progress' to process list display. sql-common/client.c: Added handling of progress messages. Removed check_license() function. sql/mysql_priv.h: Added opt_progress_report_time sql/mysqld.cc: Added progress_report_time time variable to configure how often progress reports is sent to client sql/protocol.cc: Added net_send_progress_packet() sql/protocol.h: New prototypes sql/set_var.cc: Added variables progress_report_time and in_transaction sql/sql_acl.cc: Safety fix: Made client_capabilities ulonglong sql/sql_class.cc: Added interface functions for progress reporting sql/sql_class.h: Added varibles in THD for progress reporting. Added CF_REPORT_PROGRESS sql/sql_load.cc: Added progress reporting for LOAD DATA INFILE sql/sql_parse.cc: Added CF_REPORT_PROGRESS for top level commands for which it's safe to send progress reports to client sql/sql_show.cc: Added Progress field last to 'show processlist' Stage, Max_stage and Progress field added to information_schema.progresslist sql/sql_table.cc: Added progress reporting for ALTER TABLE Added THD as argument to copy_data_between_tables() storage/maria/ha_maria.cc: Added progress reporting for check table, repair table, analyze table Fixed a bug in start_bulk_insert() that caused alter table to always run with all keys enabled. storage/maria/ma_check.c: Added progress reporting Remember old state before starting repair. This removes some warnings from optimize_table if create-with-sort fails. storage/maria/ma_check_standalone.h: Added dummy reporting function for standalone Aria programs. storage/maria/ma_sort.c: Added progress reporting storage/maria/maria_chk.c: Updated version storage/maria/maria_def.h: Added new prototypes tests/mysql_client_test.c: Added test case for progress reporting
2011-07-01 15:08:30 +03:00
static struct progress_report_service_st progress_report_handler= {
thd_progress_init,
thd_progress_report,
thd_progress_next_stage,
thd_progress_end,
set_thd_proc_info
};
2013-01-18 19:04:23 +01:00
static struct kill_statement_service_st thd_kill_statement_handler= {
thd_kill_level
};
static struct thd_timezone_service_st thd_timezone_handler= {
thd_TIME_to_gmt_sec,
thd_gmt_sec_to_TIME
};
2017-03-06 13:06:03 +01:00
static struct my_sha2_service_st my_sha2_handler = {
my_sha224,
my_sha224_multi,
my_sha224_context_size,
my_sha224_init,
my_sha224_input,
my_sha224_result,
my_sha256,
my_sha256_multi,
my_sha256_context_size,
my_sha256_init,
my_sha256_input,
my_sha256_result,
my_sha384,
my_sha384_multi,
my_sha384_context_size,
my_sha384_init,
my_sha384_input,
my_sha384_result,
my_sha512,
my_sha512_multi,
my_sha512_context_size,
my_sha512_init,
my_sha512_input,
my_sha512_result,
};
static struct my_sha1_service_st my_sha1_handler = {
my_sha1,
my_sha1_multi,
my_sha1_context_size,
my_sha1_init,
my_sha1_input,
my_sha1_result
};
2014-09-26 20:03:20 +02:00
static struct my_md5_service_st my_md5_handler = {
my_md5,
my_md5_multi,
my_md5_context_size,
my_md5_init,
my_md5_input,
my_md5_result
};
static struct logger_service_st logger_service_handler= {
logger_init_mutexes,
logger_open,
logger_close,
logger_vprintf,
logger_printf,
logger_write,
logger_rotate
};
static struct thd_autoinc_service_st thd_autoinc_handler= {
thd_get_autoinc
};
2017-03-06 19:34:22 +01:00
static struct thd_rnd_service_st thd_rnd_handler= {
thd_rnd,
thd_create_random_password
};
2017-03-06 17:05:03 +01:00
static struct base64_service_st base64_handler= {
2017-03-30 12:48:42 +02:00
my_base64_needed_encoded_length,
my_base64_encode_max_arg_length,
my_base64_needed_decoded_length,
my_base64_decode_max_arg_length,
my_base64_encode,
my_base64_decode
2017-03-06 17:05:03 +01:00
};
static struct thd_error_context_service_st thd_error_context_handler= {
thd_get_error_message,
thd_get_error_number,
thd_get_error_row,
thd_inc_error_row,
thd_get_error_context_description
};
static struct wsrep_service_st wsrep_handler = {
2015-07-14 16:05:29 -04:00
get_wsrep_recovery,
wsrep_consistency_check,
wsrep_is_wsrep_xid,
wsrep_xid_seqno,
wsrep_xid_uuid,
wsrep_on,
2019-01-23 15:30:00 +04:00
wsrep_prepare_key_for_innodb,
wsrep_thd_LOCK,
wsrep_thd_UNLOCK,
wsrep_thd_query,
wsrep_thd_retry_counter,
wsrep_thd_ignore_table,
wsrep_thd_trx_seqno,
2019-01-23 15:30:00 +04:00
wsrep_thd_is_aborting,
wsrep_set_data_home_dir,
wsrep_thd_is_BF,
wsrep_thd_is_local,
wsrep_thd_self_abort,
wsrep_thd_append_key,
wsrep_thd_client_state_str,
wsrep_thd_client_mode_str,
wsrep_thd_transaction_state_str,
wsrep_thd_transaction_id,
wsrep_thd_bf_abort,
wsrep_thd_order_before,
wsrep_handle_SR_rollback,
wsrep_thd_skip_locking,
wsrep_get_sr_table_name,
10.4 wsrep group commit fixes (#1224) * MDEV-16509 Improve wsrep commit performance with binlog disabled Release commit order critical section early after trx_commit_low() if binlog is not transaction coordinator. In order to avoid two phase commit, binlog_hton is not registered for THD during IO_CACHE population. Implemented a test which verifies that the transactions release commit order early. This optimization will change behavior during recovery as the commit is not two phase when binlog is off. Fixed and recorded wsrep-recover-v25 and wsrep-recover to match the behavior. * MDEV-18730 Ordering for wsrep binlog group commit Previously out of order execution was allowed for wsrep commits. Established proper ordering by populating wait_for_commit for every wsrep THD and making group commit leader to wait for prior commits before proceeding to trx_group_commit_leader(). * MDEV-18730 Added a test case to verify correct commit ordering * MDEV-16509, MDEV-18730 Review fixes Use WSREP_EMULATE_BINLOG() macro to decide if the binlog_hton should be registered. Whitespace/syntax fixes and cleanups. * MDEV-16509 Require binlog for galera_var_innodb_disallow_writes test If the commit to InnoDB is done in one phase, the native InnoDB behavior is that the transaction is committed in memory before it is persisted to disk. This means that the innodb_disallow_writes=ON may not prevent transaction to become visible to other readers before commit is completely over. On the other hand, if the commit is two phase (as it is with binlog), the transaction will be blocked in prepare phase. Fixed the test to use binlog, which enforces two phase commit, which in turn makes commit to block before the changes become visible to other connections. This guarantees that the test produces expected result.
2019-03-15 07:09:13 +02:00
wsrep_get_debug,
wsrep_commit_ordered,
wsrep_thd_is_applying,
wsrep_OSU_method_get,
wsrep_thd_has_ignored_error,
MDEV-21910 Deadlock between BF abort and manual KILL command When high priority replication slave applier encounters lock conflict in innodb, it will force the conflicting lock holder transaction (victim) to rollback. This is a must in multi-master sychronous replication model to avoid cluster lock-up. This high priority victim abort (aka "brute force" (BF) abort), is started from innodb lock manager while holding the victim's transaction's (trx) mutex. Depending on the execution state of the victim transaction, it may happen that the BF abort will call for THD::awake() to wake up the victim transaction for the rollback. Now, if BF abort requires THD::awake() to be called, then the applier thread executed locking protocol of: victim trx mutex -> victim THD::LOCK_thd_data If, at the same time another DBMS super user issues KILL command to abort the same victim, it will execute locking protocol of: victim THD::LOCK_thd_data -> victim trx mutex. These two locking protocol acquire mutexes in opposite order, hence unresolvable mutex locking deadlock may occur. The fix in this commit adds THD::wsrep_aborter flag to synchronize who can kill the victim This flag is set both when BF is called for from innodb and by KILL command. Either path of victim killing will bail out if victim's wsrep_killed is already set to avoid mutex conflicts with the other aborter execution. THD::wsrep_aborter records the aborter THD's ID. This is needed to preserve the right to kill the victim from different locations for the same aborter thread. It is also good error logging, to see who is reponsible for the abort. A new test case was added in galera.galera_bf_kill_debug.test for scenario where wsrep applier thread and manual KILL command try to kill same idle victim
2020-05-19 11:12:26 +03:00
wsrep_thd_set_ignored_error,
2020-09-21 12:29:00 +03:00
wsrep_thd_set_wsrep_aborter,
wsrep_report_bf_lock_wait,
wsrep_thd_kill_LOCK,
wsrep_thd_kill_UNLOCK
};
static struct thd_specifics_service_st thd_specifics_handler=
{
thd_key_create,
thd_key_delete,
thd_getspecific,
thd_setspecific
};
static struct encryption_scheme_service_st encryption_scheme_handler=
{
encryption_scheme_encrypt,
encryption_scheme_decrypt
};
static struct my_crypt_service_st crypt_handler=
{
my_aes_crypt_init,
my_aes_crypt_update,
my_aes_crypt_finish,
my_aes_crypt,
my_aes_get_size,
my_aes_ctx_size,
my_random_bytes
};
static struct my_print_error_service_st my_print_error_handler=
{
my_error,
my_printf_error,
my_printv_error
};
struct json_service_st json_handler=
{
json_type,
json_get_array_item,
json_get_object_key,
json_get_object_nkey,
json_escape_string,
json_unescape_json
};
MDEV-16678 Prefer MDL to dict_sys.latch for innodb background tasks This is joint work with Thirunarayanan Balathandayuthapani. The MDL interface between InnoDB and the rest of the server (in storage/innobase/dict/dict0dict.cc and in include/) is my work, while most everything else is Thiru's. The collection of InnoDB persistent statistics and the defragmentation were not refactored to use MDL. They will keep relying on lower-level interlocking with fil_check_pending_operations(). The purge of transaction history and the background operations on fulltext indexes will use MDL. We will revert commit 2c4844c9e76427525e8c39a2d72686085efe89c3 (MDEV-17813) because thanks to MDL, purge cannot conflict with DDL operations anymore. For a similar reason, we will remove the MDEV-16222 test case from gcol.innodb_virtual_debug_purge. Purge is essentially replacing all use of the global dict_sys.latch with MDL. Purge will skip the undo log records for tables whose names start with #sql-ib or #sql2. Theoretically, such tables might be renamed back to visible table names if TRUNCATE fails to create a new table, or the final rename in ALTER TABLE...ALGORITHM=COPY fails. In that case, purge could permanently leave some garbage in the table. Such garbage will be tolerated; the table would not be considered corrupted. To avoid repeated MDL releases and acquisitions, trx_purge_attach_undo_recs() will sort undo log records by table_id, and purge_node_t will keep the MDL and table handle open for multiple successive undo log records. get_purge_table(): A new accessor, used during the purge of history for indexed virtual columns. This interface should ideally not exist at all. thd_mdl_context(): Accessor of THD::mdl_context. Wrapped in a new thd_mdl_service. dict_get_db_name_len(): Define inline. dict_acquire_mdl_shared(): Acquire explicit shared MDL on a table name if needed. dict_table_open_on_id(): Return MDL_ticket, if requested. dict_table_close(): Release MDL ticket, if requested. dict_fts_index_syncing(), dict_index_t::index_fts_syncing: Remove. row_drop_table_for_mysql() no longer needs to check these, because MDL guarantees that a fulltext index sync will not be in progress while MDL_EXCLUSIVE is protecting a DDL operation. dict_table_t::parse_name(): Parse the table name for acquiring MDL. purge_node_t::undo_recs: Change the type to std::list<trx_purge_rec_t*> (different container, and storing also roll_ptr). purge_node_t: Add mdl_ticket, last_table_id, purge_thd, mdl_hold_recs for acquiring MDL and for keeping the table open across multiple undo log records. purge_vcol_info_t, row_purge_store_vsec_cur(), row_purge_restore_vsec_cur(): Remove. We will acquire the MDL earlier. purge_sys_t::heap: Added, for reading undo log records. fts_sync_during_ddl(): Invoked during ALGORITHM=INPLACE operations to ensure that fts_sync_table() will not conflict with MDL_EXCLUSIVE. Uses fts_t::sync_message for bookkeeping.
2019-12-10 15:42:50 +02:00
static struct thd_mdl_service_st thd_mdl_handler=
{
thd_mdl_context
};
static struct st_service_ref list_of_services[]=
{
{ "base64_service", VERSION_base64, &base64_handler },
{ "debug_sync_service", VERSION_debug_sync, 0 }, // updated in plugin_init()
{ "encryption_scheme_service", VERSION_encryption_scheme, &encryption_scheme_handler },
{ "encryption_service", VERSION_encryption, &encryption_handler },
{ "logger_service", VERSION_logger, &logger_service_handler },
{ "my_crypt_service", VERSION_my_crypt, &crypt_handler},
{ "my_md5_service", VERSION_my_md5, &my_md5_handler},
{ "my_print_error_service", VERSION_my_print_error, &my_print_error_handler},
{ "my_sha1_service", VERSION_my_sha1, &my_sha1_handler},
{ "my_sha2_service", VERSION_my_sha2, &my_sha2_handler},
2013-01-18 19:04:23 +01:00
{ "my_snprintf_service", VERSION_my_snprintf, &my_snprintf_handler },
{ "progress_report_service", VERSION_progress_report, &progress_report_handler },
{ "thd_alloc_service", VERSION_thd_alloc, &thd_alloc_handler },
{ "thd_autoinc_service", VERSION_thd_autoinc, &thd_autoinc_handler },
{ "thd_error_context_service", VERSION_thd_error_context, &thd_error_context_handler },
{ "thd_kill_statement_service", VERSION_kill_statement, &thd_kill_statement_handler },
2017-03-06 19:34:22 +01:00
{ "thd_rnd_service", VERSION_thd_rnd, &thd_rnd_handler },
{ "thd_specifics_service", VERSION_thd_specifics, &thd_specifics_handler },
{ "thd_timezone_service", VERSION_thd_timezone, &thd_timezone_handler },
{ "thd_wait_service", VERSION_thd_wait, &thd_wait_handler },
{ "wsrep_service", VERSION_wsrep, &wsrep_handler },
MDEV-16678 Prefer MDL to dict_sys.latch for innodb background tasks This is joint work with Thirunarayanan Balathandayuthapani. The MDL interface between InnoDB and the rest of the server (in storage/innobase/dict/dict0dict.cc and in include/) is my work, while most everything else is Thiru's. The collection of InnoDB persistent statistics and the defragmentation were not refactored to use MDL. They will keep relying on lower-level interlocking with fil_check_pending_operations(). The purge of transaction history and the background operations on fulltext indexes will use MDL. We will revert commit 2c4844c9e76427525e8c39a2d72686085efe89c3 (MDEV-17813) because thanks to MDL, purge cannot conflict with DDL operations anymore. For a similar reason, we will remove the MDEV-16222 test case from gcol.innodb_virtual_debug_purge. Purge is essentially replacing all use of the global dict_sys.latch with MDL. Purge will skip the undo log records for tables whose names start with #sql-ib or #sql2. Theoretically, such tables might be renamed back to visible table names if TRUNCATE fails to create a new table, or the final rename in ALTER TABLE...ALGORITHM=COPY fails. In that case, purge could permanently leave some garbage in the table. Such garbage will be tolerated; the table would not be considered corrupted. To avoid repeated MDL releases and acquisitions, trx_purge_attach_undo_recs() will sort undo log records by table_id, and purge_node_t will keep the MDL and table handle open for multiple successive undo log records. get_purge_table(): A new accessor, used during the purge of history for indexed virtual columns. This interface should ideally not exist at all. thd_mdl_context(): Accessor of THD::mdl_context. Wrapped in a new thd_mdl_service. dict_get_db_name_len(): Define inline. dict_acquire_mdl_shared(): Acquire explicit shared MDL on a table name if needed. dict_table_open_on_id(): Return MDL_ticket, if requested. dict_table_close(): Release MDL ticket, if requested. dict_fts_index_syncing(), dict_index_t::index_fts_syncing: Remove. row_drop_table_for_mysql() no longer needs to check these, because MDL guarantees that a fulltext index sync will not be in progress while MDL_EXCLUSIVE is protecting a DDL operation. dict_table_t::parse_name(): Parse the table name for acquiring MDL. purge_node_t::undo_recs: Change the type to std::list<trx_purge_rec_t*> (different container, and storing also roll_ptr). purge_node_t: Add mdl_ticket, last_table_id, purge_thd, mdl_hold_recs for acquiring MDL and for keeping the table open across multiple undo log records. purge_vcol_info_t, row_purge_store_vsec_cur(), row_purge_restore_vsec_cur(): Remove. We will acquire the MDL earlier. purge_sys_t::heap: Added, for reading undo log records. fts_sync_during_ddl(): Invoked during ALGORITHM=INPLACE operations to ensure that fts_sync_table() will not conflict with MDL_EXCLUSIVE. Uses fts_t::sync_message for bookkeeping.
2019-12-10 15:42:50 +02:00
{ "json_service", VERSION_json, &json_handler },
{ "thd_mdl_service", VERSION_thd_mdl, &thd_mdl_handler }
};