mariadb/sql/sql_plugin_services.inl

368 lines
12 KiB
Text
Raw Permalink Normal View History

/* Copyright (c) 2009, 2010, Oracle and/or its affiliates.
2020-09-23 11:32:43 +03:00
Copyright (c) 2012, 2020, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
2019-05-11 22:19:05 +03:00
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */
/* support for Services */
#include <service_versions.h>
#include <mysql/service_wsrep.h>
MDEV-16678 Prefer MDL to dict_sys.latch for innodb background tasks This is joint work with Thirunarayanan Balathandayuthapani. The MDL interface between InnoDB and the rest of the server (in storage/innobase/dict/dict0dict.cc and in include/) is my work, while most everything else is Thiru's. The collection of InnoDB persistent statistics and the defragmentation were not refactored to use MDL. They will keep relying on lower-level interlocking with fil_check_pending_operations(). The purge of transaction history and the background operations on fulltext indexes will use MDL. We will revert commit 2c4844c9e76427525e8c39a2d72686085efe89c3 (MDEV-17813) because thanks to MDL, purge cannot conflict with DDL operations anymore. For a similar reason, we will remove the MDEV-16222 test case from gcol.innodb_virtual_debug_purge. Purge is essentially replacing all use of the global dict_sys.latch with MDL. Purge will skip the undo log records for tables whose names start with #sql-ib or #sql2. Theoretically, such tables might be renamed back to visible table names if TRUNCATE fails to create a new table, or the final rename in ALTER TABLE...ALGORITHM=COPY fails. In that case, purge could permanently leave some garbage in the table. Such garbage will be tolerated; the table would not be considered corrupted. To avoid repeated MDL releases and acquisitions, trx_purge_attach_undo_recs() will sort undo log records by table_id, and purge_node_t will keep the MDL and table handle open for multiple successive undo log records. get_purge_table(): A new accessor, used during the purge of history for indexed virtual columns. This interface should ideally not exist at all. thd_mdl_context(): Accessor of THD::mdl_context. Wrapped in a new thd_mdl_service. dict_get_db_name_len(): Define inline. dict_acquire_mdl_shared(): Acquire explicit shared MDL on a table name if needed. dict_table_open_on_id(): Return MDL_ticket, if requested. dict_table_close(): Release MDL ticket, if requested. dict_fts_index_syncing(), dict_index_t::index_fts_syncing: Remove. row_drop_table_for_mysql() no longer needs to check these, because MDL guarantees that a fulltext index sync will not be in progress while MDL_EXCLUSIVE is protecting a DDL operation. dict_table_t::parse_name(): Parse the table name for acquiring MDL. purge_node_t::undo_recs: Change the type to std::list<trx_purge_rec_t*> (different container, and storing also roll_ptr). purge_node_t: Add mdl_ticket, last_table_id, purge_thd, mdl_hold_recs for acquiring MDL and for keeping the table open across multiple undo log records. purge_vcol_info_t, row_purge_store_vsec_cur(), row_purge_restore_vsec_cur(): Remove. We will acquire the MDL earlier. purge_sys_t::heap: Added, for reading undo log records. fts_sync_during_ddl(): Invoked during ALGORITHM=INPLACE operations to ensure that fts_sync_table() will not conflict with MDL_EXCLUSIVE. Uses fts_t::sync_message for bookkeeping.
2019-12-10 15:42:50 +02:00
#include <mysql/service_thd_mdl.h>
MDEV-33277 In-place upgrade causes invalid AUTO_INCREMENT values MDEV-33308 CHECK TABLE is modifying .frm file even if --read-only As noted in commit d0ef1aaf618452efea5a2e74f6d86ed0a586b63e, MySQL as well as older versions of MariaDB server would during ALTER TABLE ... IMPORT TABLESPACE write bogus values to the PAGE_MAX_TRX_ID field to pages of the clustered index, instead of letting that field remain 0. In commit 8777458a6eb73ac1d7d864ebac390ea7039e21c1 this field was repurposed for PAGE_ROOT_AUTO_INC in the clustered index root page. To avoid trouble when upgrading from MySQL or older versions of MariaDB, we will try to detect and correct bogus values of PAGE_ROOT_AUTO_INC when opening a table for the first time from the SQL layer. btr_read_autoinc_with_fallback(): Add the parameters to mysql_version,max to indicate the TABLE_SHARE::mysql_version of the .frm file and the maximum value allowed for the type of the AUTO_INCREMENT column. In case the table was originally created in MySQL or an older version of MariaDB, read also the maximum value of the AUTO_INCREMENT column from the table and reset the PAGE_ROOT_AUTO_INC if it is above the limit. dict_table_t::get_index(const dict_col_t &) const: Find an index that starts with the specified column. ha_innobase::check_for_upgrade(): Return HA_ADMIN_FAILED if InnoDB needs upgrading but is in read-only mode. In this way, the call to update_frm_version() will be skipped. row_import_autoinc(): Adjust the AUTO_INCREMENT column at the end of ALTER TABLE...IMPORT TABLESPACE. This refinement was suggested by Debarun Banerjee. The changes outside InnoDB were developed by Michael 'Monty' Widenius: Added print_check_msg() service for easy reporting of check/repair messages in ENGINE=Aria and ENGINE=InnoDB. Fixed that CHECK TABLE do not update the .frm file under --read-only. Added 'handler_flags' to HA_CHECK_OPT as a way for storage engines to store state from handler::check_for_upgrade(). Reviewed by: Debarun Banerjee
2024-02-08 10:35:45 +02:00
#include <mysql/service_print_check_msg.h>
struct st_service_ref {
const char *name;
uint version;
void *service;
};
static struct my_snprintf_service_st my_snprintf_handler = {
my_snprintf,
my_vsnprintf
};
static struct thd_alloc_service_st thd_alloc_handler= {
thd_alloc,
thd_calloc,
thd_strdup,
thd_strmake,
thd_memdup,
thd_make_lex_string
};
static struct thd_wait_service_st thd_wait_handler= {
thd_wait_begin,
thd_wait_end
};
Added progress reporting for alter table, LOAD DATA INFILE and for aria tables: check table, repair table, analyze table. - The client gets a progress report message that triggers a callback function if requested with mysql_options(MYSQL_PROGRESS_CALLBACK, function) - Added Progress field last to 'show processlist' - Stage, Max_stage and Progress field added to information_schema.progresslist - The 'mysql' client by defaults enables progress reports when the output is a tty. - Added progress_report_time time variable to configure how often progress reports is sent to client Added read only system variable 'in_transaction' which is 1 if we have executed a BEGIN statement. client/client_priv.h: Added OPT_REPORT_PROGRESS client/mysql.cc: Added option --progress-reports (on by default if not batch mode) Progress reports is written to stdout for long running commands include/Makefile.am: Added mysql/service_progress_report.h include/myisamchk.h: Added variables to be able to do progress reporting in Aria and later in MyISAM include/mysql.h: Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK include/mysql.h.pp: Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK include/mysql/plugin.h: Added functions for reporting progress. include/mysql/plugin_auth.h.pp: Added functions for reporting progress. include/mysql_com.h: Added CLIENT_PROGRESS mysql_real_connect() flag. include/sql_common.h: Added callback function for reporting progress mysql-test/r/old-mode.result: Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode. mysql-test/suite/funcs_1/datadict/datadict_priv.inc: Added new column mysql-test/suite/funcs_1/datadict/processlist_priv.inc: Test all new PROCESSLIST columns mysql-test/suite/funcs_1/r/is_columns_is.result: Updated results mysql-test/suite/funcs_1/r/is_columns_is_embedded.result: Updated results mysql-test/suite/funcs_1/r/is_columns_mysql_embedded.result: Updated results mysql-test/suite/funcs_1/r/is_tables_is_embedded.result: Updated results mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result: Updated results mysql-test/suite/funcs_1/r/processlist_priv_ps.result: Updated results mysql-test/suite/funcs_1/r/processlist_val_no_prot.result: Updated results mysql-test/suite/funcs_1/r/processlist_val_ps.result: Updated results mysql-test/suite/pbxt/r/pbxt_locking.result: Updated results mysql-test/suite/pbxt/r/skip_name_resolve.result: Updated results mysql-test/t/old-mode.test: Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode. plugin/handler_socket/handlersocket/Makefile.am: Added -lmysqlservices scripts/mytop.sh: Made 'State' field width dynamic. Added 'Progress' to process list display. sql-common/client.c: Added handling of progress messages. Removed check_license() function. sql/mysql_priv.h: Added opt_progress_report_time sql/mysqld.cc: Added progress_report_time time variable to configure how often progress reports is sent to client sql/protocol.cc: Added net_send_progress_packet() sql/protocol.h: New prototypes sql/set_var.cc: Added variables progress_report_time and in_transaction sql/sql_acl.cc: Safety fix: Made client_capabilities ulonglong sql/sql_class.cc: Added interface functions for progress reporting sql/sql_class.h: Added varibles in THD for progress reporting. Added CF_REPORT_PROGRESS sql/sql_load.cc: Added progress reporting for LOAD DATA INFILE sql/sql_parse.cc: Added CF_REPORT_PROGRESS for top level commands for which it's safe to send progress reports to client sql/sql_show.cc: Added Progress field last to 'show processlist' Stage, Max_stage and Progress field added to information_schema.progresslist sql/sql_table.cc: Added progress reporting for ALTER TABLE Added THD as argument to copy_data_between_tables() storage/maria/ha_maria.cc: Added progress reporting for check table, repair table, analyze table Fixed a bug in start_bulk_insert() that caused alter table to always run with all keys enabled. storage/maria/ma_check.c: Added progress reporting Remember old state before starting repair. This removes some warnings from optimize_table if create-with-sort fails. storage/maria/ma_check_standalone.h: Added dummy reporting function for standalone Aria programs. storage/maria/ma_sort.c: Added progress reporting storage/maria/maria_chk.c: Updated version storage/maria/maria_def.h: Added new prototypes tests/mysql_client_test.c: Added test case for progress reporting
2011-07-01 15:08:30 +03:00
static struct progress_report_service_st progress_report_handler= {
thd_progress_init,
thd_progress_report,
thd_progress_next_stage,
thd_progress_end,
set_thd_proc_info
};
2013-01-18 19:04:23 +01:00
static struct kill_statement_service_st thd_kill_statement_handler= {
thd_kill_level
};
static struct thd_timezone_service_st thd_timezone_handler= {
thd_TIME_to_gmt_sec,
thd_gmt_sec_to_TIME
};
2017-03-06 13:06:03 +01:00
static struct my_sha2_service_st my_sha2_handler = {
my_sha224,
my_sha224_multi,
my_sha224_context_size,
my_sha224_init,
my_sha224_input,
my_sha224_result,
my_sha256,
my_sha256_multi,
my_sha256_context_size,
my_sha256_init,
my_sha256_input,
my_sha256_result,
my_sha384,
my_sha384_multi,
my_sha384_context_size,
my_sha384_init,
my_sha384_input,
my_sha384_result,
my_sha512,
my_sha512_multi,
my_sha512_context_size,
my_sha512_init,
my_sha512_input,
my_sha512_result,
};
static struct my_sha1_service_st my_sha1_handler = {
my_sha1,
my_sha1_multi,
my_sha1_context_size,
my_sha1_init,
my_sha1_input,
my_sha1_result
};
2014-09-26 20:03:20 +02:00
static struct my_md5_service_st my_md5_handler = {
my_md5,
my_md5_multi,
my_md5_context_size,
my_md5_init,
my_md5_input,
my_md5_result
};
static struct logger_service_st logger_service_handler= {
logger_init_mutexes,
logger_open,
logger_close,
logger_vprintf,
logger_printf,
logger_write,
logger_rotate
};
static struct thd_autoinc_service_st thd_autoinc_handler= {
thd_get_autoinc
};
2017-03-06 19:34:22 +01:00
static struct thd_rnd_service_st thd_rnd_handler= {
thd_rnd,
thd_create_random_password
};
2017-03-06 17:05:03 +01:00
static struct base64_service_st base64_handler= {
2017-03-30 12:48:42 +02:00
my_base64_needed_encoded_length,
my_base64_encode_max_arg_length,
my_base64_needed_decoded_length,
my_base64_decode_max_arg_length,
my_base64_encode,
my_base64_decode
2017-03-06 17:05:03 +01:00
};
static struct thd_error_context_service_st thd_error_context_handler= {
thd_get_error_message,
thd_get_error_number,
thd_get_error_row,
thd_inc_error_row,
thd_get_error_context_description
};
static struct wsrep_service_st wsrep_handler = {
2015-07-14 16:05:29 -04:00
get_wsrep_recovery,
wsrep_consistency_check,
wsrep_is_wsrep_xid,
wsrep_xid_seqno,
wsrep_xid_uuid,
wsrep_on,
2019-01-23 15:30:00 +04:00
wsrep_prepare_key_for_innodb,
wsrep_thd_LOCK,
MDEV-29293 MariaDB stuck on starting commit state This commit contains a merge from 10.5-MDEV-29293-squash into 10.6. Although the bug MDEV-29293 was not reproducible with 10.6, the fix contains several improvements for wsrep KILL query and BF abort handling, and addresses the following issues: * MDEV-30307 KILL command issued inside a transaction is problematic for galera replication: This commit will remove KILL TOI replication, so Galera side transaction context is not lost during KILL. * MDEV-21075 KILL QUERY maintains nodes data consistency but breaks GTID sequence: This is fixed as well as KILL does not use TOI, and thus does not change GTID state. * MDEV-30372 Assertion in wsrep-lib state: This was caused by BF abort or KILL when local transaction was in the middle of group commit. This commit disables THD::killed handling during commit, so the problem is avoided. * MDEV-30963 Assertion failure !lock.was_chosen_as_deadlock_victim in trx0trx.h:1065: The assertion happened when the victim was BF aborted via MDL while it was committing. This commit changes MDL BF aborts so that transactions which are committing cannot be BF aborted via MDL. The RQG grammar attached in the issue could not reproduce the crash anymore. Original commit message from 10.5 fix: MDEV-29293 MariaDB stuck on starting commit state The problem seems to be a deadlock between KILL command execution and BF abort issued by an applier, where: * KILL has locked victim's LOCK_thd_kill and LOCK_thd_data. * Applier has innodb side global lock mutex and victim trx mutex. * KILL is calling innobase_kill_query, and is blocked by innodb global lock mutex. * Applier is in wsrep_innobase_kill_one_trx and is blocked by victim's LOCK_thd_kill. The fix in this commit removes the TOI replication of KILL command and makes KILL execution less intrusive operation. Aborting the victim happens now by using awake_no_mutex() and ha_abort_transaction(). If the KILL happens when the transaction is committing, the KILL operation is postponed to happen after the statement has completed in order to avoid KILL to interrupt commit processing. Notable changes in this commit: * wsrep client connections's error state may remain sticky after client connection is closed. This error message will then pop up for the next client session issuing first SQL statement. This problem raised with test galera.galera_bf_kill. The fix is to reset wsrep client error state, before a THD is reused for next connetion. * Release THD locks in wsrep_abort_transaction when locking innodb mutexes. This guarantees same locking order as with applier BF aborting. * BF abort from MDL was changed to do BF abort on server/wsrep-lib side first, and only then do the BF abort on InnoDB side. This removes the need to call back from InnoDB for BF aborts which originate from MDL and simplifies the locking. * Removed wsrep_thd_set_wsrep_aborter() from service_wsrep.h. The manipulation of the wsrep_aborter can be done solely on server side. Moreover, it is now debug only variable and could be excluded from optimized builds. * Remove LOCK_thd_kill from wsrep_thd_LOCK/UNLOCK to allow more fine grained locking for SR BF abort which may require locking of victim LOCK_thd_kill. Added explicit call for wsrep_thd_kill_LOCK/UNLOCK where appropriate. * Wsrep-lib was updated to version which allows external locking for BF abort calls. Changes to MTR tests: * Disable galera_bf_abort_group_commit. This test is going to be removed (MDEV-30855). * Make galera_var_retry_autocommit result more readable by echoing cases and expectations into result. Only one expected result for reap to verify that server returns expected status for query. * Record galera_gcache_recover_manytrx as result file was incomplete. Trivial change. * Make galera_create_table_as_select more deterministic: Wait until CTAS execution has reached MDL wait for multi-master conflict case. Expected error from multi-master conflict is ER_QUERY_INTERRUPTED. This is because CTAS does not yet have open wsrep transaction when it is waiting for MDL, query gets interrupted instead of BF aborted. This should be addressed in separate task. * A new test galera_bf_abort_registering to check that registering trx gets BF aborted through MDL. * A new test galera_kill_group_commit to verify correct behavior when KILL is executed while the transaction is committing. Co-authored-by: Seppo Jaakola <seppo.jaakola@iki.fi> Co-authored-by: Jan Lindström <jan.lindstrom@galeracluster.com> Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-04-19 16:51:55 +03:00
wsrep_thd_TRYLOCK,
wsrep_thd_UNLOCK,
wsrep_thd_query,
wsrep_thd_retry_counter,
wsrep_thd_ignore_table,
wsrep_thd_trx_seqno,
2019-01-23 15:30:00 +04:00
wsrep_thd_is_aborting,
wsrep_set_data_home_dir,
wsrep_thd_is_BF,
wsrep_thd_is_local,
wsrep_thd_self_abort,
wsrep_thd_append_key,
wsrep_thd_append_table_key,
wsrep_thd_is_local_transaction,
2019-01-23 15:30:00 +04:00
wsrep_thd_client_state_str,
wsrep_thd_client_mode_str,
wsrep_thd_transaction_state_str,
wsrep_thd_transaction_id,
wsrep_thd_bf_abort,
wsrep_thd_order_before,
wsrep_handle_SR_rollback,
wsrep_thd_skip_locking,
wsrep_get_sr_table_name,
10.4 wsrep group commit fixes (#1224) * MDEV-16509 Improve wsrep commit performance with binlog disabled Release commit order critical section early after trx_commit_low() if binlog is not transaction coordinator. In order to avoid two phase commit, binlog_hton is not registered for THD during IO_CACHE population. Implemented a test which verifies that the transactions release commit order early. This optimization will change behavior during recovery as the commit is not two phase when binlog is off. Fixed and recorded wsrep-recover-v25 and wsrep-recover to match the behavior. * MDEV-18730 Ordering for wsrep binlog group commit Previously out of order execution was allowed for wsrep commits. Established proper ordering by populating wait_for_commit for every wsrep THD and making group commit leader to wait for prior commits before proceeding to trx_group_commit_leader(). * MDEV-18730 Added a test case to verify correct commit ordering * MDEV-16509, MDEV-18730 Review fixes Use WSREP_EMULATE_BINLOG() macro to decide if the binlog_hton should be registered. Whitespace/syntax fixes and cleanups. * MDEV-16509 Require binlog for galera_var_innodb_disallow_writes test If the commit to InnoDB is done in one phase, the native InnoDB behavior is that the transaction is committed in memory before it is persisted to disk. This means that the innodb_disallow_writes=ON may not prevent transaction to become visible to other readers before commit is completely over. On the other hand, if the commit is two phase (as it is with binlog), the transaction will be blocked in prepare phase. Fixed the test to use binlog, which enforces two phase commit, which in turn makes commit to block before the changes become visible to other connections. This guarantees that the test produces expected result.
2019-03-15 07:09:13 +02:00
wsrep_get_debug,
wsrep_commit_ordered,
wsrep_thd_is_applying,
wsrep_OSU_method_get,
wsrep_thd_has_ignored_error,
MDEV-21910 Deadlock between BF abort and manual KILL command When high priority replication slave applier encounters lock conflict in innodb, it will force the conflicting lock holder transaction (victim) to rollback. This is a must in multi-master sychronous replication model to avoid cluster lock-up. This high priority victim abort (aka "brute force" (BF) abort), is started from innodb lock manager while holding the victim's transaction's (trx) mutex. Depending on the execution state of the victim transaction, it may happen that the BF abort will call for THD::awake() to wake up the victim transaction for the rollback. Now, if BF abort requires THD::awake() to be called, then the applier thread executed locking protocol of: victim trx mutex -> victim THD::LOCK_thd_data If, at the same time another DBMS super user issues KILL command to abort the same victim, it will execute locking protocol of: victim THD::LOCK_thd_data -> victim trx mutex. These two locking protocol acquire mutexes in opposite order, hence unresolvable mutex locking deadlock may occur. The fix in this commit adds THD::wsrep_aborter flag to synchronize who can kill the victim This flag is set both when BF is called for from innodb and by KILL command. Either path of victim killing will bail out if victim's wsrep_killed is already set to avoid mutex conflicts with the other aborter execution. THD::wsrep_aborter records the aborter THD's ID. This is needed to preserve the right to kill the victim from different locations for the same aborter thread. It is also good error logging, to see who is reponsible for the abort. A new test case was added in galera.galera_bf_kill_debug.test for scenario where wsrep applier thread and manual KILL command try to kill same idle victim
2020-05-19 11:12:26 +03:00
wsrep_thd_set_ignored_error,
wsrep_report_bf_lock_wait,
wsrep_thd_kill_LOCK,
wsrep_thd_kill_UNLOCK,
wsrep_thd_set_PA_unsafe,
wsrep_get_domain_id
};
static struct thd_specifics_service_st thd_specifics_handler=
{
thd_key_create,
thd_key_delete,
thd_getspecific,
thd_setspecific
};
static struct encryption_scheme_service_st encryption_scheme_handler=
{
encryption_scheme_encrypt,
encryption_scheme_decrypt
};
static struct my_crypt_service_st crypt_handler=
{
my_aes_crypt_init,
my_aes_crypt_update,
my_aes_crypt_finish,
my_aes_crypt,
my_aes_get_size,
my_aes_ctx_size,
my_random_bytes
};
static struct my_print_error_service_st my_print_error_handler=
{
my_error,
my_printf_error,
my_printv_error
};
MDEV-33277 In-place upgrade causes invalid AUTO_INCREMENT values MDEV-33308 CHECK TABLE is modifying .frm file even if --read-only As noted in commit d0ef1aaf618452efea5a2e74f6d86ed0a586b63e, MySQL as well as older versions of MariaDB server would during ALTER TABLE ... IMPORT TABLESPACE write bogus values to the PAGE_MAX_TRX_ID field to pages of the clustered index, instead of letting that field remain 0. In commit 8777458a6eb73ac1d7d864ebac390ea7039e21c1 this field was repurposed for PAGE_ROOT_AUTO_INC in the clustered index root page. To avoid trouble when upgrading from MySQL or older versions of MariaDB, we will try to detect and correct bogus values of PAGE_ROOT_AUTO_INC when opening a table for the first time from the SQL layer. btr_read_autoinc_with_fallback(): Add the parameters to mysql_version,max to indicate the TABLE_SHARE::mysql_version of the .frm file and the maximum value allowed for the type of the AUTO_INCREMENT column. In case the table was originally created in MySQL or an older version of MariaDB, read also the maximum value of the AUTO_INCREMENT column from the table and reset the PAGE_ROOT_AUTO_INC if it is above the limit. dict_table_t::get_index(const dict_col_t &) const: Find an index that starts with the specified column. ha_innobase::check_for_upgrade(): Return HA_ADMIN_FAILED if InnoDB needs upgrading but is in read-only mode. In this way, the call to update_frm_version() will be skipped. row_import_autoinc(): Adjust the AUTO_INCREMENT column at the end of ALTER TABLE...IMPORT TABLESPACE. This refinement was suggested by Debarun Banerjee. The changes outside InnoDB were developed by Michael 'Monty' Widenius: Added print_check_msg() service for easy reporting of check/repair messages in ENGINE=Aria and ENGINE=InnoDB. Fixed that CHECK TABLE do not update the .frm file under --read-only. Added 'handler_flags' to HA_CHECK_OPT as a way for storage engines to store state from handler::check_for_upgrade(). Reviewed by: Debarun Banerjee
2024-02-08 10:35:45 +02:00
static struct print_check_msg_service_st print_check_msg_handler=
{
print_check_msg
};
static struct json_service_st json_handler=
{
json_type,
json_get_array_item,
json_get_object_key,
json_get_object_nkey,
json_escape_string,
json_unescape_json
};
struct sql_service_st sql_service_handler=
{
mysql_init,
mysql_real_connect_local,
mysql_real_connect,
mysql_errno,
mysql_error,
mysql_real_query,
mysql_affected_rows,
mysql_num_rows,
mysql_store_result,
mysql_free_result,
mysql_fetch_row,
mysql_close,
2022-11-02 19:47:23 +01:00
mysql_options,
mysql_fetch_lengths,
mysql_set_character_set,
mysql_num_fields,
mysql_select_db,
mysql_ssl_set
};
MDEV-16678 Prefer MDL to dict_sys.latch for innodb background tasks This is joint work with Thirunarayanan Balathandayuthapani. The MDL interface between InnoDB and the rest of the server (in storage/innobase/dict/dict0dict.cc and in include/) is my work, while most everything else is Thiru's. The collection of InnoDB persistent statistics and the defragmentation were not refactored to use MDL. They will keep relying on lower-level interlocking with fil_check_pending_operations(). The purge of transaction history and the background operations on fulltext indexes will use MDL. We will revert commit 2c4844c9e76427525e8c39a2d72686085efe89c3 (MDEV-17813) because thanks to MDL, purge cannot conflict with DDL operations anymore. For a similar reason, we will remove the MDEV-16222 test case from gcol.innodb_virtual_debug_purge. Purge is essentially replacing all use of the global dict_sys.latch with MDL. Purge will skip the undo log records for tables whose names start with #sql-ib or #sql2. Theoretically, such tables might be renamed back to visible table names if TRUNCATE fails to create a new table, or the final rename in ALTER TABLE...ALGORITHM=COPY fails. In that case, purge could permanently leave some garbage in the table. Such garbage will be tolerated; the table would not be considered corrupted. To avoid repeated MDL releases and acquisitions, trx_purge_attach_undo_recs() will sort undo log records by table_id, and purge_node_t will keep the MDL and table handle open for multiple successive undo log records. get_purge_table(): A new accessor, used during the purge of history for indexed virtual columns. This interface should ideally not exist at all. thd_mdl_context(): Accessor of THD::mdl_context. Wrapped in a new thd_mdl_service. dict_get_db_name_len(): Define inline. dict_acquire_mdl_shared(): Acquire explicit shared MDL on a table name if needed. dict_table_open_on_id(): Return MDL_ticket, if requested. dict_table_close(): Release MDL ticket, if requested. dict_fts_index_syncing(), dict_index_t::index_fts_syncing: Remove. row_drop_table_for_mysql() no longer needs to check these, because MDL guarantees that a fulltext index sync will not be in progress while MDL_EXCLUSIVE is protecting a DDL operation. dict_table_t::parse_name(): Parse the table name for acquiring MDL. purge_node_t::undo_recs: Change the type to std::list<trx_purge_rec_t*> (different container, and storing also roll_ptr). purge_node_t: Add mdl_ticket, last_table_id, purge_thd, mdl_hold_recs for acquiring MDL and for keeping the table open across multiple undo log records. purge_vcol_info_t, row_purge_store_vsec_cur(), row_purge_restore_vsec_cur(): Remove. We will acquire the MDL earlier. purge_sys_t::heap: Added, for reading undo log records. fts_sync_during_ddl(): Invoked during ALGORITHM=INPLACE operations to ensure that fts_sync_table() will not conflict with MDL_EXCLUSIVE. Uses fts_t::sync_message for bookkeeping.
2019-12-10 15:42:50 +02:00
static struct thd_mdl_service_st thd_mdl_handler=
{
thd_mdl_context
};
#define DEFINE_warning_function(name, ret) { \
static query_id_t last_query_id= -1; \
THD *thd= current_thd; \
if((thd ? thd->query_id : 0) != last_query_id) \
{ \
my_error(ER_PROVIDER_NOT_LOADED, MYF(ME_ERROR_LOG|ME_WARNING), name); \
last_query_id= thd ? thd->query_id : 0; \
} \
return ret; \
}
#include <providers/lzma.h>
static struct provider_service_lzma_st provider_handler_lzma=
{
DEFINE_lzma_stream_buffer_decode([]) DEFINE_warning_function("LZMA compression", LZMA_PROG_ERROR),
DEFINE_lzma_easy_buffer_encode([]) DEFINE_warning_function("LZMA compression", LZMA_PROG_ERROR),
false // .is_loaded
};
struct provider_service_lzma_st *provider_service_lzma= &provider_handler_lzma;
#include <providers/lzo/lzo1x.h>
static struct provider_service_lzo_st provider_handler_lzo=
{
DEFINE_lzo1x_1_15_compress([]) DEFINE_warning_function("LZO compression", LZO_E_INTERNAL_ERROR),
DEFINE_lzo1x_decompress_safe([]) DEFINE_warning_function("LZO compression", LZO_E_INTERNAL_ERROR),
false // .is_loaded
};
struct provider_service_lzo_st *provider_service_lzo= &provider_handler_lzo;
#include <providers/bzlib.h>
static struct provider_service_bzip2_st provider_handler_bzip2=
{
DEFINE_BZ2_bzBuffToBuffCompress([]) DEFINE_warning_function("BZip2 compression", -1),
DEFINE_BZ2_bzBuffToBuffDecompress([]) DEFINE_warning_function("BZip2 compression", -1),
DEFINE_BZ2_bzCompress([]) DEFINE_warning_function("BZip2 compression", -1),
DEFINE_BZ2_bzCompressEnd([]) DEFINE_warning_function("BZip2 compression", -1),
DEFINE_BZ2_bzCompressInit([]) DEFINE_warning_function("BZip2 compression", -1),
DEFINE_BZ2_bzDecompress([]) DEFINE_warning_function("BZip2 compression", -1),
DEFINE_BZ2_bzDecompressEnd([]) DEFINE_warning_function("BZip2 compression", -1),
DEFINE_BZ2_bzDecompressInit([]) DEFINE_warning_function("BZip2 compression", -1),
false // .is_loaded
};
struct provider_service_bzip2_st *provider_service_bzip2= &provider_handler_bzip2;
#include <providers/snappy-c.h>
static struct provider_service_snappy_st provider_handler_snappy=
{
DEFINE_snappy_max_compressed_length([]) -> size_t DEFINE_warning_function("Snappy compression", 0),
DEFINE_snappy_compress([]) DEFINE_warning_function("Snappy compression", SNAPPY_INVALID_INPUT),
DEFINE_snappy_uncompressed_length([]) DEFINE_warning_function("Snappy compression", SNAPPY_INVALID_INPUT),
DEFINE_snappy_uncompress([]) DEFINE_warning_function("Snappy compression", SNAPPY_INVALID_INPUT),
false // .is_loaded
};
struct provider_service_snappy_st *provider_service_snappy= &provider_handler_snappy;
#include <providers/lz4.h>
static struct provider_service_lz4_st provider_handler_lz4=
{
DEFINE_LZ4_compressBound([]) DEFINE_warning_function("LZ4 compression", 0),
DEFINE_LZ4_compress_default([]) DEFINE_warning_function("LZ4 compression", 0),
DEFINE_LZ4_decompress_safe([]) DEFINE_warning_function("LZ4 compression", -1),
false // .is_loaded
};
struct provider_service_lz4_st *provider_service_lz4= &provider_handler_lz4;
static struct st_service_ref list_of_services[]=
{
{ "base64_service", VERSION_base64, &base64_handler },
{ "debug_sync_service", VERSION_debug_sync, 0 }, // updated in plugin_init()
{ "encryption_scheme_service", VERSION_encryption_scheme, &encryption_scheme_handler },
{ "encryption_service", VERSION_encryption, &encryption_handler },
{ "logger_service", VERSION_logger, &logger_service_handler },
{ "my_crypt_service", VERSION_my_crypt, &crypt_handler},
{ "my_md5_service", VERSION_my_md5, &my_md5_handler},
{ "my_print_error_service", VERSION_my_print_error, &my_print_error_handler},
MDEV-33277 In-place upgrade causes invalid AUTO_INCREMENT values MDEV-33308 CHECK TABLE is modifying .frm file even if --read-only As noted in commit d0ef1aaf618452efea5a2e74f6d86ed0a586b63e, MySQL as well as older versions of MariaDB server would during ALTER TABLE ... IMPORT TABLESPACE write bogus values to the PAGE_MAX_TRX_ID field to pages of the clustered index, instead of letting that field remain 0. In commit 8777458a6eb73ac1d7d864ebac390ea7039e21c1 this field was repurposed for PAGE_ROOT_AUTO_INC in the clustered index root page. To avoid trouble when upgrading from MySQL or older versions of MariaDB, we will try to detect and correct bogus values of PAGE_ROOT_AUTO_INC when opening a table for the first time from the SQL layer. btr_read_autoinc_with_fallback(): Add the parameters to mysql_version,max to indicate the TABLE_SHARE::mysql_version of the .frm file and the maximum value allowed for the type of the AUTO_INCREMENT column. In case the table was originally created in MySQL or an older version of MariaDB, read also the maximum value of the AUTO_INCREMENT column from the table and reset the PAGE_ROOT_AUTO_INC if it is above the limit. dict_table_t::get_index(const dict_col_t &) const: Find an index that starts with the specified column. ha_innobase::check_for_upgrade(): Return HA_ADMIN_FAILED if InnoDB needs upgrading but is in read-only mode. In this way, the call to update_frm_version() will be skipped. row_import_autoinc(): Adjust the AUTO_INCREMENT column at the end of ALTER TABLE...IMPORT TABLESPACE. This refinement was suggested by Debarun Banerjee. The changes outside InnoDB were developed by Michael 'Monty' Widenius: Added print_check_msg() service for easy reporting of check/repair messages in ENGINE=Aria and ENGINE=InnoDB. Fixed that CHECK TABLE do not update the .frm file under --read-only. Added 'handler_flags' to HA_CHECK_OPT as a way for storage engines to store state from handler::check_for_upgrade(). Reviewed by: Debarun Banerjee
2024-02-08 10:35:45 +02:00
{ "print_check_msg_service", VERSION_print_check_msg, &print_check_msg_handler},
{ "my_sha1_service", VERSION_my_sha1, &my_sha1_handler},
{ "my_sha2_service", VERSION_my_sha2, &my_sha2_handler},
2013-01-18 19:04:23 +01:00
{ "my_snprintf_service", VERSION_my_snprintf, &my_snprintf_handler },
{ "progress_report_service", VERSION_progress_report, &progress_report_handler },
{ "thd_alloc_service", VERSION_thd_alloc, &thd_alloc_handler },
{ "thd_autoinc_service", VERSION_thd_autoinc, &thd_autoinc_handler },
{ "thd_error_context_service", VERSION_thd_error_context, &thd_error_context_handler },
{ "thd_kill_statement_service", VERSION_kill_statement, &thd_kill_statement_handler },
2017-03-06 19:34:22 +01:00
{ "thd_rnd_service", VERSION_thd_rnd, &thd_rnd_handler },
{ "thd_specifics_service", VERSION_thd_specifics, &thd_specifics_handler },
{ "thd_timezone_service", VERSION_thd_timezone, &thd_timezone_handler },
{ "thd_wait_service", VERSION_thd_wait, &thd_wait_handler },
{ "wsrep_service", VERSION_wsrep, &wsrep_handler },
MDEV-16678 Prefer MDL to dict_sys.latch for innodb background tasks This is joint work with Thirunarayanan Balathandayuthapani. The MDL interface between InnoDB and the rest of the server (in storage/innobase/dict/dict0dict.cc and in include/) is my work, while most everything else is Thiru's. The collection of InnoDB persistent statistics and the defragmentation were not refactored to use MDL. They will keep relying on lower-level interlocking with fil_check_pending_operations(). The purge of transaction history and the background operations on fulltext indexes will use MDL. We will revert commit 2c4844c9e76427525e8c39a2d72686085efe89c3 (MDEV-17813) because thanks to MDL, purge cannot conflict with DDL operations anymore. For a similar reason, we will remove the MDEV-16222 test case from gcol.innodb_virtual_debug_purge. Purge is essentially replacing all use of the global dict_sys.latch with MDL. Purge will skip the undo log records for tables whose names start with #sql-ib or #sql2. Theoretically, such tables might be renamed back to visible table names if TRUNCATE fails to create a new table, or the final rename in ALTER TABLE...ALGORITHM=COPY fails. In that case, purge could permanently leave some garbage in the table. Such garbage will be tolerated; the table would not be considered corrupted. To avoid repeated MDL releases and acquisitions, trx_purge_attach_undo_recs() will sort undo log records by table_id, and purge_node_t will keep the MDL and table handle open for multiple successive undo log records. get_purge_table(): A new accessor, used during the purge of history for indexed virtual columns. This interface should ideally not exist at all. thd_mdl_context(): Accessor of THD::mdl_context. Wrapped in a new thd_mdl_service. dict_get_db_name_len(): Define inline. dict_acquire_mdl_shared(): Acquire explicit shared MDL on a table name if needed. dict_table_open_on_id(): Return MDL_ticket, if requested. dict_table_close(): Release MDL ticket, if requested. dict_fts_index_syncing(), dict_index_t::index_fts_syncing: Remove. row_drop_table_for_mysql() no longer needs to check these, because MDL guarantees that a fulltext index sync will not be in progress while MDL_EXCLUSIVE is protecting a DDL operation. dict_table_t::parse_name(): Parse the table name for acquiring MDL. purge_node_t::undo_recs: Change the type to std::list<trx_purge_rec_t*> (different container, and storing also roll_ptr). purge_node_t: Add mdl_ticket, last_table_id, purge_thd, mdl_hold_recs for acquiring MDL and for keeping the table open across multiple undo log records. purge_vcol_info_t, row_purge_store_vsec_cur(), row_purge_restore_vsec_cur(): Remove. We will acquire the MDL earlier. purge_sys_t::heap: Added, for reading undo log records. fts_sync_during_ddl(): Invoked during ALGORITHM=INPLACE operations to ensure that fts_sync_table() will not conflict with MDL_EXCLUSIVE. Uses fts_t::sync_message for bookkeeping.
2019-12-10 15:42:50 +02:00
{ "json_service", VERSION_json, &json_handler },
{ "sql_service", VERSION_sql_service, &sql_service_handler },
2023-12-17 16:56:38 +01:00
{ "thd_mdl_service", VERSION_thd_mdl, &thd_mdl_handler },
{ "provider_service_bzip2", VERSION_provider_bzip2, &provider_handler_bzip2 },
{ "provider_service_lz4", VERSION_provider_lz4, &provider_handler_lz4 },
{ "provider_service_lzma", VERSION_provider_lzma, &provider_handler_lzma },
{ "provider_service_lzo", VERSION_provider_lzo, &provider_handler_lzo },
{ "provider_service_snappy", VERSION_provider_snappy, &provider_handler_snappy }
};