ha_innobase::extra(): Conditionally avoid a log write that had been
added in commit e5b9dc1536 (MDEV-25910)
because it may be invoked as part of select_insert::prepare_eof()
and not only during DDL operations.
Reviewed by: Sergei Golubchik
At the start of mariadb-backup --backup, trigger a flush of the
InnoDB buffer pool, so that as little log as possible will have
to be copied.
The previously debug-build-only interface
SET GLOBAL innodb_log_checkpoint_now=ON;
will be made available on all builds, and
mariadb-backup --backup will invoke it, unless the option
--skip-innodb-log-checkpoint-now is specified.
Reviewed by: Vladislav Vaintroub
If connect engineis not able to allocate connect_work_space memory for
GetUser() it will call free() twice with the same value (g).
g was freed first in user_connect::user_init() which calls PlugExit() on
errors and then again in ~user_connect() which also calls PlugExit().
Fixed by setting g to 0 in user_init() after calling PlugExit()
This code was tested 'by hand' by setting connect.work_space=600G
Other things:
- Removed some very old not relevant comments in touched code
- Added comments to clarify how some memory was freed
- Fixed indentation in changed functions.
Some thing causes the aria_log_control file to be larger than the expected
52 bytes. The control file has the correct information but somehow it
is filled up with ox00 bytes up to 512 bytes.
This could have happened in case of a file system crash that enlarged
the file to the sector boundary.
Fixed that aria will ignore bytes outside of it's expected
Other things:
- Fixed wrong DBUG_ASSERT() in my_malloc_size_cb_func() that could cause
crashes in debug binaries during Aria recovery.
In commit 6e6a1b316c (MDEV-35000)
a race condition was exposed.
ha_innobase::check_if_incompatible_data(): If the statistics have
already been initialized for the table, skip the invocation of
innobase_copy_frm_flags_from_create_info() in order to avoid
unexpectedly ruining things for other threads that are concurrently
accessing the table.
dict_stats_save(): Add debug instrumentation that is necessary for
reproducing the interlocking of the failure scenario.
stats_deinit(): Replaces dict_stats_deinit().
Deinitialize the statistics for persistent tables,
so that they will be reloaded or recalculated
on a subsequent ha_innobase::open().
ha_innobase::rename_table(): Invoke stats_deinit() so that the
subsequent ha_innobase::open() will reload the InnoDB persistent
statistics. That is, it will remain possible to have the InnoDB
persistent statistics reloaded by executing the following:
RENAME TABLE t TO tmp, tmp TO t;
dict_table_close(table): Replaced with table->release().
There will no longer be any logic that would attempt to ensure
that the InnoDB persistent statistics will be reloaded after
FLUSH TABLES has been executed. This also fixes the problem that
dict_table_t::stat_modified_counter would be frequently reset to 0,
whenever ha_innobase::open() is invoked after the table reference
count had dropped to 0.
dict_table_close(table, thd, mdl): Remove the parameter "dict_locked".
Do not try to invalidate the statistics.
ha_innobase::statistics_init(): Replaces dict_stats_init(table).
Reviewed by: Thirunarayanan Balathandayuthapani
innodb_stats_transient_sample_pages, innodb_stats_persistent_sample_pages:
Change the type to UNSIGNED, because the number of pages in a table
is limited to 32 bits by the InnoDB file format.
btr_get_size_and_reserved(), fseg_get_n_frag_pages(),
fseg_n_reserved_pages_low(), fseg_n_reserved_pages(): Return uint32_t.
The file format limits page numbers to 32 bits.
dict_table_t::stat: An Atomic_relaxed<uint32_t> that combines a
number of metadata fields.
innodb_copy_stat_flags(): Copy the statistics flags from
TABLE_SHARE or HA_CREATE_INFO.
dict_table_t::stats_initialized(), dict_table_t::stats_is_persistent():
Accessors to dict_table_t::stat.
Reviewed by: Thirunarayanan Balathandayuthapani
Writing the redo log resized will write uninitialized data. There is
a MEM_MAKE_DEFINED construct in the code to bless this however it was
correct on the initial length, but not the changed length.
The MEM_MAKE_DEFINED is moved earlier in the code where the length
contains the correct value.
In MDEV-26345 77ed235d50 a bitmap is
introduced to skip spider GBH SELECTed constant fields when storing
the results from the data node. Unfortunately this bitmap was not used
in all applicable calls. This patch fixes it. The test covers most of
the calls, with the exception of
spider_db_store_result_for_reuse_cursor(), which is not covered in
existing tests, because it is only called when limit_mode()==1, which
is not the case for any spider backend wrapper.
recv_sys_t::parse(): Allocate decrypt_buf also for storing==BACKUP
but limit its size to correspond to 1 byte of record payload.
Ensure that last_offset=0 for storing==BACKUP.
When parsing INIT_PAGE or FREE_PAGE, avoid an unnecessary
l.copy_if_needed() for storing!=YES.
When parsing EXTENDED in storing==BACKUP, properly invoke
l.copy_if_needed() on a large enough decrypt_buf.
When parsing WRITE, MEMMOVE, MEMSET in storing==BACKUP,
skip further validation (and potential overflow of the tiny decrypt_buf),
like we used to do before commit 46aaf328ce
(MDEV-35830).
Reviewed by: Debarun Banerjee
A spider_conn may outlive its associated ha_spider (in the field
queued_ping_spider) used for connecting to and pinging the data
node (a call to spider_db_ping(), guarded by the boolean field
queued_ping). In a call to ha_spider::close() (which is often preceded
with the deletion of the ha_spider itself), many cleanups happen,
including freeing the associated spider_share, which is used by the
spider_conn in spider_db_ping. Therefore it is necessary to reset both
the queued_ping_spider and queued_ping fields, so that any further
spider interaction with the data node will not trigger the call using
the ha_spider including its freed spider_share.
Also out of caution added an assert and internal error in case a
connection has not been established (the db_conn field of type
MYSQL * is NULL), and attempt to connect is skipped because both
queued_connect and queued_ping are false. Note that this unlikely (if
not impossible) scenario would not be a regression caused by this
change, as it strictly falls under the scenario of this bug.
mysql_alter_table(): Consider ha_sequence::storage_ht() when determining
if the storage engine changed.
ha_sequence::check_if_supported_inplace_alter(): A new function, to
ensure that ha_innobase::check_if_supported_inplace_alter() will be
called on ALTER TABLE name_of_sequence SEQUENCE=0.
ha_innobase::check_if_supported_inplace_alter(): For any change of
the SEQUENCE attribute, always return HA_ALTER_INPLACE_NOT_SUPPORTED,
forcing ALGORITHM=COPY.
log_t::writer_update(): Add the parameter bool resizing,
to indicate whether log resizing is in progress.
We must enable log_writer_resizing only if resize_lsn>1,
to ensure that log_t::resize_abort() will not choose the wrong
log_sys.log_writer.
log_t::resize_initiator: The thread that successfully invoked
resize_start().
log_t::resize_start(): Simplify some logic, and assign resize_initiator
if we successfully started log resizing.
log_t::resize_abort(): Abort log resizing if we are the
resize_initiator.
innodb_log_file_size_update(): Clean up some logic.
Reviewed by: Debarun Banerjee
dict_stats_fetch_from_ps(): Acquire dict_sys.latch as few times as
possible, and release dict_sys.latch after invoking pars_sql(),
so that we will not be unnecessarily holding dict_sys.latch while
possibly waiting for data to be read into the buffer pool.
row_create_index_for_mysql(): Tolerate DB_LOCK_TABLE_FULL better.
fts_create_one_common_table(), fts_create_one_index_table():
Do not corrupt the error state of a non-active transaction object.
fts_config_set_value(): Only run another statement if there was
no error yet.
row_parse_int(): Refactor the code and define the function static in
one compilation unit. For any negative values, we must return 0.
row_search_get_max_rec(), row_search_max_autoinc(): Moved to the same
compilation unit with row_parse_int().
We also remove a work-around of an internal compiler error when
targeting ARMv8 on GCC 4.8.5, a compiler that is no longer supported.
Reviewed by: Debarun Banerjee
This ensures that the error message is populated when the reading
fails with ER_NET_READ_INTERRUPTED, which at the client layer returns
without storing any error message as there is no corresponding CR_
error code.
Patch originally by Sergei Golubchik <serg@mariadb.com>
in _ma_unique_hash, skip_trailing_space, my_hash_sort_mb_nopad_bin and my_strnncollsp_utf8mb4_bin
UBSAN detected the nullptr-with-offset in a few places
when handling empty blobs.
Fix:
- Adding DBUG_ASSERT(source_string) into all hash_sort() implementations
to catch this problem in non-UBSAN debug builds.
- Fixing mi_unique_hash(), mi_unique_comp(),
_ma_unique_hash(), _ma_unique_comp() to replace NULL pointer to
an empty string ponter..
Note, we should also add DBUG_ASSERT(source_string != NULL) into
all implementations of strnncoll*(). But I'm afraid the patch
is going to be too long and too dangerous for 10.5.
innobase_commit(): When Galera is used with SET SQL_LOG_BIN=OFF,
some debug assertions that had been added in
commit ddd7d5d8e3 (MDEV-24035)
would fail. Let us relax those assertions for Galera transactions,
to allow an implicit commit after an internally executed XA PREPARE.
Note that trx_undo_report_row_operation() only allows undo log records
to be added to ACTIVE transactions (not after XA PREPARE). Hence, this
relaxation should be safe with respect to writes.
Problem:
=======
- Import tablespace fails to check the index fields descending
property while matching the schema given in cfg file with the
table schema.
Fix:
===
row_quiesce_write_index_fields(): Write the descending
property of the field into field fixed length field.
Since the field fixed length uses only 10 bits,
InnoDB can use 0th bit of the field fixed length
to store the descending field property.
row_import_cfg_read_index_fields(): Read the field
descending information from field fixed length.
page_encrypt_thread_key: The key for fil_crypt_thread().
All other InnoDB threads should already have been registered for
performance_schema ever since
commit a2f510fccf
page_encrypt_thread_key: The key for fil_crypt_thread().
All other InnoDB threads should already have been registered for
performance_schema ever since
commit a2f510fccf
enum rename_fk: Replaces the "bool use_fk" parameter of
row_rename_table_for_mysql() and innobase_rename_table():
RENAME_IGNORE_FK: Replaces use_fk=false when the operation cannot
involve any FOREIGN KEY constraints, that is, it is a partitioned
table or an internal table for FULLTEXT INDEX.
RENAME_REBUILD: Replaces use_fk=false when the table may contain
FOREIGN KEY constraints, which must not be modified in the data
dictionary tables SYS_FOREIGN and SYS_FOREIGN_COLS.
RENAME_ALTER_COPY: Replaces use_fk=true. This is only specified
in ha_innobase::rename_table(), which may be invoked as part of
ALTER TABLE…ALGORITHM=COPY, but also during RENAME TABLE.
An alternative value RENAME_FK could be useful to specify in
ha_innobase::rename_table() when it is executed as part of
CREATE OR REPLACE TABLE, which currently is not an atomic operation.
Reviewed by: Debarun Banerjee
innodb_convert_name(): Convert a schema or table name to
my_charset_filename compatible format.
dict_table_lookup(): Replaces dict_get_referenced_table().
Make the callers responsible for invoking innodb_convert_name().
innobase_casedn_str(): Remove. Let us invoke my_casedn_str() directly.
dict_table_rename_in_cache(): Do not duplicate a call to
dict_mem_foreign_table_name_lookup_set().
innobase_convert_to_filename_charset(): Defined static in the only
compilation unit that needs it.
dict_scan_id(): Remove the constant parameters
table_id=FALSE, accept_also_dot=TRUE. Invoke strconvert() directly.
innobase_convert_from_id(): Remove; only called from dict_scan_id().
innobase_convert_from_table_id(): Remove (dead code).
table_name_t::dblen(), table_name_t::basename(): In non-debug builds,
tolerate names that may miss a '/' separator.
Reviewed by: Debarun Banerjee
row_ins_cascade_calc_update_vec(): Skip any virtual columns in the
update vector of the parent table.
Based on mysql/mysql-server@0ac176453b
Reviewed by: Debarun Banerjee
Update conn->queue_connect_share in spider_check_trx_and_get_conn to
avoid use-after-free.
There are two branches in spider_check_trx_and_get_conn, often called
at the beginning of a spider DML, depending on whether an update of
various spider fields is needed. If it is determined to be needed, the
updating may NULL the connections associated with the spider handler,
which subsequently causes a call to spider_get_conn() which updates
conn->queued_connect_share with the SPIDER_SHARE associated with the
spider handler.
We make it so that conn->queued_connect_share is updated regardless of
the branch it enters, so that it will not be a stale and potentially
already freed one.
Each spider connection is identified with a connection key, which is
an encoding of the backend parameters.
The first byte of the key is by default 0, and in rare circumstances
it is changed to a different value: when semi_table_lock is set to 1;
and when using casual read. When this happens, often a new connection
is created with the new key. Neither case is useful: the description
of semi_table_lock has nothing to do with creation of new connections
and the parameter itself was deprecated for 10.7+ (MDEV-28829) and
marked for deletion (MDEV-28830); while new threads created by
non-zero spider_casual_read causes only threads to be idle, thus not
achieving any gain, see MDEV-26151, and the param has also been
deprecated in 11.5+ (MDEV-31789). The relevant code adds unnecessary
complexity to the spider code. This change does not reduce
parallelism, because already when bgs mode is on a background thread
is created per partition, and there is no evidence spider creates
multiple threads for one partition. If the needs of such cases arise
it will be a separate issue.
The conn_kind, which stands for "connection kind", is no longer useful
because the HandlerSocket support is deleted and Spider now has only
one connection kind, SPIDER_CONN_KIND_MYSQL. Remove conn_kind and
related code.
Signed-off-by: Yuchen Pei <yuchen.pei@mariadb.com>
Reviewed-by: Nayuta Yanagisawa <nayuta.yanagisawa@mariadb.com>
- MDEV-34392(commit cc810e64d4) adds
the check for nullability of foreign key column when foreign key
relation is of UPDATE_CASCADE or UPDATE SET NULL. This check
makes DDL fail when it violates foreign key nullability.
This patch basically does the nullability check for foreign key
column only for strict sql mode
It's supposed that the function gets the previous lock set on a record.
But if there are several locks set on a record, it will return only the
first one. Continue locks list iteration till the certain lock even if
the certain bit in lock bitmap is set.
Item:print_for_table_def() uses QT_TO_SYSTEM_CHARSET to print
the DEFAULT expression into FRM file during CREATE TABLE.
Therefore, the expression is encoded in utf8 in FRM.
get_field_default_value() erroneously used field->charset() to
print the DEFAULT expression at SHOW CREATE TABLE time.
Fixing get_field_default_value() to use &my_charset_utf8mb4_general_ci instead.
This makes DEFAULT work in the way way with:
- virtual column expressions:
if (field->vcol_info)
{
StringBuffer<MAX_FIELD_WIDTH> str(&my_charset_utf8mb4_general_ci);
field->vcol_info->print(&str);
- check constraint expressions:
if (field->check_constraint)
{
StringBuffer<MAX_FIELD_WIDTH> str(&my_charset_utf8mb4_general_ci);
field->check_constraint->print(&str);
Additional cleanup:
Fixing system_charset_info to &my_charset_utf8mb4_general_ci in a few
places to make non-BMP characters work in DEFAULT, virtual column,
check constraint expressions.
