This follows up the previous fix in
commit c3c53926c4 (MDEV-26554).
ha_innobase::delete_table(): Work around the insufficient
metadata locking (MDL) during DML operations by acquiring exclusive
InnoDB table locks on all child tables. Previously, this was only
done on TRUNCATE and ALTER.
ibuf_delete_rec(), btr_cur_optimistic_delete(): Do not invoke
lock_update_delete() during change buffer operations.
The revised trx_t::commit(std::vector<pfs_os_file_t>&) will
hold exclusive lock_sys.latch while invoking fil_delete_tablespace(),
which in turn may invoke ibuf_delete_rec().
dict_index_t::has_locking(): A new predicate, replacing the dummy
!dict_table_is_locking_disabled(index->table). Used for skipping lock
operations during ibuf_delete_rec().
trx_t::commit(std::vector<pfs_os_file_t>&): Release the locks
and remove the table from the cache while holding exclusive
lock_sys.latch.
trx_t::commit_in_memory(): Skip release_locks() if dict_operation holds.
trx_t::commit(): Reset dict_operation before invoking commit_in_memory()
via commit_persist().
lock_release_on_drop(): Release locks while lock_sys.latch is
exclusively locked.
lock_table(): Add a parameter for a pointer to the table.
We must not dereference the table before a lock_sys.latch has
been acquired. If the pointer to the table does not match the table
at that point, the table is invalid and DB_DEADLOCK will be returned.
row_ins_foreign_check_on_constraint(): Improve the checks.
Remove a bogus DB_LOCK_WAIT_TIMEOUT return that was needed
before commit c5fd9aa562 (MDEV-25919).
row_upd_check_references_constraints(),
wsrep_row_upd_check_foreign_constraints(): Simplify checks.
- InnoDB should avoid bulk insert operation when table has active
DDL. Because bulk insert writes only one undo log as TRX_UNDO_EMPTY
and logging of concurrent DML happens at commit time uses undo log
record to parse and get the value and operation.
- Removed ROW_T_EMPTY, ROW_OP_EMPTY and their associated functions
and also the test case which tries to log the ROW_OP_EMPTY
when table has active DDL.
- InnoDB DDL results in `Duplicate entry' if concurrent DML throws
duplicate key error. The following scenario explains the problem
connection con1:
ALTER TABLE t1 FORCE;
connection con2:
INSERT INTO t1(pk, uk) VALUES (2, 2), (3, 2);
In connection con2, InnoDB throws the 'DUPLICATE KEY' error because
of unique index. Alter operation will throw the error when applying
the concurrent DML log.
- Inserting the duplicate key for unique index logs the insert
operation for online ALTER TABLE. When insertion fails,
transaction does rollback and it leads to logging of
delete operation for online ALTER TABLE.
While applying the insert log entries, alter operation
encounters 'DUPLICATE KEY' error.
- To avoid the above fake duplicate scenario, InnoDB should
not write any log for online ALTER TABLE before DML transaction
commit.
- User thread which does DML can apply the online log if
InnoDB ran out of online log and index is marked as completed.
Set online log error if apply phase encountered any error.
It can also clear all other indexes log, marks the newly
added indexes as corrupted.
- Removed the old online code which was a part of DML operations
commit_inplace_alter_table() : Does apply the online log
for the last batch of secondary index log and does frees
the log for the completed index.
trx_t::apply_online_log: Set to true while writing the undo
log if the modified table has active DDL
trx_t::apply_log(): Apply the DML changes to online DDL tables
dict_table_t::is_active_ddl(): Returns true if the table
has an active DDL
dict_index_t::online_log_make_dummy(): Assign dummy value
for clustered index online log to indicate the secondary
indexes are being rebuild.
dict_index_t::online_log_is_dummy(): Check whether the online
log has dummy value
ha_innobase_inplace_ctx::log_failure(): Handle the apply log
failure for online DDL transaction
row_log_mark_other_online_index_abort(): Clear out all other
online index log after encountering the error during
row_log_apply()
row_log_get_error(): Get the error happened during row_log_apply()
row_log_online_op(): Does apply the online log if index is
completed and ran out of memory. Returns false if apply log fails
UndorecApplier: Introduced a class to maintain the undo log
record, latched undo buffer page, parse the undo log record,
maintain the undo record type, info bits and update vector
UndorecApplier::get_old_rec(): Get the correct version of the
clustered index record that was modified by the current undo
log record
UndorecApplier::clear_undo_rec(): Clear the undo log related
information after applying the undo log record
UndorecApplier::log_update(): Handle the update, delete undo
log and apply it on online indexes
UndorecApplier::log_insert(): Handle the insert undo log
and apply it on online indexes
UndorecApplier::is_same(): Check whether the given roll pointer
is generated by the current undo log record information
trx_t::rollback_low(): Set apply_online_log for the transaction
after partially rollbacked transaction has any active DDL
prepare_inplace_alter_table_dict(): After allocating the online
log, InnoDB does create fulltext common tables. Fulltext index
doesn't allow the index to be online. So removed the dead
code of online log removal
Thanks to Marko Mäkelä for providing the initial prototype and
Matthias Leich for testing the issue patiently.
The purpose of the compress() wrapper my_compress_buffer() was twofold:
silence Valgrind warnings about uninitialized memory access before
zlib 1.2.4, and have PERFORMANCE_SCHEMA instrumentation of some zlib
related memory allocation. Because of PERFORMANCE_SCHEMA, we cannot
trivially replace my_compress_buffer() with compress().
az_open(): Remove a crc32() call. Any CRC of the empty string is 0.
This is a backport of commit 4489a89c71
in order to remove the test innodb.redo_log_during_checkpoint
that would cause trouble in the DBUG subsystem invoked by
safe_mutex_lock() via log_checkpoint(). Before
commit 7cffb5f6e8
these mutexes were of different type.
The following options were introduced in
commit 2e814d4702 (mariadb-10.2.2)
and have little use:
innodb_disable_resize_buffer_pool_debug had no effect even in
MariaDB 10.2.2 or MySQL 5.7.9. It was introduced in
mysql/mysql-server@5c4094cf49
to work around a problem that was fixed in
mysql/mysql-server@2957ae4f99
(but the parameter was not removed).
innodb_page_cleaner_disabled_debug and innodb_master_thread_disabled_debug
are only used by the test innodb.redo_log_during_checkpoint
that will be removed as part of this commit.
innodb_dict_stats_disabled_debug is only used by that test,
and it is redundant because one could simply use
innodb_stats_persistent=OFF or the STATS_PERSISTENT=0 attribute
of the table in the test to achieve the same effect.
Ever since commit 685d958e38
some Perl code in the test mariabackup.huge_lsn is writing names of
non-existing files to the InnoDB redo log and testing the recovery.
We do not need any debug instrumentation to duplicate that test.
Due to 32-bit arithmetics, SRV_TMP_SPACE_ID page number 0x200002 would be
folded to 0, which is incompatible with the assumption that was made in
commit 7cffb5f6e8 (MDEV-23399).
page_id_t::fold(): Compute in the native word width instead of uint32_t.
On 64-bit platforms, an alternative would be to return the 64-bit m_id
directly, but that was measured to cause a performance regression.
fil_space_t::open(): Invoke fil_node_t::find_metadata() when the
tablespace is being created. In this way, we will actually detect
that the temporary tablespace resides on SSD. (During database
creation, also the system tablespace will correctly be detected as
residing on SSD.)
The only purpose of ibuf_bitmap_mutex is to prevent a deadlock between
two concurrent invocations of ibuf_update_free_bits_for_two_pages_low()
on the same pair of bitmap pages, but in opposite order.
The mutex is unnecessarily serializing the execution of the function
even when it is being invoked on totally different tablespaces.
To avoid deadlocks, it suffices to ensure that the two page latches
are being acquired in a deterministic (sorted) order.
The following condition has to added:
1) InnoDB fails to include the offset of the node pointer field
in non-leaf record for redundant row format.
2) If the Fixed length field does have only prefix length then
calculate the field maximum size as prefix length.
- Added the test case to test (2) and to check maximum number of
fields can exist in the index.
- While loading the indexes, InnoDB fails to clear the index if the
system record has TEMP_INDEX_PREFIX_STR. This lead to ASAN failure.
The leak was introduced by commit cc2ddde4d8
(MDEV-18518)
When we enable writes after Galera SST srv_n_fil_crypt_threads needs
to be set temporally to 0 (as was done when writes were disabled)
to make sure that encryption threads will be really started based
on old value of encryption threads.
Fix provided by Marko Mäkelä.
Analysis: There are 2 server variables- "old_mode" and "old". "old" is no
longer needed as "old_mode" has replaced it (however still used in some places
in the code). "old_mode" and "old" has same purpose- emulate behavior from
previous MariaDB versions. So they can be merged to avoid confusion.
Fix: Deprecate "old" variable and create another mode for @@old_mode to mimic
behavior of previous "old" variable. Create specific modes for specifix task
that --old sql variable was doing earlier and use the new modes instead.
The test encryption.innodb-redo-nokeys did not actually test
recovery without valid keys, because due to the setting
innodb_encrypt_tables, InnoDB refused to start up at all,
without even attempting any crash recovery.
fil_ibd_load(): If the encryption key is not available,
refuse to load the file.
Deprecate the server variable spider_internal_offset and the corresponding
table parameters "ios" and "internal_offset".
We believe this variable is not much use to users, therefore decided to
deprecate it.
Reviewed-by: Nayuta Yanagisawa <nayuta.yanagisawa@hey.com>
The variables, spider_crd_mode and spider_sts_mode, specify the
ways to fetch statistics from data nodes.
Using the SHOW command seems to work for any cases. Thus, we deprecate
the variables.
We deprecate the variable spider_xa_register_mode because there is
no need to perform a two phase commit for a read-only transaction.
Note that the variable only affects Spider's internal XA transactions.
A few PERFORMANCE_SCHEMA instrumentation keys were not exposed
in all_innodb_mutexes[]. Let us remove them.
The keys fts_pll_tokenize_mutex_key and read_view_mutex_key were
internally used. Let us make ReadView::m_mutex use the simpler
and smaller srw_mutex, hoping to improve memory access patterns.
trx_lock_t: Remove byte pad[256] and use
alignas(CPU_LEVEL1_DCACHE_LINESIZE) instead.
trx_t: Declare n_ref (the first member) aligned at cache line.
Pool: Assert that the sizes are multiples of
CPU_LEVEL1_DCACHE_LINESIZE, and invoke an aligned allocator.
Access the all_charsets[] array directly in a hot loop.
This avoids get_charset() that increments a shared counter via
my_collation_statistics_inc_use_count(), causing a scalability issue.
Instead, call get_charset() when a table is opened (and InnoDB
fills in dtype_t values) - this is enough, as charset is marked
ready (MY_CS_READY) only once, on the first get_charset() call,
after that it can be accessed directly.
This also fixes a potential bug in InnoDB. It used to access charsets
directly when filling dtype_t values. This wasn't preceded by any
get_charset() call inside InnoDB. Normally the server would call
get_charset() on reading the frm, but if the table was first opened
from a purge thread InnoDB could, theoretically, access a not-ready
charset in innobase_get_cset_width().
The variable spider_internal_limit is for specifying the number of
records acquired by Spider from each remote server.
There seems not to be much use of the variable. Thus, we deprecate it
and the corresponding table options.
Reviewed-by: Nayuta Yanagisawa <nayuta.yanagisawa@hey.com>
We can only remove a subdirectory in mtr on an installed instance
Example failure previously:
CURRENT_TEST: rocksdb.rocksdb_log_dir
mysqltest: At line 15: Path '/usr/local/mariadb-10.9.0-linux-systemd-x86_64/mysql-test/var/tmp/1' is not a subdirectory of MYSQLTEST_VARDIR '/usr/local/mariadb-10.9.0-linux-systemd-x86_64/mysql-test/var/1'or MYSQL_TMP_DIR '/usr/local/mariadb-10.9.0-linux-systemd-x86_64/mysql-test/var/tmp/1'
Parameter rocksdb_log_dir specifies the path of MyRocks error logs.
By default, the error logs are stored in the same folder with MyRocks
redo logs. Being able to put human readable logs in one place and
machine logs in another place improves usability.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
- Trigger statement initiate update statement after bulk insert
operation and leads to disable of bulk operation. During commit,
InnoDB expects transaction to be in bulk insert mode before
applying the bulk insert operation. InnoDB transaction should
apply all bulk insert operation before disabling bulk insert
operation.
The issue was that the value of MARIA_FOUND_WRONG_KEY was a value
that could be returned by ha_key_cmp.
This was already fixed in MyISAM, now using the same fix in Aria:
Setting the value to INT_MAX32, which should be impossible in any
normal cases.
I also fixed so that if there is a wrong key, we now get a proper error
message and not an assert.
