Removed trx_sys_validate_trx_list(): with rw_trx_hash elements are not
required to be ordered by transaction id. Transaction state is now guarded
by asserts in rw_trx_hash_t.
Removed trx_sys_t::n_prepared_recovered_trx: never used.
Removed trx_sys_t::n_prepared_trx: used only at shutdown, we can perfectly
get this value from rw_trx_hash.
Determine minimum transaction id by iterating rw_trx_hash, not rw_trx_list.
It is more expensive than previous implementation since it does linear
search, especially if there're many concurrent transactions running. But in
such case mutex is much bigger evil. And since it doesn't require
trx_sys->mutex protection it scales better.
For low concurrency performance difference is neglible.
Replaced UT_LIST_GET_LEN(trx_sys->rw_trx_list) with
trx_sys->rw_trx_hash.size().
Moved freeing of trx objects at shutdown to rw_trx_hash destructor.
Small clean-up in trx_rollback_recovered().
Reduce divergence between trx_sys_t::rw_trx_hash and trx_sys_t::rw_trx_list
by not adding recovered COMMITTED transactions to trx_sys_t::rw_trx_list.
Such transactions are discarded immediately without creating trx object.
This also required to split rollback and cleanup phases of recovery. To
reflect these updates the following renames happened:
trx_rollback_or_clean_all_recovered() -> trx_rollback_all_recovered()
trx_rollback_or_clean_is_active -> trx_rollback_is_active
trx_rollback_or_clean_recovered() -> trx_rollback_recovered()
trx_cleanup_at_db_startup() -> trx_cleanup_recovered()
Also removed a hack from lock_trx_release_locks(). Instead let recovery
rollback thread to skip committed XA transactions.
If InnoDB is killed while ALTER TABLE...ALGORITHM=COPY is in progress,
after recovery there could be undo log records some records that were
inserted into an intermediate copy of the table. Due to these undo log
records, InnoDB would resurrect locks at recovery, and the intermediate
table would be locked while we are trying to drop it. This would cause
a call to row_rename_table_for_mysql(), either from
row_mysql_drop_garbage_tables() or from the rollback of a RENAME
operation that was part of the ALTER TABLE.
row_rename_table_for_mysql(): Do not attempt to parse FOREIGN KEY
constraints when renaming from #sql-something to #sql-something-else,
because it does not make any sense.
row_drop_table_for_mysql(): When deferring DROP TABLE due to locks,
do not rename the table if its name already starts with the #sql-
prefix, which is what row_mysql_drop_garbage_tables() uses.
Previously, the too strict prefix #sql-ib was used, and some
tables were renamed unnecessarily.
Replace some !rw_lock_own() assertions with the stronger
!btr_search_own_any(). Remove some redundant btr_get_search_latch()
calls.
btr_search_update_hash_ref(): Remove a duplicated assertion.
btr_search_build_page_hash_index(): Remove a duplicated assertion.
rw_lock_s_lock() asserts that the latch is not being held.
btr_search_disable_ref_count(): Remove an assertion. The only caller
is acquiring all adaptive hash index latches.
innodb_init_param(): Initialize btr_ahi_parts=1 for Mariabackup.
btr_search_enabled: Let the adaptive hash index be disabled
in Mariabackup. This would potentially only matter during --export,
and --export performs a table scan, not many index lookups.
innobase_start_or_create_for_mysql(): Only start the data dictionary
and transaction subsystems in normal server startup and during
mariabackup --export.
btr_cur_update_in_place(): Read block->index only once,
so that it cannot change to NULL after the first read.
When block->index != NULL, it must be equal to index.
btr_cur_update_in_place(): The call rw_lock_x_lock(ahi_latch) must
of course be inside the if (ahi_latch) condition. This is a mistake
that I made when backporting the fix-under-development from 10.3.
btr_cur_search_to_nth_level(), btr_search_guess_on_hash(),
btr_pcur_open_with_no_init_func(), row_sel_open_pcur():
Replace the parameter has_search_latch with the ahi_latch
(passed as NULL if the caller does not hold the latch).
btr_search_update_hash_node_on_insert(),
btr_search_update_hash_on_insert(),
btr_search_build_page_hash_index(): Add the parameter ahi_latch.
btr_search_x_lock(), btr_search_x_unlock(),
btr_search_s_lock(), btr_search_s_unlock(): Remove.
btr_cur_search_to_nth_level(), row_sel(): Do not bother to yield to
waiting exclusive lock requests on the adaptive hash index latch.
When the btr_search_latch was split into an array of latches
in MySQL 5.7.8 as part of the Oracle Bug#20985298 fix, the "caching"
of the latch across storage engine API calls was removed. Thus,
X-lock requests should have a good chance of becoming served, and
starvation should not be possible.
btr_search_guess_on_hash(): Clean up a debug assertion.
Also, use 32-bit native reads to read the 32-bit aligned
FIL_PAGE_PREV and FIL_PAGE_NEXT reads, to compare them to the
byte order agnostic pattern FIL_NULL (0xffffffff).
This is mere code clean-up; the reported problem was already fixed
in commit 3fdd390791.
row_sel(): Remove the variable search_latch_locked.
row_sel_try_search_shortcut(): Remove the parameter
search_latch_locked, which was always passed as nonzero.
row_sel_try_search_shortcut(), row_sel_try_search_shortcut_for_mysql():
Do not expect the caller to acquire the AHI latch. Instead,
acquire and release it inside this function.
row_search_mvcc(): Remove a bogus condition on mysql_n_tables_locked.
When the btr_search_latch was split into an array of latches
in MySQL 5.7.8 as part of the Oracle Bug#20985298 fix, the "caching"
of the latch across storage engine API calls was removed, and
thus it is unnecessary to avoid adaptive hash index searches
during INSERT...SELECT.
This is not fixing the reported problem, but a potential problem that was
introduced in MDEV-11369.
row_sel_try_search_shortcut(), row_sel_try_search_shortcut_for_mysql():
When an adaptive hash index search lands on top of rec_is_default_row(),
we must skip the candidate and perform a normal search. This is because
the adaptive hash index latch only protects the record from being deleted
but does not prevent concurrent inserts into the page. Therefore, it is not
safe to dereference the next-record pointer.
This race condition is a regression caused by MDEV-12121.
btr_cur_update_in_place(): Determine block->index!=NULL only once
in order to determine whether an adaptive hash index bucket needs
to be exclusively locked and unlocked.
If we evaluated block->index multiple times, and the adaptive hash
index was disabled before we locked the adaptive hash index, then
we would never release the adaptive hash index bucket latch, which
would eventually lead to InnoDB hanging.
MariaDB inherits the MySQL limitation that ALGORITHM=INPLACE cannot
create more than one FULLTEXT INDEX at a time. As part of the MDEV-11369
Instant ADD COLUMN refactoring, MariaDB 10.3.2 accidentally stopped
enforcing the restriction.
Actually, it is a bug in MySQL 5.6 and MariaDB 10.0 that an ALTER TABLE
statement with multiple ADD FULLTEXT INDEX but without explicit
ALGORITHM=INPLACE would return in an error message, rather than
executing the operation with ALGORITHM=COPY.
ha_innobase::check_if_supported_inplace_alter(): Enforce the restriction
on multiple FULLTEXT INDEX.
prepare_inplace_alter_table_dict(): Replace some code with debug
assertions. A "goto error_handled" at this point would result in
another error, because the reference count of ctx->new_table would be 0.
This is a regression that was introduced in MySQL 5.7.6 in
19855664de
fil_node_open_file(): Use proper 64-bit arithmetics for truncating
size_bytes to a multiple of a file extent size.
InnoDB originally skipped the redo logging of PAGE_MAX_TRX_ID changes
until I enabled it in commit e76b873f24
that was part of MySQL 5.5.5 already.
Later, when a more complete history of the InnoDB Plugin for MySQL 5.1
(aka branches/zip in the InnoDB subversion repository) and of the
planned-to-be closed-source branches/innodb+ that became the basis of
InnoDB in MySQL 5.5 was pushed to the MySQL source repository, the
change was part of commit 509e761f06:
------------------------------------------------------------------------
r5038 | marko | 2009-05-19 22:59:07 +0300 (Tue, 19 May 2009) | 30 lines
branches/zip: Write PAGE_MAX_TRX_ID to the redo log. Otherwise,
transactions that are started before the rollback of incomplete
transactions has finished may have an inconsistent view of the
secondary indexes.
dict_index_is_sec_or_ibuf(): Auxiliary function for controlling
updates and checks of PAGE_MAX_TRX_ID: check whether an index is a
secondary index or the insert buffer tree.
page_set_max_trx_id(), page_update_max_trx_id(),
lock_rec_insert_check_and_lock(),
lock_sec_rec_modify_check_and_lock(), btr_cur_ins_lock_and_undo(),
btr_cur_upd_lock_and_undo(): Add the parameter mtr.
page_set_max_trx_id(): Allow mtr to be NULL. When mtr==NULL, do not
attempt to write to the redo log. This only occurs when creating a
page or reorganizing a compressed page. In these cases, the
PAGE_MAX_TRX_ID will be set correctly during the application of redo
log records, even though there is no explicit log record about it.
btr_discard_only_page_on_level(): Preserve PAGE_MAX_TRX_ID. This
function should be unreachable, though.
btr_cur_pessimistic_update(): Update PAGE_MAX_TRX_ID.
Add some assertions for checking that PAGE_MAX_TRX_ID is set on all
secondary index leaf pages.
rb://115 tested by Michael, fixes Issue #211
------------------------------------------------------------------------
After this fix, some bogus references to recv_recovery_is_on()
remained. Also, some references could be replaced with
references to index->is_dummy to prepare us for MDEV-14481
(background redo log apply).
- Make Rdb_binlog_manager::unpack_value to not have a stack overrun
when it is reading invalid data (which it currently does as we in
MariaDB do not store binlog coordinates under BINLOG_INFO_INDEX_NUMBER,
see comments in MDEV-14892 for details).
- We may need to store these coordinates in the future, so instead of
removing the call of this function, let's make it work properly for
all possible inputs.
In CREATE SEQUENCE or CREATE TEMPORARY SEQUENCE, we should not start
an InnoDB transaction for inserting the sequence status record into
the underlying no-rollback table. Because we did this, a debug assertion
failure would fail in START TRANSACTION WITH CONSISTENT SNAPSHOT after
CREATE TEMPORARY SEQUENCE was executed.
row_ins_step(): Do not start the transaction. Let the caller do that.
que_thr_step(): Start the transaction before calling row_ins_step().
row_ins_clust_index_entry(): Skip locking and undo logging for no-rollback
tables, even for temporary no-rollback tables.
row_ins_index_entry(): Allow trx->id==0 for no-rollback tables.
row_insert_for_mysql(): Do not start a transaction for no-rollback tables.
trx reference counter was updated under mutex and read without any
protection. This is both slow and unsafe. Use atomic operations for
reference counter accesses.
trx_sys_t::rw_trx_set is implemented as std::set, which does a few quite
expensive operations under trx_sys_t::mutex protection: e.g. malloc/free
when adding/removing elements. Traversing b-tree is not that cheap either.
This has negative scalability impact, which is especially visible when running
oltp_update_index.lua benchmark on a ramdisk.
To reduce trx_sys_t::mutex contention std::set is replaced with LF_HASH. None
of LF_HASH operations require trx_sys_t::mutex (nor any other global mutex)
protection.
Another interesting issue observed with std::set is reproducible ~2% performance
decline after benchmark is ran for ~60 seconds. With LF_HASH results are stable.
All in all this patch optimises away one of three trx_sys->mutex locks per
oltp_update_index.lua query. The other two critical sections became smaller.
Relevant clean-ups:
Replaced rw_trx_set iteration at startup with local set. The latter is needed
because values inserted to rw_trx_list must be ordered by trx->id.
Removed redundant conditions from trx_reference(): it is (and even was) never
called with transactions that have trx->state == TRX_STATE_COMMITTED_IN_MEMORY.
do_ref_count doesn't (and probably even didn't) make any sense: now it is called
only when reference counter increment is actually requested.
Moved condition out of mutex in trx_erase_lists().
trx_rw_is_active(), trx_rw_is_active_low() and trx_get_rw_trx_by_id() were
greatly simplified and replaced by appropriate trx_rw_hash_t methods.
Compared to rw_trx_set, rw_trx_hash holds transactions only in PREPARED or
ACTIVE states. Transactions in COMMITTED state were required to be found
at InnoDB startup only. They are now looked up in the local set.
Removed unused trx_assert_recovered().
Removed unused innobase_get_trx() declaration.
Removed rather semantically incorrect trx_sys_rw_trx_add().
Moved information printout from trx_sys_init_at_db_start() to
trx_lists_init_at_db_start().
