While the bug was reported as a regression of
MDEV-11025 Make number of page cleaner threads variable dynamic
in MariaDB Server 10.3, the code that MariaDB Server 10.2
inherited from MySQL 5.7.4 (WL#6642) looks prone to similar errors.
pc_flush_slot(): If there is no work to do, reset the is_requested
signal, to avoid potential busy-waiting in
buf_flush_page_cleaner_worker(). If the coordinator thread has shut
down, avoid resetting the is_requested event, to avoid a potential
hang at shutdown if there are multiple worker threads.
ibuf_merge_or_delete_for_page(): Invoke fil_space_acquire_silent()
instead of fil_space_acquire() in order to avoid displaying
a useless message.
We know perfectly well that a tablespace can be dropped while a
change buffer merge is pending, because change buffer merges skip
any transactional locks.
innobase_commit_by_xid(), innobase_rollback_by_xid(): Decrement
the reference count before freeing the transaction object to the pool.
Failure to do so might corrupt the transaction bookkeeping
if trx_create_low() returns the same object to another thread
before we are done with it.
trx_sys_close(): Detach the recovered XA PREPARE transactions from
trx_sys->rw_trx_list before freeing them.
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
The field trx_rseg_t::trx_ref_count that was added in WL#6965 in
MySQL 5.7.5 is being incremented twice if a recovered transaction
includes both undo log partitions insert_undo and update_undo.
This reference count is being used in trx_purge(), which invokes
trx_purge_initiate_truncate() to try to truncate an undo tablespace
file. Because of the double-increment, the trx_ref_count would never
reach 0.
It is possible that after the failed truncation attempt, the undo
tablespace would be disabled for logging any new transactions until
the server is restarted (hopefully after committing or rolling back
all transactions, so that no transactions would be recovered
on the next startup).
trx_resurrect_insert(), trx_resurrect_update(): Do not increment
trx_ref_count. Instead, let the caller do that.
trx_lists_init_at_db_start(): Increment rseg->trx_ref_count only
once for each recovered transaction. Adjust comments.
Finally, if innodb_force_recovery prevents the undo log scan,
do not bother iterating the empty lists.
innobase_start_or_create_for_mysql(): Only start the data dictionary
and transaction subsystems in normal server startup and during
mariabackup --export.
btr_cur_update_in_place(): Read block->index only once,
so that it cannot change to NULL after the first read.
When block->index != NULL, it must be equal to index.
btr_cur_update_in_place(): The call rw_lock_x_lock(ahi_latch) must
of course be inside the if (ahi_latch) condition. This is a mistake
that I made when backporting the fix-under-development from 10.3.
This race condition is a regression caused by MDEV-12121.
btr_cur_update_in_place(): Determine block->index!=NULL only once
in order to determine whether an adaptive hash index bucket needs
to be exclusively locked and unlocked.
If we evaluated block->index multiple times, and the adaptive hash
index was disabled before we locked the adaptive hash index, then
we would never release the adaptive hash index bucket latch, which
would eventually lead to InnoDB hanging.
This is a regression that was introduced in MySQL 5.7.6 in
19855664de
fil_node_open_file(): Use proper 64-bit arithmetics for truncating
size_bytes to a multiple of a file extent size.
The warning was originally added in
commit c67663054a
(MySQL 4.1.12, 5.0.3) to trace claimed undo log corruption that
was analyzed in https://lists.mysql.com/mysql/176250
on November 9, 2004.
Originally, the limit was 20,000 undo log headers or transactions,
but in commit 9d6d1902e0
in MySQL 5.5.11 it was increased to 2,000,000.
The message can be triggered when the progress of purge is prevented
by a long-running transaction (or just an idle transaction whose
read view was started a long time ago), by running many transactions
that UPDATE or DELETE some records, then starting another transaction
with a read view, and finally by executing more than 2,000,000
transactions that UPDATE or DELETE records in InnoDB tables. Finally,
when the oldest long-running transaction is completed, purge would
run up to the next-oldest transaction, and there would still be more
than 2,000,000 transactions to purge.
Because the message can be triggered when the database is obviously
not corrupted, it should be removed. Heavy users of InnoDB should be
monitoring the "History list length" in SHOW ENGINE INNODB STATUS;
there is no need to spam the error log.
recv_log_recover_10_3(): Determine if a log from MariaDB 10.3 is clean.
recv_find_max_checkpoint(): Allow startup with a clean 10.3 redo log.
srv_prepare_to_delete_redo_log_files(): When starting up with a 10.3 log,
display a "Downgrading redo log" message instead of "Upgrading".
innodb/buf_LRU_get_free_block
Add debug instrumentation to produce error message about
no free pages. Print error message only once and do not
enable innodb monitor.
xtradb/buf_LRU_get_free_block
Add debug instrumentation to produce error message about
no free pages. Print error message only once and do not
enable innodb monitor. Remove code that does not seem to
be used.
innodb-lru-force-no-free-page.test
New test case to force produce desired error message.
dict_foreign_find_index(): Ignore incompletely created indexes.
After a failed ADD UNIQUE INDEX, an incompletely created index
could be left behind until the next ALTER TABLE statement.
rec_get_trx_id(): Because rec is not necessarily residing in
a buffer pool page (it could be an old version of a clustered index
record, allocated from heap), remove the debug assertions that
depend on page_align(rec).
This bug affects both writing and reading encrypted redo log in
MariaDB 10.1, starting from version 10.1.3 which added support for
innodb_encrypt_log. That is, InnoDB crash recovery and Mariabackup
will sometimes fail when innodb_encrypt_log is used.
MariaDB 10.2 or Mariabackup 10.2 or later versions are not affected.
log_block_get_start_lsn(): Remove. This function would cause trouble if
a log segment that is being read is crossing a 32-bit boundary of the LSN,
because this function does not allow the most significant 32 bits of the
LSN to change.
log_blocks_crypt(), log_encrypt_before_write(), log_decrypt_after_read():
Add the parameter "lsn" for the start LSN of the block.
log_blocks_encrypt(): Remove (unused function).
The function rtr_update_mbr_field_in_place() is generating
MLOG_REC_UPDATE_IN_PLACE or MLOG_COMP_REC_UPDATE_IN_PLACE records
on non-leaf pages, even though MLOG_WRITE_STRING would perfectly
suffice for updating a fixed-length data field.
btr_cur_parse_update_in_place(): If flags==7, the record may be
from rtr_update_mbr_field_in_place(), and we must check if the
page is a leaf page. Otherwise, assume that it is.
btr_cur_update_in_place(): Assert that the page is a leaf page.
Other things, mainly to get
create_mysqld_error_find_printf_error tool to work:
- Added protection to not include mysqld_error.h twice
- Include "unireg.h" instead of "mysqld_error.h" in server
- Added protection if ER_XX messages are already defined
- Removed wrong calls to my_error(ER_OUTOFMEMORY) as
my_malloc() and my_alloc will do this automatically
- Added missing %s to ER_DUP_QUERY_NAME
- Removed old and wrong calls to my_strerror() when using
MY_ERROR_ON_RENAME (wrong merge)
- Fixed deadlock error message from Galera. Before the extra
information given to ER_LOCK_DEADLOCK was missing because
ER_LOCK_DEADLOCK doesn't provide any extra information.
I kept #ifdef mysqld_error_find_printf_error_used in sql_acl.h
to make it easy to do this kind of check again in the future
trx_undo_page_report_modify(): For SPATIAL INDEX, keep logging
updated off-page columns twice, so that
the minimum bounding rectangle (MBR) will be logged.
Avoiding the redundant logging would require larger changes
to the undo log format.
row_build_index_entry_low(): Handle SPATIAL_UNKNOWN more robustly,
by refusing to purge the record from the spatial index.
We can get this code when processing old undo log from 10.2.10 or
10.2.11 (the releases affected by MDEV-14799, which was a regression
from MDEV-14051).
trx_undo_rec_get_partial_row(): When the PRIMARY KEY includes a
column prefix of an externally stored column, the already parsed
part of the undo log record may contain a reference to
an off-page column. This is the case in the bug58912 test in
innodb.innodb.
This is a regression caused by MDEV-14051 'Undo log record is too big.'
Purge in the secondary index is wrongly skipped in
row_purge_upd_exist_or_extern() because node->row only does not contain all
indexed columns.
trx_undo_rec_get_partial_row(): Add the parameter for node->update
so that the updated columns will be copied from the initial part
of the undo log record.
row_log_table_apply_insert_low(), row_log_table_apply_update():
When reporting the error_key_num, only count the clustered index
if it corresponds to a key in the SQL layer.
The assertion failure was probably introduced by the (incomplete)
MySQL 5.6.28 bug fix
Bug #21364096 THE BOGUS DUPLICATE KEY ERROR IN ONLINE DDL
WITH INCORRECT KEY NAME
which we are improving.
Side note: the fix was incorrectly merged to MySQL 5.7.10;
incorrect key names will continue to be reported in MySQL 5.7.
These assertions were disabled in MariaDB 10.1.1 in
commit df4dd593f2
with a bogus comment referring to the function wsrep_fake_trx_id()
that was introduced in the very same commit.
Problem was that crypt_data->min_key_version is not a reliable way
to detect is tablespace encrypted and could lead that in first page
of the second (page 192 and similarly for other files if more configured)
system tablespace file used key_version is replaced with zero leading
a corruption as in next startup page is though to be corrupted.
Note that crypt_data->min_key_version is updated only after all
pages from tablespace have been processed (i.e. key rotation is done)
and flushed.
fil_write_flushed_lsn
Use crypt_data->should_encrypt() instead.
