commit_try_rebuild(): Invoke trx_t::drop_table_statistics()
with the correct (original) name of the table.
This fixes a regression that was introduced in
commit 1bd681c8b3 (MDEV-25506 part 3).
If an ALTER TABLE that affected a table with FULLTEXT INDEX was
in progress while a backup was made, InnoDB would crash or hang
during the first startup after restoring the backup,
while trying to drop the #sql-alter- table for the DDL operation.
drop_garbage_tables_after_restore(): Invoke dict_sys.unlock()
before invoking the FTS functions. Also, invoke purge_sys.stop_FTS()
in debug builds to silence debug assertions. (Purge is not yet
running at this point.)
trx_undo_mem_create_at_db_start(): Relax too strict upgrade checks
that were introduced in
commit e46f76c974 (MDEV-15912).
On commit, pages will typically be set to TRX_UNDO_CACHED state.
Having the type TRX_UNDO_INSERT in such pages is common and
unproblematic; the type would be reset in trx_undo_reuse_cached().
trx_rseg_array_init(): On failure, clean up the rollback segments
that were initialized so far, to avoid an assertion failure later
during shutdown.
init_crypt_key(): On failure, set info->key_version to
ENCRYPTION_KEY_VERSION_INVALID.
log_crypt_101_read_block(): Refuse to attempt decryption if
info->key_version is ENCRYPTION_KEY_VERSION_INVALID.
trx_undo_report_row_operation(): Fix a race condition that was introduced
in commit f74023b955 (MDEV-15090).
We must not access undo_block after the page latch has been released
in mtr_t::commit(), because the block could be evicted or replaced.
fil_crypt_rotate_page(): If a page is already marked as dirty,
do not move it in buf_pool.flush_list. Actual re-encryption
will happen in buf_flush_page(), and moving the page in
buf_pool.flush_list may only postpone the operation.
Let us simply refuse an upgrade from earlier versions if the
upgrade procedure was not followed. This simplifies the purge,
commit, and rollback of transactions.
Before upgrading to MariaDB 10.3 or later, a clean shutdown
of the server (with innodb_fast_shutdown=1 or 0) is necessary,
to ensure that any incomplete transactions are rolled back.
The undo log format was changed in MDEV-12288. There is only
one persistent undo log for each transaction.
When doing CREATE TABLE ... ENGINE=MyISAM SELECT ...
the MyISAM table was not completely flushed at commit
which gave some warnings about table not closed in the
atomic test suite.
Fixed by flushing the table in select_insert::prepare_eof().
and *never* disable tests in suite.pm based on $::opt_big_test,
this will make the test skipped both as too big (for ./mtr)
and as too small (for ./mtr --big --big).
In commit 1c5ae99194 (MDEV-25666)
we had changed Mariabackup so that it would no longer skip files
whose names start with #sql. This turned out to be wrong.
Because operations on such named files are not protected by any
locks in the server, it is not safe to copy them.
Not copying the files may make the InnoDB data dictionary
inconsistent with the file system. So, we must do something
in InnoDB to adjust for that.
If InnoDB is being started up without the redo log (ib_logfile0)
or with a zero-length log file, we will assume that the server
was restored from a backup, and adjust things as follows:
dict_check_sys_tables(), fil_ibd_open(): Do not complain about
missing #sql files if they would be dropped a little later.
dict_stats_update_if_needed(): Never add #sql tables to
the recomputing queue. This avoids a potential race condition when
dropping the garbage tables.
drop_garbage_tables_after_restore(): Try to drop any garbage tables.
innodb_ddl_recovery_done(): Invoke drop_garbage_tables_after_restore()
if srv_start_after_restore (a new flag) was set and we are not in
read-only mode (innodb_read_only=ON or innodb_force_recovery>3).
The tests and dbug_mariabackup_event() instrumentation
were developed by Vladislav Vaintroub, who also reviewed this.
In commit 1bd681c8b3 we introduced
a work-around for the missing MDL protection when the internal
tables of FULLTEXT INDEX are dropped during DDL operations.
That work-around suffered from a race condition. A purge thread
could have narrowly passed purge_sys.check_stop_FTS() and then
(while holding dict_sys.mutex) acquire a table reference right
after fts_lock_table() determined that no references were being
held.
fts_lock_table(): Protect the reference check with dict_sys.mutex.
Thanks to Thirunarayanan Balathandayuthapani for repeating the
failure and testing the fix.
Before any committed DDL operation returns control to the caller,
we must ensure that it will have been durably committed before the
ddl_log state may be changed, no matter if
innodb_flush_log_at_trx_commit=0 is being used.
Operations that would involve deleting files were already safe,
because the durable write of the FILE_DELETE record that would precede
the file deletion would also have made the commit durable.
Furthermore, when we clean up ADD INDEX stubs that were left behind
by a previous ha_innobase::commit_inplace_alter_table(commit=false)
when MDL could not be acquired, we will use the same interface as
for dropping indexes. This safety measure should be dead code,
because ADD FULLTEXT INDEX is not supported online, and dropping indexes
only involves file deletion for FULLTEXT INDEX.
ha_innobase::extra(HA_EXTRA_END_ALTER_COPY): Flush the redo log
because the main transaction will not have written any log and
thus will not initiate any log write either. But the DDL recovery
layer expects that the operation will have been committed.
- During online alter conversion from compact to redundant,
virtual column field length already set during
innobase_get_computed_value(). Skip the char(n) check for
virtual column in row_merge_buf_add()
- InnoDB fails to check DB_COMPUTE_VALUE_FAILED error in
row_merge_read_clustered_index() and wrongly asserts that
the buffer shouldn't be ran out of memory. Alter table
should give warning when the column value is being
truncated.
Problem:
=======
- InnoDB iterates the fil_system space list to encrypt the
tablespace in case of key rotation. But it is not
necessary for any encryption plugin which doesn't do
key version rotation.
Solution:
=========
- Introduce a new variable called srv_encrypt_rotate to
indicate whether encryption plugin does key rotation
fil_space_crypt_t::key_get_latest_version(): Enable the
srv_encrypt_rotate only once if current key version is
higher than innodb_encyrption_rotate_key_age
fil_crypt_must_default_encrypt(): Default encryption tables
should be added to default_encryp_tables list if
innodb_encyrption_rotate_key_age is zero and encryption
plugin doesn't do key version rotation
fil_space_create(): Add the newly created space to
default_encrypt_tables list if
fil_crypt_must_default_encrypt() returns true
Removed the nondeterministic select from
innodb-key-rotation-disable test. By default,
InnoDB adds the tablespace to the rotation list and
background crypt thread does encryption of tablespace.
So these select doesn't give reliable results.
- Rename the rotation list to default_encrypt_tables in
fil_system_t. Because rotation list naming could be
misleading when it comes to key version rotation
- Rename is_in_rotation_list to is_in_default_encrypt
in fil_space_t
- Rename keyrotate_next function to default_encrypt_next
fil_system_t::default_encrypt_next(): Find the next
suitable default encrypt table if beginning of
default_encrypt_tables list has been scheduled
to be deleted
- Replace strncpy() with strmake() to ensure things are \0 terminated
- Initialized some variables that caused compiler warnings
- Increased buffer that caused warnings from strncpy
One warning from InnoDB, others from Connect.
This is used by InnoDB to detect if CREATE...SELECT is used
Other things:
- Changed InnoDB to use thd_ddl_options()
- Removed lock checking code for create...select (Approved by Marko)
In commit 49e2c8f0a6 (MDEV-25743)
we made dict_sys_t::find() incompatible with the rest of the
table name hash table operations in case the table name contains
non-ASCII octets (using a compatibility mode that facilitates the
upgrade into the MySQL 5.0 filename-safe encoding) and the target
platform implements signed char.
ut_fold_string(): Remove; replace with my_crc32c(). This also makes
table name hash value calculations independent on whether char
is unsigned or signed.
In commit de407e7cb4 a debug assertion
was added that would not always hold: We could have TRX_STATE_PREPARED
here. But, in that case, the transaction should not have been chosen
as a deadlock victim.
Problem:
=======
When the semisync master is crashed and restarted as slave it could
recover transactions that former slaves may never have seen.
A known method existed to clear out all prepared transactions
with --tc-heuristic-recover=rollback does not care to adjust
binlog accordingly.
Fix:
===
The binlog-based recovery is made to concern of the slave semisync role of
post-crash restarted server.
No changes in behavior is done to the "normal" binloggging server
and the semisync master.
When the restarted server is configured with
--rpl-semi-sync-slave-enabled=1
the refined recovery attempts to roll back prepared transactions
and truncate binlog accordingly.
In case of a partially committed (that is committed at least
in one of the engine participants) such transaction gets committed.
It's guaranteed no (partially as well) committed transactions
exist beyond the truncate position.
In case there exists a non-transactional replication event
(being in a way a committed transaction) past the
computed truncate position the recovery ends with an error.
As after master crash and failover to slave, the demoted-to-slave
ex-master must be ready to face and accept its own (generated by)
events, without generally necessary --replicate-same-server-id.
So the acceptance conditions are relaxed for the semisync slave
to accept own events without that option.
While gtid_strict_mode ON ensures no duplicate transaction can be
(re-)executed the master_use_gtid=none slave has to be
configured with --replicate-same-server-id.
*NOTE* for reviewers.
This patch does not handle the user XA which is done
in next git commit.
also avoid an oxymoron of using `MYSQL_PLUGIN_IMPORT` under
`#ifdef MYSQL_SERVER`, and empty_clex_str is so trivial that a plugin
can define it if needed.
