mariadb/mysql-test/suite/mariabackup/incremental_ddl_during_backup.test

60 lines
2 KiB
Text
Raw Normal View History

--source include/have_debug.inc
--source include/innodb_undo_tablespaces.inc
call mtr.add_suppression("InnoDB: New log files created");
let $basedir=$MYSQLTEST_VARDIR/tmp/backup;
let $incremental_dir=$MYSQLTEST_VARDIR/tmp/backup_inc1;
CREATE TABLE t1(i INT PRIMARY KEY) ENGINE INNODB;
CREATE TABLE t2(i INT PRIMARY KEY) ENGINE INNODB;
CREATE TABLE t3(i INT) ENGINE INNODB;
CREATE TABLE t10(i INT PRIMARY KEY) ENGINE INNODB;
MDEV-24626 Remove synchronous write of page0 file during file creation During data file creation, InnoDB holds dict_sys mutex, tries to write page 0 of the file and flushes the file. This not only causing unnecessary contention but also a deviation from the write-ahead logging protocol. The clean sequence of operations is that we first start a dictionary transaction and write SYS_TABLES and SYS_INDEXES records that identify the tablespace. Then, we durably write a FILE_CREATE record to the write-ahead log and create the file. Recovery should not unnecessarily insist that the first page of each data file that is referred to by the redo log is valid. It must be enough that page 0 of the tablespace can be initialized based on the redo log contents. We introduce a new data structure deferred_spaces that keeps track of corrupted-looking files during recovery. The data structure holds the last LSN of a FILE_ record referring to the data file, the tablespace identifier, and the last known file name. There are two scenarios can happen during recovery: i) Sufficient memory: InnoDB can reconstruct the tablespace after parsing all redo log records. ii) Insufficient memory(multiple apply phase): InnoDB should store the deferred tablespace redo logs even though tablespace is not present. InnoDB should start constructing the tablespace when it first encounters deferred tablespace id. Mariabackup copies the zero filled ibd file in backup_fix_ddl() as the extension of .new file. Mariabackup test case does page flushing when it deals with DDL operation during backup operation. fil_ibd_create(): Remove the write of page0 and flushing of file fil_ibd_load(): Return FIL_LOAD_DEFER if the tablespace has zero filled page0 Datafile: Clean up the error handling, and do not report errors if we are in the middle of recovery. The caller will check Datafile::m_defer. fil_node_t::deferred: Indicates whether the tablespace loading was deferred during recovery FIL_LOAD_DEFER: Returned by fil_ibd_load() to indicate that tablespace file was cannot be loaded. recv_sys_t::recover_deferred(): Invoke deferred_spaces.create() to initialize fil_space_t based on buffered metadata and records to initialize page 0. Ignore the flags in fil_name_t, because they are intentionally invalid. fil_name_process(): Update deferred_spaces. recv_sys_t::parse(): Store the redo log if the tablespace id is present in deferred spaces recv_sys_t::recover_low(): Should recover the first page of the tablespace even though the tablespace instance is not present recv_sys_t::apply(): Initialize the deferred tablespace before applying the deferred tablespace records recv_validate_tablespace(): Skip the validation for deferred_spaces. recv_rename_files(): Moved and revised from recv_sys_t::apply(). For deferred-recovery tablespaces, do not attempt to rename the file if a deferred-recovery tablespace is associated with the name. recv_recovery_from_checkpoint_start(): Invoke recv_rename_files() and initialize all deferred tablespaces before applying redo log. fil_node_t::read_page0(): Skip page0 validation if the tablespace is deferred buf_page_create_deferred(): A variant of buf_page_create() when the fil_space_t is not available yet This is joint work with Thirunarayanan Balathandayuthapani, who implemented an initial prototype.
2021-05-17 18:12:33 +03:00
set global innodb_log_checkpoint_now = 1;
echo # Create full backup , modify table, then create incremental/differential backup;
--disable_result_log
exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$basedir;
--enable_result_log
INSERT into t1 values(1);
--let after_load_tablespaces=CREATE TABLE test.t4 ENGINE=INNODB SELECT UUID() from test.seq_1_to_10000
--let after_copy_test_t1=RENAME TABLE test.t1 TO test.t1_renamed
--let after_copy_test_t2=DROP TABLE test.t2
--let after_copy_test_t3=CREATE INDEX a_i ON test.t3(i);
--let before_copy_test_t10=DROP TABLE test.t10
MDEV-22929 MariaBackup option to report and/or continue when corruption is encountered The new option --log-innodb-page-corruption is introduced. When this option is set, backup is not interrupted if innodb corrupted page is detected. Instead it logs all found corrupted pages in innodb_corrupted_pages file in backup directory and finishes with error. For incremental backup corrupted pages are also copied to .delta file, because we can't do LSN check for such pages during backup, innodb_corrupted_pages will also be created in incremental backup directory. During --prepare, corrupted pages list is read from the file just after redo log is applied, and each page from the list is checked if it is allocated in it's tablespace or not. If it is not allocated, then it is zeroed out, flushed to the tablespace and removed from the list. If all pages are removed from the list, then --prepare is finished successfully and innodb_corrupted_pages file is removed from backup directory. Otherwise --prepare is finished with error message and innodb_corrupted_pages contains the list of the pages, which are detected as corrupted during backup, and are allocated in their tablespaces, what means backup directory contains corrupted innodb pages, and backup can not be considered as consistent. For incremental --prepare corrupted pages from .delta files are applied to the base backup, innodb_corrupted_pages is read from both base in incremental directories, and the same action is proceded for corrupted pages list as for full --prepare. innodb_corrupted_pages file is modified or removed only in base directory. If DDL happens during backup, it is also processed at the end of backup to have correct tablespace names in innodb_corrupted_pages.
2020-08-20 16:49:40 +03:00
--let wait_innodb_redo_before_copy_test_t10 = 1
# mariabackup should crash with assertion if MDEV-24026 is not fixed
exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf --backup --target-dir=$incremental_dir --incremental-basedir=$basedir --dbug=+d,mariabackup_events,mariabackup_inject_code;
--let after_load_tablespaces=
--disable_result_log
echo # Prepare full backup, apply incremental one;
MDEV-18215: mariabackup does not report unknown command line options MDEV-21298: mariabackup doesn't read from the [mariadbd] and [mariadbd-X.Y] server option groups from configuration files MDEV-21301: mariabackup doesn't read [mariadb-backup] option group in configuration file All three issues require to change the same code, that is why their fixes are joined in one commit. The fix is in invoking load_defaults_or_exit() and handle_options() for backup-specific groups separately from client-server groups to let the last handle_options() call fail on unknown backup-specific options. The order of options procesing is the following: 1) Load server groups and process server options, ignore unknown options 2) Load client groups and process client options, ignore unknown options 3) Load backup groups and process client-server options, exit on unknown option 4) Process --mysqld-args command line options, ignore unknown options New global flag my_handle_options_init_variables was added to have ability to invoke handle_options() for the same allowed options set several times without re-initialising previously set option values. --password value destroying is moved from option processing callback to mariabackup's handle_options() function to have ability to invoke server's handle_options() several times for the same possible allowed options set. Galera invokes wsrep_sst_mariabackup.sh with mysqld command line options to configure mariabackup as close to the server as possible. It is not known what server options are supported by mariabackup when the script is invoked. That is why new mariabackup option "--mysqld-args" is added, all unknown options that follow this option will be silently ignored. wsrep_sst_mariabackup.sh was also changed to: - use "--mysqld-args" mariabackup option to pass mysqld options, - remove deprecated innobackupex mode, - remove unsupported mariabackup options: --encrypt --encrypt-key --rebuild-indexes --rebuild-threads
2020-06-08 11:45:56 +03:00
exec $XTRABACKUP --prepare --target-dir=$basedir;
exec $XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir ;
echo # Restore and check results;
let $targetdir=$basedir;
-- source include/restart_and_restore.inc
--enable_result_log
# Test that t1 does not exist, but t1_renamed does
CREATE TABLE t1(i int);
DROP TABLE t1;
SELECT * from t1_renamed;
DROP TABLE t1_renamed;
# Test that t2 does not exist;
CREATE TABLE t2(i INT PRIMARY KEY) ENGINE INNODB;
DROP TABLE t2;
DROP TABLE t3;
DROP TABLE t4;
# Cleanup
rmdir $basedir;
rmdir $incremental_dir;