mariadb/mysql-test/suite/mariabackup
Marko Mäkelä 852d42e993 MDEV-34483 Backup may copy unnecessarily much log
In mariadb-backup --backup there are multiple mechanisms for ensuring that
a sufficient amount of the InnoDB write-ahead log (ib_logfile0) is being
copied at the end of the backup. The backup needs to include the latest
committed transaction. While further transaction commits are blocked by
BACKUP STAGE BLOCK_COMMIT, ongoing transactions may modify the database
contents and write log records. We were unnecessarily copying such log,
which would also cause further effort of rolling back incomplete
transactions after the backup is restored.

backup_wait_for_lsn(): Declare as static, and refactor some code
to separate functions backup_wait_for_lsn_low() and
backup_wait_timeout().

backup_wait_for_commit_lsn(): A new function to determine the current
LSN (within BACKUP STAGE BLOCK_COMMIT) and to wait for the log to be
copied until that. Invoked by BackupStages::stage_block_commit().

xtrabackup_backup_func(): Remove a condition that had already been
checked by a caller of backup_wait_timeout().

server_lsn_after_lock: Declare as a local variable in
BackupStages::stage_block_ddl().

log_copying_thread(), io_watching_thread(): Use metadata_last_lsn
instead of metadata_to_lsn as the stop condition.

BackupStages::stage_block_commit(): Ensure that the log tables
(in particular, mysql.general_log) will have been copied before
the BACKUP STAGE BLOCK_COMMIT is being followed by any further
SQL statements.

Reviewed by: Debarun Banerjee
Tested by: Matthias Leich
2024-09-09 16:47:35 +03:00
..
include
std_data
absolute_ibdata_paths.opt
absolute_ibdata_paths.result
absolute_ibdata_paths.test
alter_copy_excluded.opt
alter_copy_excluded.result
alter_copy_excluded.test
alter_copy_race.result
alter_copy_race.test
apply-log-only-incr.result
apply-log-only-incr.test
apply-log-only.result
apply-log-only.test
aria_backup.opt
aria_backup.result
aria_backup.test
aria_log.opt
aria_log_dir_path.result
aria_log_dir_path.test
aria_log_dir_path_rel.result
aria_log_dir_path_rel.test
aria_log_rotate_during_backup.opt
aria_log_rotate_during_backup.result
aria_log_rotate_during_backup.test
auth_plugin_win.opt
auth_plugin_win.result
auth_plugin_win.test
backup_grants.result
backup_grants.test
backup_lock_wait_timeout.result
backup_lock_wait_timeout.test
backup_ssl.result
backup_ssl.test
big_innodb_log.result
big_innodb_log.test
binlog.result
binlog.test
compress_qpress.result
compress_qpress.test
compression_providers_loaded,lz4.rdiff
compression_providers_loaded,lzma.rdiff
compression_providers_loaded,lzo.rdiff
compression_providers_loaded,snappy.rdiff
compression_providers_loaded.combinations
compression_providers_loaded.result
compression_providers_loaded.test
compression_providers_unloaded,lz4.rdiff
compression_providers_unloaded,lzma.rdiff
compression_providers_unloaded,lzo.rdiff
compression_providers_unloaded,snappy.rdiff
compression_providers_unloaded.combinations
compression_providers_unloaded.result
compression_providers_unloaded.test
create_during_backup.result
create_during_backup.test
create_with_data_directory_during_backup.result
create_with_data_directory_during_backup.test
data_directory.result
data_directory.test
ddl_for_common_engine.result
ddl_for_common_engine.test
defer_space.result
defer_space.test
disabled.def
drop_table_during_backup.result
drop_table_during_backup.test
encrypted_export.opt
encrypted_export.result
encrypted_export.test
encrypted_page_compressed.opt
encrypted_page_compressed.result
encrypted_page_compressed.test
encrypted_page_corruption,full_crc32.rdiff
encrypted_page_corruption.combinations
encrypted_page_corruption.opt
encrypted_page_corruption.result
encrypted_page_corruption.test
error_during_copyback.result
error_during_copyback.test
extra_lsndir.result
extra_lsndir.test
extra_lsndir_stream.result
extra_lsndir_stream.test
filekeys-data.enc
filekeys-data.key
full_backup.opt
full_backup.result
full_backup.test
full_backup_win.result
full_backup_win.test
huge_lsn,strict_crc32.rdiff
huge_lsn.combinations
huge_lsn.opt
huge_lsn.result
huge_lsn.test
incremental_backup.result
incremental_backup.test
incremental_backup_newdb_before_inc.result
incremental_backup_newdb_before_inc.test
incremental_compressed.result
incremental_compressed.test
incremental_ddl_before_backup.result
incremental_ddl_before_backup.test
incremental_ddl_during_backup.result
incremental_ddl_during_backup.test
incremental_drop_db.result
incremental_drop_db.test
incremental_encrypted.opt
incremental_encrypted.result
incremental_encrypted.test
incremental_newdb_while_backup.result
incremental_newdb_while_backup.test
incremental_page_compressed.result
incremental_page_compressed.test
incremental_rocksdb.opt
incremental_rocksdb.result
incremental_rocksdb.test
innodb_ddl_on_intermediate_table.result
innodb_ddl_on_intermediate_table.test
innodb_force_recovery.result
innodb_force_recovery.test
innodb_redo_log_overwrite.opt
innodb_redo_log_overwrite.result
innodb_redo_log_overwrite.test
innodb_redo_overwrite.opt
innodb_redo_overwrite.result
innodb_redo_overwrite.test
lock_ddl_per_table.opt
lock_ddl_per_table.result
lock_ddl_per_table.test
log_checksum_mismatch.result
log_checksum_mismatch.test
log_copy_interval.result
log_copy_interval.test
log_file_unexpected_large_number_in_name.result
log_file_unexpected_large_number_in_name.test
log_page_corruption.opt
log_page_corruption.result
log_page_corruption.test
log_tables.result
log_tables.test
mdev-14447.combinations
mdev-14447.opt
mdev-14447.result
mdev-14447.test
mdev-18438.result
mdev-18438.test
missing_ibd.result
missing_ibd.test
nolock_ddl_during_backup_end.result
nolock_ddl_during_backup_end.test
options_check.result
options_check.test
page_compression_level.result
page_compression_level.test
partial.result
partial.test
partial_exclude.opt
partial_exclude.result
partial_exclude.test
partition_datadir.result
partition_datadir.test
partition_partial.opt
partition_partial.result
partition_partial.test
recreate_table_during_backup.result
recreate_table_during_backup.test
relative_path.opt
relative_path.result
relative_path.test
rename_during_backup.result
rename_during_backup.test
rename_during_mdl_lock.result
rename_during_mdl_lock.test
row_format_redundant.result
row_format_redundant.test
rpl_clone_slave.result
rpl_clone_slave.test
rpl_slave_info.result
rpl_slave_info.test
skip_innodb.opt
skip_innodb.result
skip_innodb.test
slave_info_norpl.result
slave_info_norpl.test
slave_provision_nolock.cnf
slave_provision_nolock.result
slave_provision_nolock.test
small_ibd.result
small_ibd.test
suite.opt
suite.pm
system_versioning.result
system_versioning.test
truncate_during_backup.result
truncate_during_backup.test
undo_space_id.opt
undo_space_id.result
undo_space_id.test
undo_upgrade.result
undo_upgrade.test
unencrypted_page_compressed.result
unencrypted_page_compressed.test
unsupported_redo.result
unsupported_redo.test
xb_aws_key_management.opt
xb_aws_key_management.result
xb_aws_key_management.test
xb_compressed_encrypted.opt
xb_compressed_encrypted.result
xb_compressed_encrypted.test
xb_file_key_management.opt
xb_file_key_management.result
xb_file_key_management.test
xb_fulltext_encrypted.opt
xb_fulltext_encrypted.result
xb_fulltext_encrypted.test
xb_history.result
xb_history.test
xb_page_compress.result
xb_page_compress.test
xb_partition.result
xb_partition.test
xb_rocksdb.opt
xb_rocksdb.result
xb_rocksdb.test
xb_rocksdb_datadir.opt
xb_rocksdb_datadir.result
xb_rocksdb_datadir.test
xb_rocksdb_datadir_debug.opt
xb_rocksdb_datadir_debug.result
xb_rocksdb_datadir_debug.test
xbstream.result
xbstream.test