The InnoDB write-ahead log file in the old innodb_log_archive=OFF
format is named ib_logfile0, pre-allocated to innodb_log_file_size and
written as a ring buffer. This is good for write performance and space
management, but unsuitable for arbitrary point-in-time recovery or for
facilitating efficient incremental backup.
innodb_log_archive=ON: A new format where InnoDB will create and
preallocate files ib_%016x.log, instead of writing a circular file
ib_logfile0. Each file will be pre-allocated to innodb_log_file_size.
Once a log fills up, we will create and pre-allocate another log file,
to which log records will be written. Upon the completion of the first
log checkpoint in a recently created log file, the old log file will
be marked read-only, signaling that there will be no further writes to
that file, and that the file may safely be moved to long-term storage.
The file name includes the log sequence number (LSN) at file offset
12288 (log_t::START_OFFSET). Each checkpoint is identified by storing
a 64-bit big-endian offset into an optional sequence of FILE_MODIFY
records followed by a FILE_CHECKPOINT record, between 12288 and the
end of the file.
The innodb_encrypt_log format is identified by storing the encryption
information at the start of the log file. The first 64-bit value will
be 1, which is an invalid checkpoint offset. Each
innodb_log_archive=ON log must use the same encryption parameters.
Changing innodb_encrypt_log or related parameters is only possible by
setting innodb_log_archive=OFF and restarting the server, which will
permanently lose the history of the archived log.
The maximum number of log checkpoints that the innodb_log_archive=ON
file header can represent is limited to 12288/8=1536 when using
innodb_encrypt_log=OFF. If we run out of slots in a log file, each
subsequently completed checkpoint in that log file will overwrite the
last slot in the checkpoint header, until we switch to the next log.
innodb_log_recovery_start: The checkpoint LSN to start recovery from.
This will be useful when recovering from an archived log. This is useful
for restoring an incremental backup (applying InnoDB log files that were
copied since the previous restore).
innodb_log_recovery_target: The requested LSN to end recovery at.
When this is set, all persistent InnoDB tables will be read-only, and
no writes to the log are allowed. The intended purpose of this setting
is to prepare an incremental backup, as well as to allow data
retrieval as of a particular logical point of time.
Setting innodb_log_recovery_target>0 is much like setting
innodb_read_only=ON, with the exception that the data files may be
written to by crash recovery, and locking reads will conflict with any
incomplete transactions as necessary, and all transaction isolation
levels will work normally (not hard-wired to READ UNCOMMITTED).
srv_read_only_mode: When this is set (innodb_read_only=ON), also
recv_sys.rpo (innodb_log_recovery_target) will be set to the current LSN.
This ensures that it will suffice to check only one of these variables
when blocking writes to persistent tables.
The status variable innodb_lsn_archived will reflect the LSN
since when a complete InnoDB log archive is available. Its initial
value will be that of the new parameter innodb_log_archive_start.
If that variable is 0 (the default), the innodb_lsn_archived will
be recovered from the available log files. If innodb_log_archive=OFF,
innodb_lsn_archived will be adjusted to the latest checkpoint every
time a log checkpoint is executed. If innodb_log_archive=ON, the value
should not change.
SET GLOBAL innodb_log_archive=!@@GLOBAL.innodb_log_archive will take
effect as soon as possible, possibly after a log checkpoint has been
completed. The log file will be renamed between ib_logfile0 and
ib_%016x.log as appropriate.
When innodb_log_archive=ON, the setting SET GLOBAL innodb_log_file_size
will affect subsequently created log files when the file that is being
currently written is running out. If we are switching log files exactly
at the same time, then a somewhat misleading error message
"innodb_log_file_size change is already in progress" will be issued.
no_checkpoint_prepare.inc: A new file, to prepare for subsequent
inclusion of no_checkpoint_end.inc. We will invoke the server to
parse the log and to determine the latest checkpoint.
All --suite=encryption tests that use innodb_encrypt_log
will be skipped for innodb_log_archive=ON, because enabling
or disabling encryption on the log is not possible without
temporarily setting innodb_log_archive=OFF and restarting
the server. The idea is to add the following arguments to an
invocation of mysql-test/mtr:
--mysqld=--loose-innodb-log-archive \
--mysqld=--loose-innodb-log-recovery-start=12288 \
--mysqld=--loose-innodb-log-file-mmap=OFF \
--skip-test=mariabackup
Alternatively, specify --mysqld=--loose-innodb-log-file-mmap=ON
to cover both code paths.
The mariabackup test suite must be skipped when using the
innodb_log_archive=ON format, because mariadb-backup will only
support the old ib_logfile0 format (innodb_log_archive=OFF).
A number of tests would fail when the parameter
innodb_log_recovery_start=12288 is present, which is forcing
recovery to start from the beginning of the history
(the database creation). The affected tests have been adjusted with
explicit --innodb-log-recovery-start=0 to override that:
(0) Some injected corruption may be "healed" by replaying the log
from the beginning. Some tests expect an empty buffer pool after
a restart, with no page I/O due to crash recovery.
(1) Any test that sets innodb_read_only=ON would fail with an error
message that the setting prevents crash recovery, unless
innodb_log_recovery_start=0.
(2) Any test that changes innodb_undo_tablespaces would fail in crash
recovery, because crash recovery assumes that the undo tablespace ID
that is available from the undo* files corresponds with the start of
the log. This is an unforunate design bug which we cannot fix easily.
log_sys.first_lsn: The start of the current log file, to be consulted
in log_t::write_checkpoint() when renaming files.
log_sys.archived_lsn: New field: The value of innodb_lsn_archived.
log_sys.end_lsn: New field: The log_sys.get_lsn() when the latest
checkpoint was initiated. That is, the start LSN of a possibly empty
sequence of FILE_MODIFY records followed by FILE_CHECKPOINT.
log_sys.resize_target: The value of innodb_log_file_size that will be
used for creating the next archive log file once the current file (of
log_sys.file_size) fills up.
log_sys.archive: New field: The value of innodb_log_archive.
log_sys.next_checkpoint_no: Widen to uint16_t. There may be up to
12288/8=1536 checkpoints in the header.
log_sys.log: If innodb_log_archive=ON, this file handle will be kept
open also in the PMEM code path.
log_sys.resize_log: If innodb_log_archive=ON, we may have two log
files open both during normal operation and when parsing the log. This
will store the other handle (old or new file).
log_sys.resize_buf: In the memory-mapped code path, this will point
to the file resize_log when innodb_log_archive=ON.
recv_sys.log_archive: All innodb_log_archive=ON files that will be
considered in recovery.
recv_sys.was_archive: A flag indicating that an innodb_log_archive=ON
file is in innodb_log_archive=OFF format.
log_sys.is_pmem, log_t::is_mmap_writeable(): A new predicate.
If is_mmap_writeable(), we assert and guarantee buf_size == capacity().
log_t::checkpoint_age_max(): Return the maximum checkpoint age.
When innodb_log_archive=ON the limit is innodb_log_file_size larger.
log_t::archive_new_write(): Create and allocate a new log file, and
write the outstanding data to both the current and the new file, or
only to the new file, until write_checkpoint() completes the first
checkpoint in the new file. If the log file would overflow,
increase innodb_log_file_size to the next multiple of 4096 bytes.
log_t::archive_create(bool): Create and allocate an archive log file.
On POSIX, ensure that we get the exact correct file size by invoking
ftruncate() on POSIX, because os_file_set_size() would only
truncate files on Microsoft Windows. Remember the file handle
of the current log in resize_log, so that write_checkpoint() will be
able to make it read-only.
log_t::archived_mmap_switch_prepare(): Invoke archive_create() unless
another thread already did it.
log_t::archived_mmap_switch_complete(): Switch to the buffer that was
created in log_t::archived_mmap_switch_prepare(). Remember the old
log_sys.buf in log_sys.checkpoint_buf, to be unmapped in
log_t::write_checkpoint().
buf_flush_archive_create(): Create a spare archive log file if needed.
buf_flush_page_cleaner(): Invoke buf_flush_archive_create() about
one second after a checkpoint where we switched archived log files.
This allows any writer threads to resume activity after we completed
the previous checkpoint.
log_t::write_checkpoint(): Allow an old checkpoint to be completed in
the old log file even after a new one has been created. If we are
writing the first checkpoint in a new log file, we will mark the old
log file read-only. We will also update log_sys.first_lsn unless it
was already updated in ARCHIVED_MMAP code path. In that code path,
there is the special case where log_sys.resize_buf == nullptr and
log_sys.checkpoint_buf points to log_sys.resize_log (the old log file
that is about to be made read-only). In this case, log_sys.first_lsn
will already point to the start of the current log_sys.log, even
though the switch has not been fully completed yet.
Try to preallocate the next archive file if needed, with the goal that
when we need the file it will already be ready for use.
log_t::write_checkpoint(), log_checkpoint_low(), log_checkpoint(),
buf_flush_sync_for_checkpoint(): Return the LSN on which to invoke
archive_create(), or 0 if no spare archive log file needs to be created.
log_t::header_rewrite(my_bool): Rewrite the log file header before or
after renaming the log file, and write a message about the change,
so that there will be a chance to recover in case the server is being
killed during this operation. The recovery of the last ib_%016%.log
does tolerate also the ib_logfile0 format.
log_t::set_archive(my_bool,THD): Implement SET GLOBAL innodb_log_archive.
An error will be returned if non-archived SET GLOBAL innodb_log_file_size
(log file resizing) is in progress. Wait for checkpoint if necessary.
The current log file will be renamed to either ib_logfile0 or
ib_%016x.log, as appropriate. In SET GLOBAL innodb_log_archive=OFF,
we trigger a write-ahead of the log if necessary, to prevent overrun.
log_t::archive_rename(): Rename an archived log to ib_logfile0 on recovery
in case there had been a crash during set_archive().
log_t::archive_set_size(): A new function, to ensure that
log_sys.resize_target is set on startup.
log_checkpoint_low(): Do not prevent a checkpoint at the start of a file.
We want the first innodb_log_archive=ON file to start with a checkpoint.
log_t::create(lsn_t): Initialize last_checkpoint_lsn. Initialize the
log header as specified by log_sys.archive (innodb_log_archive).
log_write_buf(): Add the parameter max_length, the file wrap limit.
log_write_up_to(), mtr_t::commit_log_release<bool mmap=true>():
If we are switching log files, invoke buf_flush_ahead(lsn, false)
to ensure that a log checkpoint will be completed in the new file.
mtr_t::finish_writer(): Specialize for innodb_log_archive=ON.
mtr_t::commit_file(): Ensure that log archive rotation will complete.
log_t::append_prepare<log_t::ARCHIVED_MMAP>(): Special case.
log_t::append_prepare(): Remove the call to set_check_for_checkpoint()
that was duplicating logic in log_close().
log_t::get_path(): Get the name of the current log file.
log_t::get_circular_path(size_t): Get the path name of a circular file.
Replaces get_log_file_path().
log_t::get_archive_path(lsn_t): Return a name of an archived log file.
log_t::append_archive_name(): Append the archive log file name
to a path string.
log_t::checkpoint_margin(): Replaces log_checkpoint_margin().
If a new archived log file has been created, wait for the
first checkpoint in that file. Revise the formula when
log_sys.need_checkpoint needs to hold. When innodb_log_archive=ON,
we can allow the checkpoint to be older by innodb_log_file_size.
log_close(): Relax the log_sys.need_checkpoint condition
for innodb_log_archive=ON.
srv_log_rebuild_if_needed(): Never rebuild if innodb_log_archive=ON.
The setting innodb_log_file_size will affect the creation of
subsequent log files. The parameter innodb_encrypt_log cannot be
changed while the log is in the innodb_log_archive=ON format.
log_t::attach(), log_mmap(): Add the parameter log_access,
to distinguish memory-mapped or read-only access.
log_t::attach(): When disabling innodb_log_file_mmap, read
checkpoint_buf from the last innodb_log_archive=ON file.
log_t::clear_mmap(): Clear the tail of the checkpoint buffer
if is_mmap_writeable().
log_t::set_recovered(): Invoke clear_mmap(), and restore the
log buffer to the correct position.
recv_sys_t::apply(): Let log_t::clear_mmap() enable log writes.
log_file_is_zero(): Check if a log file starts with NUL bytes
(is a preallocated file).
recv_sys_t::find_checkpoint(): Find and remember the checkpoint position
in the last file when innodb_log_recovery_start points to an older file.
When innodb_log_file_mmap=OFF, restore log_sys.checkpoint_buf from
the latest log file. If the last archive log file is actually
in innodb_log_archive=OFF format despite being named ib_%016.log,
try to recover it in that format. If the circular ib_logfile0 is missing,
determine the oldest archived log file with contiguous LSN.
If innodb_log_archive=ON, refuse to start if ib_logfile0 exists.
Open non-last non-preallocated archived log file in read/write mode.
recv_sys_t::find_checkpoint_archived(): Validate each checkpoint in
the current file header, and by default aim to recover from the last
valid one. Terminate the search if the last validated checkpoint
spanned two files. If innodb_log_recovery_start has been specified,
attempt to validate it even if there is no such information stored
in the checkpoint header.
log_parse_file(): Do not invoke fil_name_process() during
recv_sys_t::find_checkpoint_archived(), when we tolerate FILE_MODIFY
records while looking for a FILE_CHECKPOINT record.
recv_scan_log(): Invoke log_t::archived_switch_recovery() upon
reaching the end of the current archived log file.
log_t::archived_switch_recovery_prepare(): Make use of
recv_sys.log_archive and open all but the last file read-only.
log_t::archived_switch_recovery(): Switch files in the pread() code path.
log_t::archived_mmap_switch_recovery_complete(): Switch files in the
memory-mapped code path.
recv_warp: A pointer wrapper for memory-mapped parsing that spans two
archive log files.
recv_sys_t::parse_mmap(): Use recv_warp for innodb_log_archive=ON.
recv_sys_t::parse(): Tweak some logic for innodb_log_archive=ON.
log_t::set_recovered_checkpoint(): Set the checkpoint on recovery.
Updates also the end_lsn.
log_t::set_recovered_lsn(): Also update flush_lock and write_lock,
to ensure that log_write_up_to() will be a no-op. Invoke
unstash_archive_file() in case there was a garbage (pre-allocated)
file at the end which was not parsed at all.
log_t::persist(): Even if the flushed_to_disk_lsn does not change,
we may want to reset the write_lsn_offset.
The mariadb_upgrade_info file was not being backed up during
mariadb-backup --backup, causing MariaDB to incorrectly prompt for
upgrade after restore.
Add mariadb_upgrade_info to the list of files backed up from the datadir
during backup operations, ensuring the upgrade state is preserved across
backup and restore cycles.
All new code of the whole pull request, including one or several files
that are that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Remove function checks that are guaranteed by C89/C99 standards to
reduce the excessive time spent during CMake configuration step. Also,
clean up related compatibility code.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Problem:
`mariabackup` ignores `my_close()` failures, resulting in false
success report with a 'completed OK!' message.
Fix:
Update `local_close()` function in `mariabackup` to check `my_close()`
failures and enusre errors are reported.
log_sys.latch: Remove the PERFORMANCE_SCHEMA instrumentation.
We already know that this is a very busy latch. All code paths
where a shared log_sys.latch is being held should already be
highly optimized. The few paths where an exclusive log_sys.latch
is being held are known to be potentially problematic, but rare.
Removing the PERFORMANCE_SCHEMA instrumentation for this latch
is expected to improve performance. On a quick Sysbench run with
performance_schema_instrument=wait/synch/rwlock/%=on
I observed an 2.9% improvement on throughput.
srw_lock_debug_simple: A non-instrumented version of srw_lock_debug,
to allow PERFORMANCE_SCHEMA to work WITH_INNODB_EXTRA_DEBUG=ON.
Reviewed by: Alessandro Vetere
Concurrent calls to log_checkpoint_low() were possible from multiple
threads, and they could cause redundant writes of FILE_CHECKPOINT
records that refer to the same checkpoint. Let us simplify the logic
by making the dedicated buf_flush_page_cleaner() thread responsible
for checkpoints.
log_t::write_checkpoint(lsn_t end_lsn): Add the parameter checkpoint,
which will replace the data member log_sys.next_checkpoint_lsn.
log_sys.checkpoint_pending: Remove. Only the buf_flush_page_cleaner()
thread will write checkpoints or initiate page writes.
log_checkpoint_low(), log_checkpoint(): Remove the return value,
because there cannot be any concurrent log checkpoint in progress.
buf_flush_wait(): Add a parameter for waiting for a full checkpoint.
This function replaces buf_flush_wait_flushed().
log_t::checkpoint_margin(): Replaces log_checkpoint_margin().
lot_t::write_buf(): Remove a call set_check_for_checkpoint(false)
that commit 7443ad1c8a (MDEV-32374)
had added. The flag should only be cleared when the checkpoint has
advanced far enough.
log_make_checkpoint(): Simply wrap buf_flush_sync_batch(0, true).
buf_flush_sync_batch(): Add the parameter bool checkpoint, to wait
for an empty FILE_CHECKPOINT record to be written. Outside recovery,
pass lsn=0.
buf_flush_sync_for_checkpoint(): On shutdown, update the systemd
watchdog and keep flushing until a final checkpoint has been written.
buf_flush_page_cleaner(): Revise the shutdown logic so that all
changes will be written out and a checkpoint with just a FILE_CHECKPOINT
record can be written.
buf_flush_buffer_pool(): Remove.
buf_flush_wait_flushed(): Require the caller to acquire
buf_pool.flush_list_mutex.
logs_empty_and_mark_files_at_shutdown(): Simplify the logic,
and return the shutdown LSN.
fil_names_clear(): Fix an off-by-one error that would prevent
removal from fil_system.named_spaces.
innodb_shutdown(): Always invoke logs_empty_and_mark_files_at_shutdown().
srv_undo_tablespaces_reinit(): Simplify the logic and remove the
fault injection point after_reinit_undo_abort, which would cause
a failure on Microsoft Windows. Changing innodb_undo_tablespaces is
not fully crash-safe.
Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam
Currently, posix_fadvise(..., 0, 0, POSIX_FADV_DONTNEED) is executed
repeatedly after each my_write. In case of pipes, it evaluates to
ESPIPE and fails. This results in millions of redundant syscalls,
introducing massive overhead.
As proposed by Daniel Black, call posix_fadvise with a length/size
of 0 once when the file is opened, rather than repetitively after
every write. This change moves the fadvise call to the respective
_open functions for ds_local, ds_stdout and ds_tmpfile datasinks.
It fails fast on pipes early on and eliminates the redundant
syscall overhead for subsequent write operations.
When a cluster donor node executes mariabackup SST, it will use same
approximation for max open files limit as was set for the mariadbd
process during the server startup. This may be a problem for installation
where mariabackup execution would need higher open file count, and might
crash for exceeding the too tight open file limit.
The reason for this behavior is that when mariadbd server calculates the
expected max open files count, it will record this open file count approximation
as system ulimit value, both as soft and hard limit. Later, when the node
operates as SST donor, it spawns mariabackup executable, which now inherits the
ulmit setting used for the mariadbd process. Mariabackup tries to enforce
open_files_limit variable value configured in [mariabackup] group in the my.cnf
fle, but this will fail if hard ulimit value is smaller.
The fix in this commit records the approximated max open file count only as soft
ulimit value. If hard ulimit is higher or unlimited, there remains head room for
the mariabackup to use higher open_files_limit configuration.
The InnoDB bootstrap is creating unnecessary log checkpoints
and even invalidating the buffer pool for no reason, making it
necessary to read pages into the buffer pool during bootstrap.
Furthermore, the page cleaner thread attempts to invoke
recv_sys.apply(true), even though most calls were unreachable
starting with commit 685d958e38 (MDEV-14425)
if not earlier.
buf_flush_sync(): Replaced with buf_flush_sync_batch(LSN_MAX),
possibly preceded by recv_sys.apply(true).
buf_pool_t::assert_all_freed(): Assert that buf_pool.mutex is
being held by the caller.
buf_dblwr_t::create(): Evict each doublewrite buffer pool page
immediately from the buffer pool, so that there will be no need
to invoke buf_pool_invalidate(). Also, remove the unnecessary
call to buf_flush_wait_flushed(); there is nothing special about
the doublewrite buffer creation ever since
commit e2c63a7062 (MDEV-38595)
made the redo logging actually crash-safe.
buf_pool_invalidate(): Moved to the same compilation unit with
the only remaining caller, recv_sys_t::apply().
srv_start(): Do not invoke buf_flush_sync(). We already created
an initial checkpoint in log_t::create(lsn_t), and we will create
the final one during shutdown. Together with the simplification of
buf_dblwr_t::create(), this should noticeably speed up the
server bootstrap. Also, simplify the log file rebuild by extending
the critical section of log_sys.latch and buf_pool.flush_list_mutex.
create_log_file(): Assert and assume that both log_sys.latch and
buf_pool.flush_list_mutex were acquired by the caller.
This prevents any race condition with buf_flush_page_cleaner()
when the log is being rebuilt.
srv_prepare_to_delete_redo_log_file(): Invoke recv_sys.apply(true)
before invoking buf_flush_sync_batch(LSN_MAX).
Remove a redundant call to fil_system.sys_space->open(false)
because srv_start() must already have successfully invoked that
in order for us to reach this code path. Remove some duplicated
assertions as well as a redundant call to log_write_up_to() from
the end. If we crash while rebuilding a latest_format log file,
the subsequent startup will perform crash recovery normally.
This avoids a redundant write to the old log.
removed:
* DES encryption
* --secure-auth
* --old
* spider table options: bfz, btt, cmd, ctp, cwg, isa, ilm, ios, smd, stc, stl
extended under old-mode:
* YEAR(2), still available when old-mode=2_DIGIT_YEAR
un-deprecated:
* keep_files_on_create, originally (MDEV-23570) the idea was to make it
TRUE and deprecate. It cannot be removed when FALSE, but TRUE breaks
mariabackup.aria_backup where a table is altered from Aria to InnoDB
during a backup, so both t.MAD/t.MAI and t.ibd gets into a backup.
Fix a missing error check when opendir() returns NULL, leading to
SIGSEGV when the directory does not exist or otherwise cannot be read.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
remove conditional compilation for old MariaDB versions.
Some were as old as MYSQL_VERSION_ID > 32300
Didn't touch engines hosted externally, like connect, mroonga,
and columnstore
The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.
Other things:
- Fixed some (not critical) memory leaks in mariabackup
Implement an improved binlog implementation that is integrated into
the storage engine. The new implementation is enabled with the
--binlog-storage-engine option. Initially the InnoDB storage engine
implements the binlog.
Integrating the binlog in the storage engine improves performance,
since it makes the InnoDB redo log the single source of truth and
avoids the need for expensive two-phase commit between binlog and
engine. It also makes it possible to disable durability (set
--innodb-flush-log-at-trx-commit=0) to further improve performance,
while still preserving the ability to recover the binlog and database
into a consistent state after a crash.
The new binlog implementation also greatly improves the internal
design and implementation of the binlog, and enables future
enhancements for replication.
This is a squash of the original 11.4-based patch series.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
compiling mariadb from a git tree
Fixed sprintf deprecation warnings compiling on MacOSX.
Replaced some sprintf calls with equivalent snprintf calls,
enough so that "normal" compile on MacOSX (as documented
in the docs) completes without warnings.
This is a combined patch of various spelling fixes originally done in
Debian.
* Fix misc typos in MariaDB Server
* Fix spelling of 'allows one to'
Fix the following Lintian nags introduced in commit
c8d040938a:
I: mariadb-backup: spelling-error-in-binary "allows to" "allows one to" [usr/bin/mariadb-backup]
I: mariadb-server-core: spelling-error-in-binary "allows to" "allows one to" [usr/sbin/mariadbd]
I: mariadb-test: spelling-error-in-binary "allows to" "allows one to" [usr/bin/mariadb-client-test-embedded]
I: mariadb-test: spelling-error-in-binary "allows to" "allows one to" [usr/bin/mariadb-test-embedded]
I: mariadb-test: spelling-error-in-binary "allows to" "allows one to" [usr/bin/test-connect-t]
Problem:
========
When an inplace ALTER operation is rolled back, InnoDB drops
intermediate tables and their associated FTS internal tables.
However, MariaBackup's DDL tracking can incorrectly report
this as a backup failure.
The issue occurs because backup_set_alter_copy_lock() downgrades the
MDL_BACKUP_DDL lock before the inplace phase of ALTER,
allowing FTS internal tables to be dropped during the
later phases of backup when DDL tracking is still active.
Solution:
========
backup_file_op_fail(): Ignore delete operations on FTS internal
tables when not using --no-lock option, preventing false
positive backup failures.
multiple file tablespace
Problem:
=======
- innochecksum was incorrectly interpreting doublewrite buffer
pages as index pages, causing confusion about stale tables
in the system tablespace.
- innochecksum fails to parse the multi-file system tablespace
Solution:
========
1. Rewrite checksum of doublewrite buffer pages
are skipped.
2. Introduced the option --tablespace-flags which can be used
to initialize page size. This option can handle the ibdata2,
ibdata3 etc without parsing ibdata1.
This is a cherry-pick of commit 9f8716ab61
It was possible that a log checkpoint was completed and the server killed
between the time that fil_ibd_create() durably wrote a FILE_CREATE record,
and the initialization of the tablespace. This could lead to a failure
to start up after the server was killed during TRUNCATE TABLE or any
table-rebuilding operation such as OPTIMIZE TABLE.
In the case of TRUNCATE TABLE, an attempt to rename a file #sql-ibNNN.ibd
(the contents of the table before truncation) to tablename.ibd would fail,
because both files existed and the file tablename.ibd would have been
filled with NUL bytes. It was possible to resume from this error by
deleting the file tablename.ibd and restarting the server.
We will prevent this class of errors by ensuring that both the FILE_CREATE
record and the records written by fsp_header_init() will be part of the
same atomic transaction, which must be durably written before any file
is created or allocated.
NOTE: There is another possible crash recovery problem, which we are not
attempting to solve here and which will be covered by the subsequent
change (MDEV-38026). If fil_ibd_create() fails to create the file and
the server were killed, recovery would not even attempt to create the file
at all.
fil_space_t::create(): Remove the DBUG_EXECUTE_IF fault injection that
was the only cause of return nullptr. This allows us to simplify
several callers.
fil_space_t::set_stopped(), fil_space_t::clear_stopped(): Accessor
functions for fil_ibd_create() for preventing any concurrent access
to an incompletely created tablespace.
fil_ibd_create(): In a single atomic mini-transaction, write the
FILE_CREATE record as well as the log for initializing the tablespace.
After durably writing the log, create the file in the file system
Only after the file has been successfully created and allocated,
open the tablespace for business. Finally, release the exclusive page
latches so that the header pages may be written to the file.
fil_ibd_open(): Move some fault injection from fil_space_t::create()
to a higher level, to the place where an existing file is being opened.
Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam
