mariadb/mysql-test/suite/encryption/r
Marko Mäkelä eca552a1a4 MDEV-34830: LSN in the future is not being treated as serious corruption
The invariant of write-ahead logging is that before any change to a
page is written to the data file, the corresponding log record must
must first have been durably written.

In crash recovery, there were some sloppy checks for this. Let us
implement accurate checks and flag an inconsistency as a hard error,
so that we can avoid further corruption of a corrupted database.
For data extraction from the corrupted database, innodb_force_recovery
can be used.

Before recovery is reading any data pages or invoking
buf_dblwr_t::recover() to recover torn pages from the
doublewrite buffer, InnoDB will have parsed the log until the
final LSN and updated log_sys.lsn to that. So, we can rely on
log_sys.lsn at all times. The doublewrite buffer recovery has been
refactored in such a way that the recv_sys.dblwr.pages may be consulted
while discovering files and their page sizes, but nothing will be
written back to data files before buf_dblwr_t::recover() is invoked.

recv_max_page_lsn, recv_lsn_checks_on: Remove.

recv_sys_t::validate_checkpoint(): Validate the write-ahead-logging
condition at the end of the recovery.

recv_dblwr_t::validate_page(): Keep track of the maximum LSN
(if we are checking a non-doublewrite copy of a page) but
do not complain LSN being in the future. The doublewrite buffer
is a special case, because it will be read early during recovery.
Besides, starting with commit 762bcb81b5
the dblwr=true copies of pages may legitimately be "too new".

recv_dblwr_t::find_page(): Find a valid page with the smallest
FIL_PAGE_LSN that is in the valid range for recovery.

recv_dblwr_t::restore_first_page(): Replaced by find_page().
Only buf_dblwr_t::recover() will write to data files.

buf_dblwr_t::recover(): Simplify the message output. Do attempt
doublewrite recovery on user page read error. Ignore doublewrite
pages whose FIL_PAGE_LSN is outside the usable bounds. Previously,
we could wrongly recover a too new page from the doublewrite buffer.
It is unlikely that this could have lead to an actual error.
Write back all recovered pages from the doublewrite buffer here,
including for the first page of any tablespace.

buf_page_is_corrupted(): Distinguish the return values
CORRUPTED_FUTURE_LSN and CORRUPTED_OTHER.

buf_page_check_corrupt(): Return the error code DB_CORRUPTION
in case the LSN is in the future.

Datafile::read_first_page_flags(): Split from read_first_page().
Take a copy of the first page as a parameter.

recv_sys_t::free_corrupted_page(): Take the file as a parameter
and return whether a message was displayed. This avoids some duplicated
and incomplete error messages.

buf_page_t::read_complete(): Remove some redundant output and always
display the name of the corrupted file. Never return DB_FAIL;
use it only in internal error handling.

IORequest::read_complete(): Assume that buf_page_t::read_complete()
will have reported any error.

fil_space_t::set_corrupted(): Return whether this is the first time
the tablespace had been flagged as corrupted.

Datafile::validate_first_page(), fil_node_open_file_low(),
fil_node_open_file(), fil_space_t::read_page0(),
fil_node_t::read_page0(): Add a parameter for a copy of the
first page, and a parameter to indicate whether the FIL_PAGE_LSN
check should be suppressed. Before buf_dblwr_t::recover() is
invoked, we cannot validate the FIL_PAGE_LSN, but we can trust the
FSP_SPACE_FLAGS and the tablespace ID that may be present in a
potentially too new copy of a page.

Reviewed by: Debarun Banerjee
2024-10-18 10:12:47 +03:00
..
aria_tiny.result MDEV-25507 CHECK on encrypted Aria table complains about "Wrong LSN" 2021-04-30 15:45:07 +03:00
bulk_insert.result MDEV-28679 After upgrade to 10.7.3-1 with enabled data-at-rest encryption unable to restore dump file. 2022-07-26 11:25:56 +05:30
compressed_import_tablespace.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
corrupted_during_recovery.result MDEV-33421 innodb.corrupted_during_recovery fails due to error that the table is corrupted 2024-02-08 14:20:42 +02:00
create_or_replace.result MDEV-23776: Re-apply the fix and make the test more robust 2020-09-22 13:08:09 +03:00
create_or_replace_big.result Merge 10.3 into 10.4 2020-09-21 17:42:02 +03:00
debug_key_management.result MDEV-29613 Improve WITH_DBUG_TRACE=OFF 2022-09-23 13:40:42 +03:00
encrypt_and_grep,undo3.rdiff MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
encrypt_and_grep.result Merge 10.2 into 10.3 2021-11-09 08:26:59 +02:00
encryption_force.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
encryption_key_corruption.result MDEV-27882 Innodb - recognise MySQL-8.0 innodb flags and give a specific error message 2022-11-11 10:21:28 +11:00
file_creation.result Problem: 2019-08-16 18:11:32 +05:30
filekeys_emptyfile.result
filekeys_encfile.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
filekeys_encfile_bad.result
filekeys_encfile_badfile.result
filekeys_encfile_file.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
filekeys_encfile_no.result
filekeys_nofile.result
filekeys_secret_openssl_rand_128bits.result MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_secret_too_long.result MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_syntax.result
filekeys_tooshort.result
filekeys_unencfile.result
import_4k.result MDEV-29692 Assertion `(writeptr + (i * size)) != local_frame' failed upon IMPORT TABLESPACE 2023-03-21 14:33:54 +02:00
innochecksum.result Merge 10.3 into 10.4 2022-04-06 08:59:09 +03:00
innodb-bad-key-change.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-bad-key-change2.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-bad-key-change3.result Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
innodb-bad-key-change4.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-checksum-algorithm,32k.rdiff MDEV-25105 Remove innodb_checksum_algorithm values none,innodb,... 2021-03-11 12:46:18 +02:00
innodb-checksum-algorithm,64k.rdiff MDEV-25105 Remove innodb_checksum_algorithm values none,innodb,... 2021-03-11 12:46:18 +02:00
innodb-checksum-algorithm.result Merge 10.3 into 10.4 2021-03-11 19:33:45 +02:00
innodb-compressed-blob.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-discard-import-change.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
innodb-discard-import.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
innodb-encr-threads.result
innodb-encryption-alter.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
innodb-encryption-disable.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-force-corrupt.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-key-rotation-disable.result Merge 10.3 into 10.4 2021-06-21 12:38:25 +03:00
innodb-missing-key.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-page_encryption-32k.result MDEV-21664 Add opt files for have_innodb_Xk.inc 2023-09-11 09:09:02 +03:00
innodb-page_encryption.result Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
innodb-page_encryption_compression.result Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
innodb-page_encryption_log_encryption.result Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
innodb-read-only.result Write information about restart in .result 2019-04-01 19:47:24 +03:00
innodb-redo-badkey.result MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-redo-nokeys.result Merge 10.6 into 10.11 2024-02-08 15:04:46 +02:00
innodb-remove-encryption,undo3.rdiff MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
innodb-remove-encryption.result Merge 10.4 into 10.5 2021-06-08 16:03:53 +03:00
innodb-spatial-index,full_crc32.rdiff Write information about restart in .result 2019-04-01 19:47:24 +03:00
innodb-spatial-index,strict_full_crc32.rdiff Write information about restart in .result 2019-04-01 19:47:24 +03:00
innodb-spatial-index.result MDEV-18644: Support full_crc32 for page_compressed 2019-03-18 14:08:43 +02:00
innodb_encrypt_freed,undo3.rdiff MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
innodb_encrypt_freed.result MDEV-24792 Assertion `!newest_lsn || fil_page_get_type(page)' failed upon MariaBackup prepare in buf_flush_init_for_writing with innodb_log_optimize_ddl=off 2021-02-14 10:11:03 +03:00
innodb_encrypt_key_rotation_age,undo3.rdiff MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
innodb_encrypt_key_rotation_age.result MDEV-28484 InnoDB encryption key rotation is not being marked completed 2022-05-06 11:23:13 +03:00
innodb_encrypt_log.result MDEV-27735 Deprecate the parameter innodb_change_buffering 2022-02-14 10:29:18 +02:00
innodb_encrypt_log_corruption.result Merge 10.7 into 10.8 2022-11-30 12:12:07 +02:00
innodb_encrypt_temporary_tables.result MDEV-20142 encryption.innodb_encrypt_temporary_tables fails 2023-12-10 13:19:21 +02:00
innodb_encryption-page-compression.result Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
innodb_encryption.result MDEV-28484 InnoDB encryption key rotation is not being marked completed 2022-05-06 11:23:13 +03:00
innodb_encryption_discard_import.result MDEV-34670 IMPORT TABLESPACE unnecessary traverses tablespace list 2024-07-31 14:13:38 +05:30
innodb_encryption_filekeys.result MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
innodb_encryption_is.result
innodb_encryption_row_compressed.result Write information about restart in .result 2019-04-01 19:47:24 +03:00
innodb_encryption_tables.result Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
innodb_first_page.result Write information about restart in .result 2019-04-01 19:47:24 +03:00
innodb_import.result Merge 10.4 into 10.5 2021-08-18 18:22:35 +03:00
innodb_onlinealter_encryption.result Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
innodb_page_encryption_key_change.result Write information about restart in .result 2019-04-01 19:47:24 +03:00
instant_alter_import.result MDEV-18543 IMPORT TABLESPACE fails after instant DROP COLUMN 2021-10-26 22:50:58 +06:00
key_version_rotation.result Merge 10.3 into 10.4 2021-06-21 12:38:25 +03:00
second_plugin-12863.result
tempfiles.result Updated mtr files to support different compiled in options 2019-09-01 19:17:35 +03:00
tempfiles_encrypted.result Merge branch '10.6' into 10.11 2024-08-03 09:15:40 +02:00