mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-07-16 00:08:14 +02:00
Code Issues Projects Releases Packages Wiki Activity
mariadb/mysql-test/suite/encryption/t
History
Exact
Marko Mäkelä eca552a1a4 MDEV-34830: LSN in the future is not being treated as serious corruption 
The invariant of write-ahead logging is that before any change to a
page is written to the data file, the corresponding log record must
must first have been durably written.

In crash recovery, there were some sloppy checks for this. Let us
implement accurate checks and flag an inconsistency as a hard error,
so that we can avoid further corruption of a corrupted database.
For data extraction from the corrupted database, innodb_force_recovery
can be used.

Before recovery is reading any data pages or invoking
buf_dblwr_t::recover() to recover torn pages from the
doublewrite buffer, InnoDB will have parsed the log until the
final LSN and updated log_sys.lsn to that. So, we can rely on
log_sys.lsn at all times. The doublewrite buffer recovery has been
refactored in such a way that the recv_sys.dblwr.pages may be consulted
while discovering files and their page sizes, but nothing will be
written back to data files before buf_dblwr_t::recover() is invoked.

recv_max_page_lsn, recv_lsn_checks_on: Remove.

recv_sys_t::validate_checkpoint(): Validate the write-ahead-logging
condition at the end of the recovery.

recv_dblwr_t::validate_page(): Keep track of the maximum LSN
(if we are checking a non-doublewrite copy of a page) but
do not complain LSN being in the future. The doublewrite buffer
is a special case, because it will be read early during recovery.
Besides, starting with commit 762bcb81b5
the dblwr=true copies of pages may legitimately be "too new".

recv_dblwr_t::find_page(): Find a valid page with the smallest
FIL_PAGE_LSN that is in the valid range for recovery.

recv_dblwr_t::restore_first_page(): Replaced by find_page().
Only buf_dblwr_t::recover() will write to data files.

buf_dblwr_t::recover(): Simplify the message output. Do attempt
doublewrite recovery on user page read error. Ignore doublewrite
pages whose FIL_PAGE_LSN is outside the usable bounds. Previously,
we could wrongly recover a too new page from the doublewrite buffer.
It is unlikely that this could have lead to an actual error.
Write back all recovered pages from the doublewrite buffer here,
including for the first page of any tablespace.

buf_page_is_corrupted(): Distinguish the return values
CORRUPTED_FUTURE_LSN and CORRUPTED_OTHER.

buf_page_check_corrupt(): Return the error code DB_CORRUPTION
in case the LSN is in the future.

Datafile::read_first_page_flags(): Split from read_first_page().
Take a copy of the first page as a parameter.

recv_sys_t::free_corrupted_page(): Take the file as a parameter
and return whether a message was displayed. This avoids some duplicated
and incomplete error messages.

buf_page_t::read_complete(): Remove some redundant output and always
display the name of the corrupted file. Never return DB_FAIL;
use it only in internal error handling.

IORequest::read_complete(): Assume that buf_page_t::read_complete()
will have reported any error.

fil_space_t::set_corrupted(): Return whether this is the first time
the tablespace had been flagged as corrupted.

Datafile::validate_first_page(), fil_node_open_file_low(),
fil_node_open_file(), fil_space_t::read_page0(),
fil_node_t::read_page0(): Add a parameter for a copy of the
first page, and a parameter to indicate whether the FIL_PAGE_LSN
check should be suppressed. Before buf_dblwr_t::recover() is
invoked, we cannot validate the FIL_PAGE_LSN, but we can trust the
FSP_SPACE_FLAGS and the tablespace ID that may be present in a
potentially too new copy of a page.

Reviewed by: Debarun Banerjee
2024-10-18 10:12:47 +03:00
..
aria_tiny.test MDEV-25507 CHECK on encrypted Aria table complains about "Wrong LSN" 2021-04-30 15:45:07 +03:00
bulk_insert.opt MDEV-28679 After upgrade to 10.7.3-1 with enabled data-at-rest encryption unable to restore dump file. 2022-07-26 11:25:56 +05:30
bulk_insert.test MDEV-28679 After upgrade to 10.7.3-1 with enabled data-at-rest encryption unable to restore dump file. 2022-07-26 11:25:56 +05:30
compressed_import_tablespace.opt MDEV-19695 Import tablespace doesn't work with ROW_FORMAT=COMPRESSED encrypted tablespace 2019-06-06 12:54:34 +05:30
compressed_import_tablespace.test Merge 10.3 into 10.4 2019-06-12 08:37:27 +03:00
corrupted_during_recovery.combinations MDEV-25105 Remove innodb_checksum_algorithm values none,innodb,... 2021-03-11 12:46:18 +02:00
corrupted_during_recovery.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
create_or_replace.opt MDEV-23776: Re-apply the fix and make the test more robust 2020-09-22 13:08:09 +03:00
create_or_replace.test MDEV-23776: Re-apply the fix and make the test more robust 2020-09-22 13:08:09 +03:00
create_or_replace_big.opt MDEV-23776: Split encryption.create_or_replace 2020-09-21 16:14:35 +03:00
create_or_replace_big.test MDEV-23776: Split encryption.create_or_replace 2020-09-21 16:14:35 +03:00
debug_key_management.opt Test debug_key_management fails sporadically in buildbot. 2015-10-29 10:35:37 +02:00
debug_key_management.test MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
encrypt_and_grep.opt MDEV-12253: Buffer pool blocks are accessed after they have been freed 2017-04-26 15:19:16 +03:00
encrypt_and_grep.test Merge branch '10.5' into 10.6 2024-03-13 21:02:34 +01:00
encryption_force.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
encryption_force.test MDEV-14219 Allow online table rebuild when encryption or compression parameters change 2017-10-31 09:10:25 +02:00
encryption_key_corruption.combinations MDEV-26747 after-merge fix for MDEV-12026, MDEV-25105 2021-11-19 09:10:40 +02:00
encryption_key_corruption.opt MDEV-26747 improve corruption check for encrypted tables on ALTER IMPORT 2021-11-17 15:49:22 +06:00
encryption_key_corruption.test Merge 10.3 into 10.4 2021-11-17 13:59:42 +02:00
file_creation.opt Problem: 2019-08-16 18:11:32 +05:30
file_creation.test Merge 10.3 into 10.4, except for MDEV-20265 2019-08-23 08:06:17 +03:00
filekeys-data-too-long.key MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys-data.enc move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys-data.key MDEV-7993 file_key_management_filekey doesn't work as expected with FILE: 2015-05-13 14:27:16 +02:00
filekeys-tooshort.enc MDEV-10888: encryption.filekeys_emptyfile fails in buildbot with valgrind 2016-09-27 07:54:27 +03:00
filekeys_badtest.inc MDEV-12762 Some files in current 10.2 tree seem to be reverted to an old state 2017-05-10 09:06:57 +03:00
filekeys_emptyfile.opt MDEV-9613: keyfile without any keys crashes mysqld on loading file_key_management plugin 2016-03-22 22:20:33 +02:00
filekeys_emptyfile.test MDEV-9613: keyfile without any keys crashes mysqld on loading file_key_management plugin 2016-03-22 22:20:33 +02:00
filekeys_encfile.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile.test move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_bad.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_bad.test move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_badfile.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_badfile.test move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_file.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_file.test move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_no.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_encfile_no.test move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_goodtest.inc Merge 10.1 into 10.2 2017-11-01 09:55:00 +02:00
filekeys_nofile.test fix the encryption.filekeys_nofile test 2015-10-10 14:19:02 +02:00
filekeys_plugin.inc misc encryption tests fixes 2015-06-27 09:40:54 +02:00
filekeys_plugin.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_plugin_exists.inc Fix test failures seen on buildbot where file_key_management plugin 2015-09-17 16:40:31 +03:00
filekeys_secret_openssl_rand_128bits.enc MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_secret_openssl_rand_128bits.key MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_secret_openssl_rand_128bits.opt MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_secret_openssl_rand_128bits.test MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_secret_too_long.opt MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_secret_too_long.test MDEV-25343 Error log message not helpful when filekey is too long 2022-10-21 15:54:17 +11:00
filekeys_syntax.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_syntax.test file_key_management plugin: complain if key id 1 is not found 2015-06-02 19:00:23 +02:00
filekeys_tooshort.opt MDEV-10888: encryption.filekeys_emptyfile fails in buildbot with valgrind 2016-09-27 07:54:27 +03:00
filekeys_tooshort.test MDEV-10888: encryption.filekeys_emptyfile fails in buildbot with valgrind 2016-09-27 07:54:27 +03:00
filekeys_unencfile.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
filekeys_unencfile.test move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
import_4k.opt MDEV-29692 Assertion `(writeptr + (i * size)) != local_frame' failed upon IMPORT TABLESPACE 2023-03-21 14:33:54 +02:00
import_4k.test MDEV-29692 Assertion `(writeptr + (i * size)) != local_frame' failed upon IMPORT TABLESPACE 2023-03-21 14:33:54 +02:00
innochecksum.test MDEV-29982 fixup: Relax the test 2022-11-17 08:33:05 +02:00
innodb-bad-key-change.combinations Add encryption.innodb-redo-badkey,strict_full_crc32 2019-10-11 08:24:30 +03:00
innodb-bad-key-change.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-bad-key-change.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-bad-key-change2.combinations Add encryption.innodb-redo-badkey,strict_full_crc32 2019-10-11 08:24:30 +03:00
innodb-bad-key-change2.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-bad-key-change2.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-bad-key-change3.combinations Add encryption.innodb-redo-badkey,strict_full_crc32 2019-10-11 08:24:30 +03:00
innodb-bad-key-change3.test MDEV-16944 Fix file sharing issues on Windows in mysqltest 2024-04-17 16:52:37 +02:00
innodb-bad-key-change4.combinations Add encryption.innodb-redo-badkey,strict_full_crc32 2019-10-11 08:24:30 +03:00
innodb-bad-key-change4.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-bad-key-change4.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-checksum-algorithm.test MDEV-25105 Remove innodb_checksum_algorithm values none,innodb,... 2021-03-11 12:46:18 +02:00
innodb-compressed-blob.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-compressed-blob.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-discard-import-change.combinations MDEV-15527 fixup for innodb_checksum_algorithm=full_crc32 2021-03-31 10:55:21 +03:00
innodb-discard-import-change.test MDEV-12762 Some files in current 10.2 tree seem to be reverted to an old state 2017-05-10 09:06:57 +03:00
innodb-discard-import.combinations MDEV-15527 fixup for innodb_checksum_algorithm=full_crc32 2021-03-31 10:55:21 +03:00
innodb-discard-import.test Merge branch '10.1' into 10.2 2018-06-21 23:47:39 +02:00
innodb-encr-threads.test InnoDB: check if scrubbing is enabled before scanning the tablespace 2015-05-15 18:12:02 +02:00
innodb-encryption-alter.test MDEV-25180 Atomic ALTER TABLE 2021-05-19 22:54:13 +02:00
innodb-encryption-disable.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-encryption-disable.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-first-page-read.opt MDEV-12610: MariaDB start is slow 2017-06-09 13:15:39 +03:00
innodb-force-corrupt.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-force-corrupt.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-key-rotation-disable.opt MDEV-11738: Mariadb uses 100% of several of my 8 cpus doing nothing 2017-03-14 16:23:10 +02:00
innodb-key-rotation-disable.test Merge 10.5 into 10.6 2023-03-16 15:48:08 +02:00
innodb-missing-key.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-missing-key.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-page_encryption-32k.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
innodb-page_encryption-32k.test MDEV-12762 Some files in current 10.2 tree seem to be reverted to an old state 2017-05-10 09:06:57 +03:00
innodb-page_encryption.test Some tests can take very long time when run with valgrind 2021-07-24 21:32:52 +03:00
innodb-page_encryption_compression.test Clean up some encryption tests 2020-10-17 13:13:01 +03:00
innodb-page_encryption_log_encryption.opt MDEV-8143: InnoDB: Database page corruption on disk or a failed file read 2015-05-14 11:32:24 +03:00
innodb-page_encryption_log_encryption.test Clean up some encryption tests 2020-10-17 13:13:01 +03:00
innodb-read-only.opt MDEV-11835: InnoDB: Failing assertion: free_slot != NULL on 2017-01-19 08:19:08 +02:00
innodb-read-only.test MDEV-11835: InnoDB: Failing assertion: free_slot != NULL on 2017-01-19 08:19:08 +02:00
innodb-redo-badkey.combinations Add encryption.innodb-redo-badkey,strict_full_crc32 2019-10-11 08:24:30 +03:00
innodb-redo-badkey.opt MDEV-32050: Deprecate&ignore innodb_purge_rseg_truncate_frequency 2023-10-25 09:11:58 +03:00
innodb-redo-badkey.test MDEV-34830: LSN in the future is not being treated as serious corruption 2024-10-18 10:12:47 +03:00
innodb-redo-nokeys.opt MDEV-9948 Failing assertion: new_state->key_version != ENCRYPTION_KEY_VERSION_INVALID in fil0crypt.cc 2022-04-15 20:28:33 +03:00
innodb-redo-nokeys.test Merge 10.6 into 10.11 2024-02-08 15:04:46 +02:00
innodb-remove-encryption.test MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
innodb-spatial-index.opt MDEV-11974: MariaDB 10.2 encryption does not support spatial indexes 2017-02-08 09:05:15 +02:00
innodb-spatial-index.test MDEV-18644: Support full_crc32 for page_compressed 2019-03-18 14:08:43 +02:00
innodb_encrypt_freed.opt MDEV-24792 Assertion `!newest_lsn || fil_page_get_type(page)' failed upon MariaBackup prepare in buf_flush_init_for_writing with innodb_log_optimize_ddl=off 2021-02-14 10:11:03 +03:00
innodb_encrypt_freed.test Merge 10.5 into 10.6 2024-04-18 07:14:56 +03:00
innodb_encrypt_key_rotation_age.opt MDEV-14398 innodb_encryption_rotate_key_age=0 causes innodb_encrypt_tables to be ignored 2019-05-02 13:31:59 +03:00
innodb_encrypt_key_rotation_age.test MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
innodb_encrypt_log.opt MDEV-14425 deprecate and ignore innodb_log_files_in_group 2020-02-19 12:21:59 +03:00
innodb_encrypt_log.test MDEV-27735 Deprecate the parameter innodb_change_buffering 2022-02-14 10:29:18 +02:00
innodb_encrypt_log_corruption.opt Updated mtr files to support different compiled in options 2019-09-01 19:17:35 +03:00
innodb_encrypt_log_corruption.test MDEV-11782: Redefine the innodb_encrypt_log format 2017-02-15 08:07:20 +02:00
innodb_encrypt_temporary_tables.combinations MDEV-20340 Encrypted temporary tables cannot be read with full_crc32 2019-08-14 10:32:52 +03:00
innodb_encrypt_temporary_tables.opt Merge 10.4 into 10.5 2019-08-13 18:57:00 +03:00
innodb_encrypt_temporary_tables.test MDEV-31005: Make working cursor-protocol 2024-09-18 18:39:26 +07:00
innodb_encryption-page-compression.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
innodb_encryption-page-compression.test Disable from valgrind big innodb tests that doesn't run well in valgrind 2020-10-21 03:09:29 +03:00
innodb_encryption.opt MDEV-9010 Encryption preset file contains different configuration preset then documentation 2015-10-29 10:46:37 +01:00
innodb_encryption.test MDEV-28484 InnoDB encryption key rotation is not being marked completed 2022-05-06 11:23:13 +03:00
innodb_encryption_discard_import.opt MDEV-23399: Performance regression with write workloads 2020-10-15 17:04:56 +03:00
innodb_encryption_discard_import.test MDEV-34670 IMPORT TABLESPACE unnecessary traverses tablespace list 2024-07-31 14:13:38 +05:30
innodb_encryption_filekeys.opt MDEV-8832: Encryption meta data should not be stored when ENCRYPTED=DEFAULT and innodb-encrypt-tables=OFF 2015-09-23 17:34:36 +03:00
innodb_encryption_filekeys.test MDEV-25998 InnoDB removes the tablespace from default encrypt list early 2021-07-26 16:01:33 +05:30
innodb_encryption_is.opt MDEV-9640: Add used key_id to INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION 2016-03-18 11:48:49 +02:00
innodb_encryption_is.test Merge 10.5 into 10.6 2023-03-16 15:48:08 +02:00
innodb_encryption_row_compressed.opt MDEV-9931: InnoDB reads first page of every .ibd file at startup 2016-09-22 16:38:24 +03:00
innodb_encryption_row_compressed.test MDEV-12762 Some files in current 10.2 tree seem to be reverted to an old state 2017-05-10 09:06:57 +03:00
innodb_encryption_tables.opt move encryption tests to a dedicate suite 2015-05-13 14:27:16 +02:00
innodb_encryption_tables.test Fixed bugs found by valgrind 2020-05-15 10:44:05 +03:00
innodb_first_page.opt Tests: removed --transaction-registry option [#387] 2017-12-21 12:51:57 +03:00
innodb_first_page.test MDEV-30119 INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION.NAME is NULL for undo tablespaces 2022-11-29 19:49:53 +05:30
innodb_import.combinations MDEV-26131 SEGV in ha_innobase::discard_or_import_tablespace 2021-08-16 15:32:07 +05:30
innodb_import.opt Merge 10.4 into 10.5 2021-08-18 18:22:35 +03:00
innodb_import.test After-merge fixup f84e28c119 2021-08-19 12:34:31 +05:30
innodb_onlinealter_encryption.opt MDEV-12253: Buffer pool blocks are accessed after they have been freed 2017-04-26 15:19:16 +03:00
innodb_onlinealter_encryption.test Remove deprecated InnoDB file format parameters 2017-06-02 09:36:14 +03:00
innodb_page_encryption_key_change.test Write information about restart in .result 2019-04-01 19:47:24 +03:00
instant_alter_import.test MDEV-18543 IMPORT TABLESPACE fails after instant DROP COLUMN 2021-10-26 22:50:58 +06:00
key_version_rotation.opt MDEV-14180 Automatically disable key rotation checks for file_key_managment plugin 2021-06-15 13:15:32 +05:30
key_version_rotation.test MDEV-14180 Automatically disable key rotation checks for file_key_managment plugin 2021-06-15 13:15:32 +05:30
second_plugin-12863.test MDEV-12863 No table can be created after second encryption plugin attempted to load 2017-08-08 14:52:08 +02:00
tempfiles.combinations MDEV-14868 MariaDB server crashes after using ROLLBACK TO when encrypt_tmp_files=ON 2018-02-08 22:54:37 +01:00
tempfiles.opt fix encryption.tempfiles to check that encrypt_tmp_files is ON 2018-02-08 22:54:37 +01:00
tempfiles.test Updated mtr files to support different compiled in options 2019-09-01 19:17:35 +03:00
tempfiles_encrypted.opt MDEV-23867: insert... select crash in compute_window_func 2020-10-23 22:36:47 +05:30
tempfiles_encrypted.test Merge 10.3 into 10.4 2020-10-29 13:38:38 +02:00