mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-29 02:05:57 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/mysql-test/suite/innodb/t/doublewrite.test
Marko Mäkelä eca552a1a4 MDEV-34830: LSN in the future is not being treated as serious corruption 
The invariant of write-ahead logging is that before any change to a
page is written to the data file, the corresponding log record must
must first have been durably written.

In crash recovery, there were some sloppy checks for this. Let us
implement accurate checks and flag an inconsistency as a hard error,
so that we can avoid further corruption of a corrupted database.
For data extraction from the corrupted database, innodb_force_recovery
can be used.

Before recovery is reading any data pages or invoking
buf_dblwr_t::recover() to recover torn pages from the
doublewrite buffer, InnoDB will have parsed the log until the
final LSN and updated log_sys.lsn to that. So, we can rely on
log_sys.lsn at all times. The doublewrite buffer recovery has been
refactored in such a way that the recv_sys.dblwr.pages may be consulted
while discovering files and their page sizes, but nothing will be
written back to data files before buf_dblwr_t::recover() is invoked.

recv_max_page_lsn, recv_lsn_checks_on: Remove.

recv_sys_t::validate_checkpoint(): Validate the write-ahead-logging
condition at the end of the recovery.

recv_dblwr_t::validate_page(): Keep track of the maximum LSN
(if we are checking a non-doublewrite copy of a page) but
do not complain LSN being in the future. The doublewrite buffer
is a special case, because it will be read early during recovery.
Besides, starting with commit 762bcb81b5
the dblwr=true copies of pages may legitimately be "too new".

recv_dblwr_t::find_page(): Find a valid page with the smallest
FIL_PAGE_LSN that is in the valid range for recovery.

recv_dblwr_t::restore_first_page(): Replaced by find_page().
Only buf_dblwr_t::recover() will write to data files.

buf_dblwr_t::recover(): Simplify the message output. Do attempt
doublewrite recovery on user page read error. Ignore doublewrite
pages whose FIL_PAGE_LSN is outside the usable bounds. Previously,
we could wrongly recover a too new page from the doublewrite buffer.
It is unlikely that this could have lead to an actual error.
Write back all recovered pages from the doublewrite buffer here,
including for the first page of any tablespace.

buf_page_is_corrupted(): Distinguish the return values
CORRUPTED_FUTURE_LSN and CORRUPTED_OTHER.

buf_page_check_corrupt(): Return the error code DB_CORRUPTION
in case the LSN is in the future.

Datafile::read_first_page_flags(): Split from read_first_page().
Take a copy of the first page as a parameter.

recv_sys_t::free_corrupted_page(): Take the file as a parameter
and return whether a message was displayed. This avoids some duplicated
and incomplete error messages.

buf_page_t::read_complete(): Remove some redundant output and always
display the name of the corrupted file. Never return DB_FAIL;
use it only in internal error handling.

IORequest::read_complete(): Assume that buf_page_t::read_complete()
will have reported any error.

fil_space_t::set_corrupted(): Return whether this is the first time
the tablespace had been flagged as corrupted.

Datafile::validate_first_page(), fil_node_open_file_low(),
fil_node_open_file(), fil_space_t::read_page0(),
fil_node_t::read_page0(): Add a parameter for a copy of the
first page, and a parameter to indicate whether the FIL_PAGE_LSN
check should be suppressed. Before buf_dblwr_t::recover() is
invoked, we cannot validate the FIL_PAGE_LSN, but we can trust the
FSP_SPACE_FLAGS and the tablespace ID that may be present in a
potentially too new copy of a page.

Reviewed by: Debarun Banerjee
2024-10-18 10:12:47 +03:00

194 lines
7.1 KiB
Text
Raw Permalink Blame History

--echo #
--echo # MDEV-32242 innodb.doublewrite test case always is skipped
--echo #
--source include/innodb_page_size.inc
--source include/not_embedded.inc
--disable_query_log
call mtr.add_suppression("InnoDB: Data file .* uses page size .* but the innodb_page_size start-up parameter is");
call mtr.add_suppression("InnoDB: adjusting FSP_SPACE_FLAGS");
call mtr.add_suppression("InnoDB: New log files created");
call mtr.add_suppression("InnoDB: Cannot create doublewrite buffer: the first file in innodb_data_file_path must be at least (3|6|12)M\\.");
call mtr.add_suppression("InnoDB: Database creation was aborted");
call mtr.add_suppression("Plugin 'InnoDB' (init function returned error|registration as a STORAGE ENGINE failed)");
call mtr.add_suppression("InnoDB: A bad Space ID was found in datafile");
call mtr.add_suppression("InnoDB: Checksum mismatch in datafile: ");
call mtr.add_suppression("InnoDB: Inconsistent tablespace ID in .*t1\\.ibd");
call mtr.add_suppression("\\[Warning\\] Found 1 prepared XA transactions");
call mtr.add_suppression("InnoDB: Header page consists of zero bytes in datafile:");
call mtr.add_suppression("InnoDB: Page \\[page id: space=[1-9][0-9]*, page number=3\\] log sequence number 18446744073709551615 is in the future!");
call mtr.add_suppression("InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the ib_logfile0");
call mtr.add_suppression("InnoDB: Plugin initialization aborted");
--enable_query_log
let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
let MYSQLD_DATADIR=`select @@datadir`;
let ALGO=`select @@innodb_checksum_algorithm`;
let SEARCH_FILE= $MYSQLTEST_VARDIR/log/mysqld.1.err;
create table t1 (f1 int primary key, f2 blob) stats_persistent=0, engine=innodb;
start transaction;
insert into t1 values(1, repeat('#',12));
insert into t1 values(2, repeat('+',12));
insert into t1 values(3, repeat('/',12));
insert into t1 values(4, repeat('-',12));
insert into t1 values(5, repeat('.',12));
commit work;
# Slow shutdown and restart to make sure ibuf merge is finished
SET GLOBAL innodb_fast_shutdown = 0;
let $shutdown_timeout=;
--source include/restart_mysqld.inc
--source ../include/no_checkpoint_start.inc
connect (dml,localhost,root,,);
XA START 'x';
insert into t1 values (6, repeat('%', @@innodb_page_size/2));
XA END 'x';
XA PREPARE 'x';
disconnect dml;
connection default;
flush table t1 for export;
let $restart_parameters=;
--let CLEANUP_IF_CHECKPOINT=XA COMMIT 'x';drop table t1;
--source ../include/no_checkpoint_end.inc
perl;
use IO::Handle;
do "$ENV{MTR_SUITE_DIR}/include/crc32.pl";
my $polynomial = 0x82f63b78; # CRC-32C
my $algo = $ENV{ALGO};
die "Unsupported innodb_checksum_algorithm=$algo\n" unless $algo =~ /crc32/;
my $fname= "$ENV{'MYSQLD_DATADIR'}test/t1.ibd";
my $page_size = $ENV{INNODB_PAGE_SIZE};
my $page;
do "$ENV{MTR_SUITE_DIR}/../innodb/include/crc32.pl";
open(FILE, "+<", $fname) or die;
sysseek(FILE, ($page_size/2), 0);
syswrite(FILE, chr(0) x ($page_size/2));
sysseek(FILE, 3*$page_size, 0);
sysread(FILE, $page, $page_size)==$page_size||die "Unable to read $name\n";
sysseek(FILE, 3*$page_size, 0)||die "Unable to seek $fname\n";
my $corrupted = $page;
# Set FIL_PAGE_LSN to the maximum
substr($corrupted, 16, 8) = chr(255) x 8;
substr($corrupted, $page_size - 8, 8) = chr(255) x 8;
if ($algo =~ /full_crc32/)
{
my $ck = mycrc32(substr($corrupted, 0, $page_size - 4), 0, $polynomial);
substr($corrupted, $page_size - 4, 4) = pack("N", $ck);
}
else
{
# Replace the innodb_checksum_algorithm=crc32 checksum
my $ck= pack("N",
mycrc32(substr($corrupted, 4, 22), 0, $polynomial) ^
mycrc32(substr($corrupted_, 38, $page_size - 38 - 8), 0,
$polynomial));
substr ($corrupted, 0, 4) = $ck;
substr ($corrupted, $page_size - 8, 4) = $ck;
}
syswrite(FILE, $corrupted);
close FILE;
# Change the flag offset of page 0 in doublewrite buffer
open(FILE, "+<", "$ENV{MYSQLD_DATADIR}ibdata1")||die "cannot open ibdata1\n";
sysseek(FILE, 6 * $page_size - 190, 0)||die "Unable to seek ibdata1\n";
sysread(FILE, $_, 12) == 12||die "Unable to read TRX_SYS\n";
my($magic,$d1,$d2)=unpack "NNN", $_;
die "magic=$magic, $d1, $d2\n" unless $magic == 536853855 && $d2 >= $d1 + 64;
sysseek(FILE, $d1 * $page_size, 0)||die "Unable to seek ibdata1\n";
# Find the page in the doublewrite buffer
for (my $d = $d1; $d < $d2 + 64; $d++)
{
sysread(FILE, $_, $page_size)==$page_size||die "Cannot read doublewrite\n";
next unless $_ eq $page;
sysseek(FILE, $d * $page_size, 0)||die "Unable to seek ibdata1\n";
# Write buggy FSP_SPACE_FLAGS to the doublewrite buffer for page
my $badflags = 0x0006FFFF;
substr ($_, 54, 4) = pack("N", $badflags);
if ($algo =~ /full_crc32/)
{
my $ck = mycrc32(substr($_, 0, $page_size - 4), 0, $polynomial);
substr($_, $page_size - 4, 4) = pack("N", $ck);
}
else
{
# Replace the innodb_checksum_algorithm=crc32 checksum
my $ck= pack("N",
mycrc32(substr($_, 4, 22), 0, $polynomial) ^
mycrc32(substr($_, 38, $page_size - 38 - 8), 0,
$polynomial));
substr ($_, 0, 4) = $ck;
substr ($_, $page_size - 8, 4) = $ck;
}
syswrite(FILE, $_, $page_size)==$page_size||die;
close(FILE);
exit 0;
}
die "Did not find the page in the doublewrite buffer ($d1,$d2)\n";
EOF
--source include/start_mysqld.inc
let SEARCH_PATTERN=InnoDB: Recovered page \\[page id: space=[1-9][0-9]*, page number=0\\];
--source include/search_pattern_in_file.inc
let SEARCH_PATTERN=InnoDB: The log was only scanned up to \\d+, while the current LSN at the time of the latest checkpoint \\d+ was 0 and the maximum LSN on a data page was 18446744073709551615!
--source include/search_pattern_in_file.inc
--error ER_XAER_NOTA
XA ROLLBACK 'x';
let $shutdown_timeout=0;
--source include/shutdown_mysqld.inc
let $shutdown_timeout=;
# Corrupt the file in a better way.
perl;
use IO::Handle;
my $fname= "$ENV{'MYSQLD_DATADIR'}test/t1.ibd";
my $page_size = $ENV{INNODB_PAGE_SIZE};
open(FILE, "+<", $fname) or die;
sysseek(FILE, ($page_size/2), 0);
syswrite(FILE, chr(0) x ($page_size/2));
sysseek(FILE, 3*$page_size, 0);
syswrite(FILE, chr(0) x ($page_size/2));
close FILE;
EOF
--source include/start_mysqld.inc
XA ROLLBACK 'x';
check table t1;
select f1, f2 from t1;
--source ../include/no_checkpoint_start.inc
connect (dml,localhost,root,,);
XA START 'x';
insert into t1 values (6, repeat('%', @@innodb_page_size/2));
XA END 'x';
XA PREPARE 'x';
disconnect dml;
connection default;
flush table t1 for export;
let $restart_parameters=;
--source ../include/no_checkpoint_end.inc
# Zero out the first page in file and try to recover from dblwr
perl;
use IO::Handle;
open(FILE, "+<", "$ENV{'MYSQLD_DATADIR'}test/t1.ibd") or die;
syswrite(FILE, chr(0) x $ENV{INNODB_PAGE_SIZE});
close FILE;
EOF
--source include/start_mysqld.inc
let SEARCH_PATTERN=InnoDB: Recovered page \\[page id: space=[1-9][0-9]*, page number=[03]\\];
--source include/search_pattern_in_file.inc
XA ROLLBACK 'x';
check table t1;
select f1, f2 from t1;
drop table t1;
--echo # End of 10.5 tests
Reference in a new issue View git blame Copy permalink