mariadb/mysql-test/suite/innodb/t/table_flags.test
Aleksey Midenkov a518e1dd42 MDEV-20865 Store foreign key info in TABLE_SHARE
1. Access foreign keys via TABLE_SHARE::foreign_keys and
   TABLE_SHARE::referenced_keys;

   foreign_keys and referenced_keys are lists in TABLE_SHARE.

2. Remove handler FK interface:

   - get_foreign_key_list()
   - get_parent_foreign_key_list()
   - referenced_by_foreign_key()

3. Invalidate referenced shares on:

   - RENAME TABLE
   - DROP TABLE
   - RENAME COLUMN
   - ADD FOREIGN KEY

   When foreign table is created or altered by the above operations
   all referenced shares are closed. This blocks the operation while
   any referenced shares are used (when at least one its TABLE
   instance is locked).

4. Update referenced shares on:

   - CREATE TABLE

   On CREATE TABLE add items to referenced_keys of referenced
   shares. States of referenced shares are restored in case of errors.

5. Invalidate foreign shares on:

   - RENAME TABLE
   - RENAME COLUMN

   The above-mentioned blocking takes effect.

6. Check foreign/referenced shares consistency on:

   - CHECK TABLE

7. Temporary change until MDEV-21051:

   InnoDB fill foreign key info at handler open().

FOREIGN_KEY_INFO is refactored to FK_info holding Lex_cstring.

On first TABLE open FK_info is loaded from storage engine into
TABLE_SHARE. All referenced shares (if any exists) are closed. This
leads to blocking of first time foreign table open while referenced
tables are used.

MDEV-21311 Converge Foreign_key and supplemental generated Key together

mysql_prepare_create_table() does data validation and such utilities
as automatic name generation. But it does that only for indexes and
ignores Foreign_key objects. Now as Foreign_key data needs to be
stored in FRM files as well this processing must be done for it like
for any other Key objects.

Replace Key::FOREIGN_KEY type with Key::foreign flag of type
Key::MULTIPLE and Key::generated set to true. Construct one object
with Key::foreign == true instead of two objects of type
Key::FOREIGN_KEY and Key::MULTIPLE.

MDEV-21051 datadict refactorings

- Move read_extra2() to datadict.cc
- Refactored extra2_fields to Extra2_info
- build_frm_image() readability

MDEV-21051 build_table_shadow_filename() refactoring

mysql_prepare_alter_table() leaks fixes

MDEV-21051 amend system tables locking restriction

Table mysql.help_relation has foreign key to mysql.help_keyword. On
bootstrap when help_relation is opened, it preopens help_keyword for
READ and fails in lock_tables_check().

If system table is opened for write then fk references are opened for
write.

Related to: Bug#25422, WL#3984
Tests: main.lock

MDEV-21051 Store and read foreign key info into/from FRM files

1. Introduce Foreign_key_io class which creates/parses binary stream
containing foreign key structures. Referenced tables store there only
hints about foreign tables (their db and name), they restore full info
from the corresponding tables.

Foreign_key_io is stored under new EXTRA2_FOREIGN_KEY_INFO field in
extra2 section of FRM file.

2. Modify mysql_prepare_create_table() to generate names for foreign
keys. Until InnoDB storage of foreign keys is removed, FK names must
be unique across the database: the FK name must be based on table
name.

3. Keep stored data in sync on DDL changes. Referenced tables update
their foreign hints after following operations on foreign tables:

  - RENAME TABLE
  - DROP TABLE
  - CREATE TABLE
  - ADD FOREIGN KEY
  - DROP FOREIGN KEY

Foreign tables update their foreign info after following operations on
referenced tables:

  - RENAME TABLE
  - RENAME COLUMN

4. To achieve 3. there must be ability to rewrite extra2 section of
FRM file without full reparse. FRM binary is built from primary
structures like HA_CREATE_INFO and cannot be built from TABLE_SHARE.

Use shadow write and rename like fast_alter_partition_table()
does. Shadow FRM is new FRM file that replaces the old one.

CREATE TABLE workflow:

  1. Foreign_key is constructed in parser, placed into
     alter_info->key_list;

  2. mysql_prepare_create_table() translates them to FK_info, assigns
     foreign_id if needed;

  3. build_frm_image() writes two FK_info lists into FRM's extra2
     section, for referenced keys it stores only table names (hints);

  4. init_from_binary_frm_image() parses extra2 section and fills
     foreign_keys and referenced_keys of TABLE_SHARE.

     It restores referenced_keys by reading hint list of table names,
     opening corresponding shares and restoring FK_info from their
     foreign_keys. Hints resolution is done only when initializing
     non-temporary shares. Usually temporary share has different
     (temporary) name and it is impossible to resolve foreign keys by
     that name (as we identify them by both foreign and referenced
     table names). Another not unimportant reason is performance: this
     saves spare share acquisitions.

ALTER TABLE workflow:

  1. Foreign_key is constructed in parser, placed into
     alter_info->key_list;

  2. mysql_prepare_alter_table() prepares action lists and share list
     of foreigns/references;

  3. mysql_prepare_alter_table() locks list of foreigns/references by
     MDL_INTENTION_EXCLUSIVE, acquires shares;

  4. prepare_create_table() converts key_list into FK_list, assigns
     foreign_id;

  5. shadow FRM of altered table is created;

  6. data is copied;

  7. altered table is locked by MDL_EXCLUSIVE;

  8. fk_handle_alter() processes action lists, creates FK backups,
     modifies shares, writes shadow FRMs;

  9. altered table is closed;

  10. shadow FRMs are installed;

  11. altered table is renamed, FRM backup deleted;

  12. (TBD in MDEV-21053) shadow FRMs installation log closed, backups
      deleted;

On FK backup system:

In case of failed DDL operation all shares that was modified must be
restored into original state. This is done by FK_ddl_backup (CREATE,
DROP), FK_rename_backup (RENAME), FK_alter_backup (ALTER).

On STL usage:

STL is used for utility not performance-critical algorithms, core
structures hold native List. A wrapper was made to convert STL
exception into bool error status or NULL value.

MDEV-20865 fk_check_consistency() in CHECK TABLE

Self-refs fix

Test table_flags fix: "debug" deviation is now gone.

FIXMEs: +16 -1
2025-09-02 13:24:36 +03:00

270 lines
10 KiB
Text

--source include/innodb_page_size.inc
# Embedded server tests do not support restarting
--source include/not_embedded.inc
# Slow shutdown may take more than 120 seconds under Valgrind,
# causing the server to be (silently) killed.
# Due to that, crash recovery could "heal" the damage that our
# Perl code is inflicting, and the SELECT statements could succeed
# instead of failing with ER_NO_SUCH_TABLE_IN_ENGINE.
--source include/not_valgrind.inc
--disable_query_log
call mtr.add_suppression("InnoDB: Table `mysql`\\.`innodb_table_stats` not found");
call mtr.add_suppression("InnoDB: incorrect flags in SYS_TABLES");
call mtr.add_suppression("InnoDB: Table test/t[cp] in InnoDB data dictionary contains invalid flags\\. SYS_TABLES\\.TYPE=(129|289|3873|1232[31]) SYS_TABLES\\.N_COLS=2147483649\\r?$");
call mtr.add_suppression("InnoDB: Table test/tr in InnoDB data dictionary contains invalid flags\\. SYS_TABLES\\.TYPE=65 SYS_TABLES\\.MIX_LEN=4294967295\\r?$");
call mtr.add_suppression("InnoDB: Refusing to load '\\..test.td\\.ibd' \\(id=3, flags=0x([2e]1)\\); dictionary contains id=3, flags=0x100\\1\\r?$");
call mtr.add_suppression("InnoDB: Refusing to load '\\..test.td\\.ibd' \\(id=3, flags=0x(1[2ae]1)\\); dictionary contains id=3, flags=0x10\\1\\r?$");
call mtr.add_suppression("InnoDB: Ignoring tablespace for `test`.`td` because it could not be opened\\.");
# FIXME: Remove the following spam due to invalid flags for test.td
call mtr.add_suppression("InnoDB: Operating system error number .* in a file operation");
call mtr.add_suppression("InnoDB: The error means the system cannot find the path specified");
call mtr.add_suppression("InnoDB: adjusting FSP_SPACE_FLAGS of file ");
call mtr.add_suppression("InnoDB: Parent table of FTS auxiliary table .* not found.");
FLUSH TABLES;
--enable_query_log
let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
let MYSQLD_DATADIR=`select @@datadir`;
let bugdir= $MYSQLTEST_VARDIR/tmp/table_flags;
--mkdir $bugdir
let undodir= $MYSQLTEST_VARDIR/tmp/undo_dir;
--mkdir $undodir
--let SEARCH_FILE = $MYSQLTEST_VARDIR/log/mysqld.1.err
--let $d=--innodb-data-home-dir=$bugdir --innodb-log-group-home-dir=$bugdir
--let $d=$d --innodb-data-file-path=ibdata1:1M:autoextend
--let $d=$d --innodb-undo-tablespaces=0
--let $d=$d --skip-innodb-fast-shutdown --innodb_undo_directory=$undodir
--let $restart_noprint=1
--let $restart_parameters=$d --innodb-stats-persistent=0
--source include/restart_mysqld.inc
CREATE TABLE tr(a INT PRIMARY KEY)ENGINE=InnoDB ROW_FORMAT=REDUNDANT;
CREATE TABLE tc(a INT PRIMARY KEY)ENGINE=InnoDB ROW_FORMAT=COMPACT;
CREATE TABLE td(a INT PRIMARY KEY)ENGINE=InnoDB ROW_FORMAT=DYNAMIC;
SET innodb_strict_mode=OFF;
CREATE TABLE tz(a INT PRIMARY KEY)ENGINE=InnoDB ROW_FORMAT=COMPRESSED
KEY_BLOCK_SIZE=1;
SET innodb_strict_mode=ON;
# PAGE_COMPRESSED is supported starting with MariaDB 10.1.0
CREATE TABLE tp(a INT PRIMARY KEY)ENGINE=InnoDB ROW_FORMAT=DYNAMIC
PAGE_COMPRESSED=1 PAGE_COMPRESSION_LEVEL=9;
--source include/shutdown_mysqld.inc
--perl
use strict;
do "$ENV{MTR_SUITE_DIR}/include/crc32.pl";
my $ps= $ENV{INNODB_PAGE_SIZE};
my $file= "$ENV{bugdir}/ibdata1";
open(FILE, "+<", $file) || die "Unable to open $file\n";
die "Unable to read $file" unless sysread(FILE, $_, 58) == 58;
my $full_crc32 = unpack("N",substr($_,54,4)) & 0x10; # FIL_SPACE_FLAGS
# Read DICT_HDR_TABLES, the root page number of CLUST_IND (SYS_TABLES.NAME).
sysseek(FILE, 7*$ps+38+32, 0) || die "Unable to seek $file";
die "Unable to read $file" unless sysread(FILE, $_, 4) == 4;
my $sys_tables_root = unpack("N", $_);
my $page;
print "SYS_TABLES clustered index root page ($sys_tables_root):\n";
sysseek(FILE, $sys_tables_root*$ps, 0) || die "Unable to seek $file";
die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
open(BACKUP, ">$ENV{bugdir}/sys_tables.bin") || die "Unable to open backup\n";
syswrite(BACKUP, $page, $ps)==$ps || die "Unable to write backup\n";
close(BACKUP) || die "Unable to close backup\n";
print "N_RECS=", unpack("n", substr($page,38+16,2));
print "; LEVEL=", unpack("n", substr($page,38+26,2));
print "; INDEX_ID=0x", unpack("H*", substr($page,38+28,8)), "\n";
my @fields=("NAME","DB_TRX_ID","DB_ROLL_PTR",
"ID","N_COLS","TYPE","MIX_ID","MIX_LEN","CLUSTER_NAME","SPACE");
for (my $offset= 0x65; $offset;
$offset= unpack("n", substr($page,$offset-2,2)))
{
print "header=0x", unpack("H*",substr($page,$offset-6,6)), " (";
my $n_fields= unpack("n", substr($page,$offset-4,2)) >> 1 & 0x3ff;
my $start= 0;
my $name;
for (my $i= 0; $i < $n_fields; $i++) {
my $end= unpack("C", substr($page, $offset-7-$i, 1));
print ",\n " if $i;
print "$fields[$i]=";
if ($end & 0x80) {
print "NULL(", ($end & 0x7f) - $start, " bytes)"
} elsif ($n_fields > 1 && $i == 0) {
$name= substr($page,$offset+$start,$end-$start);
print "'$name'"
} else {
print "0x", unpack("H*", substr($page,$offset+$start,$end-$start))
}
# Corrupt SYS_TABLES.TYPE
if ($i == 5)
{
my $flags= 0;
if ($name eq 'test/tr') {
$flags= 0x40 # DATA_DIR (largely ignored by 10.1+)
} elsif ($name eq 'test/tc') {
$flags= 0x80 # 10.1 PAGE_COMPRESSED
} elsif ($name eq 'test/td') {
$flags= 0xf00 # PAGE_COMPRESSION_LEVEL=15 (0..9 is valid)
# As part of the MDEV-12873 fix, because the
# PAGE_COMPRESSED=YES flag was not set, we will assume that
# this table was actually created with 10.2.2..10.2.6
# using PAGE_COMPRESSED=YES PAGE_COMPRESSION_LEVEL=7.
} elsif ($name eq 'test/tz') {
$flags= 0x3000 # 10.1 ATOMIC_WRITES=3 (0..2 is valid)
} elsif ($name eq 'test/tp') {
$flags= 0x880 # 10.1 PAGE_COMPRESSED, PAGE_COMPRESSION_LEVEL=8
# (in 10.2.2 through 10.2.6, this is interpreted as
# PAGE_COMPRESSION_LEVEL=4 without PAGE_COMPRESSED
# but with SHARED_SPACE, which should be invalid)
}
substr($page,$offset+$start,$end-$start)= pack(
"N", $flags ^
unpack("N", substr($page,$offset+$start,$end-$start)))
if $flags;
}
# Corrupt SYS_TABLES.MIX_LEN (ignored for ROW_FORMAT=REDUNDANT)
if ($i == 7 && $name eq 'test/tr')
{
substr($page,$offset+$start,$end-$start)= chr(255) x 4;
}
$start= $end & 0x7f;
}
print ")\n";
}
my $polynomial = 0x82f63b78; # CRC-32C
if ($full_crc32)
{
my $ck = mycrc32(substr($page, 0, $ps-4), 0, $polynomial);
substr($page, $ps-4, 4) = pack("N", $ck);
}
else
{
my $ck= pack("N",mycrc32(substr($page, 4, 22), 0, $polynomial) ^
mycrc32(substr($page, 38, $ps - 38 - 8), 0, $polynomial));
substr($page,0,4)=$ck;
substr($page,$ps-8,4)=$ck;
}
sysseek(FILE, $sys_tables_root*$ps, 0) || die "Unable to seek $file";
syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
close(FILE) || die "Unable to close $file\n";
EOF
--source include/start_mysqld.inc
--error ER_NO_SUCH_TABLE_IN_ENGINE
SHOW CREATE TABLE tr;
--error ER_NO_SUCH_TABLE_IN_ENGINE
SHOW CREATE TABLE tc;
--error ER_NO_SUCH_TABLE_IN_ENGINE
SELECT * FROM tc;
--error ER_GET_ERRNO
SHOW CREATE TABLE td;
--error ER_GET_ERRNO
SELECT * FROM td;
# This table was converted to NO_ROLLBACK due to the SYS_TABLES.TYPE change.
SHOW CREATE TABLE tz;
BEGIN;
INSERT INTO tz VALUES(42);
ROLLBACK;
SELECT * FROM tz;
--error ER_NO_SUCH_TABLE_IN_ENGINE
SHOW CREATE TABLE tp;
--source include/shutdown_mysqld.inc
let SEARCH_FILE= $MYSQLTEST_VARDIR/log/mysqld.1.err;
--let SEARCH_PATTERN= InnoDB: Table test/t[cp] in InnoDB data dictionary contains invalid flags\. SYS_TABLES\.TYPE=(129|289|3873|1232[13]) SYS_TABLES\.N_COLS=2147483649
--source include/search_pattern_in_file.inc
--let SEARCH_PATTERN= InnoDB: Table test/tr in InnoDB data dictionary contains invalid flags\. SYS_TABLES\.TYPE=65 SYS_TABLES\.MIX_LEN=4294967295\b
--source include/search_pattern_in_file.inc
# Restore the backup of the corrupted SYS_TABLES clustered index root page
--perl
use strict;
my $ps= $ENV{INNODB_PAGE_SIZE};
my $file= "$ENV{bugdir}/ibdata1";
open(FILE, "+<", $file) || die "Unable to open $file\n";
open(BACKUP, "<$ENV{bugdir}/sys_tables.bin") || die "Unable to open backup\n";
# Read DICT_HDR_TABLES, the root page number of CLUST_IND (SYS_TABLES.NAME).
sysseek(FILE, 7*$ps+38+32, 0) || die "Unable to seek $file";
die "Unable to read $file\n" unless sysread(FILE, $_, 4) == 4;
my $sys_tables_root = unpack("N", $_);
print "Restoring SYS_TABLES clustered index root page ($sys_tables_root)\n";
sysseek(FILE, $sys_tables_root*$ps, 0) || die "Unable to seek $file";
die "Unable to read backup\n" unless sysread(BACKUP, $_, $ps) == $ps;
die "Unable to restore backup\n" unless syswrite(FILE, $_, $ps) == $ps;
close(BACKUP);
close(FILE) || die "Unable to close $file\n";
EOF
--source include/start_mysqld.inc
SHOW CREATE TABLE tr;
SHOW CREATE TABLE tc;
SHOW CREATE TABLE td;
SHOW CREATE TABLE tz;
SHOW CREATE TABLE tp;
BEGIN;
INSERT INTO tr VALUES(1);
INSERT INTO tc VALUES(1);
INSERT INTO td VALUES(1);
# We cannot access tz, because due to our fiddling of the NO_ROLLBACK flag,
# it now has a record with DB_TRX_ID=0, which is invalid for
# transactional tables until MDEV-12288 is implemented.
# INSERT INTO tz VALUES(1);
INSERT INTO tp VALUES(1);
ROLLBACK;
SELECT * FROM tr;
SELECT * FROM tc;
SELECT * FROM td;
# SELECT * FROM tz;
SELECT * FROM tp;
DROP TABLE tr,tc,td,tz,tp;
--let $restart_parameters=
--source include/restart_mysqld.inc
--error 0,1
--remove_file $bugdir/ibtmp1
--error 0,1
--remove_file $bugdir/ib_buffer_pool
--list_files $bugdir
--remove_files_wildcard $bugdir
--rmdir $bugdir
--rmdir $undodir
call mtr.add_suppression("ERROR HY000: Can't create table `test`.`t1`");
--error ER_FK_NO_INDEX_PARENT
CREATE TABLE t1(f1 INT, f2 VARCHAR(1), KEY k1(f2),
FULLTEXT KEY(f2),
FOREIGN KEY (f2) REFERENCES t1(f3))ENGINE=InnoDB;
--echo #
--echo # MDEV-23199 page_compression flag is missing
--echo # for full_crc32 tablespace
--echo #
CREATE TABLE t1(f1 BIGINT PRIMARY KEY)ENGINE=InnoDB;
INSERT INTO t1 VALUES(1);
ALTER TABLE t1 PAGE_COMPRESSED = 1;
INSERT INTO t1 VALUES(2);
let $shutdown_timeout = 0;
--source include/restart_mysqld.inc
DROP TABLE t1;
--echo #
--echo # MDEV-34222 Alter operation on redundant table aborts the server
--echo #
SET @df_row = @@global.INNODB_DEFAULT_ROW_FORMAT;
SET GLOBAL INNODB_DEFAULT_ROW_FORMAT=REDUNDANT;
CREATE TABLE t1 (c CHAR(1)) ENGINE=InnoDB;
SET GLOBAL INNODB_DEFAULT_ROW_FORMAT=compact;
--error ER_ILLEGAL_HA_CREATE_OPTION
ALTER TABLE t1 PAGE_COMPRESSED=1;
DROP TABLE t1;
SET @@global.INNODB_DEFAULT_ROW_FORMAT = @df_row;