mariadb/mysql-test/suite/innodb/t/instant_alter_extend.test

211 lines
6.2 KiB
Text
Raw Normal View History

MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension This was developed by Aleksey Midenkov based on my design. In the original InnoDB storage format (that was retroactively named ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field is stored explicitly. Because of this, we can and now will allow instant conversion from VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size, as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT to INT to BIGINT (while not changing between signed and unsigned). Theoretically, we could allow changing from an unsigned integer to a bigger unsigned integer, as well as changing CHAR to VARCHAR, but that would require additional metadata and conversions whenever reading old records. Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal(): Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises HA_EXTENDED_TYPES_CONVERSION capability and we are considering the above-mentioned conversions. ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT will map to this flag. dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on. dtype_get_sql_null_size(): Keep returning the current size for BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT it will remain possible to update in place between NULL and NOT NULL values. btr_index_rec_validate(): Relax a CHECK TABLE length check for ROW_FORMAT=REDUNDANT tables. btr_cur_instant_init_low(): No longer trust fixed_len for ROW_FORMAT=REDUNDANT tables. We cannot rely on fixed_len anymore because the record can have shorter length from before instant extension. Note that importing such tablespace into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when using a .cfg file.
2019-02-13 17:39:05 +02:00
--source include/have_innodb.inc
--source include/maybe_debug.inc
-- echo #
-- echo # MDEV-15563: Instant ROW_FORMAT=REDUNDANT column type change&extension
-- echo #
# Use character-set-server in test db
create or replace database test;
use test;
set default_storage_engine=innodb;
set @save_format= @@GLOBAL.innodb_default_row_format;
SET GLOBAL innodb_default_row_format=redundant;
set @bigval= repeat('0123456789', 30);
delimiter ~~;
create or replace procedure check_table(table_name varchar(255))
begin
select table_id into @table_id
from information_schema.innodb_sys_tables
where name = concat('test/', table_name);
select name, mtype, hex(prtype) as prtype, len
from information_schema.innodb_sys_columns
where table_id = @table_id;
end~~
delimiter ;~~
--echo # VARCHAR -> CHAR, VARBINARY -> BINARY conversion
set @bigval= repeat('0123456789', 20);
create or replace table t (a varchar(300));
--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
alter table t modify a char(255), algorithm=instant;
alter table t modify a char(255), algorithm=copy;
create or replace table t (a varchar(200));
MDEV-15564 Avoid table rebuild in ALTER TABLE on collation or charset changes Allow ALGORITHM=INSTANT (or avoid touching any data) when changing the collation, or in some cases, the character set, of a non-indexed CHAR or VARCHAR column. There is no penalty for subsequent DDL or DML operations, and compatibility with older MariaDB versions will be unaffected. Character sets may be changed when the old encoding is compatible with the new one. For example, changing from ASCII to anything ASCII-based, or from 3-byte to 4-byte UTF-8 can sometimes be performed instantly. This is joint work with Eugene Kosov. The test cases as well as ALTER_CONVERT_TO, charsets_are_compatible(), Type_handler::Charsets_are_compatible() are his work. The Field_str::is_equal(), Field_varstring::is_equal() and the InnoDB changes were mostly rewritten by me due to conflicts with MDEV-15563. Limitations: Changes of indexed columns will still require ALGORITHM=COPY. We should allow ALGORITHM=NOCOPY and allow the indexes to be rebuilt inside the storage engine, without copying the entire table. Instant column size changes (in bytes) are not supported by all storage engines. Instant CHAR column changes are only allowed for InnoDB ROW_FORMAT=REDUNDANT. We could allow this for InnoDB when the CHAR internally uses a variable-length encoding, say, when converting from 3-byte UTF-8 to 4-byte UTF-8. Instant VARCHAR column changes are allowed for InnoDB ROW_FORMAT=REDUNDANT, and for others only if the size in bytes does not change from 128..255 bytes to more than 256 bytes. Inside InnoDB, this slightly changes the way how MDEV-15563 works and fixes the result of the innodb.instant_alter_extend test. We change the way how ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT is handled. All column extension, type changes and renaming now go through a common route, except when ctx->is_instant() is in effect, for example, instant ADD or DROP COLUMN has been initiated. Only in that case we will go through innobase_instant_try() and rewrite all column metadata. get_type(field, prtype, mtype, len): Convert a SQL data type into InnoDB column metadata. innobase_rename_column_try(): Remove the update of SYS_COLUMNS. innobase_rename_or_enlarge_column_try(): New function, replacing part of innobase_rename_column_try() and all of innobase_enlarge_column_try(). Also changes column types. innobase_rename_or_enlarge_columns_cache(): Also change the column type.
2019-02-14 20:17:14 +02:00
insert into t values (@bigval);
insert into t values ('z');
MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension This was developed by Aleksey Midenkov based on my design. In the original InnoDB storage format (that was retroactively named ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field is stored explicitly. Because of this, we can and now will allow instant conversion from VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size, as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT to INT to BIGINT (while not changing between signed and unsigned). Theoretically, we could allow changing from an unsigned integer to a bigger unsigned integer, as well as changing CHAR to VARCHAR, but that would require additional metadata and conversions whenever reading old records. Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal(): Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises HA_EXTENDED_TYPES_CONVERSION capability and we are considering the above-mentioned conversions. ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT will map to this flag. dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on. dtype_get_sql_null_size(): Keep returning the current size for BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT it will remain possible to update in place between NULL and NOT NULL values. btr_index_rec_validate(): Relax a CHECK TABLE length check for ROW_FORMAT=REDUNDANT tables. btr_cur_instant_init_low(): No longer trust fixed_len for ROW_FORMAT=REDUNDANT tables. We cannot rely on fixed_len anymore because the record can have shorter length from before instant extension. Note that importing such tablespace into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when using a .cfg file.
2019-02-13 17:39:05 +02:00
if ($have_debug) {
--disable_query_log
MDEV-15564 Avoid table rebuild in ALTER TABLE on collation or charset changes Allow ALGORITHM=INSTANT (or avoid touching any data) when changing the collation, or in some cases, the character set, of a non-indexed CHAR or VARCHAR column. There is no penalty for subsequent DDL or DML operations, and compatibility with older MariaDB versions will be unaffected. Character sets may be changed when the old encoding is compatible with the new one. For example, changing from ASCII to anything ASCII-based, or from 3-byte to 4-byte UTF-8 can sometimes be performed instantly. This is joint work with Eugene Kosov. The test cases as well as ALTER_CONVERT_TO, charsets_are_compatible(), Type_handler::Charsets_are_compatible() are his work. The Field_str::is_equal(), Field_varstring::is_equal() and the InnoDB changes were mostly rewritten by me due to conflicts with MDEV-15563. Limitations: Changes of indexed columns will still require ALGORITHM=COPY. We should allow ALGORITHM=NOCOPY and allow the indexes to be rebuilt inside the storage engine, without copying the entire table. Instant column size changes (in bytes) are not supported by all storage engines. Instant CHAR column changes are only allowed for InnoDB ROW_FORMAT=REDUNDANT. We could allow this for InnoDB when the CHAR internally uses a variable-length encoding, say, when converting from 3-byte UTF-8 to 4-byte UTF-8. Instant VARCHAR column changes are allowed for InnoDB ROW_FORMAT=REDUNDANT, and for others only if the size in bytes does not change from 128..255 bytes to more than 256 bytes. Inside InnoDB, this slightly changes the way how MDEV-15563 works and fixes the result of the innodb.instant_alter_extend test. We change the way how ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT is handled. All column extension, type changes and renaming now go through a common route, except when ctx->is_instant() is in effect, for example, instant ADD or DROP COLUMN has been initiated. Only in that case we will go through innobase_instant_try() and rewrite all column metadata. get_type(field, prtype, mtype, len): Convert a SQL data type into InnoDB column metadata. innobase_rename_column_try(): Remove the update of SYS_COLUMNS. innobase_rename_or_enlarge_column_try(): New function, replacing part of innobase_rename_column_try() and all of innobase_enlarge_column_try(). Also changes column types. innobase_rename_or_enlarge_columns_cache(): Also change the column type.
2019-02-14 20:17:14 +02:00
# This should not be reachable.
set @save_debug= @@SESSION.debug_dbug;
MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension This was developed by Aleksey Midenkov based on my design. In the original InnoDB storage format (that was retroactively named ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field is stored explicitly. Because of this, we can and now will allow instant conversion from VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size, as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT to INT to BIGINT (while not changing between signed and unsigned). Theoretically, we could allow changing from an unsigned integer to a bigger unsigned integer, as well as changing CHAR to VARCHAR, but that would require additional metadata and conversions whenever reading old records. Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal(): Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises HA_EXTENDED_TYPES_CONVERSION capability and we are considering the above-mentioned conversions. ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT will map to this flag. dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on. dtype_get_sql_null_size(): Keep returning the current size for BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT it will remain possible to update in place between NULL and NOT NULL values. btr_index_rec_validate(): Relax a CHECK TABLE length check for ROW_FORMAT=REDUNDANT tables. btr_cur_instant_init_low(): No longer trust fixed_len for ROW_FORMAT=REDUNDANT tables. We cannot rely on fixed_len anymore because the record can have shorter length from before instant extension. Note that importing such tablespace into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when using a .cfg file.
2019-02-13 17:39:05 +02:00
set debug_dbug= '+d,ib_instant_error';
--enable_query_log
}
alter table t modify a char(200), algorithm=instant;
select count(a) from t where a = @bigval;
select a, length(a) from t where a = 'z';
check table t extended;
call check_table('t');
--echo # CHAR enlargement
alter table t modify a char(220), algorithm=instant;
select count(a) from t where a = @bigval;
select a, length(a) from t where a = 'z';
check table t extended;
call check_table('t');
--echo # Convert from VARCHAR to a bigger CHAR
--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
alter table t modify a varchar(200), algorithm=instant;
alter table t modify a varchar(200), algorithm=copy;
alter table t modify a char(255), algorithm=instant;
select count(a) from t where a = @bigval;
select a, length(a) from t where a = 'z';
select * from t;
check table t extended;
call check_table('t');
--echo # BINARY/VARBINARY test
create or replace table t (a varbinary(300));
--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
alter table t modify a binary(255), algorithm=instant;
alter table t modify a binary(255), algorithm=copy;
create or replace table t (a varbinary(200));
insert into t values (@bigval);
insert into t values ('z');
alter table t modify a binary(200), algorithm=instant;
select count(a) from t where a = @bigval;
select length(a) from t where left(a, 1) = 'z';
check table t extended;
call check_table('t');
--echo # BINARY enlargement
alter table t modify a binary(220), algorithm=instant;
check table t extended;
call check_table('t');
--echo # Convert from VARBINARY to a bigger BINARY
--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON
alter table t modify a varbinary(220), algorithm=instant;
alter table t modify a varbinary(220), algorithm=copy;
alter table t modify a binary(255), algorithm=instant;
select count(a) from t where a = @bigval;
select a, length(a) from t where a = 'z';
select * from t;
check table t extended;
call check_table('t');
--echo # Integer conversions
create or replace table t (x tinyint);
insert into t values (127);
alter table t modify x smallint, algorithm=instant;
select * from t;
check table t extended;
call check_table('t');
update t set x= 32767;
alter table t modify x mediumint, algorithm=instant;
select * from t;
check table t extended;
call check_table('t');
update t set x= 8388607;
alter table t modify x int, algorithm=instant;
select * from t;
check table t extended;
call check_table('t');
update t set x= 2147483647;
alter table t modify x bigint, algorithm=instant;
select * from t;
check table t extended;
call check_table('t');
MDEV-15564 Avoid table rebuild in ALTER TABLE on collation or charset changes Allow ALGORITHM=INSTANT (or avoid touching any data) when changing the collation, or in some cases, the character set, of a non-indexed CHAR or VARCHAR column. There is no penalty for subsequent DDL or DML operations, and compatibility with older MariaDB versions will be unaffected. Character sets may be changed when the old encoding is compatible with the new one. For example, changing from ASCII to anything ASCII-based, or from 3-byte to 4-byte UTF-8 can sometimes be performed instantly. This is joint work with Eugene Kosov. The test cases as well as ALTER_CONVERT_TO, charsets_are_compatible(), Type_handler::Charsets_are_compatible() are his work. The Field_str::is_equal(), Field_varstring::is_equal() and the InnoDB changes were mostly rewritten by me due to conflicts with MDEV-15563. Limitations: Changes of indexed columns will still require ALGORITHM=COPY. We should allow ALGORITHM=NOCOPY and allow the indexes to be rebuilt inside the storage engine, without copying the entire table. Instant column size changes (in bytes) are not supported by all storage engines. Instant CHAR column changes are only allowed for InnoDB ROW_FORMAT=REDUNDANT. We could allow this for InnoDB when the CHAR internally uses a variable-length encoding, say, when converting from 3-byte UTF-8 to 4-byte UTF-8. Instant VARCHAR column changes are allowed for InnoDB ROW_FORMAT=REDUNDANT, and for others only if the size in bytes does not change from 128..255 bytes to more than 256 bytes. Inside InnoDB, this slightly changes the way how MDEV-15563 works and fixes the result of the innodb.instant_alter_extend test. We change the way how ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT is handled. All column extension, type changes and renaming now go through a common route, except when ctx->is_instant() is in effect, for example, instant ADD or DROP COLUMN has been initiated. Only in that case we will go through innobase_instant_try() and rewrite all column metadata. get_type(field, prtype, mtype, len): Convert a SQL data type into InnoDB column metadata. innobase_rename_column_try(): Remove the update of SYS_COLUMNS. innobase_rename_or_enlarge_column_try(): New function, replacing part of innobase_rename_column_try() and all of innobase_enlarge_column_try(). Also changes column types. innobase_rename_or_enlarge_columns_cache(): Also change the column type.
2019-02-14 20:17:14 +02:00
if ($have_debug) {
--disable_query_log
# This should not be reachable.
set debug_dbug= @save_debug;
--enable_query_log
}
MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension This was developed by Aleksey Midenkov based on my design. In the original InnoDB storage format (that was retroactively named ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field is stored explicitly. Because of this, we can and now will allow instant conversion from VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size, as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT to INT to BIGINT (while not changing between signed and unsigned). Theoretically, we could allow changing from an unsigned integer to a bigger unsigned integer, as well as changing CHAR to VARCHAR, but that would require additional metadata and conversions whenever reading old records. Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal(): Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises HA_EXTENDED_TYPES_CONVERSION capability and we are considering the above-mentioned conversions. ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT will map to this flag. dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on. dtype_get_sql_null_size(): Keep returning the current size for BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT it will remain possible to update in place between NULL and NOT NULL values. btr_index_rec_validate(): Relax a CHECK TABLE length check for ROW_FORMAT=REDUNDANT tables. btr_cur_instant_init_low(): No longer trust fixed_len for ROW_FORMAT=REDUNDANT tables. We cannot rely on fixed_len anymore because the record can have shorter length from before instant extension. Note that importing such tablespace into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when using a .cfg file.
2019-02-13 17:39:05 +02:00
--echo # Check IMPORT TABLESPACE
--let $MYSQLD_DATADIR= `select @@datadir`
create or replace table t2 (x int);
alter table t2 discard tablespace;
create or replace table t1 (x tinyint);
insert into t1 set x= 42;
alter table t1 modify x int;
flush tables t1 for export;
--move_file $MYSQLD_DATADIR/test/t1.cfg $MYSQLD_DATADIR/test/t2.cfg
--copy_file $MYSQLD_DATADIR/test/t1.ibd $MYSQLD_DATADIR/test/t2.ibd
unlock tables;
alter table t2 import tablespace;
select * from t2;
check table t2 extended;
call check_table('t2');
--echo # Check innobase_col_to_mysql() len < flen
create or replace table t1 (x mediumint);
insert into t1 values (1);
insert into t1 values (1);
alter table t1 add column y int first, modify x int, algorithm instant;
--error ER_DUP_ENTRY
alter table t1 add column z int first, add primary key (x);
--echo # Check assertion in wrong instant operation
create or replace table t1 (a varchar(26) not null) default character set utf8mb4;
alter table t1 modify a varchar(25) not null;
--echo # Check row_mysql_store_col_in_innobase_format()
create or replace table t1(x int primary key, a varchar(20));
insert into t1 (x) values (1);
update t1 set a= 'foo' where x = 2;
--echo #
--echo # MDEV-18124 PK on inplace-enlarged type fails
--echo #
create or replace table t1 (x int, y int);
insert into t1 (x, y) values (11, 22);
alter table t1 modify x bigint, algorithm instant;
alter table t1 add primary key (x), algorithm inplace;
check table t1;
create or replace table t1 (a varchar(10), y int);
insert into t1 (a, y) values ("0123456789", 33);
alter table t1 modify a char(15), algorithm instant;
alter table t1 add primary key (a), algorithm inplace;
check table t1;
create or replace table t1 (x int primary key, y int);
insert into t1 (x, y) values (44, 55);
alter table t1 modify x bigint, algorithm inplace;
check table t1;
create or replace table t1 (x int primary key, y int);
insert into t1 values (66, 77);
alter table t1 add column z int, algorithm instant;
alter table t1 drop column y, algorithm instant;
create or replace table t1 (x integer, a varchar(20));
alter table t1 add index idx3 (a);
insert into t1 (x, a) values (73, 'a');
alter table t1 modify a char(20);
create or replace database test charset latin1;
SET GLOBAL innodb_default_row_format=@save_format;