2015-09-04 14:54:20 +02:00
|
|
|
SET GLOBAL innodb_file_format = `Barracuda`;
|
|
|
|
|
SET GLOBAL innodb_file_per_table = ON;
|
|
|
|
|
CREATE TABLE t1 (id INT NOT NULL PRIMARY KEY, a VARCHAR(255)) ENGINE=InnoDB encrypted=yes;
|
|
|
|
|
CREATE TABLE t2 (id INT NOT NULL PRIMARY KEY, a VARCHAR(255)) ENGINE=InnoDB;
|
|
|
|
|
CREATE TABLE t3 (id INT, a VARCHAR(255)) ENGINE=InnoDB encrypted=yes;
|
|
|
|
|
CREATE TABLE t4 (id INT, a VARCHAR(255)) engine=InnoDB;
|
|
|
|
|
CREATE TABLE t5 (id INT NOT NULL PRIMARY KEY, a TEXT(500), b VARCHAR(255), FULLTEXT(b)) ENGINE=InnoDB encrypted=yes;
|
|
|
|
|
CREATE TABLE t6 (id INT, a TEXT(500), b VARCHAR(255), FULLTEXT(b)) ENGINE=InnoDB;
|
|
|
|
|
CREATE TABLE t7 (id INT NOT NULL PRIMARY KEY, a VARCHAR(255)) ENGINE=InnoDB row_format=compressed encrypted=yes;
|
|
|
|
|
create procedure innodb_insert_proc (repeat_count int)
|
|
|
|
|
begin
|
|
|
|
|
declare current_num int;
|
|
|
|
|
set current_num = 0;
|
|
|
|
|
while current_num < repeat_count do
|
MDEV-12253: Buffer pool blocks are accessed after they have been freed
Problem was that bpage was referenced after it was already freed
from LRU. Fixed by adding a new variable encrypted that is
passed down to buf_page_check_corrupt() and used in
buf_page_get_gen() to stop processing page read.
This patch should also address following test failures and
bugs:
MDEV-12419: IMPORT should not look up tablespace in
PageConverter::validate(). This is now removed.
MDEV-10099: encryption.innodb_onlinealter_encryption fails
sporadically in buildbot
MDEV-11420: encryption.innodb_encryption-page-compression
failed in buildbot
MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8
Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing
and replaced these with dict_table_t::file_unreadable. Table
ibd file is missing if fil_get_space(space_id) returns NULL
and encrypted if not. Removed dict_table_t::is_corrupted field.
Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(),
buf_page_decrypt_after_read(), buf_page_encrypt_before_write(),
buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats().
Added test cases when enrypted page could be read while doing
redo log crash recovery. Also added test case for row compressed
blobs.
btr_cur_open_at_index_side_func(),
btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is
NULL.
buf_page_get_zip(): Issue error if page read fails.
buf_page_get_gen(): Use dberr_t for error detection and
do not reference bpage after we hare freed it.
buf_mark_space_corrupt(): remove bpage from LRU also when
it is encrypted.
buf_page_check_corrupt(): @return DB_SUCCESS if page has
been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match. In read
case only DB_SUCCESS is possible.
buf_page_io_complete(): use dberr_t for error handling.
buf_flush_write_block_low(),
buf_read_ahead_random(),
buf_read_page_async(),
buf_read_ahead_linear(),
buf_read_ibuf_merge_pages(),
buf_read_recv_pages(),
fil_aio_wait():
Issue error if page read fails.
btr_pcur_move_to_next_page(): Do not reference page if it is
NULL.
Introduced dict_table_t::is_readable() and dict_index_t::is_readable()
that will return true if tablespace exists and pages read from
tablespace are not corrupted or page decryption failed.
Removed buf_page_t::key_version. After page decryption the
key version is not removed from page frame. For unencrypted
pages, old key_version is removed at buf_page_encrypt_before_write()
dict_stats_update_transient_for_index(),
dict_stats_update_transient()
Do not continue if table decryption failed or table
is corrupted.
dict0stats.cc: Introduced a dict_stats_report_error function
to avoid code duplication.
fil_parse_write_crypt_data():
Check that key read from redo log entry is found from
encryption plugin and if it is not, refuse to start.
PageConverter::validate(): Removed access to fil_space_t as
tablespace is not available during import.
Fixed error code on innodb.innodb test.
Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown
to innodb-bad-key-change2. Removed innodb-bad-key-change5 test.
Decreased unnecessary complexity on some long lasting tests.
Removed fil_inc_pending_ops(), fil_decr_pending_ops(),
fil_get_first_space(), fil_get_next_space(),
fil_get_first_space_safe(), fil_get_next_space_safe()
functions.
fil_space_verify_crypt_checksum(): Fixed bug found using ASAN
where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly
accessed from row compressed tables. Fixed out of page frame
bug for row compressed tables in
fil_space_verify_crypt_checksum() found using ASAN. Incorrect
function was called for compressed table.
Added new tests for discard, rename table and drop (we should allow them
even when page decryption fails). Alter table rename is not allowed.
Added test for restart with innodb-force-recovery=1 when page read on
redo-recovery cant be decrypted. Added test for corrupted table where
both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted.
Adjusted the test case innodb_bug14147491 so that it does not anymore
expect crash. Instead table is just mostly not usable.
fil0fil.h: fil_space_acquire_low is not visible function
and fil_space_acquire and fil_space_acquire_silent are
inline functions. FilSpace class uses fil_space_acquire_low
directly.
recv_apply_hashed_log_recs() does not return anything.
2017-04-26 14:19:16 +02:00
|
|
|
insert into t1 values (current_num,repeat('foobar',12));
|
|
|
|
|
insert into t2 values (current_num,repeat('tempsecret', 12));
|
2015-09-04 14:54:20 +02:00
|
|
|
insert into t3 values (current_num,repeat('barfoo',42));
|
|
|
|
|
insert into t4 values (current_num,repeat('repeat',42));
|
|
|
|
|
insert into t5 values (current_num,substring('A BC DEF GHIJ KLM NOPQRS TUV WXYZ 012 3456789', rand()*36+1, 100), repeat('author new',22));
|
|
|
|
|
insert into t6 values (current_num,substring('A BC DEF GHIJ KLM NOPQRS TUV WXYZ 012 3456789', rand()*36+1, 100), repeat('mangled old',22));
|
|
|
|
|
insert into t7 values (current_num,repeat('mysql',42));
|
|
|
|
|
set current_num = current_num + 1;
|
|
|
|
|
end while;
|
|
|
|
|
end//
|
|
|
|
|
commit;
|
|
|
|
|
set autocommit=0;
|
2017-03-14 11:56:01 +01:00
|
|
|
call innodb_insert_proc(1500);
|
2015-09-04 14:54:20 +02:00
|
|
|
commit;
|
|
|
|
|
set autocommit=1;
|
|
|
|
|
# Wait max 10 min for key encryption threads to encrypt all spaces
|
|
|
|
|
# t1 yes on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /foobar/ in t1.ibd
|
|
|
|
|
# t2 ... on expecting NOT FOUND
|
MDEV-12253: Buffer pool blocks are accessed after they have been freed
Problem was that bpage was referenced after it was already freed
from LRU. Fixed by adding a new variable encrypted that is
passed down to buf_page_check_corrupt() and used in
buf_page_get_gen() to stop processing page read.
This patch should also address following test failures and
bugs:
MDEV-12419: IMPORT should not look up tablespace in
PageConverter::validate(). This is now removed.
MDEV-10099: encryption.innodb_onlinealter_encryption fails
sporadically in buildbot
MDEV-11420: encryption.innodb_encryption-page-compression
failed in buildbot
MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8
Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing
and replaced these with dict_table_t::file_unreadable. Table
ibd file is missing if fil_get_space(space_id) returns NULL
and encrypted if not. Removed dict_table_t::is_corrupted field.
Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(),
buf_page_decrypt_after_read(), buf_page_encrypt_before_write(),
buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats().
Added test cases when enrypted page could be read while doing
redo log crash recovery. Also added test case for row compressed
blobs.
btr_cur_open_at_index_side_func(),
btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is
NULL.
buf_page_get_zip(): Issue error if page read fails.
buf_page_get_gen(): Use dberr_t for error detection and
do not reference bpage after we hare freed it.
buf_mark_space_corrupt(): remove bpage from LRU also when
it is encrypted.
buf_page_check_corrupt(): @return DB_SUCCESS if page has
been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match. In read
case only DB_SUCCESS is possible.
buf_page_io_complete(): use dberr_t for error handling.
buf_flush_write_block_low(),
buf_read_ahead_random(),
buf_read_page_async(),
buf_read_ahead_linear(),
buf_read_ibuf_merge_pages(),
buf_read_recv_pages(),
fil_aio_wait():
Issue error if page read fails.
btr_pcur_move_to_next_page(): Do not reference page if it is
NULL.
Introduced dict_table_t::is_readable() and dict_index_t::is_readable()
that will return true if tablespace exists and pages read from
tablespace are not corrupted or page decryption failed.
Removed buf_page_t::key_version. After page decryption the
key version is not removed from page frame. For unencrypted
pages, old key_version is removed at buf_page_encrypt_before_write()
dict_stats_update_transient_for_index(),
dict_stats_update_transient()
Do not continue if table decryption failed or table
is corrupted.
dict0stats.cc: Introduced a dict_stats_report_error function
to avoid code duplication.
fil_parse_write_crypt_data():
Check that key read from redo log entry is found from
encryption plugin and if it is not, refuse to start.
PageConverter::validate(): Removed access to fil_space_t as
tablespace is not available during import.
Fixed error code on innodb.innodb test.
Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown
to innodb-bad-key-change2. Removed innodb-bad-key-change5 test.
Decreased unnecessary complexity on some long lasting tests.
Removed fil_inc_pending_ops(), fil_decr_pending_ops(),
fil_get_first_space(), fil_get_next_space(),
fil_get_first_space_safe(), fil_get_next_space_safe()
functions.
fil_space_verify_crypt_checksum(): Fixed bug found using ASAN
where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly
accessed from row compressed tables. Fixed out of page frame
bug for row compressed tables in
fil_space_verify_crypt_checksum() found using ASAN. Incorrect
function was called for compressed table.
Added new tests for discard, rename table and drop (we should allow them
even when page decryption fails). Alter table rename is not allowed.
Added test for restart with innodb-force-recovery=1 when page read on
redo-recovery cant be decrypted. Added test for corrupted table where
both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted.
Adjusted the test case innodb_bug14147491 so that it does not anymore
expect crash. Instead table is just mostly not usable.
fil0fil.h: fil_space_acquire_low is not visible function
and fil_space_acquire and fil_space_acquire_silent are
inline functions. FilSpace class uses fil_space_acquire_low
directly.
recv_apply_hashed_log_recs() does not return anything.
2017-04-26 14:19:16 +02:00
|
|
|
NOT FOUND /tempsecret/ in t2.ibd
|
2015-09-04 14:54:20 +02:00
|
|
|
# t3 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /barfoo/ in t3.ibd
|
|
|
|
|
# t4 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /repeat/ in t4.ibd
|
|
|
|
|
# t5 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /author/ in t5.ibd
|
|
|
|
|
# t6 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /mangled/ in t6.ibd
|
|
|
|
|
# t7 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /mysql/ in t7.ibd
|
2017-03-14 11:56:01 +01:00
|
|
|
SET GLOBAL innodb_file_format = `Barracuda`;
|
|
|
|
|
SET GLOBAL innodb_file_per_table = ON;
|
2015-09-04 14:54:20 +02:00
|
|
|
ALTER TABLE t1 ADD COLUMN b int default 2;
|
|
|
|
|
ALTER TABLE t2 ADD COLUMN b int default 2;
|
|
|
|
|
ALTER TABLE t7 ADD COLUMN b int default 2;
|
|
|
|
|
ALTER TABLE t1 ADD KEY a(a), ADD KEY b(b);
|
|
|
|
|
ALTER TABLE t2 ADD KEY a(a), ADD KEY b(b);
|
|
|
|
|
ALTER TABLE t3 ADD COLUMN c int default 5;
|
|
|
|
|
ALTER TABLE t4 ADD COLUMN c int default 5;
|
|
|
|
|
ALTER TABLE t3 ADD KEY (a), ADD KEY c(c);
|
|
|
|
|
ALTER TABLE t4 ADD KEY (a), ADD KEY c(c);
|
|
|
|
|
ALTER TABLE t5 ADD FULLTEXT(a);
|
|
|
|
|
ALTER TABLE t6 ADD FULLTEXT(a);
|
|
|
|
|
ALTER TABLE t7 ADD KEY a(a), ADD key b(b);
|
|
|
|
|
SHOW CREATE TABLE t1;
|
|
|
|
|
Table Create Table
|
|
|
|
|
t1 CREATE TABLE `t1` (
|
|
|
|
|
`id` int(11) NOT NULL,
|
|
|
|
|
`a` varchar(255) DEFAULT NULL,
|
|
|
|
|
`b` int(11) DEFAULT '2',
|
|
|
|
|
PRIMARY KEY (`id`),
|
|
|
|
|
KEY `a` (`a`),
|
|
|
|
|
KEY `b` (`b`)
|
|
|
|
|
) ENGINE=InnoDB DEFAULT CHARSET=latin1 `encrypted`=yes
|
|
|
|
|
SHOW CREATE TABLE t2;
|
|
|
|
|
Table Create Table
|
|
|
|
|
t2 CREATE TABLE `t2` (
|
|
|
|
|
`id` int(11) NOT NULL,
|
|
|
|
|
`a` varchar(255) DEFAULT NULL,
|
|
|
|
|
`b` int(11) DEFAULT '2',
|
|
|
|
|
PRIMARY KEY (`id`),
|
|
|
|
|
KEY `a` (`a`),
|
|
|
|
|
KEY `b` (`b`)
|
|
|
|
|
) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
|
|
SHOW CREATE TABLE t3;
|
|
|
|
|
Table Create Table
|
|
|
|
|
t3 CREATE TABLE `t3` (
|
|
|
|
|
`id` int(11) DEFAULT NULL,
|
|
|
|
|
`a` varchar(255) DEFAULT NULL,
|
|
|
|
|
`c` int(11) DEFAULT '5',
|
|
|
|
|
KEY `a` (`a`),
|
|
|
|
|
KEY `c` (`c`)
|
|
|
|
|
) ENGINE=InnoDB DEFAULT CHARSET=latin1 `encrypted`=yes
|
|
|
|
|
SHOW CREATE TABLE t4;
|
|
|
|
|
Table Create Table
|
|
|
|
|
t4 CREATE TABLE `t4` (
|
|
|
|
|
`id` int(11) DEFAULT NULL,
|
|
|
|
|
`a` varchar(255) DEFAULT NULL,
|
|
|
|
|
`c` int(11) DEFAULT '5',
|
|
|
|
|
KEY `a` (`a`),
|
|
|
|
|
KEY `c` (`c`)
|
|
|
|
|
) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
|
|
SHOW CREATE TABLE t5;
|
|
|
|
|
Table Create Table
|
|
|
|
|
t5 CREATE TABLE `t5` (
|
|
|
|
|
`id` int(11) NOT NULL,
|
|
|
|
|
`a` text,
|
|
|
|
|
`b` varchar(255) DEFAULT NULL,
|
|
|
|
|
PRIMARY KEY (`id`),
|
|
|
|
|
FULLTEXT KEY `b` (`b`),
|
|
|
|
|
FULLTEXT KEY `a` (`a`)
|
|
|
|
|
) ENGINE=InnoDB DEFAULT CHARSET=latin1 `encrypted`=yes
|
|
|
|
|
SHOW CREATE TABLE t6;
|
|
|
|
|
Table Create Table
|
|
|
|
|
t6 CREATE TABLE `t6` (
|
|
|
|
|
`id` int(11) DEFAULT NULL,
|
|
|
|
|
`a` text,
|
|
|
|
|
`b` varchar(255) DEFAULT NULL,
|
|
|
|
|
FULLTEXT KEY `b` (`b`),
|
|
|
|
|
FULLTEXT KEY `a` (`a`)
|
|
|
|
|
) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
|
|
|
|
SHOW CREATE TABLE t7;
|
|
|
|
|
Table Create Table
|
|
|
|
|
t7 CREATE TABLE `t7` (
|
|
|
|
|
`id` int(11) NOT NULL,
|
|
|
|
|
`a` varchar(255) DEFAULT NULL,
|
|
|
|
|
`b` int(11) DEFAULT '2',
|
|
|
|
|
PRIMARY KEY (`id`),
|
|
|
|
|
KEY `a` (`a`),
|
|
|
|
|
KEY `b` (`b`)
|
|
|
|
|
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED `encrypted`=yes
|
|
|
|
|
# t1 yes on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /foobar/ in t1.ibd
|
|
|
|
|
# t2 ... on expecting NOT FOUND
|
MDEV-12253: Buffer pool blocks are accessed after they have been freed
Problem was that bpage was referenced after it was already freed
from LRU. Fixed by adding a new variable encrypted that is
passed down to buf_page_check_corrupt() and used in
buf_page_get_gen() to stop processing page read.
This patch should also address following test failures and
bugs:
MDEV-12419: IMPORT should not look up tablespace in
PageConverter::validate(). This is now removed.
MDEV-10099: encryption.innodb_onlinealter_encryption fails
sporadically in buildbot
MDEV-11420: encryption.innodb_encryption-page-compression
failed in buildbot
MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8
Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing
and replaced these with dict_table_t::file_unreadable. Table
ibd file is missing if fil_get_space(space_id) returns NULL
and encrypted if not. Removed dict_table_t::is_corrupted field.
Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(),
buf_page_decrypt_after_read(), buf_page_encrypt_before_write(),
buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats().
Added test cases when enrypted page could be read while doing
redo log crash recovery. Also added test case for row compressed
blobs.
btr_cur_open_at_index_side_func(),
btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is
NULL.
buf_page_get_zip(): Issue error if page read fails.
buf_page_get_gen(): Use dberr_t for error detection and
do not reference bpage after we hare freed it.
buf_mark_space_corrupt(): remove bpage from LRU also when
it is encrypted.
buf_page_check_corrupt(): @return DB_SUCCESS if page has
been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match. In read
case only DB_SUCCESS is possible.
buf_page_io_complete(): use dberr_t for error handling.
buf_flush_write_block_low(),
buf_read_ahead_random(),
buf_read_page_async(),
buf_read_ahead_linear(),
buf_read_ibuf_merge_pages(),
buf_read_recv_pages(),
fil_aio_wait():
Issue error if page read fails.
btr_pcur_move_to_next_page(): Do not reference page if it is
NULL.
Introduced dict_table_t::is_readable() and dict_index_t::is_readable()
that will return true if tablespace exists and pages read from
tablespace are not corrupted or page decryption failed.
Removed buf_page_t::key_version. After page decryption the
key version is not removed from page frame. For unencrypted
pages, old key_version is removed at buf_page_encrypt_before_write()
dict_stats_update_transient_for_index(),
dict_stats_update_transient()
Do not continue if table decryption failed or table
is corrupted.
dict0stats.cc: Introduced a dict_stats_report_error function
to avoid code duplication.
fil_parse_write_crypt_data():
Check that key read from redo log entry is found from
encryption plugin and if it is not, refuse to start.
PageConverter::validate(): Removed access to fil_space_t as
tablespace is not available during import.
Fixed error code on innodb.innodb test.
Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown
to innodb-bad-key-change2. Removed innodb-bad-key-change5 test.
Decreased unnecessary complexity on some long lasting tests.
Removed fil_inc_pending_ops(), fil_decr_pending_ops(),
fil_get_first_space(), fil_get_next_space(),
fil_get_first_space_safe(), fil_get_next_space_safe()
functions.
fil_space_verify_crypt_checksum(): Fixed bug found using ASAN
where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly
accessed from row compressed tables. Fixed out of page frame
bug for row compressed tables in
fil_space_verify_crypt_checksum() found using ASAN. Incorrect
function was called for compressed table.
Added new tests for discard, rename table and drop (we should allow them
even when page decryption fails). Alter table rename is not allowed.
Added test for restart with innodb-force-recovery=1 when page read on
redo-recovery cant be decrypted. Added test for corrupted table where
both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted.
Adjusted the test case innodb_bug14147491 so that it does not anymore
expect crash. Instead table is just mostly not usable.
fil0fil.h: fil_space_acquire_low is not visible function
and fil_space_acquire and fil_space_acquire_silent are
inline functions. FilSpace class uses fil_space_acquire_low
directly.
recv_apply_hashed_log_recs() does not return anything.
2017-04-26 14:19:16 +02:00
|
|
|
NOT FOUND /tempsecret/ in t2.ibd
|
2015-09-04 14:54:20 +02:00
|
|
|
# t3 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /barfoo/ in t3.ibd
|
|
|
|
|
# t4 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /repeat/ in t4.ibd
|
|
|
|
|
# t5 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /author/ in t5.ibd
|
|
|
|
|
# t6 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /mangled/ in t6.ibd
|
|
|
|
|
# t7 ... on expecting NOT FOUND
|
|
|
|
|
NOT FOUND /mysql/ in t7.ibd
|
|
|
|
|
DROP PROCEDURE innodb_insert_proc;
|
|
|
|
|
DROP TABLE t1, t2, t3, t4, t5, t6, t7;
|