2017-08-29 10:52:52 +03:00
|
|
|
# The test verifies a few server/engine recovery option combinations.
|
|
|
|
|
# Specifically, MDEV-13437,13438 are concerned with no crashes
|
|
|
|
|
# due to InnoDB being read-only during --tc-heuristic-recover=ROLLBACK|COMMIT.
|
|
|
|
|
#
|
|
|
|
|
# Initially the test commits a transaction and in the following proceeds
|
|
|
|
|
# throughout some phases.
|
|
|
|
|
# Within them the server is shut down and attempted to restart, to succeed
|
|
|
|
|
# that in the end.
|
|
|
|
|
# All this proves no crashes and effective rollback of the transaction.
|
|
|
|
|
#
|
|
|
|
|
--source include/have_innodb.inc
|
2017-10-04 18:32:45 +02:00
|
|
|
# The test logic really requires --log-bin.
|
2017-08-29 10:52:52 +03:00
|
|
|
--source include/have_binlog_format_mixed.inc
|
|
|
|
|
--source include/have_debug_sync.inc
|
|
|
|
|
--source include/not_embedded.inc
|
|
|
|
|
|
|
|
|
|
call mtr.add_suppression("Can't init tc log");
|
|
|
|
|
call mtr.add_suppression("Found 1 prepared transactions!");
|
|
|
|
|
call mtr.add_suppression("Aborting");
|
|
|
|
|
|
|
|
|
|
# Now take a shapshot of the last time server options.
|
|
|
|
|
#
|
|
|
|
|
# The "restart" expect-file facility can't be engaged because the server
|
|
|
|
|
# having conflicting options may not succeed to boot up.
|
2017-10-04 18:32:45 +02:00
|
|
|
--let SEARCH_FILE=$MYSQLTEST_VARDIR/log/mysqld.1.err
|
2017-08-29 10:52:52 +03:00
|
|
|
|
|
|
|
|
CREATE TABLE t1 (i INT) ENGINE=InnoDB;
|
|
|
|
|
SET GLOBAL innodb_flush_log_at_trx_commit=1;
|
|
|
|
|
FLUSH TABLES; # we need the table post crash-restart, see MDEV-8841.
|
|
|
|
|
|
|
|
|
|
# Run transaction in a separate "prey" connection
|
|
|
|
|
--connect (con1,localhost,root,,)
|
|
|
|
|
# The signal won't arrive though
|
|
|
|
|
set debug_sync='ha_commit_trans_after_prepare WAIT_FOR go';
|
|
|
|
|
--send INSERT INTO t1 VALUES (1);
|
|
|
|
|
|
|
|
|
|
--connection default
|
|
|
|
|
|
|
|
|
|
--let $table= information_schema.processlist
|
|
|
|
|
--let $where= where state = 'debug sync point: ha_commit_trans_after_prepare'
|
|
|
|
|
--let $wait_condition= SELECT count(*) = 1 FROM $table $where
|
|
|
|
|
--source include/wait_condition.inc
|
|
|
|
|
|
|
|
|
|
--echo # Prove that no COMMIT or ROLLBACK occurred yet.
|
|
|
|
|
SELECT * FROM t1;
|
|
|
|
|
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
|
|
|
SELECT * FROM t1;
|
|
|
|
|
|
|
|
|
|
# TODO: MDEV-12700 Allow innodb_read_only startup without prior slow shutdown.
|
|
|
|
|
--source include/kill_mysqld.inc
|
2017-10-04 18:32:45 +02:00
|
|
|
--error 1
|
MDEV-19514 Defer change buffer merge until pages are requested
We will remove the InnoDB background operation of merging buffered
changes to secondary index leaf pages. Changes will only be merged as a
result of an operation that accesses a secondary index leaf page,
such as a SQL statement that performs a lookup via that index,
or is modifying the index. Also ROLLBACK and some background operations,
such as purging the history of committed transactions, or computing
index cardinality statistics, can cause change buffer merge.
Encryption key rotation will not perform change buffer merge.
The motivation of this change is to simplify the I/O logic and to
allow crash recovery to happen in the background (MDEV-14481).
We also hope that this will reduce the number of "mystery" crashes
due to corrupted data. Because change buffer merge will typically
take place as a result of executing SQL statements, there should be
a clearer connection between the crash and the SQL statements that
were executed when the server crashed.
In many cases, a slight performance improvement was observed.
This is joint work with Thirunarayanan Balathandayuthapani
and was tested by Axel Schwenke and Matthias Leich.
The InnoDB monitor counter innodb_ibuf_merge_usec will be removed.
On slow shutdown (innodb_fast_shutdown=0), we will continue to
merge all buffered changes (and purge all undo log history).
Two InnoDB configuration parameters will be changed as follows:
innodb_disable_background_merge: Removed.
This parameter existed only in debug builds.
All change buffer merges will use synchronous reads.
innodb_force_recovery will be changed as follows:
* innodb_force_recovery=4 will be the same as innodb_force_recovery=3
(the change buffer merge cannot be disabled; it can only happen as
a result of an operation that accesses a secondary index leaf page).
The option used to be capable of corrupting secondary index leaf pages.
Now that capability is removed, and innodb_force_recovery=4 becomes 'safe'.
* innodb_force_recovery=5 (which essentially hard-wires
SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED)
becomes safe to use. Bogus data can be returned to SQL, but
persistent InnoDB data files will not be corrupted further.
* innodb_force_recovery=6 (ignore the redo log files)
will be the only option that can potentially cause
persistent corruption of InnoDB data files.
Code changes:
buf_page_t::ibuf_exist: New flag, to indicate whether buffered
changes exist for a buffer pool page. Pages with pending changes
can be returned by buf_page_get_gen(). Previously, the changes
were always merged inside buf_page_get_gen() if needed.
ibuf_page_exists(const buf_page_t&): Check if a buffered changes
exist for an X-latched or read-fixed page.
buf_page_get_gen(): Add the parameter allow_ibuf_merge=false.
All callers that know that they may be accessing a secondary index
leaf page must pass this parameter as allow_ibuf_merge=true,
unless it does not matter for that caller whether all buffered
changes have been applied. Assert that whenever allow_ibuf_merge
holds, the page actually is a leaf page. Attempt change buffer
merge only to secondary B-tree index leaf pages.
btr_block_get(): Add parameter 'bool merge'.
All callers of btr_block_get() should know whether the page could be
a secondary index leaf page. If it is not, we should avoid consulting
the change buffer bitmap to even consider a merge. This is the main
interface to requesting index pages from the buffer pool.
ibuf_merge_or_delete_for_page(), recv_recover_page(): Replace
buf_page_get_known_nowait() with much simpler logic, because
it is now guaranteed that that the block is x-latched or read-fixed.
mlog_init_t::mark_ibuf_exist(): Renamed from mlog_init_t::ibuf_merge().
On crash recovery, we will no longer merge any buffered changes
for the pages that we read into the buffer pool during the last batch
of applying log records.
buf_page_get_gen_known_nowait(), BUF_MAKE_YOUNG, BUF_KEEP_OLD: Remove.
btr_search_guess_on_hash(): Merge buf_page_get_gen_known_nowait()
to its only remaining caller.
buf_page_make_young_if_needed(): Define as an inline function.
Add the parameter buf_pool.
buf_page_peek_if_young(), buf_page_peek_if_too_old(): Add the
parameter buf_pool.
fil_space_validate_for_mtr_commit(): Remove a bogus comment
about background merge of the change buffer.
btr_cur_open_at_rnd_pos_func(), btr_cur_search_to_nth_level_func(),
btr_cur_open_at_index_side_func(): Use narrower data types and scopes.
ibuf_read_merge_pages(): Replaces buf_read_ibuf_merge_pages().
Merge the change buffer by invoking buf_page_get_gen().
2019-10-11 17:28:15 +03:00
|
|
|
--exec $MYSQLD_LAST_CMD --log-bin=master-bin --binlog-format=mixed --core-file --loose-debug-sync-timeout=300 --debug_dbug="+d,innobase_xa_fail"
|
2017-08-29 10:52:52 +03:00
|
|
|
|
|
|
|
|
--let SEARCH_PATTERN= was in the XA prepared state
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= Found 1 prepared transactions!
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= \\[ERROR\\] Can\\'t init tc log
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
|
2017-10-04 18:32:45 +02:00
|
|
|
--error 1
|
MDEV-19514 Defer change buffer merge until pages are requested
We will remove the InnoDB background operation of merging buffered
changes to secondary index leaf pages. Changes will only be merged as a
result of an operation that accesses a secondary index leaf page,
such as a SQL statement that performs a lookup via that index,
or is modifying the index. Also ROLLBACK and some background operations,
such as purging the history of committed transactions, or computing
index cardinality statistics, can cause change buffer merge.
Encryption key rotation will not perform change buffer merge.
The motivation of this change is to simplify the I/O logic and to
allow crash recovery to happen in the background (MDEV-14481).
We also hope that this will reduce the number of "mystery" crashes
due to corrupted data. Because change buffer merge will typically
take place as a result of executing SQL statements, there should be
a clearer connection between the crash and the SQL statements that
were executed when the server crashed.
In many cases, a slight performance improvement was observed.
This is joint work with Thirunarayanan Balathandayuthapani
and was tested by Axel Schwenke and Matthias Leich.
The InnoDB monitor counter innodb_ibuf_merge_usec will be removed.
On slow shutdown (innodb_fast_shutdown=0), we will continue to
merge all buffered changes (and purge all undo log history).
Two InnoDB configuration parameters will be changed as follows:
innodb_disable_background_merge: Removed.
This parameter existed only in debug builds.
All change buffer merges will use synchronous reads.
innodb_force_recovery will be changed as follows:
* innodb_force_recovery=4 will be the same as innodb_force_recovery=3
(the change buffer merge cannot be disabled; it can only happen as
a result of an operation that accesses a secondary index leaf page).
The option used to be capable of corrupting secondary index leaf pages.
Now that capability is removed, and innodb_force_recovery=4 becomes 'safe'.
* innodb_force_recovery=5 (which essentially hard-wires
SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED)
becomes safe to use. Bogus data can be returned to SQL, but
persistent InnoDB data files will not be corrupted further.
* innodb_force_recovery=6 (ignore the redo log files)
will be the only option that can potentially cause
persistent corruption of InnoDB data files.
Code changes:
buf_page_t::ibuf_exist: New flag, to indicate whether buffered
changes exist for a buffer pool page. Pages with pending changes
can be returned by buf_page_get_gen(). Previously, the changes
were always merged inside buf_page_get_gen() if needed.
ibuf_page_exists(const buf_page_t&): Check if a buffered changes
exist for an X-latched or read-fixed page.
buf_page_get_gen(): Add the parameter allow_ibuf_merge=false.
All callers that know that they may be accessing a secondary index
leaf page must pass this parameter as allow_ibuf_merge=true,
unless it does not matter for that caller whether all buffered
changes have been applied. Assert that whenever allow_ibuf_merge
holds, the page actually is a leaf page. Attempt change buffer
merge only to secondary B-tree index leaf pages.
btr_block_get(): Add parameter 'bool merge'.
All callers of btr_block_get() should know whether the page could be
a secondary index leaf page. If it is not, we should avoid consulting
the change buffer bitmap to even consider a merge. This is the main
interface to requesting index pages from the buffer pool.
ibuf_merge_or_delete_for_page(), recv_recover_page(): Replace
buf_page_get_known_nowait() with much simpler logic, because
it is now guaranteed that that the block is x-latched or read-fixed.
mlog_init_t::mark_ibuf_exist(): Renamed from mlog_init_t::ibuf_merge().
On crash recovery, we will no longer merge any buffered changes
for the pages that we read into the buffer pool during the last batch
of applying log records.
buf_page_get_gen_known_nowait(), BUF_MAKE_YOUNG, BUF_KEEP_OLD: Remove.
btr_search_guess_on_hash(): Merge buf_page_get_gen_known_nowait()
to its only remaining caller.
buf_page_make_young_if_needed(): Define as an inline function.
Add the parameter buf_pool.
buf_page_peek_if_young(), buf_page_peek_if_too_old(): Add the
parameter buf_pool.
fil_space_validate_for_mtr_commit(): Remove a bogus comment
about background merge of the change buffer.
btr_cur_open_at_rnd_pos_func(), btr_cur_search_to_nth_level_func(),
btr_cur_open_at_index_side_func(): Use narrower data types and scopes.
ibuf_read_merge_pages(): Replaces buf_read_ibuf_merge_pages().
Merge the change buffer by invoking buf_page_get_gen().
2019-10-11 17:28:15 +03:00
|
|
|
--exec $MYSQLD_LAST_CMD --log-bin=master-bin --binlog-format=mixed --core-file --loose-debug-sync-timeout=300 --debug_dbug="+d,innobase_xa_fail" --tc-heuristic-recover=COMMIT
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= was in the XA prepared state
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= Found 1 prepared transactions!
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= \\[ERROR\\] Can\\'t init tc log
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= Please restart mysqld without --tc-heuristic-recover
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
|
2017-10-04 18:32:45 +02:00
|
|
|
--error 1
|
|
|
|
|
--exec $MYSQLD_LAST_CMD --log-bin=master-bin --binlog-format=mixed --core-file --loose-debug-sync-timeout=300 --tc-heuristic-recover=ROLLBACK
|
2017-08-29 10:52:52 +03:00
|
|
|
|
|
|
|
|
--let SEARCH_PATTERN= was in the XA prepared state
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= Found 1 prepared transactions!
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= \\[ERROR\\] Can\\'t init tc log
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= Please restart mysqld without --tc-heuristic-recover
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
|
|
|
|
|
--source include/start_mysqld.inc
|
|
|
|
|
|
|
|
|
|
--let SEARCH_PATTERN= was in the XA prepared state
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
--let SEARCH_PATTERN= Found 1 prepared transactions!
|
2017-08-31 09:28:59 +03:00
|
|
|
--source include/search_pattern_in_file.inc
|
2017-08-29 10:52:52 +03:00
|
|
|
|
|
|
|
|
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
|
|
|
|
|
SELECT * FROM t1;
|
|
|
|
|
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
|
|
|
SELECT * FROM t1;
|
|
|
|
|
#
|
|
|
|
|
# Cleanup
|
|
|
|
|
#
|
|
|
|
|
DROP TABLE t1;
|