mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-02-09 23:24:11 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/mysql-test/suite/rpl/r/rpl_xa_survive_disconnect.result
Marko Mäkelä 3cef4f8f0f MDEV-515 Reduce InnoDB undo logging for insert into empty table 
We implement an idea that was suggested by Michael 'Monty' Widenius
in October 2017: When InnoDB is inserting into an empty table or partition,
we can write a single undo log record TRX_UNDO_EMPTY, which will cause
ROLLBACK to clear the table.

For this to work, the insert into an empty table or partition must be
covered by an exclusive table lock that will be held until the transaction
has been committed or rolled back, or the INSERT operation has been
rolled back (and the table is empty again), in lock_table_x_unlock().

Clustered index records that are covered by the TRX_UNDO_EMPTY record
will carry DB_TRX_ID=0 and DB_ROLL_PTR=1<<55, and thus they cannot
be distinguished from what MDEV-12288 leaves behind after purging the
history of row-logged operations.

Concurrent non-locking reads must be adjusted: If the read view was
created before the INSERT into an empty table, then we must continue
to imagine that the table is empty, and not try to read any records.
If the read view was created after the INSERT was committed, then
all records must be visible normally. To implement this, we introduce
the field dict_table_t::bulk_trx_id.

This special handling only applies to the very first INSERT statement
of a transaction for the empty table or partition. If a subsequent
statement in the transaction is modifying the initially empty table again,
we must enable row-level undo logging, so that we will be able to
roll back to the start of the statement in case of an error (such as
duplicate key).

INSERT IGNORE will continue to use row-level logging and locking, because
implementing it would require the ability to roll back the latest row.
Since the undo log that we write only allows us to roll back the entire
statement, we cannot support INSERT IGNORE. We will introduce a
handler::extra() parameter HA_EXTRA_IGNORE_INSERT to indicate to storage
engines that INSERT IGNORE is being executed.

In many test cases, we add an extra record to the table, so that during
the 'interesting' part of the test, row-level locking and logging will
be used.

Replicas will continue to use row-level logging and locking until
MDEV-24622 has been addressed. Likewise, this optimization will be
disabled in Galera cluster until MDEV-24623 enables it.

dict_table_t::bulk_trx_id: The latest active or committed transaction
that initiated an insert into an empty table or partition.
Protected by exclusive table lock and a clustered index leaf page latch.

ins_node_t::bulk_insert: Whether bulk insert was initiated.

trx_t::mod_tables: Use C++11 style accessors (emplace instead of insert).
Unlike earlier, this collection will cover also temporary tables.

trx_mod_table_time_t: Add start_bulk_insert(), end_bulk_insert(),
is_bulk_insert(), was_bulk_insert().

trx_undo_report_row_operation(): Before accessing any undo log pages,
invoke trx->mod_tables.emplace() in order to determine whether undo
logging was disabled, or whether this is the first INSERT and we are
supposed to write a TRX_UNDO_EMPTY record.

row_ins_clust_index_entry_low(): If we are inserting into an empty
clustered index leaf page, set the ins_node_t::bulk_insert flag for
the subsequent trx_undo_report_row_operation() call.

lock_rec_insert_check_and_lock(), lock_prdt_insert_check_and_lock():
Remove the redundant parameter 'flags' that can be checked in the caller.

btr_cur_ins_lock_and_undo(): Simplify the logic. Correctly write
DB_TRX_ID,DB_ROLL_PTR after invoking trx_undo_report_row_operation().

trx_mark_sql_stat_end(), ha_innobase::extra(HA_EXTRA_IGNORE_INSERT),
ha_innobase::external_lock(): Invoke trx_t::end_bulk_insert() so that
the next statement will not be covered by table-level undo logging.

ReadView::changes_visible(trx_id_t) const: New accessor for the case
where the trx_id_t is not read from a potentially corrupted index page
but directly from the memory. In this case, we can skip a sanity check.

row_sel(), row_sel_try_search_shortcut(), row_search_mvcc():
row_sel_try_search_shortcut_for_mysql(),
row_merge_read_clustered_index(): Check dict_table_t::bulk_trx_id.

row_sel_clust_sees(): Replaces lock_clust_rec_cons_read_sees().

lock_sec_rec_cons_read_sees(): Replaced with lower-level code.

btr_root_page_init(): Refactored from btr_create().

dict_index_t::clear(), dict_table_t::clear(): Empty an index or table,
for the ROLLBACK of an INSERT operation.

ROW_T_EMPTY, ROW_OP_EMPTY: Note a concurrent ROLLBACK of an INSERT
into an empty table.

This is joint work with Thirunarayanan Balathandayuthapani,
who created a working prototype.
Thanks to Matthias Leich for extensive testing.
2021-01-25 18:41:27 +02:00

324 lines
12 KiB
Text
Raw Permalink Blame History

include/master-slave.inc
[connection master]
connection master;
call mtr.add_suppression("Found 2 prepared XA transactions");
CREATE VIEW v_processlist as SELECT * FROM performance_schema.threads where type = 'FOREGROUND';
CREATE DATABASE d1;
CREATE DATABASE d2;
CREATE TABLE d1.t (a INT) ENGINE=innodb;
CREATE TABLE d2.t (a INT) ENGINE=innodb;
INSERT INTO d1.t VALUES(100);
connect master_conn1, 127.0.0.1,root,,test,$MASTER_MYPORT,;
SET @@session.binlog_format= statement;
XA START '1-stmt';
INSERT INTO d1.t VALUES (1);
XA END '1-stmt';
XA PREPARE '1-stmt';
disconnect master_conn1;
connection master;
connect master_conn2, 127.0.0.1,root,,test,$MASTER_MYPORT,;
SET @@session.binlog_format= row;
XA START '1-row';
INSERT INTO d2.t VALUES (1);
XA END '1-row';
XA PREPARE '1-row';
disconnect master_conn2;
connection master;
XA START '2';
INSERT INTO d1.t VALUES (2);
XA END '2';
XA PREPARE '2';
XA COMMIT '2';
XA COMMIT '1-row';
XA COMMIT '1-stmt';
include/show_binlog_events.inc
Log_name Pos Event_type Server_id End_log_pos Info
master-bin.000001 # Gtid # # BEGIN GTID #-#-#
master-bin.000001 # Query # # use `mtr`; INSERT INTO test_suppressions (pattern) VALUES ( NAME_CONST('pattern',_latin1'Found 2 prepared XA transactions' COLLATE 'latin1_swedish_ci'))
master-bin.000001 # Query # # COMMIT
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # use `test`; CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost` SQL SECURITY DEFINER VIEW `v_processlist` AS SELECT * FROM performance_schema.threads where type = 'FOREGROUND'
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # CREATE DATABASE d1
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # CREATE DATABASE d2
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # use `test`; CREATE TABLE d1.t (a INT) ENGINE=innodb
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # use `test`; CREATE TABLE d2.t (a INT) ENGINE=innodb
master-bin.000001 # Gtid # # BEGIN GTID #-#-#
master-bin.000001 # Query # # use `test`; INSERT INTO d1.t VALUES(100)
master-bin.000001 # Xid # # COMMIT /* XID */
master-bin.000001 # Gtid # # XA START X'312d73746d74',X'',1 GTID #-#-#
master-bin.000001 # Query # # use `test`; INSERT INTO d1.t VALUES (1)
master-bin.000001 # Query # # XA END X'312d73746d74',X'',1
master-bin.000001 # XA_prepare # # XA PREPARE X'312d73746d74',X'',1
master-bin.000001 # Gtid # # XA START X'312d726f77',X'',1 GTID #-#-#
master-bin.000001 # Annotate_rows # # INSERT INTO d2.t VALUES (1)
master-bin.000001 # Table_map # # table_id: # (d2.t)
master-bin.000001 # Write_rows_v1 # # table_id: # flags: STMT_END_F
master-bin.000001 # Query # # XA END X'312d726f77',X'',1
master-bin.000001 # XA_prepare # # XA PREPARE X'312d726f77',X'',1
master-bin.000001 # Gtid # # XA START X'32',X'',1 GTID #-#-#
master-bin.000001 # Query # # use `test`; INSERT INTO d1.t VALUES (2)
master-bin.000001 # Query # # XA END X'32',X'',1
master-bin.000001 # XA_prepare # # XA PREPARE X'32',X'',1
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # XA COMMIT X'32',X'',1
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # XA COMMIT X'312d726f77',X'',1
master-bin.000001 # Gtid # # GTID #-#-#
master-bin.000001 # Query # # XA COMMIT X'312d73746d74',X'',1
include/sync_slave_sql_with_master.inc
include/stop_slave.inc
connection master;
connect master2, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master2;
SET @@session.binlog_format= statement;
XA START '3-stmt';
INSERT INTO d1.t VALUES (3);
XA END '3-stmt';
XA PREPARE '3-stmt';
disconnect master2;
connect master2, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master2;
SET @@session.binlog_format= row;
XA START '3-row';
INSERT INTO d2.t VALUES (4);
XA END '3-row';
XA PREPARE '3-row';
disconnect master2;
connection master;
connect master2, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master2;
XA START '4';
SELECT * FROM d1.t;
a
100
1
2
XA END '4';
XA PREPARE '4';
disconnect master2;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_10';
INSERT INTO d1.t VALUES (10);
INSERT INTO d2.t VALUES (10);
XA END 'bulk_trx_10';
XA PREPARE 'bulk_trx_10';
disconnect master_bulk_conn10;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_9';
INSERT INTO d1.t VALUES (9);
INSERT INTO d2.t VALUES (9);
XA END 'bulk_trx_9';
XA PREPARE 'bulk_trx_9';
disconnect master_bulk_conn9;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_8';
INSERT INTO d1.t VALUES (8);
INSERT INTO d2.t VALUES (8);
XA END 'bulk_trx_8';
XA PREPARE 'bulk_trx_8';
disconnect master_bulk_conn8;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_7';
INSERT INTO d1.t VALUES (7);
INSERT INTO d2.t VALUES (7);
XA END 'bulk_trx_7';
XA PREPARE 'bulk_trx_7';
disconnect master_bulk_conn7;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_6';
INSERT INTO d1.t VALUES (6);
INSERT INTO d2.t VALUES (6);
XA END 'bulk_trx_6';
XA PREPARE 'bulk_trx_6';
disconnect master_bulk_conn6;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_5';
INSERT INTO d1.t VALUES (5);
INSERT INTO d2.t VALUES (5);
XA END 'bulk_trx_5';
XA PREPARE 'bulk_trx_5';
disconnect master_bulk_conn5;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_4';
INSERT INTO d1.t VALUES (4);
INSERT INTO d2.t VALUES (4);
XA END 'bulk_trx_4';
XA PREPARE 'bulk_trx_4';
disconnect master_bulk_conn4;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_3';
INSERT INTO d1.t VALUES (3);
INSERT INTO d2.t VALUES (3);
XA END 'bulk_trx_3';
XA PREPARE 'bulk_trx_3';
disconnect master_bulk_conn3;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_2';
INSERT INTO d1.t VALUES (2);
INSERT INTO d2.t VALUES (2);
XA END 'bulk_trx_2';
XA PREPARE 'bulk_trx_2';
disconnect master_bulk_conn2;
connection master;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START 'bulk_trx_1';
INSERT INTO d1.t VALUES (1);
INSERT INTO d2.t VALUES (1);
XA END 'bulk_trx_1';
XA PREPARE 'bulk_trx_1';
disconnect master_bulk_conn1;
connection master;
connection slave;
include/start_slave.inc
connection master;
include/sync_slave_sql_with_master.inc
include/stop_slave.inc
connection master;
XA COMMIT 'bulk_trx_10';
XA ROLLBACK 'bulk_trx_9';
XA COMMIT 'bulk_trx_8';
XA ROLLBACK 'bulk_trx_7';
XA COMMIT 'bulk_trx_6';
XA ROLLBACK 'bulk_trx_5';
XA COMMIT 'bulk_trx_4';
XA ROLLBACK 'bulk_trx_3';
XA COMMIT 'bulk_trx_2';
XA ROLLBACK 'bulk_trx_1';
include/rpl_restart_server.inc [server_number=1]
connection slave;
include/start_slave.inc
connection master;
*** '3-stmt','3-row' xa-transactions must be in the list ***
XA RECOVER;
formatID gtrid_length bqual_length data
1 6 0 3-stmt
1 5 0 3-row
XA COMMIT '3-stmt';
XA ROLLBACK '3-row';
include/sync_slave_sql_with_master.inc
connection master;
connect master_conn2, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START '0123456789012345678901234567890123456789012345678901234567890124','0123456789012345678901234567890123456789012345678901234567890124',2147483647;
INSERT INTO d1.t VALUES (64);
XA END '0123456789012345678901234567890123456789012345678901234567890124','0123456789012345678901234567890123456789012345678901234567890124',2147483647;
XA PREPARE '0123456789012345678901234567890123456789012345678901234567890124','0123456789012345678901234567890123456789012345678901234567890124',2147483647;
disconnect master_conn2;
connection master;
connect master_conn3, 127.0.0.1,root,,test,$MASTER_MYPORT,;
XA START X'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF',X'00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',0;
INSERT INTO d1.t VALUES (0);
XA END X'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF',X'00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',0;
XA PREPARE X'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF',X'00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',0;
disconnect master_conn3;
connection master;
disconnect master_conn4;
connection master;
XA COMMIT '0123456789012345678901234567890123456789012345678901234567890124','0123456789012345678901234567890123456789012345678901234567890124',2147483647;
XA COMMIT X'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF',X'00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',0;
XA COMMIT 'RANDOM XID'
include/sync_slave_sql_with_master.inc
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn10;
XA START 'one_phase_10';
INSERT INTO d1.t VALUES (10);
INSERT INTO d2.t VALUES (10);
XA END 'one_phase_10';
XA COMMIT 'one_phase_10' ONE PHASE;
disconnect master_bulk_conn10;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn9;
XA START 'one_phase_9';
INSERT INTO d1.t VALUES (9);
INSERT INTO d2.t VALUES (9);
XA END 'one_phase_9';
XA COMMIT 'one_phase_9' ONE PHASE;
disconnect master_bulk_conn9;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn8;
XA START 'one_phase_8';
INSERT INTO d1.t VALUES (8);
INSERT INTO d2.t VALUES (8);
XA END 'one_phase_8';
XA COMMIT 'one_phase_8' ONE PHASE;
disconnect master_bulk_conn8;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn7;
XA START 'one_phase_7';
INSERT INTO d1.t VALUES (7);
INSERT INTO d2.t VALUES (7);
XA END 'one_phase_7';
XA COMMIT 'one_phase_7' ONE PHASE;
disconnect master_bulk_conn7;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn6;
XA START 'one_phase_6';
INSERT INTO d1.t VALUES (6);
INSERT INTO d2.t VALUES (6);
XA END 'one_phase_6';
XA COMMIT 'one_phase_6' ONE PHASE;
disconnect master_bulk_conn6;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn5;
XA START 'one_phase_5';
INSERT INTO d1.t VALUES (5);
INSERT INTO d2.t VALUES (5);
XA END 'one_phase_5';
XA COMMIT 'one_phase_5' ONE PHASE;
disconnect master_bulk_conn5;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn4;
XA START 'one_phase_4';
INSERT INTO d1.t VALUES (4);
INSERT INTO d2.t VALUES (4);
XA END 'one_phase_4';
XA COMMIT 'one_phase_4' ONE PHASE;
disconnect master_bulk_conn4;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn3;
XA START 'one_phase_3';
INSERT INTO d1.t VALUES (3);
INSERT INTO d2.t VALUES (3);
XA END 'one_phase_3';
XA COMMIT 'one_phase_3' ONE PHASE;
disconnect master_bulk_conn3;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn2;
XA START 'one_phase_2';
INSERT INTO d1.t VALUES (2);
INSERT INTO d2.t VALUES (2);
XA END 'one_phase_2';
XA COMMIT 'one_phase_2' ONE PHASE;
disconnect master_bulk_conn2;
connect master_bulk_conn$i, 127.0.0.1,root,,test,$MASTER_MYPORT,;
connection master_bulk_conn1;
XA START 'one_phase_1';
INSERT INTO d1.t VALUES (1);
INSERT INTO d2.t VALUES (1);
XA END 'one_phase_1';
XA COMMIT 'one_phase_1' ONE PHASE;
disconnect master_bulk_conn1;
connection master;
include/sync_slave_sql_with_master.inc
include/diff_tables.inc [master:d1.t, slave:d1.t]
include/diff_tables.inc [master:d2.t, slave:d2.t]
connection master;
DELETE FROM d1.t;
DELETE FROM d2.t;
DROP TABLE d1.t, d2.t;
DROP DATABASE d1;
DROP DATABASE d2;
DROP VIEW v_processlist;
include/sync_slave_sql_with_master.inc
include/rpl_end.inc
Reference in a new issue View git blame Copy permalink