mirror of
https://github.com/MariaDB/server.git
synced 2025-01-17 20:42:30 +01:00
79e3ee00fa
New Feature:
===========
This commit extends the mariadb-binlog capabilities to allow events
to be filtered by GTID ranges. More specifically, the
--start-position and --stop-position arguments have been extended to
accept values formatted as a list of GTID positions, e.g.
--start-position=0-1-0,1-2-55. The following specific capabilities
are addressed:
1) GTIDs can be used to filter results on local binlog files
2) GTIDs can be used to filter results from remote servers
3) Implemented --gtid-strict-mode that ensures the GTID event
stream in each domain is monotonically increasing
4) Added new level of verbosity in mysqlbinlog -vvv to print
additional diagnostic information/warnings about invalid GTID
states
5) For a given GTID range, its start and stop position parameters
aim to mimic the behaviors of
CHANGE MASTER TO MASTER_USE_GTID=slave_pos and
START SLAVE UNTIL master_gtid_pos=<GTID>, respectively. In
particular, the start-position list expresses a gtid state of
the server, similarly to how @@global.gtid_slave_pos expresses
the gtid state of a slave server when connecting to a master
with MASTER_USE_GTID=slave_pos.
The GTID start-position list is exclusive and the
stop-position list is inclusive. This allows users to receive
events strictly after those that they already have, and is
useful in cases of point in (logical) time recovery including
1) events were received out of order and should be re-sent, or
2) specifying the gtid state of a slave to get events newer
than their current state. If a seq_no is 0 for start-position,
it means to include the entirety of the domain. If a seq_no is
0 for stop-position, it means to exclude all events from that
domain. The GTIDs provided in a start position argument must
match with the GTID state of the first processed log (i.e.
those listed in the Gtid_list event). If a stop position is
provided, the events that are output are limited to only those
with domain ids listed in the argument. When specifying
combinations of start and stop positions, the following
behaviors are expected:
[--start-position without --stop-position]: Events that have domain
ids in the start position are output if their seq_no occurs after
the respective start position. Events with domain ids that are
unspecified in the start position list are also output. Note that if
the Gtid_list event of the first binary log is populated (i.e.
non-empty), each domain in the Gtid_list must be present in the
start-position list with a seq_no at or after the listed value.
This behavior mimics how a slave only processes events after the
state provided by @@global.gtid_slave_pos when connecting to a
master with CHANGE MASTER TO MASTER_USE_GTID=slave_pos.
[--stop-position without --start-position]: Output is limited to
only events with both 1) domain ids that are present in the given
stop position list and 2) seq_nos that are less than or equal to
their respective stop GTID. Once all GTIDs in the stop position
list have been processed, the program will stop processing log
files. This behavior mimics how
START SLAVE UNTIL master_gtid_pos=<G>
has a slave only process events with domain ids present in G with
their seq_nos at or before the respective gtid.
[--start-position and --stop-position]: Output consists of the
intersection between the events permitted by both the start and stop
position rules. More concretely, the output can be defined by a
union of the following rules:
1. For domains which exist in both the start and stop position
lists, the events which exist in-between these positions
(exclusive start, inclusive stop) are output
2. For all other events, the rules of
[--stop-position without --start-position] are followed
This is due to the implicit filtering within each individual rule.
Even though the start position rule always includes events from
unspecified domains, the stop position rule takes precedence because
it always excludes events from unspecified domains. In other words,
events which the start position rule would have included would then
always be excluded by the stop position rule.
[neither --start-position nor --stop-position]: Events are not
omitted based on GTID positioning; however, --gtid-strict-mode and
-vvv can still analyze gtid correctness for warning and error
reporting.
[repeated specification of --start-position or --stop-position]:
Subsequent specifications of start and stop positions completely
override previous ones. E.g., if invoked as
mysqlbinlog --start-position=<G1> --start-position=<G2> ...
All GTIDs specified in G1 are ignored and only those specified in G2
are used for the start position.
A few additional notes:
1) this commit squashes together the commits:
f4319661120e-78a9d49907ba
2) Changed rpl.rpl_blackhole_row_annotate test because it has
out of order GTIDs in its binlog, so I added
--skip-gtid-strict-mode
3) After all binlog events have been written, the session server
id and domain id are reset to their values in the global state
Reviewed By:
===========
Andrei Elkin: <andrei.elkin@mariadb.com>
|
||
|---|---|---|
| .. | ||
| backup.result | ||
| binlog_1pc.result | ||
| binlog_admin_cmd_kill.result | ||
| binlog_base64_flag.result | ||
| binlog_bug23533.result | ||
| binlog_bug36391.result | ||
| binlog_checkpoint.result | ||
| binlog_checkpoint_flush_logs.result | ||
| binlog_checksum.result | ||
| binlog_commit_wait.result | ||
| binlog_database.result | ||
| binlog_delete_and_flush_index.result | ||
| binlog_dmls_on_tmp_tables_readonly.result | ||
| binlog_drop_if_exists.result | ||
| binlog_empty_xa_prepared.result | ||
| binlog_expire_logs_seconds.result | ||
| binlog_expire_warnings.result | ||
| binlog_flush_binlogs_delete_domain.result | ||
| binlog_format_switch_in_tmp_table.result | ||
| binlog_grant.result | ||
| binlog_gtid_delete_domain_debug.result | ||
| binlog_implicit_commit.result | ||
| binlog_incident.result | ||
| binlog_index.result | ||
| binlog_innodb.result | ||
| binlog_innodb_row.result | ||
| binlog_innodb_stm.result | ||
| binlog_invalid_read_in_rotate.result | ||
| binlog_ioerr.result | ||
| binlog_killed.result | ||
| binlog_killed_simulate.result | ||
| binlog_max_binlog_stmt_cache_size.result | ||
| binlog_max_extension.result | ||
| binlog_mdev342.result | ||
| binlog_mdev717.result | ||
| binlog_mixed.result | ||
| binlog_mixed_cache_stat.result | ||
| binlog_mixed_load_data.result | ||
| binlog_mysqlbinlog-cp932.result | ||
| binlog_mysqlbinlog2.result | ||
| binlog_mysqlbinlog_base64.result | ||
| binlog_mysqlbinlog_gtid_strict_mode.result | ||
| binlog_mysqlbinlog_gtid_window.result | ||
| binlog_mysqlbinlog_row.result | ||
| binlog_mysqlbinlog_row_frag.result | ||
| binlog_mysqlbinlog_row_innodb.result | ||
| binlog_mysqlbinlog_row_myisam.result | ||
| binlog_mysqlbinlog_row_trans.result | ||
| binlog_mysqlbinlog_stop_never.result | ||
| binlog_mysqlbinlog_suppress_O_TMPFILE.result | ||
| binlog_no_uniqfile_crash.result | ||
| binlog_old_versions.result | ||
| binlog_parallel_replication_marks_row.result | ||
| binlog_parallel_replication_marks_stm_mix.result | ||
| binlog_query_filter_rules.result | ||
| binlog_recover_checksum_error.result | ||
| binlog_rotate_perf.result | ||
| binlog_row_annotate.result | ||
| binlog_row_binlog.result | ||
| binlog_row_cache_stat.result | ||
| binlog_row_ctype_cp932.result | ||
| binlog_row_ctype_ucs.result | ||
| binlog_row_drop_tbl.result | ||
| binlog_row_drop_tmp_tbl.result | ||
| binlog_row_insert_select.result | ||
| binlog_row_mix_innodb_myisam.result | ||
| binlog_row_mysqlbinlog_db_filter.result | ||
| binlog_row_mysqlbinlog_options.result | ||
| binlog_row_mysqlbinlog_verbose.result | ||
| binlog_server_id.result | ||
| binlog_sf.result | ||
| binlog_show_binlog_event_random_pos.result | ||
| binlog_spurious_ddl_errors.result | ||
| binlog_sql_mode.result | ||
| binlog_start_comment.result | ||
| binlog_statement_insert_delayed.result | ||
| binlog_stm_binlog.result | ||
| binlog_stm_blackhole.result | ||
| binlog_stm_cache_stat.result | ||
| binlog_stm_ctype_cp932.result | ||
| binlog_stm_ctype_ucs.result | ||
| binlog_stm_datetime_ranges_mdev15289.result | ||
| binlog_stm_do_db.result | ||
| binlog_stm_drop_tbl.result | ||
| binlog_stm_drop_tmp_tbl.result | ||
| binlog_stm_insert_select.result | ||
| binlog_stm_mix_innodb_myisam.result | ||
| binlog_stm_ps.result | ||
| binlog_stm_row.result | ||
| binlog_stm_sp.result | ||
| binlog_stm_sp_type_row.result | ||
| binlog_stm_unsafe_warning.result | ||
| binlog_stm_user_variables.result | ||
| binlog_switch_inside_trans.result | ||
| binlog_table_map_optional_metadata.result | ||
| binlog_table_map_optional_metadata_binary.result | ||
| binlog_table_map_optional_metadata_ucs2.result | ||
| binlog_table_map_optional_metadata_utf32.result | ||
| binlog_tmp_table.result | ||
| binlog_tmp_table_row.result | ||
| binlog_trigger.result | ||
| binlog_truncate_active_log.result | ||
| binlog_truncate_innodb.result | ||
| binlog_truncate_kill.result | ||
| binlog_truncate_multi_engine.result | ||
| binlog_truncate_multi_log.result | ||
| binlog_truncate_multi_log_unsafe.result | ||
| binlog_truncate_myisam.result | ||
| binlog_unsafe.result | ||
| binlog_variables_log_bin.result | ||
| binlog_variables_log_bin_index.result | ||
| binlog_variables_relay_log.result | ||
| binlog_variables_relay_log_index.result | ||
| binlog_write_error.result | ||
| binlog_xa_checkpoint.result | ||
| binlog_xa_prepared.result | ||
| binlog_xa_prepared_disconnect.result | ||
| binlog_xa_recover.result | ||
| flashback-largebinlog.result | ||
| flashback.result | ||
| foreign_key.result | ||
| load_data_stm_view.result | ||
| mysqladmin.result | ||
| read_only.result | ||
| read_only_statement.result | ||
| show_concurrent_rotate.result | ||
| temptable_uservar_disconnect-7938.result | ||