mariadb/libmysqld
Kristian Nielsen d039346a7a MDEV-4991: GTID binlog indexing
Improve the performance of slave connect using B+-Tree indexes on each binlog
file. The index allows fast lookup of a GTID position to the corresponding
offset in the binlog file, as well as lookup of a position to find the
corresponding GTID position.

This eliminates a costly sequential scan of the starting binlog file
to find the GTID starting position when a slave connects. This is
especially costly if the binlog file is not cached in memory (IO
cost), or if it is encrypted or a lot of slaves connect simultaneously
(CPU cost).

The size of the index files is generally less than 1% of the binlog data, so
not expected to be an issue.

Most of the work writing the index is done as a background task, in
the binlog background thread. This minimises the performance impact on
transaction commit. A simple global mutex is used to protect index
reads and (background) index writes; this is fine as slave connect is
a relatively infrequent operation.

Here are the user-visible options and status variables. The feature is on by
default and is expected to need no tuning or configuration for most users.

binlog_gtid_index
  On by default. Can be used to disable the indexes for testing purposes.

binlog_gtid_index_page_size (default 4096)
  Page size to use for the binlog GTID index. This is the size of the nodes
  in the B+-tree used internally in the index. A very small page-size (64 is
  the minimum) will be less efficient, but can be used to stress the
  BTree-code during testing.

binlog_gtid_index_span_min (default 65536)
  Control sparseness of the binlog GTID index. If set to N, at most one
  index record will be added for every N bytes of binlog file written.
  This can be used to reduce the number of records in the index, at
  the cost only of having to scan a few more events in the binlog file
  before finding the target position

Two status variables are available to monitor the use of the GTID indexes:

  Binlog_gtid_index_hit
  Binlog_gtid_index_miss

The "hit" status increments for each successful lookup in a GTID index.
The "miss" increments when a lookup is not possible. This indicates that the
index file is missing (eg. binlog written by old server version
without GTID index support), or corrupt.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-01-27 12:09:54 +01:00
..
examples MDEV-21303 Make executables MariaDB named 2020-03-21 20:20:29 +01:00
client_settings.h Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
CMakeLists.txt MDEV-4991: GTID binlog indexing 2024-01-27 12:09:54 +01:00
emb_qcache.cc MDEV-27595 Backport SQL service, introduced by MDEV-19275. 2023-11-05 23:35:32 +04:00
emb_qcache.h MDEV-19275 Provide SQL service to plugins. 2020-08-14 21:04:25 +04:00
embedded_priv.h fix query cache in embedded 2022-01-30 13:56:22 +01:00
lib_sql.cc Merge branch '10.6' into 10.11 2023-12-18 11:19:04 +01:00
libmysql.c MDEV-27595 Backport SQL service, introduced by MDEV-19275. 2023-11-05 23:35:31 +04:00
libmysqld.c Merge branch '10.5' into 10.6 2023-08-01 15:08:52 +02:00
libmysqld.def merge with 5.3 2011-10-19 21:45:18 +02:00
libmysqld.rc
resource.h