mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-16 12:02:42 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/extra/mariabackup/xtrabackup.h

199 lines
5.7 KiB
C
Raw Normal View History

MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
/******************************************************
Copyright (c) 2011-2015 Percona LLC and/or its affiliates.
Declarations for xtrabackup.cc
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Merge branch '5.5' into 10.1
2019-05-11 21:19:05 +02:00
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
*******************************************************/
#ifndef XB_XTRABACKUP_H
#define XB_XTRABACKUP_H
#include <my_getopt.h>
#include "datasink.h"
#include "xbstream.h"
#include "changed_page_bitmap.h"
MDEV-12548 Initial implementation of Mariabackup for MariaDB 10.2 InnoDB I/O and buffer pool interfaces and the redo log format have been changed between MariaDB 10.1 and 10.2, and the backup code has to be adjusted accordingly. The code has been simplified, and many memory leaks have been fixed. Instead of the file name xtrabackup_logfile, the file name ib_logfile0 is being used for the copy of the redo log. Unnecessary InnoDB startup and shutdown and some unnecessary threads have been removed. Some help was provided by Vladislav Vaintroub. Parameters have been cleaned up and aligned with those of MariaDB 10.2. The --dbug option has been added, so that in debug builds, --dbug=d,ib_log can be specified to enable diagnostic messages for processing redo log entries. By default, innodb_doublewrite=OFF, so that --prepare works faster. If more crash-safety for --prepare is needed, double buffering can be enabled. The parameter innodb_log_checksums=OFF can be used to ignore redo log checksums in --backup. Some messages have been cleaned up. Unless --export is specified, Mariabackup will not deal with undo log. The InnoDB mini-transaction redo log is not only about user-level transactions; it is actually about mini-transactions. To avoid confusion, call it the redo log, not transaction log. We disable any undo log processing in --prepare. Because MariaDB 10.2 supports indexed virtual columns, the undo log processing would need to be able to evaluate virtual column expressions. To reduce the amount of code dependencies, we will not process any undo log in prepare. This means that the --export option must be disabled for now. This also means that the following options are redundant and have been removed: xtrabackup --apply-log-only innobackupex --redo-only In addition to disabling any undo log processing, we will disable any further changes to data pages during --prepare, including the change buffer merge. This means that restoring incremental backups should reliably work even when change buffering is being used on the server. Because of this, preparing a backup will not generate any further redo log, and the redo log file can be safely deleted. (If the --export option is enabled in the future, it must generate redo log when processing undo logs and buffered changes.) In --prepare, we cannot easily know if a partial backup was used, especially when restoring a series of incremental backups. So, we simply warn about any missing files, and ignore the redo log for them. FIXME: Enable the --export option. FIXME: Improve the handling of the MLOG_INDEX_LOAD record, and write a test that initiates a backup while an ALGORITHM=INPLACE operation is creating indexes or rebuilding a table. An error should be detected when preparing the backup. FIXME: In --incremental --prepare, xtrabackup_apply_delta() should ensure that if FSP_SIZE is modified, the file size will be adjusted accordingly.
2017-06-30 09:49:37 +02:00
struct xb_delta_info_t
{
MDEV-18493 Remove page_size_t MySQL 5.7 introduced the class page_size_t and increased the size of buffer pool page descriptors by introducing this object to them. Maybe the intention of this exercise was to prepare for a future where the buffer pool could accommodate multiple page sizes. But that future never arrived, not even in MySQL 8.0. It is much easier to manage a pool of a single page size, and typically all storage devices of an InnoDB instance benefit from using the same page size. Let us remove page_size_t from MariaDB Server. This will make it easier to remove support for ROW_FORMAT=COMPRESSED (or make it a compile-time option) in the future, just by removing various occurrences of zip_size.
2019-02-06 18:50:11 +01:00
xb_delta_info_t(ulint page_size, ulint zip_size, ulint space_id)
: page_size(page_size), zip_size(zip_size), space_id(space_id) {}
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
MDEV-18493 Remove page_size_t MySQL 5.7 introduced the class page_size_t and increased the size of buffer pool page descriptors by introducing this object to them. Maybe the intention of this exercise was to prepare for a future where the buffer pool could accommodate multiple page sizes. But that future never arrived, not even in MySQL 8.0. It is much easier to manage a pool of a single page size, and typically all storage devices of an InnoDB instance benefit from using the same page size. Let us remove page_size_t from MariaDB Server. This will make it easier to remove support for ROW_FORMAT=COMPRESSED (or make it a compile-time option) in the future, just by removing various occurrences of zip_size.
2019-02-06 18:50:11 +01:00
ulint page_size;
ulint zip_size;
ulint space_id;
MDEV-12548 Initial implementation of Mariabackup for MariaDB 10.2 InnoDB I/O and buffer pool interfaces and the redo log format have been changed between MariaDB 10.1 and 10.2, and the backup code has to be adjusted accordingly. The code has been simplified, and many memory leaks have been fixed. Instead of the file name xtrabackup_logfile, the file name ib_logfile0 is being used for the copy of the redo log. Unnecessary InnoDB startup and shutdown and some unnecessary threads have been removed. Some help was provided by Vladislav Vaintroub. Parameters have been cleaned up and aligned with those of MariaDB 10.2. The --dbug option has been added, so that in debug builds, --dbug=d,ib_log can be specified to enable diagnostic messages for processing redo log entries. By default, innodb_doublewrite=OFF, so that --prepare works faster. If more crash-safety for --prepare is needed, double buffering can be enabled. The parameter innodb_log_checksums=OFF can be used to ignore redo log checksums in --backup. Some messages have been cleaned up. Unless --export is specified, Mariabackup will not deal with undo log. The InnoDB mini-transaction redo log is not only about user-level transactions; it is actually about mini-transactions. To avoid confusion, call it the redo log, not transaction log. We disable any undo log processing in --prepare. Because MariaDB 10.2 supports indexed virtual columns, the undo log processing would need to be able to evaluate virtual column expressions. To reduce the amount of code dependencies, we will not process any undo log in prepare. This means that the --export option must be disabled for now. This also means that the following options are redundant and have been removed: xtrabackup --apply-log-only innobackupex --redo-only In addition to disabling any undo log processing, we will disable any further changes to data pages during --prepare, including the change buffer merge. This means that restoring incremental backups should reliably work even when change buffering is being used on the server. Because of this, preparing a backup will not generate any further redo log, and the redo log file can be safely deleted. (If the --export option is enabled in the future, it must generate redo log when processing undo logs and buffered changes.) In --prepare, we cannot easily know if a partial backup was used, especially when restoring a series of incremental backups. So, we simply warn about any missing files, and ignore the redo log for them. FIXME: Enable the --export option. FIXME: Improve the handling of the MLOG_INDEX_LOAD record, and write a test that initiates a backup while an ALGORITHM=INPLACE operation is creating indexes or rebuilding a table. An error should be detected when preparing the backup. FIXME: In --incremental --prepare, xtrabackup_apply_delta() should ensure that if FSP_SIZE is modified, the file size will be adjusted accordingly.
2017-06-30 09:49:37 +02:00
};
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
/* value of the --incremental option */
extern lsn_t incremental_lsn;
extern char *xtrabackup_target_dir;
extern char *xtrabackup_incremental_dir;
extern char *xtrabackup_incremental_basedir;
extern char *innobase_data_home_dir;
extern char *innobase_buffer_pool_filename;
MDEV-13608 : set client plugin directory with mysql_options() if plugin_dir is specified. Also, allow to specify protocol (e.g pipe)
2017-08-21 19:16:12 +02:00
extern char *xb_plugin_dir;
MDEV-13122 Backup myrocksdb with mariabackup.
2018-06-07 16:13:54 +02:00
extern char *xb_rocksdb_datadir;
extern my_bool xb_backup_rocksdb;
MDEV-13608 : set client plugin directory with mysql_options() if plugin_dir is specified. Also, allow to specify protocol (e.g pipe)
2017-08-21 19:16:12 +02:00
extern uint opt_protocol;
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
extern ds_ctxt_t *ds_meta;
extern ds_ctxt_t *ds_data;
/* The last checkpoint LSN at the backup startup time */
extern lsn_t checkpoint_lsn_start;
extern xb_page_bitmap *changed_page_bitmap;
extern char *xtrabackup_incremental;
extern my_bool xtrabackup_incremental_force_scan;
extern lsn_t metadata_to_lsn;
extern xb_stream_fmt_t xtrabackup_stream_fmt;
extern ibool xtrabackup_stream;
extern char *xtrabackup_tables;
extern char *xtrabackup_tables_file;
extern char *xtrabackup_databases;
extern char *xtrabackup_databases_file;
Xtrabackup 2.3.8
2017-04-19 15:09:03 +02:00
extern char *xtrabackup_tables_exclude;
extern char *xtrabackup_databases_exclude;
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
MDEV-18212 mariabackup: Make output format uniform whenever possible
2019-01-14 22:28:23 +01:00
extern uint xtrabackup_compress;
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
extern my_bool xtrabackup_backup;
extern my_bool xtrabackup_prepare;
extern my_bool xtrabackup_copy_back;
extern my_bool xtrabackup_move_back;
extern my_bool xtrabackup_decrypt_decompress;
extern char *innobase_data_file_path;
extern longlong innobase_page_size;
extern int xtrabackup_parallel;
extern my_bool xb_close_files;
extern const char *xtrabackup_compress_alg;
MDEV-18212 mariabackup: Make output format uniform whenever possible
2019-01-14 22:28:23 +01:00
extern uint xtrabackup_compress_threads;
extern ulonglong xtrabackup_compress_chunk_size;
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
extern my_bool xtrabackup_export;
extern char *xtrabackup_extra_lsndir;
extern ulint xtrabackup_log_copy_interval;
extern char *xtrabackup_stream_str;
extern long xtrabackup_throttle;
extern longlong xtrabackup_use_memory;
extern my_bool opt_galera_info;
extern my_bool opt_slave_info;
extern my_bool opt_no_lock;
extern my_bool opt_safe_slave_backup;
extern my_bool opt_rsync;
extern my_bool opt_force_non_empty_dirs;
extern my_bool opt_noversioncheck;
extern my_bool opt_no_backup_locks;
extern my_bool opt_decompress;
extern my_bool opt_remove_original;
Cleanup recent mariabackup validation patches. - Refactor code to isolate page validation in page_is_corrupted() function. - Introduce --extended-validation parameter(default OFF) for mariabackup --backup to enable decryption of encrypted uncompressed pages during backup. - mariabackup would still always check checksum on encrypted data, it is needed to detect partially written pages.
2018-12-20 14:31:18 +01:00
extern my_bool opt_extended_validation;
MDEV-18356 Renamed backup-encrypted option introduced in 7158edcba3af3766e9329f9927ce4adfd2a40bf8 Rename it because it caused parser warning whenever --backup was used.
2019-01-23 13:00:12 +01:00
extern my_bool opt_encrypted_backup;
MDEV-15636 mariabackup --lock-ddl-per-table hangs if ALTER table is running concurrently. There is a deadlock between C1 mariabackup's connection that holds MDL locks C2 Online ALTER TABLE that wants to have MDL exclusively and tries to upgrade its mdl lock. C3 another mariabackup's connection that does FLUSH TABLES (or FTWRL) C3 waits waits for C2, which waits for C1, which waits for C3, thus the deadlock. MDL locks cannot be released until FLUSH succeeds, because otherwise it would allow ALTER to sneak in, causing backup to abort and breaking lock-ddl-per-table's promise. The fix here workarounds the deadlock, by killing connections in "Waiting for metadata lock" status (i.e ALTER). This killing continues until FTWRL succeeds. Killing connections is skipped in case --no-locks parameter was passed to backup, because there won't be a FLUSH. For the reference,in Percona's xtrabackup --lock-ddl-per-connection silently implies --no-lock ie FLUSH is always skipped there. A rather large part of fix is introducing DBUG capability to start a query the new connection at the right moment of backup compensating somewhat for mariabackup' lack of send_query or DBUG_SYNC.
2018-03-30 00:13:01 +02:00
extern my_bool opt_lock_ddl_per_table;
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
extern char *opt_incremental_history_name;
extern char *opt_incremental_history_uuid;
extern char *opt_user;
extern char *opt_password;
extern char *opt_host;
extern char *opt_defaults_group;
extern char *opt_socket;
extern uint opt_port;
extern char *opt_log_bin;
extern const char *query_type_names[];
enum query_type_t {QUERY_TYPE_ALL, QUERY_TYPE_UPDATE,
QUERY_TYPE_SELECT};
extern TYPELIB query_type_typelib;
extern ulong opt_lock_wait_query_type;
extern ulong opt_kill_long_query_type;
extern uint opt_kill_long_queries_timeout;
extern uint opt_lock_wait_timeout;
extern uint opt_lock_wait_threshold;
extern uint opt_debug_sleep_before_unlock;
extern uint opt_safe_slave_backup_timeout;
extern const char *opt_history;
MDEV-18917 Don't create xtrabackup_binlog_pos_innodb with Mariabackup
2019-03-14 16:59:27 +01:00
enum binlog_info_enum { BINLOG_INFO_OFF, BINLOG_INFO_ON,
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
BINLOG_INFO_AUTO};
extern ulong opt_binlog_info;
void xtrabackup_io_throttling(void);
my_bool xb_write_delta_metadata(const char *filename,
const xb_delta_info_t *info);
/************************************************************************
Checks if a table specified as a name in the form "database/name" (InnoDB 5.6)
or "./database/name.ibd" (InnoDB 5.5-) should be skipped from backup based on
the --tables or --tables-file options.
@return TRUE if the table should be skipped. */
my_bool
check_if_skip_table(
/******************/
const char* name); /*!< in: path to the table */
Xtrabackup 2.3.8
2017-04-19 15:09:03 +02:00
/************************************************************************
Checks if a database specified by path should be skipped from backup based on
the --databases, --databases_file or --databases_exclude options.
@return TRUE if the table should be skipped. */
my_bool
check_if_skip_database_by_path(
const char* path /*!< in: path to the db directory. */
);
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
/************************************************************************
Check if parameter is set in defaults file or via command line argument
@return true if parameter is set. */
bool
check_if_param_set(const char *param);
MDEV-9566 Port Percona Xtrabackup to MariaDB as mariabackup - Modify backup code to work with MariaDB's 10.1 xtradb - Remove crypt encryption, version_check.pl, "compact backup" - Add encryption plugin
2017-04-18 21:05:57 +02:00
#if defined(HAVE_OPENSSL)
extern my_bool opt_ssl_verify_server_cert;
#endif
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
my_bool
xb_get_one_option(int optid,
const struct my_option *opt __attribute__((unused)),
char *argument);
const char*
xb_get_copy_action(const char *dflt = "Copying");
MDEV-13563 lock DDL for mariabackup in 10.2 Implement lock-ddl-per-table option that locks tables before it is copied to backup, and helds the lock until backup finished The "DDL-lock" itself is implemented as "SELECT * from <table> LIMIT 0", inside a transaction, and "COMMIT" of this transaction is the DDL-unlock.
2017-09-11 18:45:36 +02:00
void mdl_lock_init();
void mdl_lock_table(ulint space_id);
void mdl_unlock_all();
mariabackup : Fix enumerate_ibd_files() to include .isl
2017-09-13 14:10:48 +02:00
bool ends_with(const char *str, const char *suffix);
MDEV-9566 Add Percona Xtrabackup 2.3.7
2017-04-18 20:43:20 +02:00
#endif /* XB_XTRABACKUP_H */
Reference in a new issue Copy permalink