Mariabackup (mariadb-backup) supports the --use-memory option that
sets the buffer pool size for innodb. However, current SST scripts
do not use this option. This commit adds support for this option,
the value for which can be specified via the "use_memory" parameter
in the configuration file in the [sst], [mariabackup] or [xtrabackup]
sections (supported only for compatibility with old configurations).
In addition, if the innodb_buffer_pool_size option is specified in
the user configuration (in the main server configuration sections)
or passed to the SST scripts or the server via arguments, its value
is also passed to mariadb-backup as the value for the --use-memory
option.
A new section name [mariabackup] has also been added, which can
be used instead of the deprecated [xtrabackup] (the section name
"mariabackup" was specified in the documentation, but was not
actually supported by SST scripts before this commit).
The limit of socket length on unix according to libc is 108, see
sockaddr_un::sun_path, but in the table it is a string of max length
64, which results in truncation of socket and failure to connect by
plugins using servers such as spider.
Added get_footprint() implementation for FreeBSD (and for other
non-Linux systems), and added "apparent file size" mode for Linux
to take into account the real file size (without compression) when
used with filesystems like ZFS.
This commit fixes some functions in wsrep_sst_common
to ensure that now and in the future return codes from
a number of helper functions will be zero on success.
Fixed some issues in the script code, mainly related
to handling situations when a failure occurs:
1) the signal handler in the mariadb-backup SST script
was using an uninitialized variable when trying to kill
a hung streaming process;
2) inaccurate error messages were being logged sometime;
3) after completing SST, temporary or old (extra) files
could remain in database directories.
The lsof utility is prone to blocking on system calls that
it uses to obtain information about sockets (or files, devices,
etc.). This behavior is described in its own documentation.
It has a '-b' option (in combination with warnings suppression
via '-w') that reduces the probability of blocking, introducing
new problems (luckily probably not relevant for our use case).
However, there is no guarantee that it will not hang on some
distributions, with some TCP/IP stack implementations, or with
some filesystems, etc. Also, of the three utilities that are
suitable for our purposes, lsof is the slowest. So if there
are other utilities that we use during SST, such as 'ss' or
'sockstat', it is reasonable to use them instead of lsof.
This commit changes the prioritization of utilities, it does
not need additional tests (besides the numerous SST tests
already available in the galera suites). If the system still
need to use lsof, this commit adds the '-b' and '-w' options
to it command line - to reduce the likelihood of blocking.
Removed handling of the long-unsupported xtrabackup_pid file,
as it is not even created by modern versions of mariabackup.
Instead, added stopping of the asynchronous process that
mariabackup runs (if it is still active) to the exception
handler.
This commit makes the SST script for mariabackup more
resilient to unexpected terminations or hangs while
mariabackup or when SST scripts in a previous session
are still running (in reality they were hung while
waiting for something).
With wsrep_sst_rsync, node goes into endless loop when trying
to establish connection to donor for IST/SST if the database
is bind on specific IP address, not the "*".
This commit fixes this problem. Separate tests are not
required - the problem can occur in normal configurations
on a number of systems when selecting a bing address other
than "*", especially on FreeBSD and with the IPv6 addresses.
Specifying filters (filter_status, filter_user, etc) in the mytop config
previously wouldn't work, because any filter specified here was added to
the config hash as a literal string.
This change fixes that - if filter_* is defined in the config and matches
an existing filter_* key, then run the value through StringOrRegex() and
assign to the config hash.
Reason for the error is that --defaults-group-suffix is passed twice
to the command line client, and option parser is not prepared for this.
Double occurence of comes from 2 invocations of parse_arguments(), which
appends unparsed arguments each time it is called.
Fixed by treating --defaults-group-suffix like other "--defaults-"
(--defaults-file/--defaults-extra-file). it will be parsed, and thus
passed only once to the command line client.
The variable `sbindir` is never set for cmake. This adds borked paths to
`galera_recovery`, though it dit not break as the systemd unit changes
the dir to make the relative path work anyway.
Let's fix this nevertheless...
wsrep_sst_mariabackup should use the tmpdir defined by
the user under the '[mysqld]' section of the configuration
file rather than the default '/tmp' directory.
This commit contains a fix for the code that extracts and parses
the CN (common name, domain name) record from certificates using
the openssl utility. This code is also made common to the rsync
and mariabackup scripts. There is also some systematization of
the use of 'printf' and 'echo' builtins/utilities.
This commit fixes GTID inconsistency which was injected by mariabackup SST.
Donor node now writes new info file: donor_galera_info, which is streamed
along the mariabackup donation to the joiner node. The donor_galera_info
file contains both GTID and gtid domain_id, and joiner will use these to
initialize the GTID state.
Commit has new mtr test case: galera_3nodes.galera_gtid_consistency, which
exercises potentially harmful mariabackup SST scenarios. The test has also
scenario with IST joining.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
Also in the startup, lets not "Error" on attempting to install a
mysql.plugin that is already there. We use the 'if_not_exists'
parameter to true to downgrade this to a "Note".
Also corrects: MDEV-32041 "plugin already loaded" should be a Warning, not an Error
This commit fixes a bug where IST could be rejected in favor of SST
when ssl-mode=VERIFY_CA and when mariabackup is used. It also contains
a test and small code simplifications that will make it easier to find
bugs in the future.
Without pam compiled there will be no auth_pam_tool_dir, so check this
before attempting something that will error.
Reviewer: Sergei Golubchik / Daniel Black
Due to the different command line format of the timeout
utility on FreeBSD and Linux, SST scripts for mariabackup
may not work on FreeBSD. This commit fixes this problem
by adding a different command to test options on FreeBSD
and adding proper formatting for the utility options.
SST for mariabackup may not destroy old files if datadir or
other working directory is declared as a symlink due to the lack
of the "-L" option among the find utility options, similarly SST
for rsync in some cases may not transfer data directories if they
are created as symlinks. This fix adds the missing option and
generally unifies the work with find utility options to avoid
failures in the interpretation of directories and regular
expressions.
.snapshot exists as a directory on NetApp storage and
should not be copied during the sst process.
Thanks Daniel Czadek for the bug report.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
The table structure from MySQL-5.1.14 is:
CREATE TABLE `slow_log` (
`start_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`user_host` mediumtext NOT NULL,
`query_time` time NOT NULL,
`lock_time` time NOT NULL,
`rows_sent` int(11) NOT NULL,
`rows_examined` int(11) NOT NULL,
`db` varchar(512) DEFAULT NULL,
`last_insert_id` int(11) DEFAULT NULL,
`insert_id` int(11) DEFAULT NULL,
`server_id` int(11) DEFAULT NULL,
`sql_text` mediumtext NOT NULL
) ENGINE=CSV DEFAULT CHARSET=utf8 COMMENT='Slow log'
Even as far back as MySQL-5.5.40 this table could be created as NULLs
where not permitted in the CSV table time, but it seems they
where allowed sometime.
As the first part of mariadb-upgrade adds the column thread_id without
correcting the 'NULL'able status of existing columns it fails.
We reorder the sql statements in the ugprade as follows:
ALTER TABLE slow_log MODIFY {columns} {new types} NOT NULL,....
As thread_id doesn't exist in the above statement it was removed from
the first ALTER TABLE statement to prevent failure.
Previous ALTER TABLE slow_log where moved later appending thread_id
and rows_affected, and also enforces the type of thread_id if it
was incorrectly like the now first ALTER STATEMENT slow_log used
to do.
