2022-02-03 12:06:25 +00:00
#!/usr/bin/env bash
set -ue
2015-06-06 01:08:41 +03:00
# Copyright (C) 2009-2015 Codership Oy
MDEV-24962: Galera SST innobackupex-move ignores Environment settings
After switching to the new mariabackup interface (instead of
the outdated innobackupex interface, which is supported for
compatibility), we need to explicitly pass a path to the datadir
directory as a parameter, since in the new interface the value
of this option is not automatically set in such a way that it
always matches the SST/IST logic. This commit adds passing this
option as an explicit parameter to mariabackup. This commit also
removed unnecessary options that are not used and not supported
by mariabackup.
Also, numerous flaws in the common wsrep_sst_common script have
been fixed:
1) There are many bash-specific constructs in the script that
may not be supported by other interpreters, which can lead
to the most unexpected errors during SST, because failures
in the interpretation of bash-specific constructs lead to
incorrect parsing of arguments;
2) There is parse_cnf() function which is often called by other
scripts for the "mysqld" or "--mysqld" group, but it does not
take into account the default group suffix, which leads to
reading values only from the default group, which then leads
to errors due to reading the default values instead of the
values for a specific group;
3) Some options such as --user, --innodb-data-home-dir or --datadir
are not removed from the --mysqld-args list, although they are
processed inside scripts (and passing of these options funther
may cause problems for mariabackup);
4) If an argument that the script understands is present in
the --mysqld-args list twice, then this causes SST to fail,
instead of reading the most recent value;
5) The "--host" parameter is technically still supported among
the arguments of the SST scripts, but in reality scripts do not
work with it as expected, especially if it has an IPv6 address;
6) If the port number is absent in the --address parameter value,
but the port number is explicitly passed through the --port
argument, then the scripts for mariabackup and xtrabackup-v2
fail;
7) If a new address interface is used (with the --address parameter),
then automatic default port substitution is not performed, although
it is supported for the legacy --host/--port interface.
8) If there are spaces in the parameter values after --mysqld_args,
then their further transfer does not occur correctly, which
causes mariabackup to fail during SST - the space splits
the argument in such a way that it breaks the parsing of the
following parameters;
9) If most of the parameters that are names or paths to the files
or directories contain spaces, then SST scripts fail in an
unpredictable way due to incorrect variable substitutions;
10) If the --log-bin option is passed among the arguments of myqlds
(--mysqld-args) without a parameter, and the --binlog option
is not specified, then the script cannot substitute the default
name for binlog and cannot construct binlog name using the
--log-basename argument (which is against server specifications);
11) Tail slashes are not removed from the directory names, which,
upon further substitution, leads to the appearance of a double
slash in the file paths;
12) The explicit --binlog parameter (which is now always transmitted
from the server side) and the "hidden" --log-bin parameter in the
list of arguments after --mysqld-args are perceived as two different
parameters in different parts of the scripts, and if they are do not
match for some reason, this will lead to failures during SST;
Also, all new changes from the 10.6 branch have been migrated here,
including the latest pull requests for authentication (only the part
that concerns SST scripts).
It also fixes dozens of other bugs in all SST scripts.
2021-04-28 01:39:31 +02:00
# Copyright (C) 2017-2021 MariaDB
2014-08-06 15:39:15 +03:00
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING. If not, write to the
# Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston
2019-05-11 22:19:05 +03:00
# MA 02110-1335 USA.
2014-08-06 15:39:15 +03:00
# This is a reference script for mysqldump-based state snapshot tansfer
MDEV-23580: WSREP_SST: [ERROR] rsync daemon port has been taken
This commit contains a large set of further bug fixes and
improvements to SST scripts for Galera, continuing the work
that was started in MDEV-24962 to make SST scripts work smoothly
in different network configurations (especially using ipv6) and
with different environment settings:
1) The ipv6 addresses were incorrectly handled in the SST script
for rsync (incorrect address substitution for establishing a
connection, incorrect address substitution for bind, and so on);
2) Checking the locality of the ip-address in SST scripts did not
support ipv6 addresses (such as "[::1]"), which were falsely
identified as non-local ip, which further did not allow running
two SSTs on different local addresses on the same machine.
On the other hand, this bug masked some other errors (related
to handling ipv6 addresses);
3) The code for checking the locality of the ip address was different
in the SST scripts for rsync and for mysqldump, with individual
flaws. This code is now made common and moved to wsrep_sst_common;
4) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
did not process ipv6 addresses correctly in all cases (not for all
branches);
5) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
for some code branches could give a false positive result due to
the textual match of prefixes in the port number and/or PID of
the process;
6) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) was supported through different utilities in SST scripts
for mariabackup and for rsync, and with various minor flaws in
the code. Now the code is still different in these scripts, but
it supports a common set of utilities (lsof, ss, sockstat) and
is synchronized across patterns that used to check the output
of these utilities;
7) In SST via mariabackup, the signal about readiness to receive data
is sometimes sent too early - immediately after listen(), and not
after accept() (which are called by socat or netcat utility).
8) Checking availability of the some options of some utilities was
done using the grep pattern, which easily gives false positives;
9) Common name (CN) for local addresses, if not explicitly specified,
is now always replaced to "localhost" to avoid the need to generate
many separate certificates for local addresses of one machine and
not to depend on which the local address is currently used in test
(ipv4 or ipv6, etc.);
10) In tests galera_sst_mariabackup_encrypt_with_key_server and
galera_sst_rsync_encrypt_with_key_server the correct certificate
is selected to avoid commonname (CN) mismatch problems;
11) Further refactoring to protect against spaces in file names.
12) Further general refactoring to eliminate bash-specific constructs
or to improve code readability;
13) The code for setting options for the nc (netcat) utility was
different in different scripts for SST - now it is made identical.
14) Fixed long-time broken encryption via xbcrypt in combination with
mariabackup and added support for key-based encryption via openssl
utility, which is now enabled by default for encrypt=1 mode (this
default mode can be changed using a new configuration file option
"encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
[sst] or in the [xtrabackup] section) - this change will allow us
to use and to test the encypt=1 encryption without installing
non-standard third-party utilities.
2021-05-10 04:27:16 +02:00
. $( dirname " $0 " ) /wsrep_sst_common
2014-08-06 15:39:15 +03:00
EINVAL = 22
if test -z " $WSREP_SST_OPT_HOST " ; then wsrep_log_error "HOST cannot be nil" ; exit $EINVAL ; fi
if test -z " $WSREP_SST_OPT_PORT " ; then wsrep_log_error "PORT cannot be nil" ; exit $EINVAL ; fi
if test -z " $WSREP_SST_OPT_LPORT " ; then wsrep_log_error "LPORT cannot be nil" ; exit $EINVAL ; fi
if test -z " $WSREP_SST_OPT_SOCKET " ; then wsrep_log_error "SOCKET cannot be nil" ; exit $EINVAL ; fi
if test -z " $WSREP_SST_OPT_GTID " ; then wsrep_log_error "GTID cannot be nil" ; exit $EINVAL ; fi
MDEV-23580: WSREP_SST: [ERROR] rsync daemon port has been taken
This commit contains a large set of further bug fixes and
improvements to SST scripts for Galera, continuing the work
that was started in MDEV-24962 to make SST scripts work smoothly
in different network configurations (especially using ipv6) and
with different environment settings:
1) The ipv6 addresses were incorrectly handled in the SST script
for rsync (incorrect address substitution for establishing a
connection, incorrect address substitution for bind, and so on);
2) Checking the locality of the ip-address in SST scripts did not
support ipv6 addresses (such as "[::1]"), which were falsely
identified as non-local ip, which further did not allow running
two SSTs on different local addresses on the same machine.
On the other hand, this bug masked some other errors (related
to handling ipv6 addresses);
3) The code for checking the locality of the ip address was different
in the SST scripts for rsync and for mysqldump, with individual
flaws. This code is now made common and moved to wsrep_sst_common;
4) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
did not process ipv6 addresses correctly in all cases (not for all
branches);
5) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
for some code branches could give a false positive result due to
the textual match of prefixes in the port number and/or PID of
the process;
6) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) was supported through different utilities in SST scripts
for mariabackup and for rsync, and with various minor flaws in
the code. Now the code is still different in these scripts, but
it supports a common set of utilities (lsof, ss, sockstat) and
is synchronized across patterns that used to check the output
of these utilities;
7) In SST via mariabackup, the signal about readiness to receive data
is sometimes sent too early - immediately after listen(), and not
after accept() (which are called by socat or netcat utility).
8) Checking availability of the some options of some utilities was
done using the grep pattern, which easily gives false positives;
9) Common name (CN) for local addresses, if not explicitly specified,
is now always replaced to "localhost" to avoid the need to generate
many separate certificates for local addresses of one machine and
not to depend on which the local address is currently used in test
(ipv4 or ipv6, etc.);
10) In tests galera_sst_mariabackup_encrypt_with_key_server and
galera_sst_rsync_encrypt_with_key_server the correct certificate
is selected to avoid commonname (CN) mismatch problems;
11) Further refactoring to protect against spaces in file names.
12) Further general refactoring to eliminate bash-specific constructs
or to improve code readability;
13) The code for setting options for the nc (netcat) utility was
different in different scripts for SST - now it is made identical.
14) Fixed long-time broken encryption via xbcrypt in combination with
mariabackup and added support for key-based encryption via openssl
utility, which is now enabled by default for encrypt=1 mode (this
default mode can be changed using a new configuration file option
"encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
[sst] or in the [xtrabackup] section) - this change will allow us
to use and to test the encypt=1 encryption without installing
non-standard third-party utilities.
2021-05-10 04:27:16 +02:00
if is_local_ip " $WSREP_SST_OPT_HOST_UNESCAPED " && \
2014-08-06 15:39:15 +03:00
[ " $WSREP_SST_OPT_PORT " = " $WSREP_SST_OPT_LPORT " ]
then
wsrep_log_error \
" destination address ' $WSREP_SST_OPT_HOST : $WSREP_SST_OPT_PORT ' matches source address. "
exit $EINVAL
fi
# Check client version
2018-01-14 18:57:48 +11:00
if ! $MYSQL_CLIENT --version | grep 'Distrib 10\.[1-9]' >/dev/null
2014-08-06 15:39:15 +03:00
then
2015-07-14 16:05:29 -04:00
$MYSQL_CLIENT --version >& 2
2018-01-14 18:57:48 +11:00
wsrep_log_error "this operation requires MySQL client version 10.1 or newer"
2014-08-06 15:39:15 +03:00
exit $EINVAL
fi
2021-05-21 03:11:48 +02:00
AUTH = ""
usrst = 0
if [ -n " $WSREP_SST_OPT_USER " ] ; then
AUTH = " -u $WSREP_SST_OPT_USER "
usrst = 1
fi
2015-06-06 01:08:41 +03:00
# Refs https://github.com/codership/mysql-wsrep/issues/141
# Passing password in MYSQL_PWD environment variable is considered
# "extremely insecure" by MySQL Guidelines for Password Security
# (https://dev.mysql.com/doc/refman/5.6/en/password-security-user.html)
# that is even less secure than passing it on a command line! It is doubtful:
2015-06-06 01:08:41 +03:00
# the whole command line is easily observable by any unprivileged user via ps,
# whereas (at least on Linux) unprivileged user can't see process environment
# that he does not own. So while it may be not secure in the NSA sense of the
# word, it is arguably more secure than passing password on the command line.
2021-05-21 03:11:48 +02:00
if [ -n " $WSREP_SST_OPT_PSWD " ] ; then
export MYSQL_PWD = " $WSREP_SST_OPT_PSWD "
elif [ $usrst -eq 1 ] ; then
# Empty password, used for testing, debugging etc.
unset MYSQL_PWD
fi
2014-08-06 15:39:15 +03:00
2021-05-21 03:11:48 +02:00
STOP_WSREP = 'SET wsrep_on=OFF;'
2014-08-06 15:39:15 +03:00
# mysqldump cannot restore CSV tables, fix this issue
CSV_TABLES_FIX = "
set sql_mode = '' ;
USE mysql;
SET @cond = ( SELECT ( SUPPORT = 'YES' or SUPPORT = 'DEFAULT' ) FROM INFORMATION_SCHEMA.ENGINES WHERE ENGINE = 'csv' ) ;
2021-05-21 03:11:48 +02:00
SET @stmt = IF ( @cond = '1' , 'CREATE TABLE IF NOT EXISTS general_log ( event_time timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6), user_host mediumtext NOT NULL, thread_id bigint(21) unsigned NOT NULL, server_id int(10) unsigned NOT NULL, command_type varchar(64) NOT NULL, argument mediumtext NOT NULL) ENGINE=CSV DEFAULT CHARSET=utf8mb3 COMMENT=\"General log\"' , 'SET @dummy = 0' ) ;
2014-08-06 15:39:15 +03:00
PREPARE stmt FROM @stmt;
EXECUTE stmt;
DROP PREPARE stmt;
2021-05-21 03:11:48 +02:00
SET @stmt = IF ( @cond = '1' , 'CREATE TABLE IF NOT EXISTS slow_log ( start_time timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6), user_host mediumtext NOT NULL, query_time time(6) NOT NULL, lock_time time(6) NOT NULL, rows_sent int(11) NOT NULL, rows_examined int(11) NOT NULL, db varchar(512) NOT NULL, last_insert_id int(11) NOT NULL, insert_id int(11) NOT NULL, server_id int(10) unsigned NOT NULL, sql_text mediumtext NOT NULL, thread_id bigint(21) unsigned NOT NULL) ENGINE=CSV DEFAULT CHARSET=utf8mb3 COMMENT=\"Slow log\"' , 'SET @dummy = 0' ) ;
2014-08-06 15:39:15 +03:00
PREPARE stmt FROM @stmt;
EXECUTE stmt;
DROP PREPARE stmt; "
SET_START_POSITION = " SET GLOBAL wsrep_start_position=' $WSREP_SST_OPT_GTID '; "
2015-02-27 22:33:41 -05:00
SET_WSREP_GTID_DOMAIN_ID = ""
2021-12-13 02:15:57 +01:00
if [ -n $WSREP_SST_OPT_GTID_DOMAIN_ID ] ; then
2021-04-15 13:53:28 +02:00
SET_WSREP_GTID_DOMAIN_ID = "
SET @val = ( SELECT GLOBAL_VALUE FROM INFORMATION_SCHEMA.SYSTEM_VARIABLES WHERE VARIABLE_NAME = 'WSREP_GTID_STRICT_MODE' AND GLOBAL_VALUE > 0) ;
SET @stmt = IF ( @val IS NOT NULL, 'SET GLOBAL WSREP_GTID_DOMAIN_ID=$WSREP_SST_OPT_GTID_DOMAIN_ID' , 'SET @dummy = 0' ) ;
PREPARE stmt FROM @stmt;
EXECUTE stmt;
DROP PREPARE stmt; "
2015-02-27 22:33:41 -05:00
fi
2021-12-13 02:15:57 +01:00
MYSQL = " $MYSQL_CLIENT $WSREP_SST_OPT_CONF_UNQUOTED " \
MDEV-23580: WSREP_SST: [ERROR] rsync daemon port has been taken
This commit contains a large set of further bug fixes and
improvements to SST scripts for Galera, continuing the work
that was started in MDEV-24962 to make SST scripts work smoothly
in different network configurations (especially using ipv6) and
with different environment settings:
1) The ipv6 addresses were incorrectly handled in the SST script
for rsync (incorrect address substitution for establishing a
connection, incorrect address substitution for bind, and so on);
2) Checking the locality of the ip-address in SST scripts did not
support ipv6 addresses (such as "[::1]"), which were falsely
identified as non-local ip, which further did not allow running
two SSTs on different local addresses on the same machine.
On the other hand, this bug masked some other errors (related
to handling ipv6 addresses);
3) The code for checking the locality of the ip address was different
in the SST scripts for rsync and for mysqldump, with individual
flaws. This code is now made common and moved to wsrep_sst_common;
4) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
did not process ipv6 addresses correctly in all cases (not for all
branches);
5) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
for some code branches could give a false positive result due to
the textual match of prefixes in the port number and/or PID of
the process;
6) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) was supported through different utilities in SST scripts
for mariabackup and for rsync, and with various minor flaws in
the code. Now the code is still different in these scripts, but
it supports a common set of utilities (lsof, ss, sockstat) and
is synchronized across patterns that used to check the output
of these utilities;
7) In SST via mariabackup, the signal about readiness to receive data
is sometimes sent too early - immediately after listen(), and not
after accept() (which are called by socat or netcat utility).
8) Checking availability of the some options of some utilities was
done using the grep pattern, which easily gives false positives;
9) Common name (CN) for local addresses, if not explicitly specified,
is now always replaced to "localhost" to avoid the need to generate
many separate certificates for local addresses of one machine and
not to depend on which the local address is currently used in test
(ipv4 or ipv6, etc.);
10) In tests galera_sst_mariabackup_encrypt_with_key_server and
galera_sst_rsync_encrypt_with_key_server the correct certificate
is selected to avoid commonname (CN) mismatch problems;
11) Further refactoring to protect against spaces in file names.
12) Further general refactoring to eliminate bash-specific constructs
or to improve code readability;
13) The code for setting options for the nc (netcat) utility was
different in different scripts for SST - now it is made identical.
14) Fixed long-time broken encryption via xbcrypt in combination with
mariabackup and added support for key-based encryption via openssl
utility, which is now enabled by default for encrypt=1 mode (this
default mode can be changed using a new configuration file option
"encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
[sst] or in the [xtrabackup] section) - this change will allow us
to use and to test the encypt=1 encryption without installing
non-standard third-party utilities.
2021-05-10 04:27:16 +02:00
" $AUTH -h $WSREP_SST_OPT_HOST_UNESCAPED " \
2017-03-03 20:28:27 +00:00
" -P $WSREP_SST_OPT_PORT --disable-reconnect --connect_timeout=10 "
2014-08-06 15:39:15 +03:00
2015-02-27 22:30:38 -05:00
# Check if binary logging is enabled on the joiner node.
# Note: SELECT cannot be used at this point.
2021-05-21 03:11:48 +02:00
LOG_BIN = $( echo "set statement wsrep_sync_wait=0 for SHOW VARIABLES LIKE 'log_bin'" | $MYSQL | \
2015-02-27 22:30:38 -05:00
tail -1 | awk -F ' ' '{ print $2 }' )
# Check the joiner node's server version.
2021-05-21 03:11:48 +02:00
SERVER_VERSION = $( echo "set statement wsrep_sync_wait=0 for SHOW VARIABLES LIKE 'version'" | $MYSQL | \
2015-02-27 22:30:38 -05:00
tail -1 | awk -F ' ' '{ print $2 }' )
2018-06-15 18:20:36 +02:00
# Retrieve the donor's @@global.gtid_binlog_state.
2021-05-21 03:11:48 +02:00
GTID_BINLOG_STATE = $( echo "SHOW GLOBAL VARIABLES LIKE 'gtid_binlog_state'" | $MYSQL | \
2018-06-15 18:20:36 +02:00
tail -1 | awk -F ' ' '{ print $2 }' )
2015-02-27 22:30:38 -05:00
RESET_MASTER = ""
SET_GTID_BINLOG_STATE = ""
SQL_LOG_BIN_OFF = ""
# Safety check
2021-12-13 02:15:57 +01:00
if [ ${ SERVER_VERSION %%.* } -gt 5 ] ; then
2021-04-15 13:53:28 +02:00
# If binary logging is enabled on the joiner node, we need to copy donor's
# gtid_binlog_state to joiner. In order to do that, a RESET MASTER must be
# executed to erase binary logs (if any). Binary logging should also be
# turned off for the session so that gtid state does not get altered while
# the dump gets replayed on joiner.
MDEV-23580: WSREP_SST: [ERROR] rsync daemon port has been taken
This commit contains a large set of further bug fixes and
improvements to SST scripts for Galera, continuing the work
that was started in MDEV-24962 to make SST scripts work smoothly
in different network configurations (especially using ipv6) and
with different environment settings:
1) The ipv6 addresses were incorrectly handled in the SST script
for rsync (incorrect address substitution for establishing a
connection, incorrect address substitution for bind, and so on);
2) Checking the locality of the ip-address in SST scripts did not
support ipv6 addresses (such as "[::1]"), which were falsely
identified as non-local ip, which further did not allow running
two SSTs on different local addresses on the same machine.
On the other hand, this bug masked some other errors (related
to handling ipv6 addresses);
3) The code for checking the locality of the ip address was different
in the SST scripts for rsync and for mysqldump, with individual
flaws. This code is now made common and moved to wsrep_sst_common;
4) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
did not process ipv6 addresses correctly in all cases (not for all
branches);
5) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) in the wait_for_listen() and check_pid_and_port() functions
for some code branches could give a false positive result due to
the textual match of prefixes in the port number and/or PID of
the process;
6) Waiting for the start of the transport channel (socat, nc, rsync,
stunnel) was supported through different utilities in SST scripts
for mariabackup and for rsync, and with various minor flaws in
the code. Now the code is still different in these scripts, but
it supports a common set of utilities (lsof, ss, sockstat) and
is synchronized across patterns that used to check the output
of these utilities;
7) In SST via mariabackup, the signal about readiness to receive data
is sometimes sent too early - immediately after listen(), and not
after accept() (which are called by socat or netcat utility).
8) Checking availability of the some options of some utilities was
done using the grep pattern, which easily gives false positives;
9) Common name (CN) for local addresses, if not explicitly specified,
is now always replaced to "localhost" to avoid the need to generate
many separate certificates for local addresses of one machine and
not to depend on which the local address is currently used in test
(ipv4 or ipv6, etc.);
10) In tests galera_sst_mariabackup_encrypt_with_key_server and
galera_sst_rsync_encrypt_with_key_server the correct certificate
is selected to avoid commonname (CN) mismatch problems;
11) Further refactoring to protect against spaces in file names.
12) Further general refactoring to eliminate bash-specific constructs
or to improve code readability;
13) The code for setting options for the nc (netcat) utility was
different in different scripts for SST - now it is made identical.
14) Fixed long-time broken encryption via xbcrypt in combination with
mariabackup and added support for key-based encryption via openssl
utility, which is now enabled by default for encrypt=1 mode (this
default mode can be changed using a new configuration file option
"encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
[sst] or in the [xtrabackup] section) - this change will allow us
to use and to test the encypt=1 encryption without installing
non-standard third-party utilities.
2021-05-10 04:27:16 +02:00
if [ " $LOG_BIN " = 'ON' ] ; then
MDEV-24962: Galera SST innobackupex-move ignores Environment settings
After switching to the new mariabackup interface (instead of
the outdated innobackupex interface, which is supported for
compatibility), we need to explicitly pass a path to the datadir
directory as a parameter, since in the new interface the value
of this option is not automatically set in such a way that it
always matches the SST/IST logic. This commit adds passing this
option as an explicit parameter to mariabackup. This commit also
removed unnecessary options that are not used and not supported
by mariabackup.
Also, numerous flaws in the common wsrep_sst_common script have
been fixed:
1) There are many bash-specific constructs in the script that
may not be supported by other interpreters, which can lead
to the most unexpected errors during SST, because failures
in the interpretation of bash-specific constructs lead to
incorrect parsing of arguments;
2) There is parse_cnf() function which is often called by other
scripts for the "mysqld" or "--mysqld" group, but it does not
take into account the default group suffix, which leads to
reading values only from the default group, which then leads
to errors due to reading the default values instead of the
values for a specific group;
3) Some options such as --user, --innodb-data-home-dir or --datadir
are not removed from the --mysqld-args list, although they are
processed inside scripts (and passing of these options funther
may cause problems for mariabackup);
4) If an argument that the script understands is present in
the --mysqld-args list twice, then this causes SST to fail,
instead of reading the most recent value;
5) The "--host" parameter is technically still supported among
the arguments of the SST scripts, but in reality scripts do not
work with it as expected, especially if it has an IPv6 address;
6) If the port number is absent in the --address parameter value,
but the port number is explicitly passed through the --port
argument, then the scripts for mariabackup and xtrabackup-v2
fail;
7) If a new address interface is used (with the --address parameter),
then automatic default port substitution is not performed, although
it is supported for the legacy --host/--port interface.
8) If there are spaces in the parameter values after --mysqld_args,
then their further transfer does not occur correctly, which
causes mariabackup to fail during SST - the space splits
the argument in such a way that it breaks the parsing of the
following parameters;
9) If most of the parameters that are names or paths to the files
or directories contain spaces, then SST scripts fail in an
unpredictable way due to incorrect variable substitutions;
10) If the --log-bin option is passed among the arguments of myqlds
(--mysqld-args) without a parameter, and the --binlog option
is not specified, then the script cannot substitute the default
name for binlog and cannot construct binlog name using the
--log-basename argument (which is against server specifications);
11) Tail slashes are not removed from the directory names, which,
upon further substitution, leads to the appearance of a double
slash in the file paths;
12) The explicit --binlog parameter (which is now always transmitted
from the server side) and the "hidden" --log-bin parameter in the
list of arguments after --mysqld-args are perceived as two different
parameters in different parts of the scripts, and if they are do not
match for some reason, this will lead to failures during SST;
Also, all new changes from the 10.6 branch have been migrated here,
including the latest pull requests for authentication (only the part
that concerns SST scripts).
It also fixes dozens of other bugs in all SST scripts.
2021-04-28 01:39:31 +02:00
RESET_MASTER = "SET GLOBAL wsrep_on=OFF; RESET MASTER; SET GLOBAL wsrep_on=ON;"
SET_GTID_BINLOG_STATE = " SET GLOBAL wsrep_on=OFF; SET @@global.gtid_binlog_state=' $GTID_BINLOG_STATE '; SET GLOBAL wsrep_on=ON; "
2021-04-15 13:53:28 +02:00
SQL_LOG_BIN_OFF = "SET @@session.sql_log_bin=OFF;"
fi
2015-02-27 22:30:38 -05:00
fi
# NOTE: we don't use --routines here because we're dumping mysql.proc table
2021-12-13 02:15:57 +01:00
MYSQLDUMP = " $MYSQLDUMP $WSREP_SST_OPT_CONF_UNQUOTED $AUTH -S $WSREP_SST_OPT_SOCKET \
2015-02-27 22:30:38 -05:00
--add-drop-database --add-drop-table --skip-add-locks --create-options \
--disable-keys --extended-insert --skip-lock-tables --quick --set-charset \
2015-09-09 20:51:39 -04:00
--skip-comments --flush-privileges --all-databases --events"
2015-02-27 22:30:38 -05:00
2014-08-06 15:39:15 +03:00
if [ $WSREP_SST_OPT_BYPASS -eq 0 ]
then
2021-05-21 03:11:48 +02:00
# need to disable logging when loading the dump
# reason is that dump contains ALTER TABLE for log tables, and
# this causes an error if logging is enabled
GENERAL_LOG_OPT = $( $MYSQL --skip-column-names -e " $STOP_WSREP SELECT @@GENERAL_LOG " )
SLOW_LOG_OPT = $( $MYSQL --skip-column-names -e " $STOP_WSREP SELECT @@SLOW_QUERY_LOG " )
LOG_OFF = "SET GLOBAL GENERAL_LOG=OFF; SET GLOBAL SLOW_QUERY_LOG=OFF;"
# commands to restore log settings
RESTORE_GENERAL_LOG = " SET GLOBAL GENERAL_LOG= $GENERAL_LOG_OPT ; "
RESTORE_SLOW_QUERY_LOG = " SET GLOBAL SLOW_QUERY_LOG= $SLOW_LOG_OPT ; "
( echo " $STOP_WSREP " && echo " $LOG_OFF " && echo " $RESET_MASTER " && \
echo " $SET_GTID_BINLOG_STATE " && echo " $SQL_LOG_BIN_OFF " && \
echo " $STOP_WSREP " && $MYSQLDUMP && echo " $CSV_TABLES_FIX " && \
echo " $RESTORE_GENERAL_LOG " && echo " $RESTORE_SLOW_QUERY_LOG " && \
echo " $SET_START_POSITION " && echo " $SET_WSREP_GTID_DOMAIN_ID " \
2015-02-27 22:33:41 -05:00
|| echo "SST failed to complete;" ) | $MYSQL
2014-08-06 15:39:15 +03:00
else
wsrep_log_info "Bypassing state dump."
2021-05-21 03:11:48 +02:00
echo " $SET_START_POSITION " | $MYSQL
2014-08-06 15:39:15 +03:00
fi
#