Some users and some scripts (for example, mysqld_multi.sh) use special
option groups with names like [mysqld1], [mysqld2], ..., [mysqldN].
But SST scripts can't currently fully support these option groups.
The only option group-related value it gets from the server is
--defaults-group-suffix, if that option was set for mysqld when
the server was started.
However, the SST scripts does not get told by the server to read
these option groups, so this means that the SST script will fail
to read options like innodb-data-home-dir when it is in a option
group like [mysqld1]...[mysqldN].
Moreover, SST scripts ignore many parameters that can be passed
to them explicitly and cannot transfer them further, for example,
to the input of mariabackup utility. Ideally, we want to transfer
all the parameters of the original mysqld call to utilities such
as mariabackup, however the SST script does not receive these
parameters from the server and therefore cannot transfer them to
mariabackup.
To correct these shortcomings, we need to transfer to the scripts
all of the parameters of the original mysqld call, and in the SST
scripts themselves provide for the transfer all of these parameters
to utilities such as mariabackup. To prevent these parameters from
mixing with the script's own parameters, they should be transferred
to SST script after the special option "--mysqld-args", followed by
the string argument with the original parameters, as it received by
the mysqld call at the time of launch (further all these parameters
will be passed to mariabackup, for example).
In addition, the SST scripts themselves must be refined so that
they can read the parameters from the user-selected group, not just
from the global mysqld configuration group. And also so that they
can receive the parameters (which important for their work) as
command-line arguments.
Some users and some scripts (for example, mysqld_multi.sh) use special
option groups with names like [mysqld1], [mysqld2], ..., [mysqldN].
But SST scripts can't currently fully support these option groups.
The only option group-related value it gets from the server is
--defaults-group-suffix, if that option was set for mysqld when
the server was started.
However, the SST scripts does not get told by the server to read
these option groups, so this means that the SST script will fail
to read options like innodb-data-home-dir when it is in a option
group like [mysqld1]...[mysqldN].
Moreover, SST scripts ignore many parameters that can be passed
to them explicitly and cannot transfer them further, for example,
to the input of mariabackup utility. Ideally, we want to transfer
all the parameters of the original mysqld call to utilities such
as mariabackup, however the SST script does not receive these
parameters from the server and therefore cannot transfer them to
mariabackup.
To correct these shortcomings, we need to transfer to the scripts
all of the parameters of the original mysqld call, and in the SST
scripts themselves provide for the transfer all of these parameters
to utilities such as mariabackup. To prevent these parameters from
mixing with the script's own parameters, they should be transferred
to SST script after the special option "--mysqld-args", followed by
the string argument with the original parameters, as it received by
the mysqld call at the time of launch (further all these parameters
will be passed to mariabackup, for example).
In addition, the SST scripts themselves must be refined so that
they can read the parameters from the user-selected group, not just
from the global mysqld configuration group. And also so that they
can receive the parameters (which important for their work) as
command-line arguments.
Most of the mtr tests in the galera_3nodes suite fail
for a variety of reasons with a variety of errors.
This patch fixes several substantial flaws
in the galera_3nodes suite tests and in the mtr framework
service files, adapting the tests from galera_3nodes
for the current version of MariaDB.
This patch also synchronizes some galera_3nodes-related
files with the latest changes made for MDEV-17835 (v2 patch)
and for MDEV-18379 in other branches (10.2 and 10.3).
Closes#1161
This patch contains the port of the MDEV-18379 patch
for 10.2 branch, but also includes a number of changes
made within MDEV-17835, which are necessary for the
normal operation of tests that use IPv6:
1) Currently, the three-node mtr suite for Galera (galera_3nodes)
uses a separate IPv6 availability check using the "have_ipv6.inc"
file. This check duplicates a more accurate check at suite.pm
level, which can be used by including the file "check_ipv6.inc".
This patch removes this discrepancy between suites.
2) Fixed numerous bugs in the SST scripts and in the mtr test
files (galera_3nodes mtr suite) that prevented the use of Galera
with IPv6 addresses.
3) Fixed flaws in the galera_3nodes mtr suite control scripts,
because of which they could not work with mariabackup.
4) Fixed flaws in the rsync and mysqldump tests (for galera_3nodes
mtr tests suite). These tests were not performed successfully
without these fixes.
5) GAL-501 test in the galera_3nodes suite does not contain the
option "--bind-address=::" that is needed for the test to work
correctly with IPv6 (at least on some systems), since without
it the server will not wait for connections on the IPv6
interface.
https://jira.mariadb.org/browse/MDEV-18379
and partially https://jira.mariadb.org/browse/MDEV-17835
This patch contains the port of the MDEV-18379 patch
for 10.1 branch, but also includes a number of changes
made within MDEV-17835, which are necessary for the
normal operation of tests that use IPv6:
1) Fixed flaws in the galera_3nodes mtr suite control scripts,
because of which they could not work with mariabackup.
2) Fixed numerous bugs in the SST scripts and in the mtr test
files (galera_3nodes mtr suite) that prevented the use of Galera
with IPv6 addresses.
3) Fixed flaws in tests for rsync and mysqldump (for galera_3nodes
mtr tests suite). These tests were not performed successfully
without these fixes.
4) Currently, the three-node mtr suite for Galera (galera_3nodes)
uses a separate IPv6 availability check using the "have_ipv6.inc"
file. This check duplicates a more accurate check at suite.pm
level, which can be used by including the file "check_ipv6.inc".
This patch removes this discrepancy between suites.
5) GAL-501 test in the galera_3nodes suite does not contain the
option "--bind-address=::" which is needed for the test to work
correctly with IPv6 (at least on some systems), since without
it the server will not wait for connections on the IPv6 interface.
https://jira.mariadb.org/browse/MDEV-18379
and partially https://jira.mariadb.org/browse/MDEV-17835
The second line of changes related to replacing xtrabackup with
mariabackup:
1) All unnecessary references to xtrabackup are removed from
the documentation, from some comments, from the control files
that are used to prepare the packages.
2) Made corrections of the tests from the galera_3nodes suite
that mentioned xtrabackup or the old (associated with xtrabackup)
version of innobackupex.
3) Fixed flaws in the galera_3nodes mtr suite control scripts,
because of which they could not work with mariabackup.
4) Fixed numerous bugs in the SST scripts and in the mtr test
files (galera_3nodes mtr suite) that prevented the use of Galera
with IPv6 addresses.
5) Fixed flaws in tests for rsync and mysqldump (for galera_3nodes
mtr tests suite). These tests were not performed successfully without
these fixes.
https://jira.mariadb.org/browse/MDEV-17835
Problem was that in SST log_bin_index name and directory was not
handled and passed to rsync SST script.
wsrep_sst_common.sh
Read binlog index dirname and filename if --binlog-index
parameter is provided. Read binlog filenames from that file
from donor and write transfered binlog filenames to that
file in joiner.
mysqld.cc, mysqld.h
Moved opt_binlog_index_name from static to global and added
it to extern.
wsrep_sst.cc
generate_binlog_index_opt_val
New function to generate binlog index name if opt_binlog_index_name is
given on configuration.
sst_prepare_other
Add binlog index configuration to SST command.
wsrep_sst.h
Add new SST parameter --binlog-index
Add test case.
Current versions of xtrabackup-v2 and mariabackup support the option
--innodb-data-home-dir, but this parameter is not passed to them from
the SST script, since the SST script does not receive this information
from mysqld. The transfer of this information to the SST is already
fixed by the MDEV-10754 patch, but we need to process it in the SST
script. Also, we should take into account that on the joiner side
the corresponding information is not read yet from the configuration
file (at the mysqld side) during the start of SST, so the script must
itself read it.
https://jira.mariadb.org/browse/MDEV-10756
My conflict resolution for the script did not work out after all,
and apparently I was testing a wrong version. Revert MDEV-15511
from MariaDB 10.2 for now.
If --innodb-undo-tablespaces is used, then InnoDB stores undo in a
separate file(s) which whould also be replicated.
This fixes
Issue#337 This filter will cause sst failed at applying undo...
https://github.com/codership/mysql-wsrep/issues/337
If innodb_data_home_dir path specified in the configuration file
then rsync IST/SST will fail because the wsrep_sst_rsync.sh script
does not read this parameter from the configuration file and then
tries to find the data files in the default directory.
To fix this error, we need to add reading of the innodb_data_home_dir
configuration parameter to the rsync-related SST script.
(from 10.1 to 10.0-galera)
This conflicted signficantly with 7d550c76be
which added --defaults-group-suffix support.
Took the approach of 4bb49d84a9 and adapted the
--defaults-group-suffix handling to be consistent.
The following changes as follows:
SST scripts now use $MY_PRINT_DEFAULTS rather than the lowercase for
consistency and this include all required --default arguements.
Backport/merge by Daniel Black <daniel@linux.vnet.ibm.com>
Corrects the following error with rsync sst on FreeBSD:
WSREP_SST: [ERROR] rsync daemon port '4444' has been taken
The FreeBSD version of grep doesn't play nicely with the -w parameter mixed with [[:space:]] and ends up matching nothing. This causes the script to loop and attempt to spawn more than one instance of rsync, which fails with the initial instance already having the port open. Because of this, the entire script fails and no sst occurs resulting in the error message listed above appearing in the log file.
For running the Galera tests, the variable my_disable_leak_check
was set to true in order to avoid assertions due to memory leaks
at shutdown.
Some adjustments due to MDEV-13625 (merge InnoDB tests from MySQL 5.6)
were performed. The most notable behaviour changes from 10.0 and 10.1
are the following:
* innodb.innodb-table-online: adjustments for the DROP COLUMN
behaviour change (MDEV-11114, MDEV-13613)
* innodb.innodb-index-online-fk: the removal of a (1,NULL) record
from the result; originally removed in MySQL 5.7 in the
Oracle Bug #16244691 fix
377774689b
* innodb.create-index-debug: disabled due to MDEV-13680
(the MySQL Bug #77497 fix was not merged from 5.6 to 5.7.10)
* innodb.innodb-alter-autoinc: MariaDB 10.2 behaves like MySQL 5.6/5.7,
while MariaDB 10.0 and 10.1 assign different values when
auto_increment_increment or auto_increment_offset are used.
Also MySQL 5.6/5.7 exhibit different behaviour between
LGORITHM=INPLACE and ALGORITHM=COPY, so something needs to be tested
and fixed in both MariaDB 10.0 and 10.2.
* innodb.innodb-wl5980-alter: disabled because it would trigger an
InnoDB assertion failure (MDEV-13668 may need additional effort in 10.2)
