Commit graph

57 commits

Author SHA1 Message Date
Daniel Black
04ed70fa55 MDEV-8743: wsrep_sst_common close FDs > 2
To prevent accidential overwriting of files due to mysqld file
descriptors being open we ensure where that only FDs 1, 2 and 3
are available to SST scripts. Assumed is mysqld has thse still
mapped to stdin (unused), stdout and stderr.
2018-03-08 18:55:01 +11:00
Marko Mäkelä
b006d2ead4 Merge bb-10.2-ext into 10.3 2018-02-15 10:22:03 +02:00
Sergei Golubchik
355ee6877b MDEV-13946 Server RPMs have dependency on "which"
cleanup. use "command -v" instead of "which"
simplify some checks.
2018-02-12 12:46:34 +01:00
Alexander Barkov
3cad31f2a7 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2018-02-08 19:06:25 +04:00
Sergei Golubchik
b4fbb4183b remove bash-ish from SST scripts
replace with awk (yuck!), avoiding gensub (which wasn't recognized)
2018-02-06 14:52:01 +01:00
Sergei Golubchik
4771ae4b22 Merge branch 'github/10.1' into 10.2 2018-02-06 14:50:50 +01:00
Sergei Golubchik
c383418cbf Merge branch 'github/10.0-galera' into 10.1 2018-02-03 08:09:06 +01:00
Daniel Black
51ea696c8e wsrep_sst_common: keep WSREP_SST_OPT_HOST_UNESCAPED for IPv4
change wsrep_sst_mysqldump.sh to use WSREP_SST_OPT_HOST_UNESCAPED
directly.
2018-01-14 23:31:14 +11:00
Daniel Black
e78e308e81 wsrep_sst_common: parse --address and split WSREP_SST_OPT_PATH
Use the string functionality of sh to split the address:

IPv6 test:
. scripts/wsrep_sst_common.sh --address [ipv6]:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
$ set | grep WSREP_SST
WSREP_SST_OPT_ADDR='[ipv6]:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2'
WSREP_SST_OPT_AUTH=
WSREP_SST_OPT_BINLOG=
WSREP_SST_OPT_BYPASS=0
WSREP_SST_OPT_CONF='  '
WSREP_SST_OPT_DATA=
WSREP_SST_OPT_DEFAULT=
WSREP_SST_OPT_EXTRA_DEFAULT=
WSREP_SST_OPT_HOST='[ipv6]'
WSREP_SST_OPT_HOST_UNESCAPED=ipv6
WSREP_SST_OPT_LSN=lsn_version
WSREP_SST_OPT_MODULE=module
WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_PORT=port
WSREP_SST_OPT_PSWD=
WSREP_SST_OPT_SST_VER=sst_ver
WSREP_SST_OPT_SUFFIX_DEFAULT=
WSREP_SST_OPT_USER=

IPv4 test:
. scripts/wsrep_sst_common.sh --address ipv4:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
$ set | grep WSREP_SST
WSREP_SST_OPT_ADDR=ipv4:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_AUTH=
WSREP_SST_OPT_BINLOG=
WSREP_SST_OPT_BYPASS=0
WSREP_SST_OPT_CONF='  '
WSREP_SST_OPT_DATA=
WSREP_SST_OPT_DEFAULT=
WSREP_SST_OPT_EXTRA_DEFAULT=
WSREP_SST_OPT_HOST=ipv4
WSREP_SST_OPT_LSN=lsn_version
WSREP_SST_OPT_MODULE=module
WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_PORT=port
WSREP_SST_OPT_PSWD=
WSREP_SST_OPT_SST_VER=sst_ver
WSREP_SST_OPT_SUFFIX_DEFAULT=
WSREP_SST_OPT_USER=
2018-01-14 23:31:13 +11:00
Daniel Black
95e5fe6732 wsrep_sst_common: parse_cnf - use awk rather than grep/cut/tail excessiveness
Test cases:

f($var) = awk -v var="${var}"  \
     'BEGIN { OFS=FS="="}
     { gsub(/_/,"-",$1); if ( $1=="--"var ) lastval=substr($0,length($1)+2) }
     END { print lastval }'

Missing input is blank:

$ echo '--var_aa=something' | f(var-b-not-ther)
(blank as expected)

All RHS of = is unmunged:

$ echo '--var_aa=password==_-$' | f(var-aa)
password==_-$

Mixed - and _ in var name:
$ echo '--var_aa-bb_cc=1' | f(var-aa-bb-cc)
1

No value returns blank line:
$ echo '--var_aa-bb_cc' | f(var-aa-bb-cc)
(blank line as expected)

Multiples return the last:
$  echo -e "--bb=cc\n--bb=dd" | f(bb)
dd
2018-01-14 23:31:01 +11:00
Sergei Golubchik
943c62a5d4 Backport 4bb49d84a9, correct handling on defaults[-extra]-file is SST scripts
(from 10.1 to 10.0-galera)

This conflicted signficantly with 7d550c76be
which added --defaults-group-suffix support.

Took the approach of 4bb49d84a9 and adapted the
--defaults-group-suffix handling to be consistent.

The following changes as follows:

SST scripts now use $MY_PRINT_DEFAULTS rather than the lowercase for
consistency and this include all required --default arguements.

Backport/merge by Daniel Black <daniel@linux.vnet.ibm.com>
2018-01-14 22:48:02 +11:00
Martynas Bendorius
1c2f59f7fb MDEV-13969 sst mysqldump and xtrabackup-v2 handle WSREP_SST_OPT_CONF incorrectly
wrep_sst_common: Setting "-c ''" for my_print_defaults just takes no values from config at all. $MY_PRINT_DEFAULTS is already set at the top of the script to have --defaults-file and --defaults-extra-file. If WSREP_SST_OPT_CONF if set to "--defaults-file=/etc/my.cnf --defaults-extra-file=/etc/my.extra.cnf", then "my_print_defaults -c "" --defaults-file=/etc/my.cnf" succeeds, but if WSREP_SST_OPT_CONF is empty - no default values are taken at all.
wsrep_sst_xtrabackup-v2: innobackupex does not support --defaults-extra-file, so ${WSREP_SST_OPT_CONF} cannot be used as an argument, it has been changed to ${WSREP_SST_OPT_DEFAULT}. Removed --defaults-file= from INNOMOVE line, because WSREP_SST_OPT_CONF already includes it (INNOBACKUP was fine, INNOMOVE - not).
2017-12-18 16:41:40 +01:00
Alexander Barkov
5b697c5a23 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-11-29 12:06:48 +04:00
Sergei Golubchik
8c422bf48d MDEV-14256 MariaDB 10.2.10 can't SST with xtrabackup-v2
another followup for 4c2c057d40.

there are six possible cases:
--port can be set or not.
--address can be set, not set, or set but without a port number

The correct behavior is:
1 both --port and --address have a port number
  - use it if it's the same, otherwise an error
2 only --port has the number (--address isn't set)
  - use the value from --port
3 only --port has the number (--address is set, but has no port)
  - use the value from --port
4 --port is unset, --address has the port number
  - use the value from --address
5 --port is unset, --address has no port number
  - use the value from --address, that is, port is empty string
6 --port is unset, --address is unset
  - port is unset (an error somewhere later)

case 5 wasn't handled correctly
2017-11-21 20:03:57 +01:00
Sergei Golubchik
7f1900705b Merge branch '10.1' into 10.2 2017-11-21 19:47:46 +01:00
Sergei Golubchik
2a4e4335c4 Merge branch 'github/10.0-galera' into 10.1 2017-11-10 01:38:03 +01:00
Alexander Barkov
003cb2f424 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2017-10-30 16:42:46 +04:00
Sergei Golubchik
f7090df712 MDEV-13968 sst fails with "WSREP_SST_OPT_PORT: readonly variable"
followup for 4c2c057d40

fix broken script (set -u)
2017-10-25 01:03:19 +02:00
Sergei Golubchik
4c2c057d40 MDEV-13968 sst fails with "WSREP_SST_OPT_PORT: readonly variable" 2017-10-18 22:15:26 +02:00
Marko Mäkelä
1641879387 Merge 10.2 into bb-10.2-ext 2017-10-03 17:02:18 +03:00
Vincent Milum Jr
98e09ee4b6 MDEV-13909 Fix wsrep_sst_rsync fails on debian
Fix for the following error messages during SST:
/usr/local/mysql/bin/wsrep_sst_rsync: 258: /usr/local/mysql/bin/wsrep_sst_rsync: [[: not found
/usr/local/mysql/bin/wsrep_sst_rsync: 263: /usr/local/mysql/bin/wsrep_sst_rsync: [[: not found
/usr/local/mysql/bin/wsrep_sst_rsync: 268: /usr/local/mysql/bin/wsrep_sst_rsync: [[: not found
2017-10-02 13:21:00 -07:00
Martynas Bendorius
36ef89c999 wrep_sst_common: Setting "-c ''" for my_print_defaults just takes no values from config at all. $MY_PRINT_DEFAULTS is already set at the top of the script to have --defaults-file and --defaults-extra-file. If WSREP_SST_OPT_CONF if set to "--defaults-file=/etc/my.cnf --defaults-extra-file=/etc/my.extra.cnf", then "my_print_defaults -c "" --defaults-file=/etc/my.cnf" succeeds, but if WSREP_SST_OPT_CONF is empty - no default values are taken at all.
wsrep_sst_xtrabackup-v2: innobackupex does not support --defaults-extra-file, so ${WSREP_SST_OPT_CONF} cannot be used as an argument, it has been changed to ${WSREP_SST_OPT_DEFAULT}. Removed --defaults-file= from INNOMOVE line, because WSREP_SST_OPT_CONF already includes it (INNOBACKUP was fine, INNOMOVE - not).
2017-10-01 15:50:25 +02:00
Marko Mäkelä
6749d39a95 Merge 10.2 into bb-10.2-ext 2017-09-01 08:47:55 +03:00
Marko Mäkelä
a36c369bda Merge 10.1 into 10.2
For running the Galera tests, the variable my_disable_leak_check
was set to true in order to avoid assertions due to memory leaks
at shutdown.

Some adjustments due to MDEV-13625 (merge InnoDB tests from MySQL 5.6)
were performed. The most notable behaviour changes from 10.0 and 10.1
are the following:

* innodb.innodb-table-online: adjustments for the DROP COLUMN
behaviour change (MDEV-11114, MDEV-13613)

* innodb.innodb-index-online-fk: the removal of a (1,NULL) record
from the result; originally removed in MySQL 5.7 in the
Oracle Bug #16244691 fix
377774689b

* innodb.create-index-debug: disabled due to MDEV-13680
(the MySQL Bug #77497 fix was not merged from 5.6 to 5.7.10)

* innodb.innodb-alter-autoinc: MariaDB 10.2 behaves like MySQL 5.6/5.7,
while MariaDB 10.0 and 10.1 assign different values when
auto_increment_increment or auto_increment_offset are used.
Also MySQL 5.6/5.7 exhibit different behaviour between
LGORITHM=INPLACE and ALGORITHM=COPY, so something needs to be tested
and fixed in both MariaDB 10.0 and 10.2.

* innodb.innodb-wl5980-alter: disabled because it would trigger an
InnoDB assertion failure (MDEV-13668 may need additional effort in 10.2)
2017-08-31 09:30:40 +03:00
Jan Lindström
c23efc7d50 Merge remote-tracking branch 'origin/10.0-galera' into 10.1 2017-08-21 13:35:00 +03:00
Alexey Yurchenko
83664e21e4 MW-366 Improved support for IPv6 networks - made mysqld and SST scripts to recognize []-escaped IPv6 addresses - pulled in latest Percona and MariaDB updates to SST scripts - instruct netcat and socat in wsrep_sst_xtrabackup-v2 to listen on IPv6 socket via sockopt parameter in the [sst] section of my.cnf
In summary, wsrep_node_address and wsrep_sst_receive_address can now
be set to IPv6 addresses escaped by []. Rsync SST works out ouf the
box thanks to rsync daemon listening on both IPv4 and IPv6 sockets by
default. For xtrabackup SST onver IPv6 one needs to set sockopt in
the [sst] section of joiner's configuration file to ",pf=ip6" if
using socat as a streamer or to "-6" if using netcat.
2017-08-11 15:00:18 +03:00
Alexander Barkov
9bc3225642 Merge tag 'mariadb-10.2.6' into bb-10.2-ext 2017-05-26 19:32:28 +04:00
Marko Mäkelä
14c6f00a9f Merge 10.1 into 10.2
Also, include fixes by Vladislav Vaintroub to the
aws_key_management plugin. The AWS C++ SDK specifically depends on
OPENSSL_LIBRARIES, not generic SSL_LIBRARIES (such as YaSSL).
2017-05-06 14:36:46 +03:00
Sergei Golubchik
2c3f578789 don't generate wsrep_sst_common in-place
rename the source to wsrep_sst_common.sh
2017-04-27 19:12:37 +02:00
Sergey Vojtovich
fa137476ff MDEV-11941 - Lintian complains about executable bits
Revoked executable bit from files that are not supposed to be executed directly.
Removed interpreted from files that are not supposed to be executed directly.
Added interpreter to files that are supposed to be executed directly.
2017-03-07 22:28:33 +04:00
Nirbhay Choubey
90266e8a0e Merge branch '10.0-galera' into bb-10.1-serg 2016-08-25 15:39:39 -04:00
sjaakola
58386ca04d refs codership/mysql-wsrep#239
Synced xtrabackup SST scripts from PXC source tree as of PXC 5.6.27-25.13
- PXC#480: xtrabackup-v2 SST fails with multiple log_bin directives in my.cn
- PXC#460: wsrep_sst_auth don't work in Percona-XtraDB-Cluster-56-5.6.25-25.
- PXC-416: Fix SST related issues.
- PXC-389: Merge remote-tracking branch 'wsrep/5.6' into 5.6-wsrep-pxc389
- Bug #1431101: SST does not clobber backup-my.cnf
2016-08-21 16:16:58 -04:00
Nirbhay Choubey
370a2cbe96 Fix syntax error in wsrep_sst_common.sh. 2015-11-20 19:49:16 -05:00
Sergei Golubchik
7f19330c59 Merge branch 'github/10.0-galera' into 10.1 2015-11-19 17:48:36 +01:00
Alexey Yurchenko
0ccbbffb78 Slight cleanup improvement on a previous commit. 2015-09-09 22:30:51 -04:00
Philip Stoev
bc796c29b3 Refs codership/mysql-wsrep#143 . Account for the case where the SST password is empty 2015-09-09 22:30:50 -04:00
Alexey Yurchenko
86ee30cea9 Refs codership/mysql-wsrep#141: this commit
1. Passes wsrep_sst_auth_value to SST scripts via WSREP_SST_OPT_AUTH envronmental variable, so it never appears on the command line
  2. In mysqldump and xtrabackup* SST scripts which rely on MySQL authentication, instead of passing password on the command line, SST script sets MYSQL_PWD environment variable, so that password also never appears on the mysqldump/innobackupex command line.
2015-09-09 22:29:03 -04:00
Alexey Yurchenko
1b1410c449 Slight cleanup improvement on a previous commit. 2015-09-09 17:03:31 -04:00
Philip Stoev
62c2539376 Refs codership/mysql-wsrep#143 . Account for the case where the SST password is empty 2015-09-09 17:03:31 -04:00
Alexey Yurchenko
a7ea3ec34d Synced xtrabackup SST fixes from Percona tree (as of PXC 5.6.24-25.11 release). This fixes/adresses the following LP bugs:
- LP1380697: wsrep_sst_xtrabackup-v2 doesn't stop when mysql is SIGKILLed. (full fix for this (as engineeered by Percona) requires Linux-specific patch that we don't carry, but keep xtrabackup scripts as close as possible)
  - LP1399134: Log the innobackupex/SST logs in SST to syslog if possible. (fixed)
  - LP1405668: Race condition between donor and joiner in PXB SST. (fixed)
  - LP1405985: Fail early if xtrabackup_checkkpoints is missing. (fixed)
  - LP1407599: wsrep_sst_xtrabackup-v2 script causes innobackupex to print a false positive stack trace into the log. (fixed)
  - LP1441762: IST Fails with SST script error. (fixed)
  - LP1451670: Fail when move-back fails in xtrabackup SST. (fixed)
2015-09-09 17:03:23 -04:00
Alexey Yurchenko
d78110e7fa Refs codership/mysql-wsrep#141: this commit
1. Passes wsrep_sst_auth_value to SST scripts via WSREP_SST_OPT_AUTH envronmental variable, so it never appears on the command line
  2. In mysqldump and xtrabackup* SST scripts which rely on MySQL authentication, instead of passing password on the command line, SST script sets MYSQL_PWD environment variable, so that password also never appears on the mysqldump/innobackupex command line.
2015-09-09 17:02:26 -04:00
Nirbhay Choubey
dced5146bd Merge branch '10.0-galera' into 10.1 2015-07-14 16:05:29 -04:00
Nirbhay Choubey
1f4a89b10c Do print SST log messages. 2015-06-24 23:38:04 -04:00
Alexey Yurchenko
822c00536d Going more MTR-friendly - for SST prefer to use mysql client, mysqldump and my_print_defaults from the local build tree. 2015-05-04 13:04:51 -04:00
Nirbhay Choubey
34d86ac9ff MDEV-6594: Use separate domain_id for Galera transactions 2015-02-27 22:33:41 -05:00
Sergei Golubchik
4bb49d84a9 correct handling on defaults[-extra]-file is SST scripts
pass --defaults-file and --defaults-extra-file
(whatever was specified, or none)
from mysqld down to SST scripts.

parse these options in SST scripts and pass them down
to mysqldump, my_print_defaults, and xtrabackup
2014-10-01 23:38:27 +02:00
Jan Lindström
df4dd593f2 MDEV-6247: Merge 10.0-galera to 10.1.
Merged lp:maria/maria-10.0-galera up to revision 3879.

Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
2014-08-26 15:43:46 +03:00
Nirbhay Choubey
c29cbac2eb bzr merge -r3985..3997 codership/5.5 2014-06-10 17:00:32 -04:00
Nirbhay Choubey
90ead99572 bzr merge -r3980..3984 codership/5.5 2014-06-10 16:33:57 -04:00
Jan Lindström
fa18dc3944 Merge lp:codership-mysql/5.5 -r3961..3980. 2014-04-16 13:08:29 +03:00