Commit graph

4089 commits

Author SHA1 Message Date
Sergei Golubchik
db947b7599 Merge branch '10.0-galera' into 10.1 2018-09-07 15:25:27 +02:00
Oleksandr Byelkin
31081593aa Merge branch '11.0' into 10.1 2018-09-06 22:45:19 +02:00
Jan Lindström
653038ccad
Merge pull request #855 from tempesta-tech/sysprg/10.1-MDEV-10756
MDEV-10756: wsrep_sst_xtrabackup-v2 does not support innodb_data_home_dir
2018-09-06 14:10:29 +03:00
Jan Lindström
b44b9f71bd MDEV-15436: If log_bin and log_bin_index is different SST with rsync fails.
Problem was that in SST log_bin_index name and directory was not
handled and passed to rsync SST script.

wsrep_sst_common.sh
	Read binlog index dirname and filename if --binlog-index
	parameter is provided. Read binlog filenames from that file
	from donor and write transfered binlog filenames to that
	file in joiner.

mysqld.cc, mysqld.h
	Moved opt_binlog_index_name from static to global and added
	it to extern.

wsrep_sst.cc

generate_binlog_index_opt_val
	New function to generate binlog index name if opt_binlog_index_name is
	given on configuration.

sst_prepare_other
	Add binlog index configuration to SST command.

wsrep_sst.h
	Add new SST parameter --binlog-index

Add test case.
2018-09-05 10:34:36 +03:00
Daniel Black
eda88e60fb galera_new_cluster.sh: unused variables 2018-09-03 09:53:05 +10:00
Jan Lindström
104089e182 MDEV-15512 - Fix sh parse error when [sst] config value has spaces.
Merged https://github.com/MariaDB/server/pull/617 authored by
Colin Mollenhour.
2018-08-29 19:30:03 +03:00
Jan Lindström
f4454a35fc
Merge pull request #539 from grooverdan/10.1-wsrep_sst_rsync_read_MYSQL_BASE_VERSION_config
MDEV-14851: galera_recovery to read correct configuration file sections
2018-08-28 13:48:43 +03:00
Julius Goryavsky
c11fb374b5 MDEV-10756: wsrep_sst_xtrabackup-v2 does not support innodb_data_home_dir
Current versions of xtrabackup-v2 and mariabackup support the option
--innodb-data-home-dir, but this parameter is not passed to them from
the SST script, since the SST script does not receive this information
from mysqld. The transfer of this information to the SST is already
fixed by the MDEV-10754 patch, but we need to process it in the SST
script. Also, we should take into account that on the joiner side
the corresponding information is not read yet from the configuration
file (at the mysqld side) during the start of SST, so the script must
itself read it.

https://jira.mariadb.org/browse/MDEV-10756
2018-08-27 16:24:29 +02:00
Anel Husakovic
4ba6327f95 Fix typo in --srcdir option in echo message status of mysql_install_db 2018-08-25 17:51:15 +03:00
Julius Goryavsky
4ff7f14fef Fixes of the base patch for compatibility with the 10.1 branch 2018-08-17 15:54:55 +02:00
Julius Goryavsky
36832711c1 Reverting changes made to support the mtr under the root 2018-08-13 10:40:47 +02:00
Julius Goryavsky
7e8ed15b95 Fixes after review and correction of the problems caused by the fact that during the SST innodb plugin is not yet initialized, as well as problems with running tests from the root user (not directly related to the MDEV-10754). 2018-08-09 02:24:12 +00:00
Julius Goryavsky
6d5b71e02a Merge branch '10.1' of https://github.com/MariaDB/server into sysprg/10.1-MDEV-10754 2018-08-08 15:47:57 +02:00
Jan Lindström
b9f0112248
Merge pull request #609 from benrubson/stunnel
MDEV-15511 - Use stunnel during rsync SST if available
2018-08-05 09:27:13 +03:00
Vasil Dimov
3f0cd66a2b Also include InnoDB undo tablespaces in rsync sst
If --innodb-undo-tablespaces is used, then InnoDB stores undo in a
separate file(s) which whould also be replicated.

This fixes
Issue#337 This filter will cause sst failed at applying undo...
https://github.com/codership/mysql-wsrep/issues/337
2018-08-03 12:31:57 +03:00
Vasil Dimov
62e290923e Put one filter per line in wsrep_sst_rsync.sh
This way it is more readable and easy to change, also if a new entry is
added or one removed, the diff will be easier to read.
2018-08-03 12:31:25 +03:00
Jan Lindström
c5a8583b31 Merge tag 'mariadb-10.0.36' into 10.0-galera 2018-08-02 11:44:02 +03:00
Julius Goryavsky
46d5e1f2fd MDEV-10754 wsrep_sst_rsync does not support innodb_data_home_dir
If innodb_data_home_dir path specified in the configuration file
then rsync IST/SST will fail because the wsrep_sst_rsync.sh script
does not read this parameter from the configuration file and then
tries to find the data files in the default directory.

To fix this error, we need to add reading of the innodb_data_home_dir
configuration parameter to the rsync-related SST script.
2018-07-26 15:44:28 +02:00
Oleksandr Byelkin
cb5952b506 Merge branch '10.0' into bb-10.1-merge-sanja 2018-07-25 22:24:40 +02:00
Oleksandr Byelkin
304440b014 Merge branch '5.5' into bb-10.0-merge-sanja 2018-07-23 11:55:18 +02:00
Oleksandr Byelkin
0896d7ebc3 Merge branch '10.0' into bb-10.1-merge 2018-07-19 12:55:54 +02:00
Sergei Golubchik
5c744bb535 MDEV-14882 mysql_upgrade performs unnecessary conversions back and forth
avoid round-robin conversions, if the column is MODIFY-ed,
it should always be modified to its final definition, not to some
intermediate state.

also avoid other unconditional changes, like
  ALTER TABLE event DROP PRIMARY KEY;
  ALTER TABLE event ADD PRIMARY KEY(db, name);
2018-07-19 11:35:39 +02:00
Sergei Golubchik
33eccb5776 MDEV-11790 WITHOUT_SERVER installs mysqld_safe_helper
Don't install server files if WITHOUT_SERVER is specified.
"Server files" are defined as files going into the MariaDB-Server RPM,
that is files in the components Server, ManPagesServer, Server_Scripts,
IniFiles, SuportFiles, and Readme.
2018-07-19 11:35:38 +02:00
Jan Lindström
71e0ba4ae6
Merge pull request #645 from grooverdan/10.0-wsrep_sst_common_bashism
MDEV-15549 - wsrep_sst_common: fix per shellcheck
2018-07-19 07:04:40 +03:00
Jan Lindström
312de43f40
Merge pull request #786 from codership/10.1-MDEV-14612
MDEV-14612 wsrep_sst_mariabackup unnecessarily converts address to host name
2018-07-18 10:25:35 +03:00
Oleksandr Byelkin
e5c26fdfab Merge branch '5.5' into bb-10.0-merge 2018-07-17 16:56:21 +02:00
Daniel Black
8c45eb3ea5 MDEV-15050 scripts: mysql_install_db.{sh|pl}, mysqld_multi - mysqld is in @sbindir@
Closes #551
2018-07-16 18:36:28 +02:00
Michael Widenius
faef2e6a44 Added more help text in case mysql_install_db fails. 2018-06-27 13:41:19 +03:00
mkaruza
7a7a61998c MDEV-15442 xtrabackup-v2 SST donor stuck in DONOR/DESYNCED state when joiner is killed
SST transfer scripts rsync and maribackup now monitor parent process and will cleanup if process dies
2018-06-20 13:23:38 +02:00
mkaruza
e5c6580178 MDEV-15442 xtrabackup-v2 SST donor stuck in DONOR/DESYNCED state when joiner is killed
Monitor mysqld process and if it is terminated during sst stream kill xtrabackup-v2 script and children
2018-06-19 16:09:31 +02:00
Sergei Golubchik
7fdb7d4058 more sst test failures
* xtrabackup no longer support --compact
* wsrep_sst_mysqldump wasn't always using --default-file
* wsrep_sst.cc overquoted --default-file for wsrep_sst_mysqldump

also remove redundant lines config and test lines, compiler warnings,
and mark big tests as big.
2018-06-15 22:32:07 +02:00
Sergei Golubchik
cf648afd5b fix galera sst tests
note that ${A#foo} is $A if there's no prefix foo. That's why
galera nodes tried to connect to 127.0.0.1:127.0.0.1 if there was
no port in the address

Followup for 2b35db5ac4
2018-06-15 14:28:08 +03:00
Sergei Golubchik
c22ab56f0d fix galera sst tests
note that ${A#foo} is $A if there's no prefix foo. That's why
galera nodes tried to connect to 127.0.0.1:127.0.0.1 if there was
no port in the address

Followup for 2b35db5ac4
2018-06-14 18:06:52 +02:00
Vicențiu Ciorbaru
6e55236c0a Merge branch '10.0-galera' into 10.1 2018-06-12 19:39:37 +03:00
sjaakola
15c6d6a94a MDEV-14612 wsrep_sst_mariabackup unnecessarily converts address to host name
SST script for mariabackup (wsrep_sst_mariabackup.sh) does reverse DNS to convert
the joiner's IP address to domain name, and this is not always a wanted feature.

It is not understood why this remote host DNS conversion is in the script,
in the first place, but conversion was originally added by commit:

commit 3d8aacba86
Author: Nirbhay Choubey <nirbhay.choubey@gmail.com>
Date:   Wed Feb 22 15:58:45 2017 -0500

In this commit, REMOTEHOST variable is removed and replaced by REMOTEIP. All
references to joiner is therefore by IP address only.
2018-06-07 17:12:38 +03:00
Jan Lindström
215d652c66 MDEV-15351: wsrep_sst_xtrabackup is broken in 10.1.31
Remove the setup_ports function call. This is related to
https://github.com/MariaDB/server/pull/717

Thanks to Daniel Black and Bart S.
2018-05-14 10:36:55 +03:00
Daniel Black
82f26dafcb MDEV-13968: wsrep_log_error not defined until later in wsrep_sst_common
Fix for 990c4318a7

dash -vx './wsrep_sst_common.sh' --address '128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2' --port 1111  2>&1  | more
....
+ readonly WSREP_SST_OPT_ADDR=128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_HOST=128.0.0.5
+ readonly WSREP_SST_OPT_HOST_UNESCAPED=128.0.0.5
+ remain=3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_ADDR_PORT=3333
+ remain=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_MODULE=module
+ readonly WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_LSN=lsn_version
+ remain=sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_SST_VER=sst_ver
+ shift
+ shift
+ [ 2 -gt 0 ]
+ readonly WSREP_SST_OPT_PORT=1111
+ shift
+ shift
+ [ 0 -gt 0 ]
readonly WSREP_SST_OPT_BYPASS
+ readonly WSREP_SST_OPT_BYPASS
readonly WSREP_SST_OPT_BINLOG
+ readonly WSREP_SST_OPT_BINLOG

if [ -n "${WSREP_SST_OPT_ADDR_PORT:-}" ]; then
  if [ -n "${WSREP_SST_OPT_PORT:-}" ]; then
    if [ "$WSREP_SST_OPT_PORT" != "$WSREP_SST_OPT_ADDR_PORT" ]; then
      echo "WSREP_SST: [ERROR] port in --port=$WSREP_SST_OPT_PORT differs from port in --address=$WSREP_SST_OPT_ADDR" >&2
      exit 2
    fi
  else
    readonly WSREP_SST_OPT_PORT="$WSREP_SST_OPT_ADDR_PORT"
  fi
fi
+ [ -n 3333 ]
+ [ -n 1111 ]
+ [ 1111 != 3333 ]
+ echo WSREP_SST: [ERROR] port in --port=1111 differs from port in --address=128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST: [ERROR] port in --port=1111 differs from port in --address=128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ exit 2

dash -vx './wsrep_sst_common.sh' --address '128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2'  2>&1  | more
...
+ [ 2 -gt 0 ]
+ readonly WSREP_SST_OPT_ADDR=128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_HOST=128.0.0.5
+ readonly WSREP_SST_OPT_HOST_UNESCAPED=128.0.0.5
+ remain=3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_ADDR_PORT=3333
+ remain=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_MODULE=module
+ readonly WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_LSN=lsn_version
+ remain=sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_SST_VER=sst_ver
+ shift
+ shift
+ [ 0 -gt 0 ]
readonly WSREP_SST_OPT_BYPASS
+ readonly WSREP_SST_OPT_BYPASS
readonly WSREP_SST_OPT_BINLOG
+ readonly WSREP_SST_OPT_BINLOG

if [ -n "${WSREP_SST_OPT_ADDR_PORT:-}" ]; then
  if [ -n "${WSREP_SST_OPT_PORT:-}" ]; then
    if [ "$WSREP_SST_OPT_PORT" != "$WSREP_SST_OPT_ADDR_PORT" ]; then
      echo "WSREP_SST: [ERROR] port in --port=$WSREP_SST_OPT_PORT differs from port in --address=$WSREP_SST_OPT_ADDR" >&2
      exit 2
    fi
  else
    readonly WSREP_SST_OPT_PORT="$WSREP_SST_OPT_ADDR_PORT"
  fi
fi
+ [ -n 3333 ]
+ [ -n  ]
+ readonly WSREP_SST_OPT_PORT=3333
2018-05-14 10:36:55 +03:00
Sergei Golubchik
fe3c4a4182 MDEV-13968 sst fails with "WSREP_SST_OPT_PORT: readonly variable"
followup for 990c4318a7

fix broken script (set -u)
2018-05-14 10:36:55 +03:00
Sergei Golubchik
e6b31df6df MDEV-13968 sst fails with "WSREP_SST_OPT_PORT: readonly variable" 2018-05-14 10:36:55 +03:00
Daniel Black
24ab82e675 MDEV-15541: wsrep_sst_common - WSREP_SST_OPT_PORT set twice (--address and --port) 2018-05-14 10:36:55 +03:00
Daniel Black
2b35db5ac4 MDEV-15496: wsrep_sst_common - parse IPv6 correct
Fix to e78e308e81, IPv6 got split at :

Also got rid of awk (yuck!) to parse port (commit: b4fbb4183b)

Tested with dash to avoid bashisms:

$ dash -vx './wsrep_sst_common.sh' --address '128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2' 2>&1  | more
+ readonly WSREP_SST_OPT_ADDR=128.0.0.5:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_HOST=128.0.0.5
+ readonly WSREP_SST_OPT_HOST_UNESCAPED=128.0.0.5
+ remain=3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_PORT=3333
+ remain=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_MODULE=module
+ readonly WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_LSN=lsn_version
+ remain=sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_SST_VER=sst_ver

$ dash -vx './wsrep_sst_common.sh' --address '[2001:db8::1]:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2' 2>&1  | more
+ readonly WSREP_SST_OPT_ADDR=[2001:db8::1]:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ addr_no_bracket=2001:db8::1]:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_HOST_UNESCAPED=2001:db8::1
+ readonly WSREP_SST_OPT_HOST=[2001:db8::1]
+ remain=:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_PORT=3333
+ remain=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_MODULE=module
+ readonly WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_LSN=lsn_version
+ remain=sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_SST_VER=sst_ver

And bash as well:

$ bash -vx './wsrep_sst_common.sh' --address '127.0.0.3:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2' 2>&1 | more
+ readonly WSREP_SST_OPT_ADDR=127.0.0.3:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ WSREP_SST_OPT_ADDR=127.0.0.3:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ case "${WSREP_SST_OPT_ADDR}" in
+ readonly WSREP_SST_OPT_HOST=127.0.0.3
+ WSREP_SST_OPT_HOST=127.0.0.3
+ readonly WSREP_SST_OPT_HOST_UNESCAPED=127.0.0.3
+ WSREP_SST_OPT_HOST_UNESCAPED=127.0.0.3
+ remain=3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_PORT=3333
+ WSREP_SST_OPT_PORT=3333
+ remain=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_MODULE=module
+ WSREP_SST_OPT_MODULE=module
+ readonly WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_LSN=lsn_version
+ WSREP_SST_OPT_LSN=lsn_version
+ remain=sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_SST_VER=sst_ver
+ WSREP_SST_OPT_SST_VER=sst_ver

$ bash -vx './wsrep_sst_common.sh' --address '[2001:db8::1]:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2' 2>&1 | more
+ readonly 'WSREP_SST_OPT_ADDR=[2001:db8::1]:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2'
+ WSREP_SST_OPT_ADDR='[2001:db8::1]:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2'
+ case "${WSREP_SST_OPT_ADDR}" in
+ addr_no_bracket='2001:db8::1]:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2'
+ readonly WSREP_SST_OPT_HOST_UNESCAPED=2001:db8::1
+ WSREP_SST_OPT_HOST_UNESCAPED=2001:db8::1
+ readonly 'WSREP_SST_OPT_HOST=[2001:db8::1]'
+ WSREP_SST_OPT_HOST='[2001:db8::1]'
+ remain=:3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=3333/module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_PORT=3333
+ WSREP_SST_OPT_PORT=3333
+ remain=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_MODULE=module
+ WSREP_SST_OPT_MODULE=module
+ readonly WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
+ remain=lsn_version/sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_LSN=lsn_version
+ WSREP_SST_OPT_LSN=lsn_version
+ remain=sst_ver/some_future_option1/some_future_option2
+ readonly WSREP_SST_OPT_SST_VER=sst_ver
+ WSREP_SST_OPT_SST_VER=sst_ver
2018-05-14 10:36:55 +03:00
Jan Lindström
b2fc197b56 MDEV-15351: wsrep_sst_xtrabackup is broken in 10.1.31
Remove the setup_ports function call. This is related to
https://github.com/MariaDB/server/pull/717

Thanks to Daniel Black and Bart S.
2018-05-09 09:16:20 +03:00
Jan Lindström
648cf7176c Merge remote-tracking branch 'origin/5.5-galera' into 10.0-galera 2018-05-07 13:49:14 +03:00
Jan Lindström
231c02f7b9 MariaDB adjustments. 2018-04-24 13:58:42 +03:00
Jan Lindström
a5001a2ad7 Merge tag 'mariadb-5.5.60' into 5.5-galera 2018-04-24 13:34:57 +03:00
Jan Lindström
7b11518198
Merge pull request #557 from grooverdan/10.0-galera-wsrep_sst_mysqldump-safety
MDEV-15023 - wsrep_sst_mysqldump: safer test of version != 5
2018-04-06 07:31:44 +03:00
Sergei Golubchik
d3681c18f9 followup for 89b0d5cb6e, backport 8c422bf48d 2018-03-23 17:40:09 +01:00
Sergei Golubchik
8f1014e9a0 MDEV-15409 make sure every sst script is tested in buildbot
fix galera.galera_sst_mysqldump test to work:
* must connect to 127.0.0.1, where mysqld is listening
* disable wsrep_sync_wait in wsrep_sst_mysqldump, otherwise
  sst can deadlock
* allow 127.0.0.1 for bind_address and wsrep_sst_receive_address.
  (it's useful in tests, or when two nodes are on the same box,
  or when nodes are on different boxes, but the connection is
  tunelled, or whatever. Don't judge user's setup). MDEV-14070
* don't wait for client connections to die when doing
  mysqldump sst. they'll die in a due time, and if needed mysql
  will wait on locks until they do. MDEV-14069

Also don't mark it big, to make sure it's sufficiently tested
2018-03-23 00:55:20 +01:00
Sergei Golubchik
a15ab358fc wsrep_sst scripts: support traditional netcat 2018-03-23 00:55:19 +01:00
Sergei Golubchik
89b0d5cb6e MDEV-13968 sst fails with "WSREP_SST_OPT_PORT: readonly variable"
Backport from 10.2: 4c2c057d40 and f7090df712
2018-03-23 00:55:19 +01:00