Commit graph

4140 commits

Author SHA1 Message Date
Sergei Golubchik
b1818dccf7 Merge branch '10.2' into 10.3 2018-03-28 17:31:57 +02:00
Sergei Golubchik
c764bc0a78 Merge branch '10.1' into 10.2 2018-03-25 13:02:52 +02:00
Sergei Golubchik
d3681c18f9 followup for 89b0d5cb6e, backport 8c422bf48d 2018-03-23 17:40:09 +01:00
Sergei Golubchik
8f1014e9a0 MDEV-15409 make sure every sst script is tested in buildbot
fix galera.galera_sst_mysqldump test to work:
* must connect to 127.0.0.1, where mysqld is listening
* disable wsrep_sync_wait in wsrep_sst_mysqldump, otherwise
  sst can deadlock
* allow 127.0.0.1 for bind_address and wsrep_sst_receive_address.
  (it's useful in tests, or when two nodes are on the same box,
  or when nodes are on different boxes, but the connection is
  tunelled, or whatever. Don't judge user's setup). MDEV-14070
* don't wait for client connections to die when doing
  mysqldump sst. they'll die in a due time, and if needed mysql
  will wait on locks until they do. MDEV-14069

Also don't mark it big, to make sure it's sufficiently tested
2018-03-23 00:55:20 +01:00
Sergei Golubchik
a15ab358fc wsrep_sst scripts: support traditional netcat 2018-03-23 00:55:19 +01:00
Sergei Golubchik
89b0d5cb6e MDEV-13968 sst fails with "WSREP_SST_OPT_PORT: readonly variable"
Backport from 10.2: 4c2c057d40 and f7090df712
2018-03-23 00:55:19 +01:00
Jan Lindström
85ddd9e8ce Revert pull request #640 from grooverdan/10.3-MDEV-8743-wsrep_sst_common-close-fds 2018-03-17 12:56:23 +02:00
Daniel Black
04ed70fa55 MDEV-8743: wsrep_sst_common close FDs > 2
To prevent accidential overwriting of files due to mysqld file
descriptors being open we ensure where that only FDs 1, 2 and 3
are available to SST scripts. Assumed is mysqld has thse still
mapped to stdin (unused), stdout and stderr.
2018-03-08 18:55:01 +11:00
Alexander Barkov
583eb96c24 MDEV-11952 Oracle-style packages: stage#5
- CREATE PACKAGE [BODY] statements are now
  entirely written to mysql.proc with type='PACKAGE' and type='PACKAGE BODY'.
- CREATE PACKAGE BODY now supports IF NOT EXISTS
- DROP PACKAGE BODY now supports IF EXISTS
- CREATE OR REPLACE PACKAGE [BODY] is now supported
- CREATE PACKAGE [BODY] now support the DEFINER clause:

    CREATE DEFINER user@host PACKAGE pkg ... END;
    CREATE DEFINER user@host PACKAGE BODY pkg ... END;

- CREATE PACKAGE [BODY] now supports SQL SECURITY and COMMENT clauses, e.g.:

    CREATE PACKAGE p1 SQL SECURITY INVOKER COMMENT "comment" AS ... END;

- Package routines are now created from the package CREATE PACKAGE BODY
  statement and don't produce individual records in mysql.proc.

- CREATE PACKAGE BODY now supports package-wide variables.
  Package variables can be read and set inside package routines.
  Package variables are stored in a separate sp_rcontext,
  which is cached in THD on the first packate routine call.

- CREATE PACKAGE BODY now supports the initialization section.

- All public routines (i.e. declared in CREATE PACKAGE)
  must have implementations in CREATE PACKAGE BODY

- Only public package routines are available outside of the package

- {CREATE|DROP} PACKAGE [BODY] now respects CREATE ROUTINE and ALTER ROUTINE
  privileges

- "GRANT EXECUTE ON PACKAGE BODY pkg" is now supported

- SHOW CREATE PACKAGE [BODY] is now supported

- SHOW PACKAGE [BODY] STATUS is now supported

- CREATE and DROP for PACKAGE [BODY] now works for non-current databases

- mysqldump now supports packages

- "SHOW {PROCEDURE|FUNCTION) CODE pkg.routine" now works for package routines

- "SHOW PACKAGE BODY CODE pkg" now works (the package initialization section)

- A new package body level MDL was added

- Recursive calls for package procedures are now possible

- Routine forward declarations in CREATE PACKATE BODY are now supported.

- Package body variables now work as SP OUT parameters

- Package body variables now work as SELECT INTO targets

- Package body variables now support ROW, %ROWTYPE, %TYPE
2018-02-25 21:08:19 +04:00
Sergei Golubchik
2732fcc608 Merge branch 'bb-10.2-ext' into 10.3 2018-02-23 08:43:34 +01:00
Sergei Golubchik
b8af22af15 Merge branch '10.2' into bb-10.2-ext 2018-02-22 19:29:52 +01:00
Sergei Golubchik
2daa005800 Merge branch '10.1' into 10.2 2018-02-22 08:39:24 +01:00
Daniel Black
4e6dab94d0 MDEV-10.1.31 does not join an existing cluster with SST xtrabackup-v2
Analysis:- The problem  is the change in the implementation of wait_for_listen
in wsrep_sst_xtrabackup-v2.sh. The new script uses lsof which will always
exit with an error code if it can't find all the items, and because the
script has the -e option set in the hashbang line (#!/bin/bash -ue), the
script will abort right after running lsof if lsof can't find even a single
item among all the items listed in its arguments. This will happen even if
socat is running and listening, because it can't find nc. The loop in
wait_for_listen will therefore always quit after one iteration without
writing the "ready" line to signal the parent.

Solution:- We will or the lsof with true.

Patch Credit :Daniel Black and David Wang
2018-02-21 19:48:18 +05:30
Marko Mäkelä
b006d2ead4 Merge bb-10.2-ext into 10.3 2018-02-15 10:22:03 +02:00
Sergei Golubchik
f7621f17bd rename mysql.user and mysql.db column
Truncate_versioning_priv->Delete_history_priv

because the command and the privilege were renamed
2018-02-12 23:43:48 +01:00
Jerome Brauge
d943d7f712 MDEV-13417 UPDATE produces wrong values if an updated column is later used as an update source
Standard compatible behavior for UPDATE: all assignments in SET
are executed "simultaneously", not left-to-right. And `SET a=b,b=a`
will swap the values.
2018-02-12 13:14:23 +01:00
Sergei Golubchik
355ee6877b MDEV-13946 Server RPMs have dependency on "which"
cleanup. use "command -v" instead of "which"
simplify some checks.
2018-02-12 12:46:34 +01:00
Alexander Barkov
3cad31f2a7 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext 2018-02-08 19:06:25 +04:00
Monty
6ba06cf763 On upgrade Truncate_versioning_privilege was not correct set
Fixed that Truncate_versioning_privilege works as any other privilege
during upgrade:

- If the privilege field does not exists, add it to the user and db tables.
  If the user had super_privilege then the user will also get the new
  Truncate_versioning_privilege.

This is done to ensure that if one has GRANT ALL PRIVILEGE before, one
will continue to have it after running mysql_upgrade.

This also fixes a bug where the Truncate_versioning_privilege
2018-02-07 02:39:40 +02:00
Sergei Golubchik
b4fbb4183b remove bash-ish from SST scripts
replace with awk (yuck!), avoiding gensub (which wasn't recognized)
2018-02-06 14:52:01 +01:00
Sergei Golubchik
4771ae4b22 Merge branch 'github/10.1' into 10.2 2018-02-06 14:50:50 +01:00
Sergei Golubchik
c383418cbf Merge branch 'github/10.0-galera' into 10.1 2018-02-03 08:09:06 +01:00
Sergei Golubchik
d4df7bc9b1 Merge branch 'github/10.0' into 10.1 2018-02-02 10:09:44 +01:00
Jan Lindström
c7e5feb259 Merge tag 'mariadb-10.0.34' into 10.0-galera
Conflicts:
	storage/innobase/lock/lock0lock.cc
	storage/xtradb/lock/lock0lock.cc
	storage/xtradb/lock/lock0wait.cc
	support-files/mysql.server.sh
2018-02-01 14:09:48 +02:00
Michal Schorm
fb24eb87a8 MDEV-12301: galera_recovery use @sbindir@ macro for mysqld 2018-01-30 21:28:16 +01:00
Vicențiu Ciorbaru
d833bb65d5 Merge remote-tracking branch '5.5' into 10.0 2018-01-24 12:29:31 +02:00
Jan Lindström
72542ffd77
Merge pull request #550 from grooverdan/10.1-wsrep_sst_mysqldump-no-max-version
MDEV-15045 - wsrep_sst_mysqldump: enforce a minimum version only
2018-01-24 10:31:00 +02:00
Daniel Black
9ee372736f mysql_install_db: correct hosting/source/maillist information 2018-01-23 15:03:30 +04:00
Daniel Black
c98906e4fe mysql_install_db: correct --skip-grant-tables help 2018-01-23 15:03:30 +04:00
Daniel Black
be83785d73 MDEV-13789: FreeBSD wsrep_sst_xtrabackup-v2 - find compatibilty +lsof
Based off #451 by angeloudy <angeloudy@yahoo.com> and
Tao Zhou <taozhou@ip-179.ish.com.au>
2018-01-18 22:06:17 +11:00
Daniel Black
c73b4b1698 wsrep_sst_mysqldump: enforce a minimum version only 2018-01-17 20:16:59 +11:00
Daniel Black
c4b7074e72 wsrep_sst_xtrabackup*: use mysqld defaults arguments
It was assumed in
https://bugs.launchpad.net/codership-mysql/+bug/1378355 that xtrabackup
didn't have the groups-defaults-suffix because it wasn't listed in the
help output.

Also --defaults-group was always just an additional group despite the
misleading description implying it was a replacement.

So actually we can use the default* mysqld arguements to xtrabackup as
the standard mysqld configuration parser was used.
2018-01-16 14:44:39 +11:00
Daniel Black
a2a038152e wsrep_sst_xtrabackup*: use wsrep_sst_common parsed vars 2018-01-14 23:31:14 +11:00
Daniel Black
51ea696c8e wsrep_sst_common: keep WSREP_SST_OPT_HOST_UNESCAPED for IPv4
change wsrep_sst_mysqldump.sh to use WSREP_SST_OPT_HOST_UNESCAPED
directly.
2018-01-14 23:31:14 +11:00
Daniel Black
722df90534 wsrep_sst_xtrabackup*: read all sections of config not nust mysqld
--mysqld parsed to parse_cnf aka my_print_defaults will parse all
sections which the server read and not just mysqld.
2018-01-14 23:31:13 +11:00
Daniel Black
cc8abb21e3 wsrep_sst_xtrabackup*: du -s removed lessens output
and only displays the summary. As this is the only bit
that is used there will be less du,kernel,awk CPU usage.
2018-01-14 23:31:13 +11:00
Daniel Black
e78e308e81 wsrep_sst_common: parse --address and split WSREP_SST_OPT_PATH
Use the string functionality of sh to split the address:

IPv6 test:
. scripts/wsrep_sst_common.sh --address [ipv6]:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
$ set | grep WSREP_SST
WSREP_SST_OPT_ADDR='[ipv6]:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2'
WSREP_SST_OPT_AUTH=
WSREP_SST_OPT_BINLOG=
WSREP_SST_OPT_BYPASS=0
WSREP_SST_OPT_CONF='  '
WSREP_SST_OPT_DATA=
WSREP_SST_OPT_DEFAULT=
WSREP_SST_OPT_EXTRA_DEFAULT=
WSREP_SST_OPT_HOST='[ipv6]'
WSREP_SST_OPT_HOST_UNESCAPED=ipv6
WSREP_SST_OPT_LSN=lsn_version
WSREP_SST_OPT_MODULE=module
WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_PORT=port
WSREP_SST_OPT_PSWD=
WSREP_SST_OPT_SST_VER=sst_ver
WSREP_SST_OPT_SUFFIX_DEFAULT=
WSREP_SST_OPT_USER=

IPv4 test:
. scripts/wsrep_sst_common.sh --address ipv4:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
$ set | grep WSREP_SST
WSREP_SST_OPT_ADDR=ipv4:port/module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_AUTH=
WSREP_SST_OPT_BINLOG=
WSREP_SST_OPT_BYPASS=0
WSREP_SST_OPT_CONF='  '
WSREP_SST_OPT_DATA=
WSREP_SST_OPT_DEFAULT=
WSREP_SST_OPT_EXTRA_DEFAULT=
WSREP_SST_OPT_HOST=ipv4
WSREP_SST_OPT_LSN=lsn_version
WSREP_SST_OPT_MODULE=module
WSREP_SST_OPT_PATH=module/lsn_version/sst_ver/some_future_option1/some_future_option2
WSREP_SST_OPT_PORT=port
WSREP_SST_OPT_PSWD=
WSREP_SST_OPT_SST_VER=sst_ver
WSREP_SST_OPT_SUFFIX_DEFAULT=
WSREP_SST_OPT_USER=
2018-01-14 23:31:13 +11:00
Daniel Black
95e5fe6732 wsrep_sst_common: parse_cnf - use awk rather than grep/cut/tail excessiveness
Test cases:

f($var) = awk -v var="${var}"  \
     'BEGIN { OFS=FS="="}
     { gsub(/_/,"-",$1); if ( $1=="--"var ) lastval=substr($0,length($1)+2) }
     END { print lastval }'

Missing input is blank:

$ echo '--var_aa=something' | f(var-b-not-ther)
(blank as expected)

All RHS of = is unmunged:

$ echo '--var_aa=password==_-$' | f(var-aa)
password==_-$

Mixed - and _ in var name:
$ echo '--var_aa-bb_cc=1' | f(var-aa-bb-cc)
1

No value returns blank line:
$ echo '--var_aa-bb_cc' | f(var-aa-bb-cc)
(blank line as expected)

Multiples return the last:
$  echo -e "--bb=cc\n--bb=dd" | f(bb)
dd
2018-01-14 23:31:01 +11:00
Sergei Golubchik
943c62a5d4 Backport 4bb49d84a9, correct handling on defaults[-extra]-file is SST scripts
(from 10.1 to 10.0-galera)

This conflicted signficantly with 7d550c76be
which added --defaults-group-suffix support.

Took the approach of 4bb49d84a9 and adapted the
--defaults-group-suffix handling to be consistent.

The following changes as follows:

SST scripts now use $MY_PRINT_DEFAULTS rather than the lowercase for
consistency and this include all required --default arguements.

Backport/merge by Daniel Black <daniel@linux.vnet.ibm.com>
2018-01-14 22:48:02 +11:00
Sergey Vojtovich
b75d767689 Fixed mysql_install_db --no-defaults
Regression after 5ea2801.
2018-01-13 13:05:09 +04:00
Daniel Black
5ea28015d5 mysql_install_db: Use --defaults-group-suffix if specified
Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>
2018-01-12 12:09:06 +04:00
Aleksey Midenkov
c59c1a0736 System Versioning 1.0 pre8
Merge branch '10.3' into trunk
2018-01-10 12:36:55 +03:00
Marko Mäkelä
145ae15a33 Merge bb-10.2-ext into 10.3 2018-01-04 09:22:59 +02:00
Monty
fbab79c9b8 Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext
Conflicts:
	cmake/make_dist.cmake.in
	mysql-test/r/func_json.result
	mysql-test/r/ps.result
	mysql-test/t/func_json.test
	mysql-test/t/ps.test
	sql/item_cmpfunc.h
2018-01-01 19:39:59 +02:00
Vicențiu Ciorbaru
985d2d393c Merge remote-tracking branch 'origin/10.1' into 10.2 2017-12-22 12:23:39 +02:00
Sachin Setiya
2a4faa8ab6 MDEV-13478 Full SST sync fails because of the error in the cleaning part
Problem:
 The command was:
    find $paths -mindepth 1 -regex $cpat -prune -o -exec rm -rf {} \+
 Which was supposed to work as
    * skipping $paths directories themselves (-mindepth 1)
    * see if the dir/file name matches $cpat (-regex)
    * if yes - don't dive into the directory, skip it (-prune)
    * otherwise (-o)
    * remove it and everything inside (-exec)
 Now -exec ... \+ works like this:
    every new found path is appended to the end of the command line.
    when accumulated command line length reaches `getconf ARG_MAX` (~2Gb)
    it's executed, and find continues, appending to a new command line.

 What happens here, find appends some directory to the command line,
 then dives into it, and starts appending files from that directory.
 At some point command line overflows, rm -rf gets executed and removes
 the whole directory. Now find tries to continue scanning the directory
 that was already removed.

Fix: don't dive into directories that will be recursively removed
anyway, use -prune for them. Basically, we should be pruning both paths
that have matched $cpat and paths that have not matched it. This is
achived by pruning unconditionally, before the regex is tested:
    find $paths -mindepth 1 -prune -regex $cpat -o -exec rm -rf {} \+

Patch Credit:- Serg
2017-12-20 18:51:30 +01:00
Vicențiu Ciorbaru
e3d89652e5 Merge branch '10.0' into 10.1 2017-12-20 13:30:05 +02:00
Vicențiu Ciorbaru
042f763268 Merge remote-tracking branch '5.5' into 10.0 2017-12-20 12:51:57 +02:00
Sergei Golubchik
beabe6b216 MDEV-13969 sst mysqldump and xtrabackup-v2 handle WSREP_SST_OPT_CONF incorrectly
$WSREP_SST_OPT_CONF already includes --defaults-extra-file= prefix.
2017-12-18 16:41:40 +01:00
Martynas Bendorius
1c2f59f7fb MDEV-13969 sst mysqldump and xtrabackup-v2 handle WSREP_SST_OPT_CONF incorrectly
wrep_sst_common: Setting "-c ''" for my_print_defaults just takes no values from config at all. $MY_PRINT_DEFAULTS is already set at the top of the script to have --defaults-file and --defaults-extra-file. If WSREP_SST_OPT_CONF if set to "--defaults-file=/etc/my.cnf --defaults-extra-file=/etc/my.extra.cnf", then "my_print_defaults -c "" --defaults-file=/etc/my.cnf" succeeds, but if WSREP_SST_OPT_CONF is empty - no default values are taken at all.
wsrep_sst_xtrabackup-v2: innobackupex does not support --defaults-extra-file, so ${WSREP_SST_OPT_CONF} cannot be used as an argument, it has been changed to ${WSREP_SST_OPT_DEFAULT}. Removed --defaults-file= from INNOMOVE line, because WSREP_SST_OPT_CONF already includes it (INNOBACKUP was fine, INNOMOVE - not).
2017-12-18 16:41:40 +01:00