Problem:
The command was:
find $paths -mindepth 1 -regex $cpat -prune -o -exec rm -rf {} \+
Which was supposed to work as
* skipping $paths directories themselves (-mindepth 1)
* see if the dir/file name matches $cpat (-regex)
* if yes - don't dive into the directory, skip it (-prune)
* otherwise (-o)
* remove it and everything inside (-exec)
Now -exec ... \+ works like this:
every new found path is appended to the end of the command line.
when accumulated command line length reaches `getconf ARG_MAX` (~2Gb)
it's executed, and find continues, appending to a new command line.
What happens here, find appends some directory to the command line,
then dives into it, and starts appending files from that directory.
At some point command line overflows, rm -rf gets executed and removes
the whole directory. Now find tries to continue scanning the directory
that was already removed.
Fix: don't dive into directories that will be recursively removed
anyway, use -prune for them. Basically, we should be pruning both paths
that have matched $cpat and paths that have not matched it. This is
achived by pruning unconditionally, before the regex is tested:
find $paths -mindepth 1 -prune -regex $cpat -o -exec rm -rf {} \+
Patch Credit:- Serg
Problem:
The command was:
find $paths -mindepth 1 -regex $cpat -prune -o -exec rm -rf {} \+
Which was supposed to work as
* skipping $paths directories themselves (-mindepth 1)
* see if the dir/file name matches $cpat (-regex)
* if yes - don't dive into the directory, skip it (-prune)
* otherwise (-o)
* remove it and everything inside (-exec)
Now -exec ... \+ works like this:
every new found path is appended to the end of the command line.
when accumulated command line length reaches `getconf ARG_MAX` (~2Gb)
it's executed, and find continues, appending to a new command line.
What happens here, find appends some directory to the command line,
then dives into it, and starts appending files from that directory.
At some point command line overflows, rm -rf gets executed and removes
the whole directory. Now find tries to continue scanning the directory
that was already removed.
Fix: don't dive into directories that will be recursively removed
anyway, use -prune for them. Basically, we should be pruning both paths
that have matched $cpat and paths that have not matched it. This is
achived by pruning unconditionally, before the regex is tested:
find $paths -mindepth 1 -prune -regex $cpat -o -exec rm -rf {} \+
Patch Credit:- Serg
wrep_sst_common: Setting "-c ''" for my_print_defaults just takes no values from config at all. $MY_PRINT_DEFAULTS is already set at the top of the script to have --defaults-file and --defaults-extra-file. If WSREP_SST_OPT_CONF if set to "--defaults-file=/etc/my.cnf --defaults-extra-file=/etc/my.extra.cnf", then "my_print_defaults -c "" --defaults-file=/etc/my.cnf" succeeds, but if WSREP_SST_OPT_CONF is empty - no default values are taken at all.
wsrep_sst_xtrabackup-v2: innobackupex does not support --defaults-extra-file, so ${WSREP_SST_OPT_CONF} cannot be used as an argument, it has been changed to ${WSREP_SST_OPT_DEFAULT}. Removed --defaults-file= from INNOMOVE line, because WSREP_SST_OPT_CONF already includes it (INNOBACKUP was fine, INNOMOVE - not).
another followup for 4c2c057d40.
there are six possible cases:
--port can be set or not.
--address can be set, not set, or set but without a port number
The correct behavior is:
1 both --port and --address have a port number
- use it if it's the same, otherwise an error
2 only --port has the number (--address isn't set)
- use the value from --port
3 only --port has the number (--address is set, but has no port)
- use the value from --port
4 --port is unset, --address has the port number
- use the value from --address
5 --port is unset, --address has no port number
- use the value from --address, that is, port is empty string
6 --port is unset, --address is unset
- port is unset (an error somewhere later)
case 5 wasn't handled correctly
- Expand paths also for jemalloc
- Test also if tcmalloc or jemalloc is in /usr/lib64
- Take into account that .so has a version
- Remove automatic adding of flavors ( _minial, _debug). Better to
have user specify these directly
- Changed documentation link to MariaDB
- Don't give extra error if mysqld_safe_helper doesn't exist
always search in compile-time specified paths
INSTALL_BINDIR, INSTALL_SBINDIR, INSTALL_MYSQLSHAREDIR. User
can set them to arbitrary values, it's not enough to search only
in their usual values of bin, sbin and libexec, share and share/mysql.
1. detect resolveip location, don' assume it's in $basedir/bin
2. don't guess $scriptdir to (incorrectly) construct the $0 path
3. rename find_in_basedir -> find_in_dirs, don't prepend $basedir
automatically. This allows to use identical path lists in
find_in_dirs and in cannot_find_file.
4. move search path lists to CMakeLists.txt to avoid specifying the
same path list twice (in find_in_dirs and in cannot_find_file).
Fix for the following error messages during SST:
/usr/local/mysql/bin/wsrep_sst_rsync: 258: /usr/local/mysql/bin/wsrep_sst_rsync: [[: not found
/usr/local/mysql/bin/wsrep_sst_rsync: 263: /usr/local/mysql/bin/wsrep_sst_rsync: [[: not found
/usr/local/mysql/bin/wsrep_sst_rsync: 268: /usr/local/mysql/bin/wsrep_sst_rsync: [[: not found
This is regression caused by patch of mdev-10767.
1st problem :- mktmp is invoked without '-t' or specifing tmp directory.
2nd problem :- Since eval_log_error redirect stderr to stdout '2>' will
return nothing. and hence $wr_logfile will be empty.
Patch Credit:- Andrii Nikitin
wsrep_sst_xtrabackup-v2: innobackupex does not support --defaults-extra-file, so ${WSREP_SST_OPT_CONF} cannot be used as an argument, it has been changed to ${WSREP_SST_OPT_DEFAULT}. Removed --defaults-file= from INNOMOVE line, because WSREP_SST_OPT_CONF already includes it (INNOBACKUP was fine, INNOMOVE - not).
Problem:- To create file in /tmp dir mysqld require permission initrc_tmp_t.
And mysqld does not have his permission.
Solution:- Instead of giving mysqld permission of initrc_tmp_t , we redirected
log to file in /tmp dir through shell. I also removed a earlier workarround
in mysqld_safe.sh , which create tmp log file in datadir.
Corrects the following error with rsync sst on FreeBSD:
WSREP_SST: [ERROR] rsync daemon port '4444' has been taken
The FreeBSD version of grep doesn't play nicely with the -w parameter mixed with [[:space:]] and ends up matching nothing. This causes the script to loop and attempt to spawn more than one instance of rsync, which fails with the initial instance already having the port open. Because of this, the entire script fails and no sst occurs resulting in the error message listed above appearing in the log file.
For running the Galera tests, the variable my_disable_leak_check
was set to true in order to avoid assertions due to memory leaks
at shutdown.
Some adjustments due to MDEV-13625 (merge InnoDB tests from MySQL 5.6)
were performed. The most notable behaviour changes from 10.0 and 10.1
are the following:
* innodb.innodb-table-online: adjustments for the DROP COLUMN
behaviour change (MDEV-11114, MDEV-13613)
* innodb.innodb-index-online-fk: the removal of a (1,NULL) record
from the result; originally removed in MySQL 5.7 in the
Oracle Bug #16244691 fix
377774689b
* innodb.create-index-debug: disabled due to MDEV-13680
(the MySQL Bug #77497 fix was not merged from 5.6 to 5.7.10)
* innodb.innodb-alter-autoinc: MariaDB 10.2 behaves like MySQL 5.6/5.7,
while MariaDB 10.0 and 10.1 assign different values when
auto_increment_increment or auto_increment_offset are used.
Also MySQL 5.6/5.7 exhibit different behaviour between
LGORITHM=INPLACE and ALGORITHM=COPY, so something needs to be tested
and fixed in both MariaDB 10.0 and 10.2.
* innodb.innodb-wl5980-alter: disabled because it would trigger an
InnoDB assertion failure (MDEV-13668 may need additional effort in 10.2)
In summary, wsrep_node_address and wsrep_sst_receive_address can now
be set to IPv6 addresses escaped by []. Rsync SST works out ouf the
box thanks to rsync daemon listening on both IPv4 and IPv6 sockets by
default. For xtrabackup SST onver IPv6 one needs to set sockopt in
the [sst] section of joiner's configuration file to ",pf=ip6" if
using socat as a streamer or to "-6" if using netcat.
This happens because on line 59 of /usr/bin/wsrep_sst_mysqldump
we have a check :
"if ! $MYSQL_CLIENT --version | grep 'Distrib 10.1' >/dev/null"
but the client is 10.2 so it doesnt match the grep expression.
Fixed check to be:
"if ! $MYSQL_CLIENT --version | grep 'Distrib 10.' >/dev/null"
so that every 10. version is accepted.
InnoDB I/O and buffer pool interfaces and the redo log format
have been changed between MariaDB 10.1 and 10.2, and the backup
code has to be adjusted accordingly.
The code has been simplified, and many memory leaks have been fixed.
Instead of the file name xtrabackup_logfile, the file name ib_logfile0
is being used for the copy of the redo log. Unnecessary InnoDB startup and
shutdown and some unnecessary threads have been removed.
Some help was provided by Vladislav Vaintroub.
Parameters have been cleaned up and aligned with those of MariaDB 10.2.
The --dbug option has been added, so that in debug builds,
--dbug=d,ib_log can be specified to enable diagnostic messages
for processing redo log entries.
By default, innodb_doublewrite=OFF, so that --prepare works faster.
If more crash-safety for --prepare is needed, double buffering
can be enabled.
The parameter innodb_log_checksums=OFF can be used to ignore redo log
checksums in --backup.
Some messages have been cleaned up.
Unless --export is specified, Mariabackup will not deal with undo log.
The InnoDB mini-transaction redo log is not only about user-level
transactions; it is actually about mini-transactions. To avoid confusion,
call it the redo log, not transaction log.
We disable any undo log processing in --prepare.
Because MariaDB 10.2 supports indexed virtual columns, the
undo log processing would need to be able to evaluate virtual column
expressions. To reduce the amount of code dependencies, we will not
process any undo log in prepare.
This means that the --export option must be disabled for now.
This also means that the following options are redundant
and have been removed:
xtrabackup --apply-log-only
innobackupex --redo-only
In addition to disabling any undo log processing, we will disable any
further changes to data pages during --prepare, including the change
buffer merge. This means that restoring incremental backups should
reliably work even when change buffering is being used on the server.
Because of this, preparing a backup will not generate any further
redo log, and the redo log file can be safely deleted. (If the
--export option is enabled in the future, it must generate redo log
when processing undo logs and buffered changes.)
In --prepare, we cannot easily know if a partial backup was used,
especially when restoring a series of incremental backups. So, we
simply warn about any missing files, and ignore the redo log for them.
FIXME: Enable the --export option.
FIXME: Improve the handling of the MLOG_INDEX_LOAD record, and write
a test that initiates a backup while an ALGORITHM=INPLACE operation
is creating indexes or rebuilding a table. An error should be detected
when preparing the backup.
FIXME: In --incremental --prepare, xtrabackup_apply_delta() should
ensure that if FSP_SIZE is modified, the file size will be adjusted
accordingly.
Currently galera has capability of using delta transfer algorithm (rsync)
for SST. But for using delta transfer we have change/copy wsrep_sst_rsync
wsrep_sst_rsync_wan. This patch creates a symbolic link of
wsrep_sst_rsync_wan to wsrep_sst_rsync.
Backport to 5.5
Current MySQL builds, even on Pushbuild, are not reproducible; they return
different results depending on which directory they are built from (and
Pushbuild uses several different directories). This is because absolute paths
leak into debug information, and even worse, __FILE__. The latter moves code
around enough that we've actually seen sysbench changes on the order of 4% in
some tests.
CMake seemingly insists on using absolute paths, but we can insert our own
layer between CMake and GCC to relativize all paths. Also give the right flags
to get debug information reproducible and turn off build stamping. This makes
the mysqld build 100% bit-for-bit reproducible between runs on my machine,
even when run from different directories.
Also, include fixes by Vladislav Vaintroub to the
aws_key_management plugin. The AWS C++ SDK specifically depends on
OPENSSL_LIBRARIES, not generic SSL_LIBRARIES (such as YaSSL).
FROM THE CURRENT DIRECTORY
DESCRIPTION
===========
When 'mysqlaccess' tool is run, it reads (and executes) the
content of its configuration file 'mysqlaccess.conf' from
the current directory. This is not a recommended behaviour
as someone with ill intentions can insert malicious
instructions into this file which could be executed
whenever this tool is run.
ANALYSIS
========
The configuration file is presently looked for, in the
following folders (in given order):
1. Current directory
2. SYSCONFDIR //This gets expanded
3. /etc/
Owing to the reasons mentioned above, we should not permit
the file to be in the current directory. Since the other
two folders are assumed to be accessible only to authorized
people, the config file is safe to be read from there.
FIX
===
Modified the script so that it looks for the config file
now in the following two folders (in the given order):
1. SYSCONFDIR
2. /etc/
If it's absent from above locations but present in current
directory, an error is thrown asking the user to move the
file to one of the above locations and retry.
NOTE
====
The location paths and their precedence are not documented
for this tool. It needs to be noted as part of the
associated documentation.
mysqld_safe is working on real files, however passing these file paths
as is to mysqld as options gives different meaning when paths are
relative.
mysqld_safe uses current working directory as basedir for relative paths,
while mysqld uses $datadir as basedir.
Revoked executable bit from files that are not supposed to be executed directly.
Removed interpreted from files that are not supposed to be executed directly.
Added interpreter to files that are supposed to be executed directly.
Fatal error: mysql.user table is damaged or in unsupported 3.20 format
The problem stems from MySQL 5.7.6. According to MySQL documentation:
In MySQL 5.7.6, the Password column was removed and all credentials are
stored in the authentication_string column.
If opening a MySQL 5.7.6 (and up) datadir with MariaDB 10.2, the user table
appears corrupted. In order to fix this, the server must be started with
--skip-grant-tables and then a subsequent mysql_upgrade command must be
issued.
This patch updates the mysql_upgrade command to also add the removed
Password column. The password column is necessary, otherwise
the mysql_upgrade script fails due to the Event_scheduler not being able
to start, as it can't find Event_priv in the table where it ought to be.
MySQL's version has column position 28 (0 index) vs our datadir version
expects position 29.
Fix of Bug#25088048 caused paths to be relative, not absolute, this
proved to be problematic.
Fix is to still ignore current working directory, however switch to
using full path of basedir, which is set to parent directory of bin/
directory where mysqld_safe is located.
References to legacy tool mysql_print_defaults are removed, only
my_print_defaults is used these days.
This will also fix:
Bug#11745176 (11192) MYSQLD_SAFE ONLY EVALUATES --DEFAULTS-FILE OPTION WHEN IT IS THE FIRST OP
Bug#23013510 (80866) MYSQLD_SAFE SHOULD NOT SEARCH $MY_BASEDIR_VERSION/VAR AS DATADIR
Bug#25244898 (84173) MYSQLD_SAFE --NO-DEFAULTS & SILENTLY DOES NOT WORK ANY MORE
Bug#25261472 (84219) INITSCRIPT ERRORS WHEN LAUCHING MYSQLD_SAFE IN NON DEFAULT BASEDIR
Bug#25319392 (84263) MYSQL.SERVER (MYSQL SERVER STARTUP SCRIPT) CAN NOT WORK,AND EXPORT SOME ERROR.
Bug#25319457 MYSQLD_SAFE MIGHT FAIL IF $DATADIR HAS TRAILING /
Bug#25341981 MYSQLD_SAFE ASSUMES INCORRECT BASEDIR WHEN EXECUTED WITH ABSOLUTE PATH
Bug#25356221 (84427) MYSQLD_SAFE FAILS TO START WHEN USING A FIFO FOR LOG-ERROR (REGRESSION)
Bug#25365194 (84447) MYSQLD_SAFE DOESN'T CHECK EXISTENCE OF GIVEN BASEDIR PARAMETER
Bug#25377815 ERRORS WHILE STARTING MYSQLD_SAFE WITH SYM LINK ENABLED
In Debian, the default install is made more secure by omitting the anonymous
user and by making the root account authenticate by unix socket
authentication instead of the default password-less root. However, Debian
hard-codes this change in mysql_install_db, which breaks that program for
other users.
This commit instead implements new general options for mysql_install_db that
can be used by anyone to similarly perform a more secure install:
--skip-auth-anonymous-user: omits the anonymous user.
--auth-root-authentication-method=normal: Keeps the existing behaviour
with a password-less root account. Currently on by default.
--auth-root-socket-user=USER
--auth-root-authentication-method=socket: creates the MariaDB root user
with the name USER (defaults to 'root') and using unix socket
authentication. This way, only that user has MariaDB root access
after install.
The idea with --auth-root-authentication-method=normal is that
applications that need this behaviour can give that option explicitly.
Then eventually we could make --auth-root-authentication-method=socket
the default, giving a more secure default installation.
Note that it is perfectly possible to do a secure install with
--auth-root-authentication-method=normal. For example, installing a
private server just for local access by a single OS-level user, by
using --skip-networking and putting the connection socket in a
location without public access. So it is important to preserve this
API for backwards compatibility.
* Remove duplicate lines from tests
* Use thd instead of current_thd
* Remove extra wsrep_binlog_format_names
* Correctly merge union patch from 5.5 wrt duplicate rows.
* Correctly merge SELinux changes into 10.1
* Update mysqld_safe script to remove duplicated parameter --crash-script
* Make --core-file-size accept underscores as well as dashes correctly.
* Add mysqld_safe_helper to Debian and Ubuntu files.
* Update innodb minor version to 35
IS STARTING: CONFUSING ERROR
DESCRIPTION
===========
When mysql server processes transactions but has not yet
committed and shuts down abnormally (due to crash, external
killing etc.), a recovery is due from Storage engine side
which takes place the next time mysql server (either
through mysqld or mysqld_safe) is run.
While the 1st server is in mid of recovery, if another
instance of mysqld_safe is made to run, it may result into
2nd instance killing the 1st one after a moment.
ANALYSIS
========
In the "while true" loop, we've a check (which is done
after the server stops) for the existence of pid file to
enquire if it was a normal shutdown or not. If the file is
absent, it means that the graceful exit of server had
removed this file.
However if the file is present, the scripts makes a plain
assumption that this file is leftover of the "current"
server. It misses to consider that it could be a valid pid
file belonging to another running mysql server.
We need to add more checks in the latter case. The script
should extract the PID from this existing file and check if
its running or not. If yes, it means an older instance of
mysql server is running and hence the script should abort.
FIX
===
Checking the status of process (alive or not) by adding a
@CHECK_PID@ in such a case. Aborting if its alive. Detailed
logic is as follows:
- The mysqld_safe script would quit at start only as soon
as it finds that there is an active PID i.e. a mysql server
is already running.
- The PID file creation takes place after InnoDb recovery,
which means in rare case (when PID file isn't created yet)
it may happen that more than 1 server can come up but even
in that case others will have to wait till the 1st server
has released the acquired InnoDb lock. In this case all
these servers will either TIMEOUT waiting for InnoDb lock
or after this they would find that the 1st server is
already running (by reading $pid_file) and would abort.
- Our core fix is that we now check the status of mysql
server process (alive or not) after the server stops
running within the loop of "run -> shutdown/kill/abort ->
run ... ", so that only the script who owns the mysql
server would be able to bring it down if required.
NOTE
====
Removed the deletion of pid file and socket file from entry
of the loop, as it may result in 2nd instance deleting
these files created by 1st instance in RACE condition.
Compensated this by deleting these files at end of the loop
Reverted the changes made in patch to Bug#16776528. So
after this patch is pushed, the concept of mysqld_safe.pid
would go altogether. This was required as the script was
deleting other instance's mysqld_safe.pid allowing multiple
mysqld_safe instances to run in parallel. This patch would
fix Bug#16776528 as well as the resources would be guarded
anyway by InnoDb lock + our planned 5.7 patch.
- Replace #!/bin/bash with #!/bin/sh
- Split username:password using POSIX compat %% and ##
- Don't use array for FILTERS
- Replace == tests with POSIX-compat =
Don't read --ledir option from config file.
Ignore current working for finding location of mysqld
Remove use of chown/chmod in scripts.
Be helpful only when basedir is /var/log or /var/lib.
Removed unused systemd files for SLES.
Set explicit basedir in scripts.
- Remove use of touch and chmod.
- Restrict usage of chown to cases where target directory is /var/log.
- Due to limited feature set in /bin/sh on Solaris, /bin/bash will be
used on this platform.
- Give error if directory for UNIX socket file is missing.
- Privileged user should not log to files owned by different user
(mysqld will log as before).
* Out of tree build: mysql_install_db to see all .sql files
Since MDEV-7875 (da0991c6), not all sql source files are in the source
directory, maria_add_gis_sp_bootstrap.sql is in the build directory.
This corrects mysql_install_db{.sh} to be aware of the differing
locations.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
* Out of tree build: scripts/mysql_install_db.pl.in
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
* Fix undefined database test error when running mysql_install_db
When using mariaDb in docker mode it can fail as it calls mysql_install_db but as we are going through a slightly different install process the test database has not been created, therefore we should fall back to the mysql database as per https://mariadb.com/kb/en/mariadb/mariadb-10112-mysql_install_db-aborts-on-unkown-file-test/
* Also fix mysql_install_db.pl.in
Synced xtrabackup SST scripts from PXC source tree as of PXC 5.6.27-25.13
- PXC#480: xtrabackup-v2 SST fails with multiple log_bin directives in my.cn
- PXC#460: wsrep_sst_auth don't work in Percona-XtraDB-Cluster-56-5.6.25-25.
- PXC-416: Fix SST related issues.
- PXC-389: Merge remote-tracking branch 'wsrep/5.6' into 5.6-wsrep-pxc389
- Bug #1431101: SST does not clobber backup-my.cnf
Argument to malloc-lib must be included in restricted list of
directories, symlink guards added, and mysqld and mysqld-version
options restricted to command line only. Don't redirect errors to
stderr.
In rsync based SST method, during third phase of data transfer,
'lost+found' should be filtered out while recursively transferring
files from various directories under data directory.
During wsrep position recovery, galera_recovery.sh script
redirected mysqld's error log to a temporary file in order
to find the start position. This, however, will not work
if --log-error is configured for the server.
Fixed by using --log-error in command line instead of
redirection.
[Patch contributed by Philippe MARASSE]
Using IP address in donor's socat with TLS/SSL and certificate
which doesn't contain IP address in CN or SubjectAltName causes
transfer to fail with message:
socat[5799] E certificate is valid but its commonName does not
match hostname.
This patch tries to reverse resolve IP address to hostname and
use it for transfer. If reverse resolution fails, IP address is
still used as fall-back, so proper A/AAAA and PTR records are
important, but not mandatory.
Certain certificates cannot contain IP addresses, e.g. FreeIPA's
Dogtag doesn't allow it, so in my case I would need to use self-
signed certificates instead, use verify=0 with socat or don't use
TLS/SSL at all. Issue is mentioned in MDEV-9403.
Galera recovery process works in two phases. In the first
phase, mysqld is started as non-daemon with --wsrep-recover
to recover and fetch the last logged global transaction ID.
This ID is then used in second phase as the start position
(--wsrep-start-position=XX) to start mysqld as daemon.
As this process was implemented in mysqld_safe script, the
recovery did not work when server was started using systemd.
Fixed by introducing a shell script (wsrep_recovery.sh) that
mimics the first phase of the recovery process.
Added mysql_to_mariadb.sql script, to change mysql.user tables from
MySQL 5.7 to MariaDB.
After this script is run, one can get the other tables fixed by running
mysql_upgrade
[Fix taken from https://github.com/percona/percona-xtradb-
cluster/commit/b3ee75949ed82b88f355ca2e26431350cc1c89ac]
During SST, the receiver node creates .sst directory under
datadir to process/prepare the received data and removes it
at the end of the process. In case of error, this directory,
however, was not removed, which later caused subsequent SSTs
to fail. Fixed by removing this directory at the beginning
of SST if it existed.
- Removed some QQ markers
- Removed some rows not compatible with valgrind 3.9.0
- Made mysql_install_db.sh more silent by default. --verbose now gives more information
- Added assert that auto-increment doesn't generate 0 (safety)
- Removed thd->set_time() in some places as it's set in init_for_queries()
- Fixed some --big tests in tokudb
- Fixed a bug in mysql_client_test.cc where sql_mode was not properly reset
IS NOT FOUND
DESCRIPTION
===========
If script mysqld_multi and utility my_print_defaults are in
the same folder (not included in $PATH) and the former is
made to run, it complaints that the mysqld binary is absent
eventhough the binary exists.
ANALYSIS
========
We've a subroutine my_which() mimicking the behaviour of
POSIX "which" command. Its current behaviour is to check
for a given argument as follows:
- Step 1: Assume the argument to be a command having full
fledged absolute path. If it exists "as-is", return the
argument (which will be pathname), else proceed to Step 2.
- Step 2: Assume the argument to be a plain command with no
aboslute path. Try locating it in all of the paths
(mentioned in $PATH) one by one. If found return the
pathname. If found nowhere, return NULL.
Currently when my_which(my_print_defaults) is called, it
returns from Step 1 (since utlity exists in current
folder) and doesn't proceed to Step 2. This is wrong since
the returned value is same as the argument i.e.
'my_print_default' which defies the purpose of this
subroutine whose job is to return a pathname either in Step
1 or Step 2.
Later when the utility is executed in subroutine
defaults_for_group(), it evaluates to NULL and returns the
same. This is because the plain command 'my_print_defaults
{options} ...' would execute properly only if
my_print_defaults exists in one of the paths (in $PATH). In
such a case, in the course of the flow it looks onto the
variable $mysqld_found which comes out to be NULL and
hence ethe error.
In this case, call to my_which should fail resulting in
script being aborted and thus avoiding this mess.
FIX
===
This utility my_print_defaults should be tested only in
Step 2 since it does not have an absolute path. Thus added
a condition in Step 1 so that is gets executed iff not
called for my_print_defaults thus bypassing it to proceed
to Step 2 where the check is made for various paths (in
$PATH)
During SST, since wsrep_sst_rsync waits for mysqld to create
"tables_flushed" file after it has successfully executed FTWRL,
it would wait forever if FTWRL fails.
Fixed by introducing a mechanism to report failure to the script.
If any given variable the xtrabackup-v2 sst script looks for is specified
multiple times in cnf file then it tend to pick both of them causing
some of the follow-up command to fail.
Avoid this programatic mistake by honoring only the last variable assigned
setting as done by mysqld too.
Check https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1362830
Semantics:
---------
* Generally end-user will create a separate user with needed
privileges for
performing DONOR action.
* This user credentials are specified using wsrep_sst_auth.
* Along with this user there could be other user(s) created on the
server
that sysadmin may use for normal or other operations
* Credentials for these user(s) can be specified in same
cluster/server
cnf file as part of [client] section
When cluster act as DONOR and if wsrep_sst_auth is provided then it
should
strictly use it for performing SST based action.
What if end-user has same credentials for performing both SST action
and
normal admin work ?
* Then end-user can simply specify these credentials as part of
[client]
section in cnf file and skip providing wsrep_sst_auth.
Issue:
-----
MySQL client user/password parsing preference order is as follows:
* command line (through --user/--password)
* cnf file
* MYSQL_PWD enviornment variable.
Recent change tried passing sst user password through MYSQL_PWD
(and user though --user command line param as before).
On the system where-in admin had another user for performing non-SST
actions,
credentials for such user were present in cnf file under [client]
section.
Due to mysql client preference order, SST user name was used (as it
was
passed through command line) but password of other user (meant for
non-SST)
action was being used as it was passed through cnf file.
Password passed through MYSQL_PWD was completely ignored causing
user-name/password mismatch.
Solution:
---------
* If user has specified credentials for SST then pass them through
command
line so that they are used in priority.
(There could be security concern on passing things through command
line but
when I tried passing user-name and password through command line to
mysql
client and then did ps I saw this
./bin/mysql --user=sstuser --password=x xxxxxxxx -S /tmp/n1.sock
so seems like password is not shown)
- Add CA validation to wsrep_sst_xtrabackup-v2.sh.
- Also added a few {} around tpem for consistency.
- Abort if encryption is requested but socat is not ssl-enabled.
Patch contributed by : Klaas Demter
mysqlbug has been obsolete since MySQL 5.5, and has been removed in MySQL 5.7.
It's also of no use for reporting MariaDB bugs. The script and the associated
man page removed from MariaDB.
MTR drops the datadir in the event of a test failure. In case mysqld
is running as a Galera node and a failure occurs while the SST is in
progress, the rsync pid file gets removed as part of the cleanup and
wsrep_sst_rsync, which relies on this file, fails to kill the rsync.
Fixed by using the cached $RSYNC_REAL_PID to kill rsync daemon.
Patch from Daniel Black:
- Change the charset of mysql.column_stats.{min_value, max_value} from
utf8_bin varchar to varbinary
- Adjust the code that saves/reads the data accordingly.
- Also provide upgrade statement in mysql_system_tables_fix.sql
Post-fix:
Reverting the patch for MDEV-6069 brought some ALTERs with
ENGINE=MYISAM back into the mysql_system_tables_fix.sql
script. As a result, running mysql_upgrade with global
enforce_storage_engine=INNODB (or any other non-MyISAM
engine, for that matter) would fail.
Fixed by locally unsetting enforce_storage_engine in the
upgrade script.
Use galera_new_cluster instead. systemctl start mariadb@bootstrap
will generate error message, use_galera_new_cluster.conf is the name
of the file that will generate this error.
Output:
Job for mariadb@bootstrap.service failed. See "systemctl status
mariadb@bootstrap.service" and "journalctl -xe" for details.
● mariadb@bootstrap.service - MariaDB database server
Loaded: loaded (/usr/lib/systemd/system/mariadb@.service; disabled;
vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/mariadb@bootstrap.service.d
└─use_galera_new_cluster.conf
Active: failed (Result: exit-code) since Thu 2015-10-15 19:27:52
CEST; 5s ago
Process: 24334 ExecStart=/usr/bin/false (code=exited,
status=1/FAILURE)
Process: 24330 ExecStart=/usr/bin/echo Please use galera_new_cluster
to start the mariadb service with --wsrep-new-cluster (code=exited,
status=0/SUCCESS)
Main PID: 24334 (code=exited, status=1/FAILURE)
Oct 15 19:27:52 spaceman systemd[1]: Starting MariaDB database server...
Oct 15 19:27:52 spaceman systemd[1]: mariadb@bootstrap.service: main
process exited, code=exited, status=1/FAILURE
Oct 15 19:27:52 spaceman systemd[1]: Failed to start MariaDB database
server.
Oct 15 19:27:52 spaceman systemd[1]: Unit mariadb@bootstrap.service
entered failed state.
Oct 15 19:27:52 spaceman systemd[1]: mariadb@bootstrap.service failed.
mysql_secure_installation used incorrect path while looking up for "mysql"
client tool: $basedir/$basedir/bin instead of $basedir/bin.
This patch adapts "my_print_defaults" lookup algorithm for "mysql" client tool.
new features:
set event_scheduler=ON|OFF will now try to init event scheduler
if it's not enabled
set event_scheduler=default will try to enable it based on
the value of the event_scheduler when mysqld was started
Systemd config files need absolute paths.
LimitCore was ment to be LimitCORE
Oct 14 07:28:04 spaceman systemd[1]: [/etc/systemd/system/mariadb.service.d/migrated-from-my.cnf-settings.conf:7] Unknown lvalue 'LimitCore' in section 'Service'
Oct 14 07:28:04 spaceman systemd[1]: [/etc/systemd/system/mariadb.service.d/migrated-from-my.cnf-settings.conf:9] Executable path is not absolute, ignoring: sync
Oct 14 07:28:04 spaceman systemd[1]: [/etc/systemd/system/mariadb.service.d/migrated-from-my.cnf-settings.conf:10] Executable path is not absolute, ignoring: sysctl -q -w vm.drop_caches=3
During the review process OPTIONS was converted to MYSQLD_OPTS.
In the script mariadb-service convert, the ExecStart of the system
also uses this setting.
1. Passes wsrep_sst_auth_value to SST scripts via WSREP_SST_OPT_AUTH envronmental variable, so it never appears on the command line
2. In mysqldump and xtrabackup* SST scripts which rely on MySQL authentication, instead of passing password on the command line, SST script sets MYSQL_PWD environment variable, so that password also never appears on the mysqldump/innobackupex command line.
