In rsync based SST method, during third phase of data transfer,
'lost+found' should be filtered out while recursively transferring
files from various directories under data directory.
During wsrep position recovery, galera_recovery.sh script
redirected mysqld's error log to a temporary file in order
to find the start position. This, however, will not work
if --log-error is configured for the server.
Fixed by using --log-error in command line instead of
redirection.
[Patch contributed by Philippe MARASSE]
Using IP address in donor's socat with TLS/SSL and certificate
which doesn't contain IP address in CN or SubjectAltName causes
transfer to fail with message:
socat[5799] E certificate is valid but its commonName does not
match hostname.
This patch tries to reverse resolve IP address to hostname and
use it for transfer. If reverse resolution fails, IP address is
still used as fall-back, so proper A/AAAA and PTR records are
important, but not mandatory.
Certain certificates cannot contain IP addresses, e.g. FreeIPA's
Dogtag doesn't allow it, so in my case I would need to use self-
signed certificates instead, use verify=0 with socat or don't use
TLS/SSL at all. Issue is mentioned in MDEV-9403.
Galera recovery process works in two phases. In the first
phase, mysqld is started as non-daemon with --wsrep-recover
to recover and fetch the last logged global transaction ID.
This ID is then used in second phase as the start position
(--wsrep-start-position=XX) to start mysqld as daemon.
As this process was implemented in mysqld_safe script, the
recovery did not work when server was started using systemd.
Fixed by introducing a shell script (wsrep_recovery.sh) that
mimics the first phase of the recovery process.
Added mysql_to_mariadb.sql script, to change mysql.user tables from
MySQL 5.7 to MariaDB.
After this script is run, one can get the other tables fixed by running
mysql_upgrade
[Fix taken from https://github.com/percona/percona-xtradb-
cluster/commit/b3ee75949ed82b88f355ca2e26431350cc1c89ac]
During SST, the receiver node creates .sst directory under
datadir to process/prepare the received data and removes it
at the end of the process. In case of error, this directory,
however, was not removed, which later caused subsequent SSTs
to fail. Fixed by removing this directory at the beginning
of SST if it existed.
IS NOT FOUND
DESCRIPTION
===========
If script mysqld_multi and utility my_print_defaults are in
the same folder (not included in $PATH) and the former is
made to run, it complaints that the mysqld binary is absent
eventhough the binary exists.
ANALYSIS
========
We've a subroutine my_which() mimicking the behaviour of
POSIX "which" command. Its current behaviour is to check
for a given argument as follows:
- Step 1: Assume the argument to be a command having full
fledged absolute path. If it exists "as-is", return the
argument (which will be pathname), else proceed to Step 2.
- Step 2: Assume the argument to be a plain command with no
aboslute path. Try locating it in all of the paths
(mentioned in $PATH) one by one. If found return the
pathname. If found nowhere, return NULL.
Currently when my_which(my_print_defaults) is called, it
returns from Step 1 (since utlity exists in current
folder) and doesn't proceed to Step 2. This is wrong since
the returned value is same as the argument i.e.
'my_print_default' which defies the purpose of this
subroutine whose job is to return a pathname either in Step
1 or Step 2.
Later when the utility is executed in subroutine
defaults_for_group(), it evaluates to NULL and returns the
same. This is because the plain command 'my_print_defaults
{options} ...' would execute properly only if
my_print_defaults exists in one of the paths (in $PATH). In
such a case, in the course of the flow it looks onto the
variable $mysqld_found which comes out to be NULL and
hence ethe error.
In this case, call to my_which should fail resulting in
script being aborted and thus avoiding this mess.
FIX
===
This utility my_print_defaults should be tested only in
Step 2 since it does not have an absolute path. Thus added
a condition in Step 1 so that is gets executed iff not
called for my_print_defaults thus bypassing it to proceed
to Step 2 where the check is made for various paths (in
$PATH)
During SST, since wsrep_sst_rsync waits for mysqld to create
"tables_flushed" file after it has successfully executed FTWRL,
it would wait forever if FTWRL fails.
Fixed by introducing a mechanism to report failure to the script.
If any given variable the xtrabackup-v2 sst script looks for is specified
multiple times in cnf file then it tend to pick both of them causing
some of the follow-up command to fail.
Avoid this programatic mistake by honoring only the last variable assigned
setting as done by mysqld too.
Check https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1362830
Semantics:
---------
* Generally end-user will create a separate user with needed
privileges for
performing DONOR action.
* This user credentials are specified using wsrep_sst_auth.
* Along with this user there could be other user(s) created on the
server
that sysadmin may use for normal or other operations
* Credentials for these user(s) can be specified in same
cluster/server
cnf file as part of [client] section
When cluster act as DONOR and if wsrep_sst_auth is provided then it
should
strictly use it for performing SST based action.
What if end-user has same credentials for performing both SST action
and
normal admin work ?
* Then end-user can simply specify these credentials as part of
[client]
section in cnf file and skip providing wsrep_sst_auth.
Issue:
-----
MySQL client user/password parsing preference order is as follows:
* command line (through --user/--password)
* cnf file
* MYSQL_PWD enviornment variable.
Recent change tried passing sst user password through MYSQL_PWD
(and user though --user command line param as before).
On the system where-in admin had another user for performing non-SST
actions,
credentials for such user were present in cnf file under [client]
section.
Due to mysql client preference order, SST user name was used (as it
was
passed through command line) but password of other user (meant for
non-SST)
action was being used as it was passed through cnf file.
Password passed through MYSQL_PWD was completely ignored causing
user-name/password mismatch.
Solution:
---------
* If user has specified credentials for SST then pass them through
command
line so that they are used in priority.
(There could be security concern on passing things through command
line but
when I tried passing user-name and password through command line to
mysql
client and then did ps I saw this
./bin/mysql --user=sstuser --password=x xxxxxxxx -S /tmp/n1.sock
so seems like password is not shown)
- Add CA validation to wsrep_sst_xtrabackup-v2.sh.
- Also added a few {} around tpem for consistency.
- Abort if encryption is requested but socat is not ssl-enabled.
Patch contributed by : Klaas Demter
MTR drops the datadir in the event of a test failure. In case mysqld
is running as a Galera node and a failure occurs while the SST is in
progress, the rsync pid file gets removed as part of the cleanup and
wsrep_sst_rsync, which relies on this file, fails to kill the rsync.
Fixed by using the cached $RSYNC_REAL_PID to kill rsync daemon.
Patch from Daniel Black:
- Change the charset of mysql.column_stats.{min_value, max_value} from
utf8_bin varchar to varbinary
- Adjust the code that saves/reads the data accordingly.
- Also provide upgrade statement in mysql_system_tables_fix.sql
Post-fix:
Reverting the patch for MDEV-6069 brought some ALTERs with
ENGINE=MYISAM back into the mysql_system_tables_fix.sql
script. As a result, running mysql_upgrade with global
enforce_storage_engine=INNODB (or any other non-MyISAM
engine, for that matter) would fail.
Fixed by locally unsetting enforce_storage_engine in the
upgrade script.
Use galera_new_cluster instead. systemctl start mariadb@bootstrap
will generate error message, use_galera_new_cluster.conf is the name
of the file that will generate this error.
Output:
Job for mariadb@bootstrap.service failed. See "systemctl status
mariadb@bootstrap.service" and "journalctl -xe" for details.
● mariadb@bootstrap.service - MariaDB database server
Loaded: loaded (/usr/lib/systemd/system/mariadb@.service; disabled;
vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/mariadb@bootstrap.service.d
└─use_galera_new_cluster.conf
Active: failed (Result: exit-code) since Thu 2015-10-15 19:27:52
CEST; 5s ago
Process: 24334 ExecStart=/usr/bin/false (code=exited,
status=1/FAILURE)
Process: 24330 ExecStart=/usr/bin/echo Please use galera_new_cluster
to start the mariadb service with --wsrep-new-cluster (code=exited,
status=0/SUCCESS)
Main PID: 24334 (code=exited, status=1/FAILURE)
Oct 15 19:27:52 spaceman systemd[1]: Starting MariaDB database server...
Oct 15 19:27:52 spaceman systemd[1]: mariadb@bootstrap.service: main
process exited, code=exited, status=1/FAILURE
Oct 15 19:27:52 spaceman systemd[1]: Failed to start MariaDB database
server.
Oct 15 19:27:52 spaceman systemd[1]: Unit mariadb@bootstrap.service
entered failed state.
Oct 15 19:27:52 spaceman systemd[1]: mariadb@bootstrap.service failed.
mysql_secure_installation used incorrect path while looking up for "mysql"
client tool: $basedir/$basedir/bin instead of $basedir/bin.
This patch adapts "my_print_defaults" lookup algorithm for "mysql" client tool.
new features:
set event_scheduler=ON|OFF will now try to init event scheduler
if it's not enabled
set event_scheduler=default will try to enable it based on
the value of the event_scheduler when mysqld was started
Systemd config files need absolute paths.
LimitCore was ment to be LimitCORE
Oct 14 07:28:04 spaceman systemd[1]: [/etc/systemd/system/mariadb.service.d/migrated-from-my.cnf-settings.conf:7] Unknown lvalue 'LimitCore' in section 'Service'
Oct 14 07:28:04 spaceman systemd[1]: [/etc/systemd/system/mariadb.service.d/migrated-from-my.cnf-settings.conf:9] Executable path is not absolute, ignoring: sync
Oct 14 07:28:04 spaceman systemd[1]: [/etc/systemd/system/mariadb.service.d/migrated-from-my.cnf-settings.conf:10] Executable path is not absolute, ignoring: sysctl -q -w vm.drop_caches=3
During the review process OPTIONS was converted to MYSQLD_OPTS.
In the script mariadb-service convert, the ExecStart of the system
also uses this setting.
1. Passes wsrep_sst_auth_value to SST scripts via WSREP_SST_OPT_AUTH envronmental variable, so it never appears on the command line
2. In mysqldump and xtrabackup* SST scripts which rely on MySQL authentication, instead of passing password on the command line, SST script sets MYSQL_PWD environment variable, so that password also never appears on the mysqldump/innobackupex command line.
- LP1380697: wsrep_sst_xtrabackup-v2 doesn't stop when mysql is SIGKILLed. (full fix for this (as engineeered by Percona) requires Linux-specific patch that we don't carry, but keep xtrabackup scripts as close as possible)
- LP1399134: Log the innobackupex/SST logs in SST to syslog if possible. (fixed)
- LP1405668: Race condition between donor and joiner in PXB SST. (fixed)
- LP1405985: Fail early if xtrabackup_checkkpoints is missing. (fixed)
- LP1407599: wsrep_sst_xtrabackup-v2 script causes innobackupex to print a false positive stack trace into the log. (fixed)
- LP1441762: IST Fails with SST script error. (fixed)
- LP1451670: Fail when move-back fails in xtrabackup SST. (fixed)
1. Passes wsrep_sst_auth_value to SST scripts via WSREP_SST_OPT_AUTH envronmental variable, so it never appears on the command line
2. In mysqldump and xtrabackup* SST scripts which rely on MySQL authentication, instead of passing password on the command line, SST script sets MYSQL_PWD environment variable, so that password also never appears on the mysqldump/innobackupex command line.
Let mysqld_safe_syslog.cnf force disable error log so that logging to syslog is
not affected by previous log_error setting.
Added handling of --skip-log-error to mysqld_safe.
in cmake tests let's treat clang like gcc (same options,
same builtins) in many cases.
* don't check the compiler when
* testing for -fvisibility=hidden support
* testing for HAVE_ABI_CXA_DEMANGLE
* testing for HAVE_GCC_ATOMIC_BUILTINS
* when removing options with string(replace)
* when running ${CC} --version (ignore the error instead)
* run ABI checks for clang
* use "canonical" gcc flags for clang
* fix groonga too
Also:
* add cmake detection for gcc __atomic_* builtins. they might be
supported (__ATOMIC_SEQ_CST is defined), but not for all operand
sizes. In particular, 64-bit atomic load is problematic on i386
* cache check results for Windows
* remove the test for HAVE_CXXABI_H (HAVE_ABI_CXA_DEMANGLE is
suffifient)
* mysqld_safe: Since wsrep_on variable is mandatory in 10.1, skip wsrep
position recovery if its OFF.
* mysqld: Remove "-wsrep" from server version
* mysqld: Remove wsrep patch version from @@version_comment
* mysqld: Introduce @@wsrep_patch_version
* improves MySQL client version check making it no less than required as opposed to exactly as required.
* adds event table copying to ensure same results as with rsync SST.
Fix for failing tests.
* Update mysql_system_tables_fix.sql to makeup the differences in system
tables in 5.1.17 (main.system_mysql_db_fix50117)
* Removed system_mysql_db tests for versions 5.0.30 & 4.1.23.
Removed 'galera-sst-mode' option from mysqldump and added logic
in wsrep_sst_mysqldump script to retrieve gtid_binlog_state from
donor node and send it to the joiner node.
Removed 'galera-sst-mode' option from mysqldump and added logic
in wsrep_sst_mysqldump script to retrieve gtid_binlog_state from
donor node and send it to the joiner node.
some checks were "if [ -n "$err_log" ]", others were "if [ $want_syslog -eq 0 ]",
so when both are set, error log file was only partially initialized.
To avoid this ambiguity we reset want_syslog when error log file is used.
xargs may split the input and execute them in batches for
large number of files. As a result xtrabackup sst script
may feed incorrect size to pv.
Fixed by piping the files to du directly (du --files0-from=-).
[Based on suggestions from Brad Jorgensen]
xargs may split the input and execute them in batches for
large number of files. As a result xtrabackup sst script
may feed incorrect size to pv.
Fixed by piping the files to du directly (du --files0-from=-).
[Based on suggestions from Brad Jorgensen]
The GEOMETRY field metadata is stored in the FRM file.
SRID for a spatial column now can be stored, it was added to the CREATE TABLE syntax,
so the AddGeometryData() stored procedure is now possible. Script adding the required Add/DropGeometryColumn sp-s added.
two bugs in mysql_config:
1. flags like -Werror=format were not stipped out
2. one-pass s/// command was used that could not remove all matching flags
(e.g. s/ A */ /g cannot remove all A's in " 1 2 A A A A A 4 5 ")
When building RPMs and using the "rpmbuild" automatic scanning for
Perl dependencies, it might interpret "use hostnames" in the
"mysql_config.pl" script usage comment as a Perl "use" statement.
And then makes the resulting RPMs depends on a non-existing module
Perl "hostnames". The comment was changed to prevent this.
Description: Certain operations using the client tool
'mysql\_setpermisson' which uses username of the format
"xxx-yyy" results in a syntax error.
Analysis: The user name variable ( $user ) was not quoted
properly in the 'mysql\_setpermission' script. As a result,
while granting user privileges for a user with username of
the format 'xxx-yyy', results in a syntax error.
Fix: The escape sequence has been applied for the user
variable in the script file.
Note: Test case is not added for this fix.
'mysql\_setpermission' executable resides in different
directories on pb2 machines and on local boxes. Hence the
test case cannot be made generic.
Description: Certain operations using the client tool
'mysql\_setpermisson' which uses username of the format
"xxx-yyy" results in a syntax error.
Analysis: The user name variable ( $user ) was not quoted
properly in the 'mysql\_setpermission' script. As a result,
while granting user privileges for a user with username of
the format 'xxx-yyy', results in a syntax error.
Fix: The escape sequence has been applied for the user
variable in the script file.
Note: Test case is not added for this fix.
'mysql\_setpermission' executable resides in different
directories on pb2 machines and on local boxes. Hence the
test case cannot be made generic.
Bug#17959689: MAKE GCC AND CLANG GIVE CONSISTENT COMPILATION WARNINGS
Bug#18313717: ENABLE -WERROR IN MAINTANER MODE WHEN COMPILING WITH CLANG
Bug#18510941: REMOVE CMAKE WORKAROUNDS FOR OLDER VERSIONS OF OS X/XCODE
Backport from mysql-5.6 to mysql-5.5
