Also clarify which --{no-,}default* options, must be first.
Sample output:
$ client/mysql --help
client/mysql Ver 15.1 Distrib 5.5.59-MariaDB, for Linux (x86_64) using readline 5.1
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Usage: client/mysql [OPTIONS] [database]
Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
The following groups are read: mysql client client-server client-mariadb
The following options may be given as the first argument:
--print-defaults Print the program argument list and exit.
--no-defaults Don't read default options from any option file.
The following specify which files/groups are read (specified before other options):
--defaults-file=# Only read default options from the given file #.
--defaults-extra-file=# Read this file after the global files are read.
--defaults-group-suffix=# Additionally read default groups with # appended as a suffix.
tests running from build directory:
TEST: print defaults ignored as not first
$ sql/mysqld --no-defaults --print-defaults --lc-messages-dir=${PWD}/sql/share
TEST: no startup occurs as --print-defaults specified
$ sql/mysqld --print-defaults --lc-messages-dir=${PWD}/sql/share
sql/mysqld would have been started with the following arguments:
--lc-messages-dir=/home/dan/repos/build-mariadb-5.5/sql/share
TEST: default args can't be anywhere
$ client/mysql --user=bob --defaults-file=/etc/my.cnf
client/mysql: unknown variable 'defaults-file=/etc/my.cnf'
$ client/mysql --user=bob --defaults-group-suffix=.group
client/mysql: unknown variable 'defaults-group-suffix=.group'
/etc/my.cnf:
[client-server.group]
socket=/var/lib/mysql-multi/group/mysqld.sock
user=bob
/etc/my.other.cnf:
socket=/var/lib/mysql-other/mysqld.sock
TEST: defaults file read and suffix also applied
$ client/mysql --defaults-file=/etc/my.other.cnf --defaults-group-suffix=.group
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql-other/mysqld.sock' (2)
TEST: defaults extra file
$ client/mysql --defaults-extra-file=/etc/my.other.cnf --defaults-group-suffix=.group
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql-other/mysqld.sock' (2)
For galera compatibility, the main thing is to ensure the FD 1, 2 are
not opened with O_CLOEXEC otherwise galera sst errors don't appear in
the error.log
Files without O_CLOEXEC from the test below:
0 -> /dev/pts/9
1 -> /tmp/error.log (intended)
2 -> /tmp/error.log (intended)
5 -> /tmp/datadir
6 -> /tmp/datadir/aria_log.00000001
(Innodb temp files)
8 -> /tmp/ibIIrhFL (deleted)
9 -> /tmp/ibfx1vai (deleted)
10 -> /tmp/ibAQUKFO (deleted)
11 -> /tmp/ibWBQSHR (deleted)
15 -> /tmp/ibEXEcfo (deleted)
20 -> /tmp/datadir/mysql/host.MYD
22 -> /tmp/datadir/mysql/user.MYD
... (rest of MYD files)
Test for this and the previous commit.
sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --log-error=/tmp/error.log --general-log-file=/tmp/general.log --general-log=1 --slow-query-log-file=/tmp/slow.log --slow-query-log=1
180302 10:56:41 [Note] sql/mysqld (mysqld 5.5.60-MariaDB-wsrep) starting as process 26056 ...
$ cd /proc/26056
$ ls -la --sort=none fd
total 0
dr-x------. 2 dan dan 0 Mar 2 10:57 .
dr-xr-xr-x. 9 dan dan 0 Mar 2 10:56 ..
lrwx------. 1 dan dan 64 Mar 2 10:57 0 -> /dev/pts/9
l-wx------. 1 dan dan 64 Mar 2 10:57 1 -> /tmp/error.log
l-wx------. 1 dan dan 64 Mar 2 10:57 2 -> /tmp/error.log
lrwx------. 1 dan dan 64 Mar 2 10:57 3 -> /tmp/datadir/mysqlbin.index
lrwx------. 1 dan dan 64 Mar 2 10:57 4 -> /tmp/datadir/aria_log_control
lr-x------. 1 dan dan 64 Mar 2 10:57 5 -> /tmp/datadir
lrwx------. 1 dan dan 64 Mar 2 10:57 6 -> /tmp/datadir/aria_log.00000001
lrwx------. 1 dan dan 64 Mar 2 10:57 7 -> /tmp/datadir/ibdata1
lrwx------. 1 dan dan 64 Mar 2 10:57 8 -> /tmp/ibIIrhFL (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 9 -> /tmp/ibfx1vai (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 10 -> /tmp/ibAQUKFO (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 11 -> /tmp/ibWBQSHR (deleted)
lrwx------. 1 dan dan 64 Mar 2 10:57 12 -> /tmp/datadir/ib_logfile0
lrwx------. 1 dan dan 64 Mar 2 10:57 13 -> /tmp/datadir/ib_logfile1
l-wx------. 1 dan dan 64 Mar 2 10:57 14 -> /tmp/slow.log
lrwx------. 1 dan dan 64 Mar 2 10:57 15 -> /tmp/ibEXEcfo (deleted)
l-wx------. 1 dan dan 64 Mar 2 10:57 16 -> /tmp/general.log
lrwx------. 1 dan dan 64 Mar 2 10:57 17 -> socket:[1897356]
lrwx------. 1 dan dan 64 Mar 2 10:57 18 -> socket:[45335]
l-wx------. 1 dan dan 64 Mar 2 10:57 19 -> /tmp/datadir/mysqlbin.000004
lrwx------. 1 dan dan 64 Mar 2 10:57 20 -> /tmp/datadir/mysql/host.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 21 -> /tmp/datadir/mysql/host.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 22 -> /tmp/datadir/mysql/user.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 23 -> /tmp/datadir/mysql/user.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 24 -> /tmp/datadir/mysql/db.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 25 -> /tmp/datadir/mysql/db.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 26 -> /tmp/datadir/mysql/proxies_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 27 -> /tmp/datadir/mysql/proxies_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 28 -> /tmp/datadir/mysql/tables_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 29 -> /tmp/datadir/mysql/tables_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 30 -> /tmp/datadir/mysql/columns_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 31 -> /tmp/datadir/mysql/columns_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 32 -> /tmp/datadir/mysql/procs_priv.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 33 -> /tmp/datadir/mysql/procs_priv.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 34 -> /tmp/datadir/mysql/servers.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 35 -> /tmp/datadir/mysql/servers.MYI
lrwx------. 1 dan dan 64 Mar 2 10:57 36 -> /tmp/datadir/mysql/event.MYD
lrwx------. 1 dan dan 64 Mar 2 10:57 37 -> /tmp/datadir/mysql/event.MYI
O_CLOEXEC files are those with flags 02000000
/usr/include/bits/fcntl-linux.h:# define __O_CLOEXEC 02000000
/usr/include/bits/fcntl-linux.h:# define O_CLOEXEC __O_CLOEXEC /* Set close_on_exec. */
$ find fdinfo/ -type f -ls -exec cat {} \; | more
1924720 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/0
pos: 0
flags: 0100002
mnt_id: 25
1924721 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/1
pos: 9954
flags: 0102001
mnt_id: 82
1924722 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/2
pos: 10951
flags: 0102001
mnt_id: 82
1924723 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/3
pos: 116
flags: 02100002
mnt_id: 82
1924724 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/4
pos: 52
flags: 0100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866365 0 EOF
1924725 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/5
pos: 0
flags: 0100000
mnt_id: 82
1924726 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/6
pos: 16384
flags: 0100002
mnt_id: 82
1924727 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/7
pos: 0
flags: 02100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866491 0 EOF
1924728 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/8
pos: 0
flags: 0100002
mnt_id: 82
1924729 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/9
pos: 0
flags: 0100002
mnt_id: 82
1924730 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/10
pos: 0
flags: 0100002
mnt_id: 82
1924731 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/11
pos: 0
flags: 0100002
mnt_id: 82
1924732 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/12
pos: 0
flags: 02100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866492 0 EOF
1924733 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/13
pos: 0
flags: 02100002
mnt_id: 82
lock: 1: POSIX ADVISORY WRITE 26056 00:2c:1866493 0 EOF
1924734 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/14
pos: 763
flags: 02102001
mnt_id: 82
1924735 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/15
pos: 0
flags: 0100002
mnt_id: 82
1924736 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/16
pos: 473
flags: 02102001
mnt_id: 82
1924737 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/17
pos: 0
flags: 02000002
mnt_id: 9
1924738 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/18
pos: 0
flags: 02
mnt_id: 9
1924739 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/19
pos: 245
flags: 02100001
mnt_id: 82
1924740 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/20
pos: 0
flags: 0100002
mnt_id: 82
1924741 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/21
pos: 503
flags: 0500002
mnt_id: 82
1924742 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/22
pos: 324
flags: 0100002
mnt_id: 82
1924743 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/23
pos: 642
flags: 0500002
mnt_id: 82
1924744 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/24
pos: 880
flags: 0100002
mnt_id: 82
1924745 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/25
pos: 581
flags: 0500002
mnt_id: 82
1924746 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/26
pos: 1386
flags: 0100002
mnt_id: 82
1924747 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/27
pos: 498
flags: 0500002
mnt_id: 82
1924748 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/28
pos: 0
flags: 0100002
mnt_id: 82
1924749 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/29
pos: 513
flags: 0500002
mnt_id: 82
1924750 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/30
pos: 0
flags: 0100002
mnt_id: 82
1924751 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/31
pos: 494
flags: 0500002
mnt_id: 82
1924752 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/32
pos: 0
flags: 0100002
mnt_id: 82
1924753 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/33
pos: 535
flags: 0500002
mnt_id: 82
1924754 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/34
pos: 0
flags: 0100002
mnt_id: 82
1924755 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/35
pos: 396
flags: 0500002
mnt_id: 82
1924756 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/36
pos: 0
flags: 0100002
mnt_id: 82
1924757 0 -r-------- 1 dan dan 0 Mar 2 10:57 fdinfo/37
pos: 517
flags: 0500002
mnt_id: 82
escape all charecters less or equal 0x1F (control symbols)
(shorter sequence are not used to make code simple, long encoding is always legal according to the rfc4627)
TRASH was mapped to TRASH_FREE and was supposed to be used for memory
that should not be accessed anymore, while TRASH_ALLOC() is to be
used for uninitialized but to-be-used memory.
But sometimes TRASH() was used in the latter sense.
Remove TRASH() macro, always use explicit TRASH_ALLOC() or TRASH_FREE().
Resolving a stacktrace including functions in dynamic libraries requires
us to look inside the libraries for the symbols. Addr2line needs to be
started with the correct binary for each address on the stack. To do this,
figure out which library it is using dladdr, then if the addr2line
binary was started with a different binary, fork it again with the
correct one.
We only have one addr2line process running at any point during the
stacktrace resolving step. The maximum number of forks for addr2line should
generally be around 6.
One for server stacktrace code, one for plugin code, one when going back
into server code, one for pthread library, one for libc, one for the
_start function in the server. More can come up if plugin calls server
function which goes back to a plugin, etc.
(from 10.1 to 10.0-galera)
This conflicted signficantly with 7d550c76be
which added --defaults-group-suffix support.
Took the approach of 4bb49d84a9 and adapted the
--defaults-group-suffix handling to be consistent.
The following changes as follows:
SST scripts now use $MY_PRINT_DEFAULTS rather than the lowercase for
consistency and this include all required --default arguements.
Backport/merge by Daniel Black <daniel@linux.vnet.ibm.com>
find_type_or_exit() client helper did exit(1) on error, exit(1) moved to
clients.
mysql_read_default_options() did exit(1) on error, error is passed through and
handled now.
my_str_malloc_default() did exit(1) on error, replaced my_str_ allocator
functions with normal my_malloc()/my_realloc()/my_free().
sql_connect.cc did many exit(1) on hash initialisation failure. Removed error
check since my_hash_init() never fails.
my_malloc() did exit(1) on error. Replaced with abort().
my_load_defaults() did exit(1) on error, replaced with return 2.
my_load_defaults() still does exit(0) when invoked with --print-defaults.
MDL_CONTEXT::TRY_ACQUIRE_LOCK_IMPL
ANALYSIS:
=========
Server sometimes exited when multiple threads tried to
acquire and release metadata locks simultaneously (for
example, necessary to access a table). The same problem
could have occurred when new objects were registered/
deregistered in Performance Schema.
The problem was caused by a bug in LF_HASH - our lock free
hash implementation which is used by metadata locking
subsystem in 5.7 branch. In 5.5 and 5.6 we only use LF_HASH
in Performance Schema Instrumentation implementation. So
for these versions, the problem was limited to P_S.
The problem was in my_lfind() function, which searches for
the specific hash element by going through the elements
list. During this search it loads information about element
checked such as key pointer and hash value into local
variables. Then it confirms that they are not corrupted by
concurrent delete operation (which will set pointer to 0)
by checking if element is still in the list. The latter
check did not take into account that compiler (and
processor) can reorder reads in such a way that load of key
pointer will happen after it, making result of the check
invalid.
FIX:
====
This patch fixes the problem by ensuring that no such
reordering can take place. This is achieved by using
my_atomic_loadptr() which contains compiler and processor
memory barriers for the check mentioned above and other
similar places.
The default (for non-Windows systems) implementation of
my_atomic*() relies on old __sync intrisics and implements
my_atomic_loadptr() as read-modify operation. To avoid
scalability/performance penalty associated with addition of
my_atomic_loadptr()'s we change the my_atomic*() to use
newer __atomic intrisics when available. This new default
implementation doesn't have such a drawback.
Reintroduce environment variable MYSQL_GROUP_SUFFIX to be used as
--default-group-suffix value if not already set.
The environment variable was accidentally renamed to DEFAULT_GROUP_SUFFIX_ENV
in MySQL server 5.5.
ERROR_FILE_SYSTEM_LIMITATION was seen by support when backing up large
file. However mariabackup error message was not very helpful,
since it mapped the error to generic catch-all EINVAL.
With the patch, ERROR_FILE_SYSTEM_LIMITATION will be mapped to more
appropriate EFBIG. Also add mapping from ERROR_NO_SYSTEM_RESOURCES
to ENOMEM.
