Also clarify which --{no-,}default* options, must be first.
Sample output:
$ client/mysql --help
client/mysql Ver 15.1 Distrib 5.5.59-MariaDB, for Linux (x86_64) using readline 5.1
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Usage: client/mysql [OPTIONS] [database]
Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
The following groups are read: mysql client client-server client-mariadb
The following options may be given as the first argument:
--print-defaults Print the program argument list and exit.
--no-defaults Don't read default options from any option file.
The following specify which files/groups are read (specified before other options):
--defaults-file=# Only read default options from the given file #.
--defaults-extra-file=# Read this file after the global files are read.
--defaults-group-suffix=# Additionally read default groups with # appended as a suffix.
tests running from build directory:
TEST: print defaults ignored as not first
$ sql/mysqld --no-defaults --print-defaults --lc-messages-dir=${PWD}/sql/share
TEST: no startup occurs as --print-defaults specified
$ sql/mysqld --print-defaults --lc-messages-dir=${PWD}/sql/share
sql/mysqld would have been started with the following arguments:
--lc-messages-dir=/home/dan/repos/build-mariadb-5.5/sql/share
TEST: default args can't be anywhere
$ client/mysql --user=bob --defaults-file=/etc/my.cnf
client/mysql: unknown variable 'defaults-file=/etc/my.cnf'
$ client/mysql --user=bob --defaults-group-suffix=.group
client/mysql: unknown variable 'defaults-group-suffix=.group'
/etc/my.cnf:
[client-server.group]
socket=/var/lib/mysql-multi/group/mysqld.sock
user=bob
/etc/my.other.cnf:
socket=/var/lib/mysql-other/mysqld.sock
TEST: defaults file read and suffix also applied
$ client/mysql --defaults-file=/etc/my.other.cnf --defaults-group-suffix=.group
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql-other/mysqld.sock' (2)
TEST: defaults extra file
$ client/mysql --defaults-extra-file=/etc/my.other.cnf --defaults-group-suffix=.group
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql-other/mysqld.sock' (2)
escape all charecters less or equal 0x1F (control symbols)
(shorter sequence are not used to make code simple, long encoding is always legal according to the rfc4627)
TRASH was mapped to TRASH_FREE and was supposed to be used for memory
that should not be accessed anymore, while TRASH_ALLOC() is to be
used for uninitialized but to-be-used memory.
But sometimes TRASH() was used in the latter sense.
Remove TRASH() macro, always use explicit TRASH_ALLOC() or TRASH_FREE().
Resolving a stacktrace including functions in dynamic libraries requires
us to look inside the libraries for the symbols. Addr2line needs to be
started with the correct binary for each address on the stack. To do this,
figure out which library it is using dladdr, then if the addr2line
binary was started with a different binary, fork it again with the
correct one.
We only have one addr2line process running at any point during the
stacktrace resolving step. The maximum number of forks for addr2line should
generally be around 6.
One for server stacktrace code, one for plugin code, one when going back
into server code, one for pthread library, one for libc, one for the
_start function in the server. More can come up if plugin calls server
function which goes back to a plugin, etc.
MDL_CONTEXT::TRY_ACQUIRE_LOCK_IMPL
ANALYSIS:
=========
Server sometimes exited when multiple threads tried to
acquire and release metadata locks simultaneously (for
example, necessary to access a table). The same problem
could have occurred when new objects were registered/
deregistered in Performance Schema.
The problem was caused by a bug in LF_HASH - our lock free
hash implementation which is used by metadata locking
subsystem in 5.7 branch. In 5.5 and 5.6 we only use LF_HASH
in Performance Schema Instrumentation implementation. So
for these versions, the problem was limited to P_S.
The problem was in my_lfind() function, which searches for
the specific hash element by going through the elements
list. During this search it loads information about element
checked such as key pointer and hash value into local
variables. Then it confirms that they are not corrupted by
concurrent delete operation (which will set pointer to 0)
by checking if element is still in the list. The latter
check did not take into account that compiler (and
processor) can reorder reads in such a way that load of key
pointer will happen after it, making result of the check
invalid.
FIX:
====
This patch fixes the problem by ensuring that no such
reordering can take place. This is achieved by using
my_atomic_loadptr() which contains compiler and processor
memory barriers for the check mentioned above and other
similar places.
The default (for non-Windows systems) implementation of
my_atomic*() relies on old __sync intrisics and implements
my_atomic_loadptr() as read-modify operation. To avoid
scalability/performance penalty associated with addition of
my_atomic_loadptr()'s we change the my_atomic*() to use
newer __atomic intrisics when available. This new default
implementation doesn't have such a drawback.
Reintroduce environment variable MYSQL_GROUP_SUFFIX to be used as
--default-group-suffix value if not already set.
The environment variable was accidentally renamed to DEFAULT_GROUP_SUFFIX_ENV
in MySQL server 5.5.
ERROR_FILE_SYSTEM_LIMITATION was seen by support when backing up large
file. However mariabackup error message was not very helpful,
since it mapped the error to generic catch-all EINVAL.
With the patch, ERROR_FILE_SYSTEM_LIMITATION will be mapped to more
appropriate EFBIG. Also add mapping from ERROR_NO_SYSTEM_RESOURCES
to ENOMEM.
A few tests assumes that the CYCLE timer is always available,
which is not true on some platforms (e.g. ARM).
Fixing the tests not to reply on the CYCLE availability.
SYMLINK CHECK RACE CONDITIONS
ANALYSIS:
=========
A potential defect exists in the handling of CREATE
TABLE .. DATA DIRECTORY/ INDEX DIRECTORY which gives way to
the user to gain access to another user table or a system
table.
FIX:
====
The lstat and fstat output of the target files are now
stored which help in determining the identity of the target
files thus preventing the unauthorized access to other
files.
mysql_client uses some inline assembly code to switch thread stacks.
This works, however tools that perform backtrace get confused to fix
this we write a specific constant to signify bottom of stack. This
constant is needed when compiling with CLang as well.
my_readline can fail due to missing file. Make my_readline report this
condition separately so that we can catch it and report an appropriate
error message to the user.
The reason for this is that stop slave takes LOCK_active_mi over the
whole operation while some slave operations will also need LOCK_active_mi
which causes deadlocks.
Fixed by introducing object counting for Master_info and not taking
LOCK_active_mi over stop slave or even stop_all_slaves()
Another benefit of this approach is that it allows:
- Multiple threads can run SHOW SLAVE STATUS at the same time
- START/STOP/RESET/SLAVE STATUS on a slave will not block other slaves
- Simpler interface for handling get_master_info()
- Added some missing unlock of 'log_lock' in error condtions
- Moved rpl_parallel_inactivate_pool(&global_rpl_thread_pool) to end
of stop_slave() to not have to use LOCK_active_mi inside
terminate_slave_threads()
- Changed argument for remove_master_info() to Master_info, as we always
have this available
- Fixed core dump when doing FLUSH TABLES WITH READ LOCK and parallel
replication. Problem was that waiting for pause_for_ftwrl was not done
when deleting rpt->current_owner after a force_abort.
it was race condition prone. instead use either a pair of my_delete()
calls with already resolved paths, or a safe high-level function
my_handler_delete_with_symlink(), like MyISAM and Aria already do.
TOCTOU bug. The path is checked to be valid, symlinks are resolved.
Then the resolved path is opened. Between the check and the open,
there's a window when one can replace some path component with a
symlink, bypassing validity checks.
Fix: after we resolved all symlinks in the path, don't allow open()
to resolve symlinks, there should be none.
Compared to the old MyISAM/Aria code:
* fastpath. Opening of not-symlinked files is just one open(),
no fn_format() and lstat() anymore.
* opening of symlinked tables doesn't do fn_format() and lstat() either.
it also doesn't to realpath() (which was lstat-ing every path
component), instead if opens every path component with O_PATH.
* share->data_file_name stores realpath(path) not readlink(path). So,
SHOW CREATE TABLE needs to do lstat/readlink() now (see ::info()),
and certain error messages (cannot open file "XXX") show the real
file path with all symlinks resolved.
Don't let my_register_filename() fail because strdup() failed. Better to
have NULL for a filename, then to fail the already successful open().
Filenames are only used for error reporting and there was already code
to ignore OOMs (my_fdopen()) and to cope with missing filenames
(my_filename()).
