it was race condition prone. instead use either a pair of my_delete()
calls with already resolved paths, or a safe high-level function
my_handler_delete_with_symlink(), like MyISAM and Aria already do.
TOCTOU bug. The path is checked to be valid, symlinks are resolved.
Then the resolved path is opened. Between the check and the open,
there's a window when one can replace some path component with a
symlink, bypassing validity checks.
Fix: after we resolved all symlinks in the path, don't allow open()
to resolve symlinks, there should be none.
Compared to the old MyISAM/Aria code:
* fastpath. Opening of not-symlinked files is just one open(),
no fn_format() and lstat() anymore.
* opening of symlinked tables doesn't do fn_format() and lstat() either.
it also doesn't to realpath() (which was lstat-ing every path
component), instead if opens every path component with O_PATH.
* share->data_file_name stores realpath(path) not readlink(path). So,
SHOW CREATE TABLE needs to do lstat/readlink() now (see ::info()),
and certain error messages (cannot open file "XXX") show the real
file path with all symlinks resolved.
my_thread_global_init() + my_thrad_global_end() repeatadily.
This caused THR_KEY_mysys to be allocated multiple times.
Deletion of THR_KEY_mysys was originally in my_thread_global_end() but was
moved to my_end() as DBUG uses THR_KEY_mysys and DBUG is released after
my_thread_global_end() is called.
Releasing DBUG before my_thread_global_end() and move THR_KEY_mysys back
into my_thread_global_end() could be a solution, but as safe_mutex and other
things called by my_thread_global_end is using DBUG it may not be completely
safe.
To solve this, I used the simple solution to add a marker that THR_KEY_mysys
is created and not re-create it in my_thread_global_init if it already
exists.
it was supposed to be used in command-line tools only.
Different fix for 4e5473862e:
Bug#24388746: PRIVILEGE ESCALATION AND RACE CONDITION USING CREATE TABLE
Both aria and myisam storage engines feature a logic path in
thr_find_all_keys that leads to undefined behaviour by bypassing the
initialization code of variables after my_thread_init().
By refactoring the nested logic into a separate function, this problem
is resolved.
Fix test whether process is alive in mysqltest.
Also fix SHUT_RD definition on Windows to be SD_RECEIVE.
SD_BOTH was used instead prior to this patch, and this would
occasionally make mysql_shutdown() fail - when the socket for the current connection
is not able send the COM_SHUTDOWN response anymore.
UNIQUE::~UNIQUE | SQL/UNIQUES.CC:355
Analysis
========
Enabling the sort_buffer_size with a large value
can cause operations utilizing the sort buffer
like DELETE as mentioned in the bug report to
fail. 5.5 and 5.6 versions reports OOM error
while in 5.7+, the server crashes.
While initializing the mem_root for the sort buffer
tree, the block size for the mem_root is determined
from the 'sort_buffer_size' value. This unsigned
long value is typecasted to unsigned int, hence
it becomes zero. Further block_size computation
while initializing the mem_root results in a very
large block_size value. Hence while trying to
allocate a block during the DELETE operation,
an OOM error is reported. In case of 5.7+, the PFS
instrumentation for memory allocation, overshoots
the unsigned value and allocates a block of just
one byte. While trying to free the block of the
mem_root, the original block_size is used. This
triggers the crash since the server tries to free
unallocated memory.
Fix:
====
In order to restrict usage of such unreasonable
sort_buffer_size, the typecast of block size
to 'unsigned int' is removed and hence reports
OOM error across all versions for sizes
exceeding unsigned int range.
1. check that unused inline functions are removed
2. only allow compilation if they are or if the check if overridden
3. with CMAKE_GENERATOR=Makefiles, use all flags when testing
(e.g. both CMAKE_C_FLAGS and CMAKE_C_FLAGS_DEBUG if
CMAKE_BUILD_TYPE=Debug). This is because
- on Solaris with the SunPro compiler, default CMAKE_C_FLAGS_xxx
values contain -xO2 (for Release and RelWithDebInfo)
and -g (for RelWithDebInfo and Debug)
- proper inlining only works at -xO4 without -g
- so if CMAKE_C_FLAGS has -xO4, inlining would work in
configure.cmake (before this fix) and fail during actual compilation
also remove the outdated check for inline from myu_global.h
cherry-pick f1daf9ce from 10.0 branch
-------------------------------------
Fix build failures caused by new C runtime library
- isnan, snprintf, struct timespec are now defined, attempt to
redefine them leads
- P_tmpdir, tzname are no more defined
- lfind() and lsearch() in lf_hash.c had to be renamed, declaration
conflicts with some C runtime functions with the same name declared in
a header included by stdlib.h
Also fix couple of annoying warnings :
- remove #define NOMINMAX from config.h to avoid "redefined" compiler
warnings(NOMINMAX is already in compile flags)
- disable incremental linker in Debug as well (feature not used much
and compiler crashes often)
Also simplify package building with Wix, require Wix 3.9 or later
(VS2015 is not compatible with old Wix 3.5/3.6)
if mysqld runs as service
The bug is caused by the attempt to write to filedescriptor 2
(STDERR_FILENO), however in case of a service stderr has different fd
(debugging shows fileno(stderr) is 4 after freopen, and -1 before it)
Fixed definition of STDERR_FILENO for Windows to be fileno(stderr).
The check for UCONTEXT in cmake was somehow become broken, disabling the
fallback to ucontext. This caused the non-blocking client API to not be
available for non-x86 platforms, on which no hand-crafted assembler
implementation of my_context is available.
Problem Statement
=========
Fix various issues when building MySQL with Visual Studio 2015.
Fix:
=======
- Visual Studio 2015 adds support for timespec. Add check and
related code to use this and only use our replacement if
timespec is not defined.
- Rename lfind/lsearch to my* to avoid redefinition problems.
- Set default value for TMPDIR to "" on Windows as P_tmpdir
no longer exists.
- using VS definition of snprintf if available
- tzname are now renamed to _tzname.
Description: The newest RHEL/CentOS/SL 6.6 openssl package
(1.0.1e-30.el6_6.9; published around 6/4/2015) contains a fix for
LogJam. RedHat's fix for this was to limit the use
of any SSL DH key sizes to a minimum of 768 bits. This breaks any
DHE SSL ciphers for MySQL clients as soon as you install the
openssl update, since in vio/viosslfactories.c, the default
DHPARAM is a 512 bit one. This cannot be changed in
configuration/runtime; and needs a recompile. Because of this the
client connection with --ssl-cipher=DHE-RSA-AES256-SHA is not
able to connect the server.
Analysis: Openssl has changed Diffie-Hellman key from the 512 to
1024 due to some reasons(please see the details at
http://openssl.org/news/secadv_20150611.txt) Because of this the client
with DHE cipher is failing to connect the server. This change took
place from the openssl-1.0.1n onwards.
Fix: Similar bug fix is already pushed to mysql-5.7 under bug#18367167.
Hence we backported the same fix to mysql-5.5 and mysql-5.6.
Start the client API for low-level protocol access:
* unsigned long mysql_net_read_packet(MYSQL *mysql);
* unsigned long mysql_net_field_length(unsigned char **packet);
Problem was that repair() did lock and unlock tables, which leaved already locked tables in wrong state
include/my_check_opt.h:
Added option T_NO_LOCKS to disable locking during repair()
Fixed duplicated bit T_NO_CREATE_RENAME_LSN
mysql-test/suite/rpl/r/myisam_external_lock.result:
Test case for MDEV-6871
mysql-test/suite/rpl/t/myisam_external_lock-slave.opt:
Test case for MDEV-6871
mysql-test/suite/rpl/t/myisam_external_lock.test:
Test case for MDEV-6871
storage/maria/ha_maria.cc:
Don't lock tables during enable_indexes()
Removed some calls to current_thd
storage/myisam/ha_myisam.cc:
Don't lock tables during enable_indexes()
Removed some calls to current_thd
Problem: For every event read, mysqlbinlog calls localtime() which in turn
calls stat(/etc/localtime) which is causing kernel mutex contention.
Analysis and Fix:
localtime() calls stat(/etc/localtime) for every instance of the call
where as localtime_r() the reentrant version was optimized to store
the read only tz internal structure. Hence it will not call
stat(/etc/localtime). It will call only once at the beginning.
The mysql server is calling localtime_r() and mysqlbinlog tool is
one place where we are still using localtime().
Once the process (mysqlbinlog) is started if timezone is changed
it will be not picked up the the process and it will continue
with the same values as the beginning of the process. This
behavior is in-lined with mysql server.
Also adding localtime_r() and gmtime_r() support for windows.
Problem: For every event read, mysqlbinlog calls localtime() which in turn
calls stat(/etc/localtime) which is causing kernel mutex contention.
Analysis and Fix:
localtime() calls stat(/etc/localtime) for every instance of the call
where as localtime_r() the reentrant version was optimized to store
the read only tz internal structure. Hence it will not call
stat(/etc/localtime). It will call only once at the beginning.
The mysql server is calling localtime_r() and mysqlbinlog tool is
one place where we are still using localtime().
Once the process (mysqlbinlog) is started if timezone is changed
it will be not picked up the the process and it will continue
with the same values as the beginning of the process. This
behavior is in-lined with mysql server.
Also adding localtime_r() and gmtime_r() support for windows.
Part of this work is based on Stewart Smitch's memory barrier and lower priori
patches for power8.
- Added memory syncronization for innodb & xtradb for power8.
- Added HAVE_WINDOWS_MM_FENCE to CMakeList.txt
- Added os_isync to fix a syncronization problem on power
- Added log_get_lsn_nowait which is now used srv_error_monitor_thread to ensur
if log mutex is locked.
All changes done both for InnoDB and Xtradb
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
MDEV-6099 Bad results for DATE_ADD(.., INTERVAL 2000000000000000000.0 SECOND)
MDEV-6097 Inconsistent results for CAST(int,decimal,double AS DATETIME)
MDEV-6100 No warning on CAST(9000000 AS TIME)
SHOW PROCESSLIST, SHOW BINLOGS
Problem: A deadlock was occurring when 4 threads were
involved in acquiring locks in the following way
Thread 1: Dump thread ( Slave is reconnecting, so on
Master, a new dump thread is trying kill
zombie dump threads. It acquired thread's
LOCK_thd_data and it is about to acquire
mysys_var->current_mutex ( which LOCK_log)
Thread 2: Application thread is executing show binlogs and
acquired LOCK_log and it is about to acquire
LOCK_index.
Thread 3: Application thread is executing Purge binary logs
and acquired LOCK_index and it is about to
acquire LOCK_thread_count.
Thread 4: Application thread is executing show processlist
and acquired LOCK_thread_count and it is
about to acquire zombie dump thread's
LOCK_thd_data.
Deadlock Cycle:
Thread 1 -> Thread 2 -> Thread 3-> Thread 4 ->Thread 1
The same above deadlock was observed even when thread 4 is
executing 'SELECT * FROM information_schema.processlist' command and
acquired LOCK_thread_count and it is about to acquire zombie
dump thread's LOCK_thd_data.
Analysis:
There are four locks involved in the deadlock. LOCK_log,
LOCK_thread_count, LOCK_index and LOCK_thd_data.
LOCK_log, LOCK_thread_count, LOCK_index are global mutexes
where as LOCK_thd_data is local to a thread.
We can divide these four locks in two groups.
Group 1 consists of LOCK_log and LOCK_index and the order
should be LOCK_log followed by LOCK_index.
Group 2 consists of other two mutexes
LOCK_thread_count, LOCK_thd_data and the order should
be LOCK_thread_count followed by LOCK_thd_data.
Unfortunately, there is no specific predefined lock order defined
to follow in the MySQL system when it comes to locks across these
two groups. In the above problematic example,
there is no problem in the way we are acquiring the locks
if you see each thread individually.
But If you combine all 4 threads, they end up in a deadlock.
Fix:
Since everything seems to be fine in the way threads are taking locks,
In this patch We are changing the duration of the locks in Thread 4
to break the deadlock. i.e., before the patch, Thread 4
('show processlist' command) mysqld_list_processes()
function acquires LOCK_thread_count for the complete duration
of the function and it also acquires/releases
each thread's LOCK_thd_data.
LOCK_thread_count is used to protect addition and
deletion of threads in global threads list. While show
process list is looping through all the existing threads,
it will be a problem if a thread is exited but there is no problem
if a new thread is added to the system. Hence a new mutex is
introduced "LOCK_thd_remove" which will protect deletion
of a thread from global threads list. All threads which are
getting exited should acquire LOCK_thd_remove
followed by LOCK_thread_count. (It should take LOCK_thread_count
also because other places of the code still thinks that exit thread
is protected with LOCK_thread_count. In this fix, we are changing
only 'show process list' query logic )
(Eg: unlink_thd logic will be protected with
LOCK_thd_remove).
Logic of mysqld_list_processes(or file_schema_processlist)
will now be protected with 'LOCK_thd_remove' instead of
'LOCK_thread_count'.
Now the new locking order after this patch is:
LOCK_thd_remove -> LOCK_thd_data -> LOCK_log ->
LOCK_index -> LOCK_thread_count
SHOW PROCESSLIST, SHOW BINLOGS
Problem: A deadlock was occurring when 4 threads were
involved in acquiring locks in the following way
Thread 1: Dump thread ( Slave is reconnecting, so on
Master, a new dump thread is trying kill
zombie dump threads. It acquired thread's
LOCK_thd_data and it is about to acquire
mysys_var->current_mutex ( which LOCK_log)
Thread 2: Application thread is executing show binlogs and
acquired LOCK_log and it is about to acquire
LOCK_index.
Thread 3: Application thread is executing Purge binary logs
and acquired LOCK_index and it is about to
acquire LOCK_thread_count.
Thread 4: Application thread is executing show processlist
and acquired LOCK_thread_count and it is
about to acquire zombie dump thread's
LOCK_thd_data.
Deadlock Cycle:
Thread 1 -> Thread 2 -> Thread 3-> Thread 4 ->Thread 1
The same above deadlock was observed even when thread 4 is
executing 'SELECT * FROM information_schema.processlist' command and
acquired LOCK_thread_count and it is about to acquire zombie
dump thread's LOCK_thd_data.
Analysis:
There are four locks involved in the deadlock. LOCK_log,
LOCK_thread_count, LOCK_index and LOCK_thd_data.
LOCK_log, LOCK_thread_count, LOCK_index are global mutexes
where as LOCK_thd_data is local to a thread.
We can divide these four locks in two groups.
Group 1 consists of LOCK_log and LOCK_index and the order
should be LOCK_log followed by LOCK_index.
Group 2 consists of other two mutexes
LOCK_thread_count, LOCK_thd_data and the order should
be LOCK_thread_count followed by LOCK_thd_data.
Unfortunately, there is no specific predefined lock order defined
to follow in the MySQL system when it comes to locks across these
two groups. In the above problematic example,
there is no problem in the way we are acquiring the locks
if you see each thread individually.
But If you combine all 4 threads, they end up in a deadlock.
Fix:
Since everything seems to be fine in the way threads are taking locks,
In this patch We are changing the duration of the locks in Thread 4
to break the deadlock. i.e., before the patch, Thread 4
('show processlist' command) mysqld_list_processes()
function acquires LOCK_thread_count for the complete duration
of the function and it also acquires/releases
each thread's LOCK_thd_data.
LOCK_thread_count is used to protect addition and
deletion of threads in global threads list. While show
process list is looping through all the existing threads,
it will be a problem if a thread is exited but there is no problem
if a new thread is added to the system. Hence a new mutex is
introduced "LOCK_thd_remove" which will protect deletion
of a thread from global threads list. All threads which are
getting exited should acquire LOCK_thd_remove
followed by LOCK_thread_count. (It should take LOCK_thread_count
also because other places of the code still thinks that exit thread
is protected with LOCK_thread_count. In this fix, we are changing
only 'show process list' query logic )
(Eg: unlink_thd logic will be protected with
LOCK_thd_remove).
Logic of mysqld_list_processes(or file_schema_processlist)
will now be protected with 'LOCK_thd_remove' instead of
'LOCK_thread_count'.
Now the new locking order after this patch is:
LOCK_thd_remove -> LOCK_thd_data -> LOCK_log ->
LOCK_index -> LOCK_thread_count
INTO CLIENT_ERRORS ARRAY
Analysis:
--------
The client may crash while executing a statement due to
the missing mapping of the server error to it's equivalent
client error.
When trying to reallocate memory for the packet buffer, if
the system is out of memory or the packet buffer is large,
the server errors 'ER_OUT_OF_RESOURCES' or 'ER_PACKET_TOO_LARGE'
is returned respectively. The client error number calculated is
negative and when trying to dereference the array of client
error messages with the calculated error number, the client
crashes.
Fix:
----
Map the server error returned to it's equivalent client error
prior to dereferencing the array of client error messages.
Note: Test case is not added since it is difficult to simulate
the error condition.
INTO CLIENT_ERRORS ARRAY
Analysis:
--------
The client may crash while executing a statement due to
the missing mapping of the server error to it's equivalent
client error.
When trying to reallocate memory for the packet buffer, if
the system is out of memory or the packet buffer is large,
the server errors 'ER_OUT_OF_RESOURCES' or 'ER_PACKET_TOO_LARGE'
is returned respectively. The client error number calculated is
negative and when trying to dereference the array of client
error messages with the calculated error number, the client
crashes.
Fix:
----
Map the server error returned to it's equivalent client error
prior to dereferencing the array of client error messages.
Note: Test case is not added since it is difficult to simulate
the error condition.
