Problem Statement
=========
Fix various issues when building MySQL with Visual Studio 2015.
Fix:
=======
- Visual Studio 2015 adds support for timespec. Add check and
related code to use this and only use our replacement if
timespec is not defined.
- Rename lfind/lsearch to my* to avoid redefinition problems.
- Set default value for TMPDIR to "" on Windows as P_tmpdir
no longer exists.
- using VS definition of snprintf if available
- tzname are now renamed to _tzname.
The assertion happens when: (i) the master and slave are configured to
use the semisync plugin; (ii) the DBA disables semisync on the master;
(iii) and he also unsets the option to wait for slaves ACK even if the
semisync slave count reaches 0 during the waiting period. This
combination of factors makes the server run into an assertion as soon
as the last semisync slave disconnects and its dump thread exits.
The root of the problem is the fact that when the dump thread
disconnects and calls the observer hook transmit_stop, which ends up
calling ReplSemiSyncMaster::remove_slave, there is no check whether
the master has already disabled semisync or not. If it has, the then a
second call to the switch_off member function must be avoided.
The quick fix is to avoid calling switch_off if the DBA has disabled
the semisync plugin interactively on the master. Also, the switch_off
member function should only be called if the plugin has not been
switched off already. This is basically the pattern throughout the
rest of the semisync plugin and no other calls seem vulnerable to
similar crashes/assertions.
(This a backport of the patch to 5.5, which is also vulnerable.)
The assertion happens when: (i) the master and slave are configured to
use the semisync plugin; (ii) the DBA disables semisync on the master;
(iii) and he also unsets the option to wait for slaves ACK even if the
semisync slave count reaches 0 during the waiting period. This
combination of factors makes the server run into an assertion as soon
as the last semisync slave disconnects and its dump thread exits.
The root of the problem is the fact that when the dump thread
disconnects and calls the observer hook transmit_stop, which ends up
calling ReplSemiSyncMaster::remove_slave, there is no check whether
the master has already disabled semisync or not. If it has, the then a
second call to the switch_off member function must be avoided.
The quick fix is to avoid calling switch_off if the DBA has disabled
the semisync plugin interactively on the master. Also, the switch_off
member function should only be called if the plugin has not been
switched off already. This is basically the pattern throughout the
rest of the semisync plugin and no other calls seem vulnerable to
similar crashes/assertions.
(This a backport of the patch to 5.5, which is also vulnerable.)
RPL_SEMI_SYNC_MASTER_ENABLED OFF.
Problem:
=======
If master is waiting for a reply from slave, at this time
set global rpl_semi_sync_master_enabled=OFF, the master
server will crash.
Analysis:
========
When master is waiting for a reply from slave, at this time
if semi sync is switched off on master, during switch off if
active transactions are present the transactions will be
cleared and "active_tranxs_" variable will be set to NULL.
When the waiting master connection finds that semi sync is
switched of it tries to access "active_tranxs_" without
checking if the transaction list exists or not. Accessing
NULL transaction list causes the crash.
Fix:
===
A check has been added to see a valid list exists before
accessing the "active_tranxs_".
plugin/semisync/semisync_master.cc:
Added check for the existence of valid 'active_transx_'.
RPL_SEMI_SYNC_MASTER_ENABLED OFF.
Problem:
=======
If master is waiting for a reply from slave, at this time
set global rpl_semi_sync_master_enabled=OFF, the master
server will crash.
Analysis:
========
When master is waiting for a reply from slave, at this time
if semi sync is switched off on master, during switch off if
active transactions are present the transactions will be
cleared and "active_tranxs_" variable will be set to NULL.
When the waiting master connection finds that semi sync is
switched of it tries to access "active_tranxs_" without
checking if the transaction list exists or not. Accessing
NULL transaction list causes the crash.
Fix:
===
A check has been added to see a valid list exists before
accessing the "active_tranxs_".
If rpl_semi_sync_master_timeout is large, calculation of absolute waiting time in semi-sync plugin is inefficient. This error is specific to systems with 64 bit long values (all 64 bit Unixes)
In rpl_semi_sync_master_timeout has maximal value (= MAX_ULONGLONG), calculating abstime may require ~ 18 billion subtract operations.
The fix is to use division instead of subtraction-in-a-loop. Also fixed an integer overflow bug.
client/mysqldump.c:
Slave needs to be initialized with 0
dbug/dbug.c:
Removed not existing function
plugin/semisync/semisync_master.cc:
Fixed compiler warning
sql/opt_range.cc:
thd needs to be set early as it's used in some error conditions.
sql/sql_table.cc:
Changed to use uchar* to make array indexing portable
storage/innobase/handler/ha_innodb.cc:
Removed not used variable
storage/maria/ma_delete.c:
Fixed compiler warning
storage/maria/ma_write.c:
Fixed compiler warning
The semisync code does a fast-but-unsafe check for enabled or not without lock,
followed by a slow-but-safe check under lock. However, if the slow check failed,
the code still referenced not valid data (in an assert() expression), causing a
crash.
Fixed by not running the incorrect assert when semisync is disabled.
bug in semisync plugin. It didn't check thd->killed before waiting on mysys->current_cond,
and thus an attepmt to kill the thread (on shutdown) was lost
plugin/semisync/semisync_master.cc:
mdev:201 lp:962540
The autotools-based build system has been superseded and
is being removed in order to ease the maintenance burden on
developers tweaking and maintaining the build system.
In order to support tools that need to extract the server
version, a new file that (only) contains the server version,
called VERSION, is introduced. The file contents are human
and machine-readable. The format is:
MYSQL_VERSION_MAJOR=5
MYSQL_VERSION_MINOR=5
MYSQL_VERSION_PATCH=8
MYSQL_VERSION_EXTRA=-rc
The CMake based version extraction in cmake/mysql_version.cmake
is changed to extract the version from this file. The configure
to CMake wrapper is retained for backwards compatibility and to
support the BUILD/ scripts. Also, a new a makefile target
show-dist-name that prints the server version is introduced.
VERSION:
Add top-level version file.
cmake/mysql_version.cmake:
Get version information from the top-level VERSION file.
Do not cache the version components (MAJOR_VERSION, etc).
Add MYSQL_RPM_VERSION as a replacement for MYSQL_U_SCORE_VERSION.
The autotools-based build system has been superseded and
is being removed in order to ease the maintenance burden on
developers tweaking and maintaining the build system.
In order to support tools that need to extract the server
version, a new file that (only) contains the server version,
called VERSION, is introduced. The file contents are human
and machine-readable. The format is:
MYSQL_VERSION_MAJOR=5
MYSQL_VERSION_MINOR=5
MYSQL_VERSION_PATCH=8
MYSQL_VERSION_EXTRA=-rc
The CMake based version extraction in cmake/mysql_version.cmake
is changed to extract the version from this file. The configure
to CMake wrapper is retained for backwards compatibility and to
support the BUILD/ scripts. Also, a new a makefile target
show-dist-name that prints the server version is introduced.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
Also, update a few cases missed by the initial patch.
client/mysqltest.cc:
Remove trailing comma.
include/my_alloc.h:
Do not use wrapper.
include/mysql.h.pp:
Update ABI file.
plugin/semisync/semisync_master.h:
Initialize variable.
sql/debug_sync.cc:
Use C linkage.
Fix various mismatches between function's language linkage. Any
particular function that is declared in C++ but should be callable
from C must have C linkage. Note that function types with different
linkages are also distinct. Thus, if a function type is declared in
C code, it will have C linkage (same if declared in a extern "C"
block).
client/mysql.cc:
Mismatch between prototype and declaration.
client/mysqltest.cc:
mysqltest used to be C code. Use C linkage where appropriate.
cmd-line-utils/readline/input.c:
Isolate unreachable code.
include/my_alloc.h:
Function type must have C linkage.
include/my_base.h:
Function type must have C linkage.
include/my_global.h:
Add helper macros to avoid spurious namespace indentation.
include/mysql.h.pp:
Update ABI file.
mysys/my_gethwaddr.c:
Remove stray carriage return and fix coding style.
plugin/semisync/semisync_master_plugin.cc:
Callback function types have C linkage.
plugin/semisync/semisync_slave_plugin.cc:
Callback function types have C linkage.
sql/derror.cc:
Expected function type has C linkage.
sql/field.cc:
Use helper macro and fix indentation.
sql/handler.cc:
Expected function type has C linkage.
sql/item_sum.cc:
Correct function linkages. Remove now unnecessary cast.
sql/item_sum.h:
Add prototypes with the appropriate linkage as otherwise they
are distinct.
sql/mysqld.cc:
Wrap functions in C linkage mode.
sql/opt_range.cc:
C language linkage is ignored for class member functions.
sql/partition_info.cc:
Add wrapper functions with C linkage for class member functions.
sql/rpl_utility.h:
Use helper macro and fix indentation.
sql/sql_class.cc:
Change type of thd argument -- THD is a class.
Use helper macro and fix indentation.
sql/sql_class.h:
Change type of thd argument -- THD is a class.
sql/sql_select.cc:
Expected function type has C linkage.
sql/sql_select.h:
Move prototype to sql_test.h
sql/sql_show.cc:
Expected function type has C linkage.
sql/sql_test.cc:
Fix required function prototype and fix coding style.
sql/sql_test.h:
Removed unnecessary export and add another.
storage/myisammrg/ha_myisammrg.cc:
Expected function type has C linkage.
storage/perfschema/pfs.cc:
PSI headers are declared with C language linkage, which also
applies to function types.
