This patch fixes three bugs as follows. First, aborting the server while purging
binary logs might generate orphan files due to how the purge operation was
implemented:
(purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)
1 - register the files to be removed in a temporary buffer.
2 - update the log-bin.index.
3 - flush the log-bin.index.
4 - erase the files whose names where register in the temporary buffer
in step 1.
Thus a failure while executing step 4 would generate an orphan file. Second,
a similar issue might happen while creating a new binary as follows:
(create routine - sql/log.cc - MYSQL_BIN_LOG::open)
1 - open the new log-bin.
2 - update the log-bin.index.
Thus a failure while executing step 1 would generate an orphan file.
To fix these issues, we record the files to be purged or created before really
removing or adding them. So if a failure happens such records can be used to
automatically remove dangling files. The new steps might be outlined as follows:
(purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)
1 - register the files to be removed in the log-bin.~rec~ placed in
the data directory.
2 - update the log-bin.index.
3 - flush the log-bin.index.
4 - delete the log-bin.~rec~.
(create routine - sql/log.cc - MYSQL_BIN_LOG::open)
1 - register the file to be created in the log-bin.~rec~
placed in the data directory.
2 - open the new log-bin.
3 - update the log-bin.index.
4 - delete the log-bin.~rec~.
(recovery routine - sql/log.cc - MYSQL_BIN_LOG::open_index_file)
1 - open the log-bin.index.
2 - open the log-bin.~rec~.
3 - for each file in log-bin.~rec~.
3.1 Check if the file is in the log-bin.index and if so ignore it.
3.2 Otherwise, delete it.
The third issue can be described as follows. The purge operation was allowing
to remove a file in use thus leading to the loss of data and possible
inconsistencies between the master and slave. Roughly, the routine was only
taking into account the dump threads and so if a slave was not connect the
file might be delete even though it was in use.
implement Davi's review suggestions (post-push fixes)
include/violite.h:
Use official abbreviation for milliseconds (ms)
sql/mysqld.cc:
Fix formatting
Add error handling for the case of CreateEvent error
vio/vio.c:
Use official abbreviation for milliseconds(ms)
Remove superfluous memset
Fix formatting
vio/viosocket.c:
Use official abbreviation for milliseconds (ms)
Use size_t datatype instead of int in pipe_complete_io
If --log-bin is set to a directory name with the trailing 'FN_LIBCHAR',
which will be '/' on Unix like systems, and '\\' on Windows like systems.
the basename of the binlog is empty so that the created files named
'.000001' and '.index'. It is not expected.
The same thing happened to --log-bin-index, --relay-log and
--relay-log-index options.
To resolve the problem, in these cases the program should report an error
and abort.
sql/mysqld.cc:
Added a check for the value of the --log-bin and --log-bin-index arguments,
if it's a directory, reports an error and aborts.
sql/rpl_rli.cc:
Added a check for the value of the --relay-log and --relay-log-index arguments,
if it's a directory, reports an error and aborts.
Bug#31621: Windows server hanging during shutdown using named pipes
and idle connection
Problem: when idle pipe connection is forcefully closed with KILL
statement or when the server goes down, thread that is closing connection
would hang infinitely in CloseHandle(). The reason for the hang is that
named pipe operations are performed synchronously. In this mode all IOs
on pipe are serialized, that is CloseHandle() will not abort ReadFile()
in another thread, but wait for ReadFile() to complete.
The fix implements asynchrnous mode for named pipes, where operation of file
are not synchronized. Read/Write operation would fire an async IO and wait for
either IO completion or timeout.
Note, that with this patch timeouts are properly handled for named pipes.
Post-review: Win32 timeout code has been fixed for named pipes and shared
memory. We do not store pointer to NET in vio structure, only the read and
write timeouts.
include/violite.h:
Add pipe_overlapped to Vio structure for async IO for named pipes.
sql-common/client.c:
Use asynchronous pipe IO.
sql/mysqld.cc:
Use asynchronous pipe IO.
vio/vio.c:
-Refactor timeouts for win32 protocols: shared memory and named pipes.
Store read/write timeout in VIO structure, instead of storing pointer
to NET. New function vio_win32_timeout called indirectly via
vio_timeout changes these values.
vio/vio_priv.h:
Remove vio_ignore_timeout.
Add vio_win32_timeout to be used for named pipes and shared memory.
vio/viosocket.c:
Use async IO for named pipes.
After issuing IO, wait for either IO completion, pipe_close_event
or timeout.
Refactor timeouts for named pipe and shared memory.
Implemented the server infrastructure for the fix:
1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of
thd_query()
2. Did some encapsulation in THD : aggregated thd_query and
thd_query_length into a LEX_STRING and made accessor and mutator
methods for easy code updating.
3. Updated the server code to use the new methods where applicable.
Backport from 6.0 to 5.1.
Only those sync points are included, which are used in debug_sync.test.
The Debug Sync Facility allows to place synchronization points
in the code:
open_tables(...)
DEBUG_SYNC(thd, "after_open_tables");
lock_tables(...)
When activated, a sync point can
- Send a signal and/or
- Wait for a signal
Nomenclature:
- signal: A value of a global variable that persists
until overwritten by a new signal. The global
variable can also be seen as a "signal post"
or "flag mast". Then the signal is what is
attached to the "signal post" or "flag mast".
- send a signal: Assign the value (the signal) to the global
variable ("set a flag") and broadcast a
global condition to wake those waiting for
a signal.
- wait for a signal: Loop over waiting for the global condition until
the global value matches the wait-for signal.
Please find more information in the top comment in debug_sync.cc
or in the worklog entry.
.bzrignore:
WL#4259 - Debug Sync Facility
Added the symbolic link libmysqld/debug_sync.cc.
CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added definition for ENABLED_DEBUG_SYNC.
configure.in:
WL#4259 - Debug Sync Facility
Added definition for ENABLED_DEBUG_SYNC.
include/my_sys.h:
WL#4259 - Debug Sync Facility
Added definition for the DEBUG_SYNC_C macro.
libmysqld/CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added sql/debug_sync.cc.
libmysqld/Makefile.am:
WL#4259 - Debug Sync Facility
Added sql/debug_sync.cc.
mysql-test/include/have_debug_sync.inc:
WL#4259 - Debug Sync Facility
New include file.
mysql-test/mysql-test-run.pl:
WL#4259 - Debug Sync Facility
Added option --debug_sync_timeout.
mysql-test/r/debug_sync.result:
WL#4259 - Debug Sync Facility
New test result.
mysql-test/r/have_debug_sync.require:
WL#4259 - Debug Sync Facility
New require file.
mysql-test/t/debug_sync.test:
WL#4259 - Debug Sync Facility
New test file.
mysys/my_static.c:
WL#4259 - Debug Sync Facility
Added definition for debug_sync_C_callback_ptr.
mysys/thr_lock.c:
WL#4259 - Debug Sync Facility
Added sync point "wait_for_lock".
sql/CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added debug_sync.cc and debug_sync.h.
sql/Makefile.am:
WL#4259 - Debug Sync Facility
Added debug_sync.cc and debug_sync.h.
sql/debug_sync.cc:
WL#4259 - Debug Sync Facility
New source file.
sql/debug_sync.h:
WL#4259 - Debug Sync Facility
New header file.
sql/mysqld.cc:
WL#4259 - Debug Sync Facility
Added opt_debug_sync_timeout.
Added calls to debug_sync_init() and debug_sync_end().
Fixed a purecov comment (unrelated).
sql/set_var.cc:
WL#4259 - Debug Sync Facility
Added server variable "debug_sync".
sql/set_var.h:
WL#4259 - Debug Sync Facility
Added declaration for server variable "debug_sync".
sql/share/errmsg.txt:
WL#4259 - Debug Sync Facility
Added error messages ER_DEBUG_SYNC_TIMEOUT and ER_DEBUG_SYNC_HIT_LIMIT.
sql/sql_base.cc:
WL#4259 - Debug Sync Facility
Added sync points "after_flush_unlock" and "before_lock_tables_takes_lock".
sql/sql_class.cc:
WL#4259 - Debug Sync Facility
Added initialization for debug_sync_control to THD::THD.
Added calls to debug_sync_init_thread() and debug_sync_end_thread().
sql/sql_class.h:
WL#4259 - Debug Sync Facility
Added element debug_sync_control to THD.
storage/myisam/myisamchk.c:
Fixed a typo in an error message string (unrelated).
when compiled with Sun Studio compiler).
The thing is that Sun Studio compiler calls destructor of stack
objects when pthread_exit() is called. That triggered an assertion
in DBUG_ENTER()/DBUG_RETURN() validation logic (if DBUG_ENTER() is
used in the beginning of function, all returns should be replaced
by DBUG_RETURN/DBUG_VOID_RETURN macros).
A fix is to explicitly use DBUG_LEAVE macro.
When parsing the service installation parameter in
default_service_handling() make sure the value of the
optional parameter doesn't overwrite it's name.
- Define and pass compile time path variables as pre-processor definitions to
mimic the makefile build.
- Set new CMake version and policy requirements explicitly.
- Changed DATADIR to MYSQL_DATADIR to avoid conflicting definition in
Platform SDK header ObjIdl.h which also defines DATADIR.
Initialize LOCK_open as a adapative mutex on platforms where the
PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP macro is available. The flag
indicates that a thread should spin (busy wait) for some time on a
locked adaptive mutex before blocking (sleeping). It's intended to
to alleviate performance problems due to LOCK_open being a highly
contended mutex.
sql/mysqld.cc:
Initialize LOCK_open as a adapative mutex.
The maximum value of the max_join_size variable is set by converting
a signed type (long int) with negative value (-1) to a wider unsigned
type (unsigned long long), which yields the largest possible value of
the wider unsigned type -- as per the language conversion rules. But,
depending on build options, the type of the max_join_size might be a
shorter type (ha_rows - unsigned long) which causes the warning to be
thrown once the large value is truncated to fit.
The solution is to ensure that the maximum value of the variable is
always set to the maximum value of integer type of max_join_size.
Furthermore, it would be interesting to always have a fixed type for
this variable, but this would incur in a change of behavior which is
not acceptable for a GA version. See Bug#35346.
sql/mysqld.cc:
Set max value for type.
not logged
Errors encountered during initialization of the SSL subsystem
are printed to stderr, rather than to the error log.
This patch adds a parameter to several SSL init functions to
report the error (if any) out to the caller. The function
init_ssl() in mysqld.cc is moved after the initialization of
the log subsystem, so that any error messages can be logged to
the error log. Printing of messages to stderr has been
retained to get diagnostic output in a client context.
include/violite.h:
Adding an enumeration for the various errors that can
occur during initialization of the SSL module.
sql/mysqld.cc:
Adding more logging of SSL init errors, and moving
init_ssl() till after initialization of logging
subsystem.
vio/viosslfactories.c:
Define error strings, provide an access method for these
strings, and maintain an error parameter in several funcs
to return the error (if any) to the caller.
The reason for the crash was rotate_relay_log (mi=0x0) did not verify
the passed value of active_mi. There are more cases where active_mi
is supposed to be non-zero e.g change_master(), stop_slave(), and it's
reasonable to protect from a similar crash all of them with common
fixes.
Fixed with spliting end_slave() in slave threads release and slave
data clean-up parts (a new close_active_mi()). The new function is
invoked at the very end of close_connections() so that all users of
active_mi are proven to have left.
sql/mysqld.cc:
added the 2nd part (data) of the slave's clean up.
sql/slave.cc:
end_slave() is split in two part to release the slave threads and the remained
resources separately.
The new close_active_mi() should be called after all possible users ofactive_mi
has left, i.e at the very end of close_connections().
sql/slave.h:
interface to the new end_active_mi() function is added.
The problem: described in the bug report.
The fix:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
client/mysql.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/ha_ndbcluster.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/ha_ndbcluster_binlog.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/handler.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/log.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/mysqld.cc:
removed unnecessary line
sql/parse_file.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_acl.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_base.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_db.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_delete.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_partition.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_rename.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_show.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_table.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
sql/sql_view.cc:
--increase buffers where it's necessary
(buffers which are used in stxnmov)
--decrease buffer lengths which are used
as argument for strxnmov function
Remove custom DLL loader code from innodb plugin code, use
symbols exported from mysqld.
storage/innodb_plugin/handler/ha_innodb.cc:
Remove a Win32 workaround for current_thd.
The original problem that innodb plugin used
value of TLS variable across DLL boundaries is
solved in MySQL server (current_thd is a function
not TLS variable now)
storage/innodb_plugin/handler/handler0alter.cc:
Remove custom delay loader
storage/innodb_plugin/handler/handler0vars.h:
Remove custom delay loader
storage/innodb_plugin/handler/i_s.cc:
Remove custom delay loader
storage/innodb_plugin/handler/win_delay_loader.cc:
Remove custom delay loader
storage/innodb_plugin/plug.in:
Remove commented out MYSQL_PLUGIN_STATIC,
CMake would not parse that correctly
variable. The problem was that THD::connect_utime could be
used without being initialized when the main thread is used
to handle connections (--thread-handling=no-threads).
sql/mysqld.cc:
Set THD::start_utime even in no-threads handling mode.
sql/sql_class.cc:
Initialize variable.
sql/sql_class.h:
Rename connect_utime to prior_thr_create_utime as to
better reflect it's use intention.
sql/sql_connect.cc:
Check only if a thread was actually created.
always rollsback.
There is failure on pushbuild machines which are using old compilers complaining about
ULLONG_MAX declaration. Changing this to ULONGLONG_MAX to solve the problem.
sql/mysqld.cc:
Change ULLONG_MAX to ULONGLONG_MAX for the max_binlog_cache_size option
always rollsback.
The global variable max_binlog_cache_size cannot be set more than 4GB on
32 bit systems, limiting transactions of all storage engines to 4G of changes.
The problem is max_binlog_cache_size is declared as ulong which is 4 bytes
on 32 bit and 8 bytes on 64 bit machines.
Fixed by using ulonglong for max_binlog_cache_size which is 8bytes on 32
and 64 bit machines.The range for max_binlog_cache_size on 32 bit and 64 bit
systems is 4096-18446744073709547520 bytes.
mysql-test/r/variables.result:
Result file for BUG#10206
mysql-test/t/variables.test:
Testcase for BUG#10206
sql/mysql_priv.h:
change the extern declaration of max_binlog_cache_size to ulonglong
sql/mysqld.cc:
change the declaration of max_binlog_cache_size to ulonglong and the option is fixed to extend the range of max_binlog_cache_size
sql/set_var.cc:
change the variable declaration of max_binlog_cache_size to ulonglong
for indexes of InnoDB table
Fixes by replacing the PRNG that is used to pick random pages with a
better one.
Also adds a configuration option "innodb_use_legacy_cardinality_algorithm"
to enable the fix only when the option is set.
This patch is from http://bugs.mysql.com/file.php?id=11789
The problem is that creating a event could fail if the value of
the variable server_id didn't fit in the originator column of
the event system table. The cause is two-fold: it was possible
to set server_id to a value outside the documented range (from
0 to 2^32-1) and the originator column of the event table didn't
have enough room for values in this range.
The log tables (general_log and slow_log) also don't have a proper
column type to store the server_id and having a large server_id
value could prevent queries from being logged.
The solution is to ensure that all system tables that store the
server_id value have a proper column type (int unsigned) and that
the variable can't be set to a value that is not within the range.
mysql-test/r/events_bugs.result:
Add test case result for Bug#36540
mysql-test/r/log_tables.result:
Update column type.
mysql-test/r/system_mysql_db.result:
Update column type.
mysql-test/r/variables.result:
Add test case result for server_id value range.
mysql-test/suite/sys_vars/r/server_id_basic_64.result:
Update test case results.
mysql-test/t/events_bugs.test:
Add test case for Bug#36540
mysql-test/t/log_tables.test:
Fix column type.
mysql-test/t/variables.test:
Add test case for server_id value range.
scripts/mysql_system_tables.sql:
Columns that store the server_id value must be of type INT UNSIGNED,
fix event (originator), general_log and slow_log (server_id) tables
in accordance.
scripts/mysql_system_tables_fix.sql:
Columns that store the server_id value must be of type INT UNSIGNED,
fix event (originator), general_log and slow_log (server_id) tables
in accordance.
sql/mysqld.cc:
Set min and max values for the server_id variable.
Unfortunately we can't easily change server_id variable type
from ulong to uint32 because of the sys_var classes.
mysql-test/r/index_merge_myisam.result:
Testcases
mysql-test/t/index_merge_myisam.test:
Testcases
sql/strfunc.cc:
Change optimizer_switch from no_xxx to xxx=on/xx=off.
- Add functions to parse the new syntax
