Bug#36428: MY_MUTEX_INIT_FAST is used before initialization
On some thread implementations, we need a fake mutex attri-
bute as a placeholder, which we define as a global variable,
"my_fast_mutexattr". Well. that must be initialized before
used in any mutexes, and the ordering of initializations in
the API function my_init() was wrong.
Now, put my_thread_global_init(), which initializes the attri-
butes that mutexes require.
The problem here is that embedded server starts handle_thread manager
thread on mysql_library_init() does not stop it on mysql_library_end().
At shutdown, my_thread_global_end() waits for thread count to become 0,
but since we did not stop the thread it will give up after 5 seconds.
Solution is to move shutdown for handle_manager thread from kill_server()
(mysqld specific) to clean_up() that is used by both embedded and mysqld.
This patch also contains some refactorings - to avoid duplicate code,
start_handle_manager() and stop_handle_manager() functions are introduced.
Unused variables are eliminated. handle_manager does not rely on global
variable abort_loop anymore to stop (abort_loop is not set for embedded).
Note: Specifically on Windows and when using DBUG version of libmysqld,
the complete solution requires removing obsolete code my_thread_init()
from my_thread_var(). This has a side effect that a DBUG statement
after my_thread_end() can cause thread counter to be incremented, and
embedded will hang for some seconds. Or worse, my_thread_init() will
crash if critical sections have been deleted by the global cleanup
routine that runs in a different thread.
This patch also fixes and revert prior changes for Bug#38293
"Libmysqld crash in mysql_library_init if language file missing".
Root cause of the crash observed in Bug#38293 was bug in my_thread_init()
described above
client/mysql.cc:
sql_protocol_typelib is not exported from libmysqld
(does not make sense either)
thus excluded from embedded client
dbug/dbug.c:
revert changes for Bug#38293
include/my_dbug.h:
revert changes for Bug#38293
libmysql/libmysql.c:
Removed DBUG_POP call, because when called after my_end(), will access
THR_key_mysys that is already deleted. The result of pthread_get_specific
is not predictable in this case and hence DBUG_POP can crash.
libmysqld/examples/CMakeLists.txt:
Revert changes for Bug#38293.
libmysqld/lib_sql.cc:
code to start handle manager is factored out into
start_handle_manager() function
libmysqld/libmysqld.def:
Revert changes for Bug #38293
Remove excessive exports from libmysqld, export what API documents.
mysys/my_thr_init.c:
Remove windows-DLL-specific workaround for something (old code, no documentation for
what specifically). The problem is that even after my_thread_end() is finished,
DBUG statement can initiate my_thread_init(). This does not happen anywhere else and
should not happen on Windows either.
sql/mysql_priv.h:
- new functions start_handle_manager() and stop_handle_manager()
- move manager_thread_in_use variable to sql_manager.cc and made
it static
- remove manager_status, as it is unused
sql/mysqld.cc:
Code to start/stop handle_manager thread is factored out into start_handle_manager()
That's a Win-specific error.
When we create libmysqld.dll we have many libraries like mysys, dbug,
strings, etc linked into that dll, so the application built upon
this library shouldn't link these libraries to itself, rather use
those inside the dll.
Fixed by redirecting calls into the libmysqld.dll
per-file comments:
dbug/dbug.c
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
fake _db_something definitions added
include/my_dbug.h
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
fake _db_something declarations added
libmysqld/examples/CMakeLists.txt
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
superfluous libraries removed from linking
libmysqld/libmysqld.def
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
set of mysys functions added to the export section
japanese characters.
Fix - removed obsolvete setlocale from my_init.c . In MBCS
environments it caused unwanted character-to-byte translations
in fputc() in client code and wrong output as result.
#ifdef HAVE_purify removed
per-file comments:
mysql-test/t/partition_not_windows.test
Bug#39102 valgrind build does not compile in realpath, which make DATA/INDEX DIR fail
test reenabled
mysys/my_symlink.c
Bug#39102 valgrind build does not compile in realpath, which make DATA/INDEX DIR fail
superfluous ifdef removed, comments fixed
The problem is that MySQL's 'fast' mutex implementation uses the
random() routine to determine the spin delay. Unfortunately, the
routine interface is not thead-safe and some implementations (eg:
glibc) might use a internal lock to protect the RNG state, causing
excessive locking contention if lots of threads are spinning on
a MySQL's 'fast' mutex. The code was also misusing the value
of the RAND_MAX macro, this macro represents the largest value
that can be returned from the rand() function, not random().
The solution is to use the quite simple Park-Miller random number
generator. The initial seed is set to 1 because the previously used
generator wasn't being seeded -- the initial seed is 1 if srandom()
is not called.
Futhermore, the 'fast' mutex implementation has several shortcomings
and provides no measurable performance benefit. Therefore, its use is
not recommended unless it provides directly measurable results.
include/my_pthread.h:
Add field to keep the RNG state.
mysys/thr_mutex.c:
Use a palliative per-mutex rng state to determine the spin delay.
The RNG is not thread-safe but jumping a few sequences in the RNG
is harmless.
This patch contains fixes for two problems:
1. As originally reported, the server crashed on Mac OS X when trying to access
an EXAMPLE table after the EXAMPLE plugin was installed.
It turned out that the dynamically loaded EXAMPLE plugin called the
function hash_earch() from a Mac OS X system library, instead of
hash_earch() from MySQL's mysys library. Makefile.am in storage/example
does not include libmysys. So the Mac OS X linker arranged the hash_search()
function to be linked to the system library when the shared object is
loaded.
One possible solution would be to include libmysys into the linkage of
dynamic plugins. But then we must have a libmysys.so, which must be
used by the server too. This could have a minimal performance impact,
but foremost the change seems to bee too risky at the current state of
MySQL 5.1.
The selected solution is to rename MySQL's hash_search() to my_hash_search()
like it has been done before with hash_insert() and hash_reset().
Since this is the third time, we need to rename a hash_*() function,
I did renamed all hash_*() functions to my_hash_*().
To avoid changing a zillion calls to these functions, and announcing
this to hundreds of developers, I added defines that map the old names
to the new names.
This change is in hash.h and hash.c.
2. The other problem was improper implementation of the handlerton-to-plugin
mapping. We use a fixed-size array to hold a plugin reference for each
handlerton. On every install of a handler plugin, we allocated a new slot
of the array. On uninstall we did not free it. After some uninstall/install
cycles the array overflowed. We did not check for overflow.
One fix is to check for overflow to stop the crashes.
Another fix is to free the array slot at uninstall and search for a free slot
at plugin install.
This change is in handler.cc.
include/hash.h:
Bug#37958 - test main.plugin crash on Mac OS X when selecting from EXAMPLE engine.
Renamed hash_*() functions to my_hash_*().
Added defines that map old names to new names.
mysys/hash.c:
Bug#37958 - test main.plugin crash on Mac OS X when selecting from EXAMPLE engine.
Renamed hash_*() functions to my_hash_*().
sql/handler.cc:
Bug#37958 - test main.plugin crash on Mac OS X when selecting from EXAMPLE engine.
Protect against a failing ha_initialize_handlerton() in ha_finalize_handlerton().
Free hton2plugin slot on uninstall of a handler plugin.
Reuse freed slost of the hton2plugin array.
Protect against array overrun.
Dumping information about locks in use by sending a SIGHUP signal
to the server or by invoking the "mysqladmin debug" command may
lead to a server crash in debug builds or to undefined behavior in
production builds.
The problem was that a mutex that protects a lock object (THR_LOCK)
might have been destroyed before the lock object was actually removed
from the list of locks in use, causing a race condition with other
threads iterating over the list. The solution is to destroy the mutex
only after removing lock object from the list.
mysys/thr_lock.c:
Destroy the mutex that protects the lock object only after removing
the lock object from the list of locks in use.
test_if_data_home_dir fixed to look into real path.
Checks added to mi_open for symlinks into data home directory.
per-file messages:
include/my_sys.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
my_is_symlink interface added
include/myisam.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
myisam_test_invalid_symlink interface added
myisam/mi_check.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
mi_open_datafile calls modified
myisam/mi_open.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
code added to mi_open to check for symlinks into data home directory.
mi_open_datafile now accepts 'original' file path to check if it's
an allowed symlink.
myisam/mi_static.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
myisam_test_invlaid_symlink defined
myisam/myisamchk.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
mi_open_datafile call modified
myisam/myisamdef.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
mi_open_datafile interface modified - 'real_path' parameter added
mysql-test/r/symlink.test
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
error codes corrected as some patch now rejected pointing inside datahome
mysql-test/r/symlink.result
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
error messages corrected in the result
mysys/my_symlink.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
my_is_symlink() implementsd
my_realpath() now returns the 'realpath' even if a file isn't a symlink
sql/mysql_priv.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
test_if_data_home_dir interface
sql/mysqld.cc
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
myisam_test_invalid_symlik set with the 'test_if_data_home_dir'
sql/sql_parse.cc
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
error messages corrected
test_if_data_home_dir code fixed
portability fixes / cleanup
include/my_sys.h:
declaration should certainly match definition!
mysys/safemalloc.c:
Make the various print-functions happy:
sf_malloc_(cur|max)_memory are size_t now, might as well use %u instead
of %d. Ideally, we'd use %zu, but we can't rely on having that, so we'll
use %lu instead. Likewise, we could cast to unsigned for our poor man's
%p -- pointers are never negative, and neither is %x --, but since it
was fixed to %p with seemingly no ill effects in 6.0 anyway, we'll back
port that instead.
We could allocate chunks larger than 4GB, but did our
size-accounting in 32-bit values. This could lead to
spurious warnings, inaccurate accounting, and, in
theory, data loss.
Affected: 64-bit platforms. Debug-build (with safemalloc).
At least one buffer larger than 4GB. For potential data
loss, a re-alloc on such a buffer would be necessary.
mysys/my_static.c:
Make memory-accounting 64-bit safe.
mysys/my_static.h:
Make memory-accounting 64-bit safe.
Move in struct for better alignment when 64-bit.
Tilde expansion could fail when it was to expand to an empty string (such as
when HOME is set to an empty string), especially on systems where size_t is
unsigned.
This fix is for 5.0 only : back porting the 6.0 patch manually
The parser code in sql/sql_yacc.yy needs to be more robust to out of
memory conditions, so that when parsing a query fails due to OOM,
the thread gracefully returns an error.
Before this fix, a new/alloc returning NULL could:
- cause a crash, if dereferencing the NULL pointer,
- produce a corrupted parsed tree, containing NULL nodes,
- alter the semantic of a query, by silently dropping token values or nodes
With this fix:
- C++ constructors are *not* executed with a NULL "this" pointer
when operator new fails.
This is achieved by declaring "operator new" with a "throw ()" clause,
so that a failed new gracefully returns NULL on OOM conditions.
- calls to new/alloc are tested for a NULL result,
- The thread diagnostic area is set to an error status when OOM occurs.
This ensures that a request failing in the server properly returns an
ER_OUT_OF_RESOURCES error to the client.
- OOM conditions cause the parser to stop immediately (MYSQL_YYABORT).
This prevents causing further crashes when using a partially built parsed
tree in further rules in the parser.
No test scripts are provided, since automating OOM failures is not
instrumented in the server.
Tested under the debugger, to verify that an error in alloc_root cause the
thread to returns gracefully all the way to the client application, with
an ER_OUT_OF_RESOURCES error.
Pull out some of unpack_dirname() into normalize_dirname(); this
new function does not expand "~" to the home directory. Use this
function in unpack_dirname(), and use it during init_default_directories()
to remove duplicate entries without losing track of which directory
is a user's home dir.
Normalize directory names before adding them to default_directories.
mysys/default.c:
Normalize directory names with unpack_dirname() before adding them
to default_directories. This way, /etc/ and /etc will not count as
duplicates.
Because this entails allocating memory to store the normalized names,
add error handling and ensure that it doesn't leak memory in case
both my_print_defaults() and load_defaults() are called.
Clean up the Windows code that finds the exe's parent directory, and
pull it out into a separate function.
Reorganize the code into a single init_default_directories() function,
with internal #ifdefs, instead of init_default_directories_<system>()
functions which were accessed via a function pointer. This is more in
line with normal MySQL coding style, and easier to read for some.
Another problem is that the backtrace facility wasn't being
enabled for non-Linux targets even if the target OS has the
backtrace functions. Also, the stacktrace functions inside
mysqltest were being used without proper checks for their
presence in the build.
client/mysqltest.c:
Only use stacktrace functions if they are available.
configure.in:
Check if the compiler defines __bss_start
include/my_stacktrace.h:
Enable stacktrace if system has backtrace functions.
mysys/stacktrace.c:
Use backtrace functions if the system supports it.
sql/mysqld.cc:
Only use stacktrace functions if they are available.
The problem was that when a embedded linked version of mysqltest
crashed there was no way to obtain a stack trace if no core file
is available. Another problem is that the embedded version of
libmysql was not behaving (crash) the same as the non-embedded with
respect to sending commands to a explicitly closed connection.
The solution is to generate a mysqltest's stack trace on crash
and to enable "reconnect" if the connection handle was explicitly
closed so the behavior matches the non-embedded one.
client/CMakeLists.txt:
Link mysys to mysqltest.
client/Makefile.am:
Link mysys to mysqltest.
client/mysqltest.c:
Add fatal signal handling with backtracing for Unix and Windows.
configure.in:
Add check for weak symbols support and remove a spurious word.
include/Makefile.am:
Add new header with prototype for stack tracing functions.
include/my_stacktrace.h:
Add new header with prototype for stack tracing functions.
libmysqld/CMakeLists.txt:
stack tracing is now part of mysys.
libmysqld/Makefile.am:
stack tracing is now part of mysys.
libmysqld/lib_sql.cc:
Re-connect if connection was explicitly closed. This is
done to match the behavior of the non-embeded libmysql.
mysql-test/t/sql_low_priority_updates_func.test:
Test expects parallelism between queries that cannot be
guaranteed under embedded.
mysys/CMakeLists.txt:
Add stacktrace to mysys.
mysys/Makefile.am:
Add stacktrace to mysys.
mysys/stacktrace.c:
Move stacktrace to mysys and add weak symbol for the
C++ name de-mangling function so that it can later be
overridden in C++ code. Also add my_ prefix to exported
functions.
sql/CMakeLists.txt:
stacktrace was moved to mysys.
sql/Makefile.am:
stacktrace was moved to mysys.
sql/mysqld.cc:
Add my_ prefix to mysys functions.
We could allocate chunks larger than 4GB, but did our size-accounting in 32-bit
values. This could lead to spurious warnings, inaccurate accounting, and, in
theory, data loss.
Affected: 64-bit platforms. Debug-build (with safemalloc). At least one buffer
larger than 4GB. For potential data loss, a re-alloc on such a buffer would be
necessary.
mysys/my_static.c:
Make memory-accounting 64-bit safe.
mysys/my_static.h:
Make memory-accounting 64-bit safe.
Move in struct for better alignment when 64-bit.
into kpdesk.mysql.com:/home/thek/Development/cpp/mysql-5.1-merge
mysql-test/r/grant.result:
Auto merged
mysql-test/t/disabled.def:
Auto merged
mysql-test/t/grant.test:
Auto merged
sql/opt_range.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
storage/myisam/mi_create.c:
Auto merged
into magare.gmz:/home/kgeorge/mysql/work/merge-build-5.1-bugteam
include/my_dbug.h:
Auto merged
mysys/mf_keycache.c:
Auto merged
sql/item_func.cc:
Auto merged
sql/log.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_select.cc:
Auto merged
storage/myisam/mi_check.c:
Auto merged
storage/myisam/mi_dynrec.c:
Auto merged
storage/myisam/mi_open.c:
Auto merged
storage/myisam/mi_packrec.c:
Auto merged
storage/myisam/mi_test1.c:
Auto merged
storage/myisam/mi_test2.c:
Auto merged
storage/myisam/mi_write.c:
Auto merged
storage/myisammrg/ha_myisammrg.cc:
Auto merged
in perror and handler descriptors (BUG#25177)
Fixed problem of masking mysql error by system
error in perror (BUG#23028)
extra/perror.c:
Used common handlers error list.
Fixed BUG#23028 ignoring mysql error in case of
OS and mysql error codes overlapping.
include/my_base.h:
Added errors of maria engine.
Fixed incorrect comment in #define (can be cause of
seriouse problems)
include/my_handler.h:
Added ability to be included into C++ code.
mysys/my_handler.c:
Error texts moved to the separate files.
mysys/my_handler_errors.h:
New BitKeeper file ``mysys/my_handler_errors.h''
into a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-5.1-marvel
client/mysqldump.c:
Auto merged
client/mysqltest.c:
Auto merged
include/my_global.h:
Auto merged
mysql-test/lib/mtr_report.pl:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/handler.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/log.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.cc:
Auto merged
sql/unireg.cc:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
storage/myisam/mi_dynrec.c:
Auto merged
storage/myisam/mi_open.c:
Auto merged
storage/myisammrg/ha_myisammrg.cc:
Auto merged
include/config-win.h:
Manual merge between main 5.1 and 5.1 marvel.
mysql-test/r/change_user.result:
Manual merge between main 5.1 and 5.1 marvel.
mysql-test/t/change_user.test:
Manual merge between main 5.1 and 5.1 marvel.
sql/sql_plugin.cc:
Manual merge between main 5.1 and 5.1 marvel.
