Bug#36428: MY_MUTEX_INIT_FAST is used before initialization
On some thread implementations, we need a fake mutex attri-
bute as a placeholder, which we define as a global variable,
"my_fast_mutexattr". Well. that must be initialized before
used in any mutexes, and the ordering of initializations in
the API function my_init() was wrong.
Now, put my_thread_global_init(), which initializes the attri-
butes that mutexes require.
Bounds-checks and blocksize corrections were applied to user-input,
but constants in the server were trusted implicitly. If these values
did not actually meet the requirements, the user could not set change
a variable, then set it back to the (wonky) factory default or maximum
by explicitly specifying it (SET <var>=<value> vs SET <var>=DEFAULT).
Now checks also apply to the server's presets. Wonky values and maxima
get corrected at startup. Consequently all non-offsetted values the user
sees are valid, and users can set the variable to that exact value if
they so desire.
The problem here is that embedded server starts handle_thread manager
thread on mysql_library_init() does not stop it on mysql_library_end().
At shutdown, my_thread_global_end() waits for thread count to become 0,
but since we did not stop the thread it will give up after 5 seconds.
Solution is to move shutdown for handle_manager thread from kill_server()
(mysqld specific) to clean_up() that is used by both embedded and mysqld.
This patch also contains some refactorings - to avoid duplicate code,
start_handle_manager() and stop_handle_manager() functions are introduced.
Unused variables are eliminated. handle_manager does not rely on global
variable abort_loop anymore to stop (abort_loop is not set for embedded).
Note: Specifically on Windows and when using DBUG version of libmysqld,
the complete solution requires removing obsolete code my_thread_init()
from my_thread_var(). This has a side effect that a DBUG statement
after my_thread_end() can cause thread counter to be incremented, and
embedded will hang for some seconds. Or worse, my_thread_init() will
crash if critical sections have been deleted by the global cleanup
routine that runs in a different thread.
This patch also fixes and revert prior changes for Bug#38293
"Libmysqld crash in mysql_library_init if language file missing".
Root cause of the crash observed in Bug#38293 was bug in my_thread_init()
described above
That's a Win-specific error.
When we create libmysqld.dll we have many libraries like mysys, dbug,
strings, etc linked into that dll, so the application built upon
this library shouldn't link these libraries to itself, rather use
those inside the dll.
Fixed by redirecting calls into the libmysqld.dll
per-file comments:
dbug/dbug.c
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
fake _db_something definitions added
include/my_dbug.h
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
fake _db_something declarations added
libmysqld/examples/CMakeLists.txt
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
superfluous libraries removed from linking
libmysqld/libmysqld.def
Bug#38293 Libmysqld crash in mysql_library_init if language file missing
set of mysys functions added to the export section
japanese characters.
Fix - removed obsolvete setlocale from my_init.c . In MBCS
environments it caused unwanted character-to-byte translations
in fputc() in client code and wrong output as result.
#ifdef HAVE_purify removed
per-file comments:
mysql-test/t/partition_not_windows.test
Bug#39102 valgrind build does not compile in realpath, which make DATA/INDEX DIR fail
test reenabled
mysys/my_symlink.c
Bug#39102 valgrind build does not compile in realpath, which make DATA/INDEX DIR fail
superfluous ifdef removed, comments fixed
The problem is that MySQL's 'fast' mutex implementation uses the
random() routine to determine the spin delay. Unfortunately, the
routine interface is not thead-safe and some implementations (eg:
glibc) might use a internal lock to protect the RNG state, causing
excessive locking contention if lots of threads are spinning on
a MySQL's 'fast' mutex. The code was also misusing the value
of the RAND_MAX macro, this macro represents the largest value
that can be returned from the rand() function, not random().
The solution is to use the quite simple Park-Miller random number
generator. The initial seed is set to 1 because the previously used
generator wasn't being seeded -- the initial seed is 1 if srandom()
is not called.
Futhermore, the 'fast' mutex implementation has several shortcomings
and provides no measurable performance benefit. Therefore, its use is
not recommended unless it provides directly measurable results.
This patch contains fixes for two problems:
1. As originally reported, the server crashed on Mac OS X when trying to access
an EXAMPLE table after the EXAMPLE plugin was installed.
It turned out that the dynamically loaded EXAMPLE plugin called the
function hash_earch() from a Mac OS X system library, instead of
hash_earch() from MySQL's mysys library. Makefile.am in storage/example
does not include libmysys. So the Mac OS X linker arranged the hash_search()
function to be linked to the system library when the shared object is
loaded.
One possible solution would be to include libmysys into the linkage of
dynamic plugins. But then we must have a libmysys.so, which must be
used by the server too. This could have a minimal performance impact,
but foremost the change seems to bee too risky at the current state of
MySQL 5.1.
The selected solution is to rename MySQL's hash_search() to my_hash_search()
like it has been done before with hash_insert() and hash_reset().
Since this is the third time, we need to rename a hash_*() function,
I did renamed all hash_*() functions to my_hash_*().
To avoid changing a zillion calls to these functions, and announcing
this to hundreds of developers, I added defines that map the old names
to the new names.
This change is in hash.h and hash.c.
2. The other problem was improper implementation of the handlerton-to-plugin
mapping. We use a fixed-size array to hold a plugin reference for each
handlerton. On every install of a handler plugin, we allocated a new slot
of the array. On uninstall we did not free it. After some uninstall/install
cycles the array overflowed. We did not check for overflow.
One fix is to check for overflow to stop the crashes.
Another fix is to free the array slot at uninstall and search for a free slot
at plugin install.
This change is in handler.cc.
Dumping information about locks in use by sending a SIGHUP signal
to the server or by invoking the "mysqladmin debug" command may
lead to a server crash in debug builds or to undefined behavior in
production builds.
The problem was that a mutex that protects a lock object (THR_LOCK)
might have been destroyed before the lock object was actually removed
from the list of locks in use, causing a race condition with other
threads iterating over the list. The solution is to destroy the mutex
only after removing lock object from the list.
test_if_data_home_dir fixed to look into real path.
Checks added to mi_open for symlinks into data home directory.
per-file messages:
include/my_sys.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
my_is_symlink interface added
include/myisam.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
myisam_test_invalid_symlink interface added
myisam/mi_check.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
mi_open_datafile calls modified
myisam/mi_open.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
code added to mi_open to check for symlinks into data home directory.
mi_open_datafile now accepts 'original' file path to check if it's
an allowed symlink.
myisam/mi_static.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
myisam_test_invlaid_symlink defined
myisam/myisamchk.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
mi_open_datafile call modified
myisam/myisamdef.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
mi_open_datafile interface modified - 'real_path' parameter added
mysql-test/r/symlink.test
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
error codes corrected as some patch now rejected pointing inside datahome
mysql-test/r/symlink.result
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
error messages corrected in the result
mysys/my_symlink.c
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
my_is_symlink() implementsd
my_realpath() now returns the 'realpath' even if a file isn't a symlink
sql/mysql_priv.h
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
test_if_data_home_dir interface
sql/mysqld.cc
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
myisam_test_invalid_symlik set with the 'test_if_data_home_dir'
sql/sql_parse.cc
Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY.
error messages corrected
test_if_data_home_dir code fixed
We could allocate chunks larger than 4GB, but did our
size-accounting in 32-bit values. This could lead to
spurious warnings, inaccurate accounting, and, in
theory, data loss.
Affected: 64-bit platforms. Debug-build (with safemalloc).
At least one buffer larger than 4GB. For potential data
loss, a re-alloc on such a buffer would be necessary.
Tilde expansion could fail when it was to expand to an empty string (such as
when HOME is set to an empty string), especially on systems where size_t is
unsigned.
This fix is for 5.0 only : back porting the 6.0 patch manually
The parser code in sql/sql_yacc.yy needs to be more robust to out of
memory conditions, so that when parsing a query fails due to OOM,
the thread gracefully returns an error.
Before this fix, a new/alloc returning NULL could:
- cause a crash, if dereferencing the NULL pointer,
- produce a corrupted parsed tree, containing NULL nodes,
- alter the semantic of a query, by silently dropping token values or nodes
With this fix:
- C++ constructors are *not* executed with a NULL "this" pointer
when operator new fails.
This is achieved by declaring "operator new" with a "throw ()" clause,
so that a failed new gracefully returns NULL on OOM conditions.
- calls to new/alloc are tested for a NULL result,
- The thread diagnostic area is set to an error status when OOM occurs.
This ensures that a request failing in the server properly returns an
ER_OUT_OF_RESOURCES error to the client.
- OOM conditions cause the parser to stop immediately (MYSQL_YYABORT).
This prevents causing further crashes when using a partially built parsed
tree in further rules in the parser.
No test scripts are provided, since automating OOM failures is not
instrumented in the server.
Tested under the debugger, to verify that an error in alloc_root cause the
thread to returns gracefully all the way to the client application, with
an ER_OUT_OF_RESOURCES error.
Pull out some of unpack_dirname() into normalize_dirname(); this
new function does not expand "~" to the home directory. Use this
function in unpack_dirname(), and use it during init_default_directories()
to remove duplicate entries without losing track of which directory
is a user's home dir.
Another problem is that the backtrace facility wasn't being
enabled for non-Linux targets even if the target OS has the
backtrace functions. Also, the stacktrace functions inside
mysqltest were being used without proper checks for their
presence in the build.
The problem was that when a embedded linked version of mysqltest
crashed there was no way to obtain a stack trace if no core file
is available. Another problem is that the embedded version of
libmysql was not behaving (crash) the same as the non-embedded with
respect to sending commands to a explicitly closed connection.
The solution is to generate a mysqltest's stack trace on crash
and to enable "reconnect" if the connection handle was explicitly
closed so the behavior matches the non-embedded one.
We could allocate chunks larger than 4GB, but did our size-accounting in 32-bit
values. This could lead to spurious warnings, inaccurate accounting, and, in
theory, data loss.
Affected: 64-bit platforms. Debug-build (with safemalloc). At least one buffer
larger than 4GB. For potential data loss, a re-alloc on such a buffer would be
necessary.
