Bug#11763065 - 55730: KILL_SERVER() CALLS SETEVENT ON A NULL
HANDLE, SMEM_EVENT_CONNECT_REQUEST
Application Verifier is a Microsoft tool used for
detecting certain classes of programming errors.
In particular, MS Windows OS resource usage is
monitored for wrong usage (handles, thread local
storage, critical sections, ...)
In MySQL 5.5.x, an error was introduced where an
object on thread local storage was used before the
TLS and the object was created.
The fix has been to move the mysys initialization
to an earlier stage in the boot process when built for
Windows. For non-win builds, the init already happens
early.
Some un-tangling of calls to my_init(), my_basic_init()
and my_thread_global_init() was done. There is no
longer a need to do init in steps, so the full my_init()
is called instead of my_init_basic().
In addition, Bug#11763065 was fixed. The event handle
'smem_event_connect_request' is only created if
'opt_enable_shared_memory' is set. When killing the
server, an event was flagged on the handle
unconditionally. Added a test, so it will only be
flagged if created.
The problem from a user point of view was that on Solaris the
time related functions (e.g. NOW(), SYSDATE(), etc) would always
return a fixed time.
This bug was happening due to a logic in the time retrieving
wrapper function which would only call the time() function every
half second. This interval between calls would be calculated
using the gethrtime() and the logic relied on the fact that time
returned by it is monotonic.
Unfortunately, due to bugs in the gethrtime() implementation,
there are some cases where the time returned by it can drift
(See Solaris bug id 6600939), potentially causing the interval
calculation logic to fail.
Since newer versions of Solaris (10+) have alleviated the
performance degradation associated with time(2), the solution is
to simply directly rely on time() at each invocation.
This simplification has an upside that it allows us to eliminate
a lock which was used to control access to the variables used
to track the half second interval, thus improving the overall
scalability of timekeeping related functions (e.g. NOW()).
Benchmarks runs have shown no significant degradation associated
with this change. With this, there are actually improvements in
performance for cases involving many connections.
In summary, the changes introduced by this patch are:
a) my_time() and my_micro_time_and_time() no longer use gethrtime().
Instead, time() and gettimeofdate() are used correspondingly.
b) my_micro_time() is changed to not use gethrtime() so as to
have the same time source as my_micro_time_and_time().
There shouldn't be any performance impact from this change
since this function is used only a few times during statement
execution and, on Solaris, gettimeofday() shows acceptable
performance.
and 'THREAD_SAFE_CLIENT'.
As of MySQL 5.5, we no longer support non-threaded
builds. This patch removes all references to the
obsolete THREAD and THREAD_SAFE_CLIENT preprocessor
symbols. These were used to distinguish between
threaded and non-threaded builds.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
Prior to this fix, mysys mutexes such as THR_LOCK_lock could be initialized
twice by a call to my_init().
The root cause was out of place initialization in my_basic_init(),
calling my_thread_global_init().
With this fix,
- my_basic_init() properly initializes the mutex implementation itself,
for SAFE or FAST mutexes, and for platform dependent initializations,
before initialiazing a mutex.
- my_init() properly initializes mysys mutexes once,
when making the first call to my_thread_global_init().
In order to allow thread schedulers to be dynamically loaded,
it is necessary to make the following changes to the server:
- Two new service interfaces
- Modifications to InnoDB to inform the thread scheduler of state changes.
- Changes to the VIO subsystem for checking if data is available on a socket.
- Elimination of remains of the old thread pool implementation.
The two new service interfaces introduces are:
my_thread_scheduler
A service interface to register a thread
scheduler.
thd_wait
A service interface to inform thread scheduler
that the thread is about to start waiting.
In addition, the patch adds code that:
- Add a call to thd_wait for table locks in mysys
thd_lock.c by introducing a set function that
can be used to set a callback to be used when
waiting on a lock and resuming from waiting.
- Calling the mysys set function from the server
to set the callbacks correctly.
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/Makefile.am
Text conflict in mysql-test/collections/default.daily
Text conflict in mysql-test/r/mysqlbinlog_row_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_typeconv_innodb.result
Text conflict in mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test
Text conflict in mysql-test/suite/rpl/t/rpl_row_create_table.test
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in mysql-test/suite/rpl/t/rpl_typeconv_innodb.test
Text conflict in mysys/charset.c
Text conflict in sql/field.cc
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_func.cc
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/rpl_utility.cc
Text conflict in sql/rpl_utility.h
Text conflict in sql/set_var.cc
Text conflict in sql/share/Makefile.am
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_table.cc
Text conflict in storage/example/ha_example.h
Text conflict in storage/federated/ha_federated.cc
Text conflict in storage/myisammrg/ha_myisammrg.cc
Text conflict in storage/myisammrg/myrg_open.c
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/r/having.result
Text conflict in mysql-test/suite/rpl/t/disabled.def
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in storage/federated/ha_federated.cc
A client doing multiple mysql_library_init() and
mysql_library_end() calls over the lifetime of the process may
experience lost character set data, potentially even a
SIGSEGV.
This patch reinstates the reloading of character set data when
a mysql_library_init() is done after a mysql_library_end().
Before this fix, opening a configuration file located under "~" failed.
To evaluate the "~" path, home_dir needs to be initialized.
The 'home_dir' variable was initialized too late in my_init().
This fix:
- moved the home_dir initialization from my_init() to my_basic_init(),
using getenv("HOME"))
- moved the initialization of my_umask / my_umask_dir also to
my_basic_init(), to have all the my_umask / my_umask_dir init code in the
same place.
The second part is not strictly required, but makes the code more
maintainable.
Tested the fix manually.
No MTR tests added, because MTR should not access or modify the $HOME
directory of the user running tests.
As documented in the bug report, the double checked locking
pattern has inherent issues, and cannot guarantee correct
initialization.
This patch replaces the logic in init_available_charsets()
with the use of pthread_once(3). A wrapper function,
my_pthread_once(), is introduced and is used in lieu of direct
calls to init_available_charsets(). Related defines
MY_PTHREAD_ONCE_* are also introduced.
For the Windows platform, the implementation in lp:sysbench is
ported. For single-thread use, a simple define calls the
function and sets the pthread_once control variable.
Charset initialization is modified to use my_pthread_once().
http://lists.mysql.com/commits/57725
Vladislav Vaintroub 2008-11-03
Cleanup CMakeLists.txt(s) - remove winsock2 (ws2_32) from
TARGET_LINK_LIBRARIES.
Every exe or dll linked with mysys needs ws2_32, because
mysys uses winsock function WSAStartup in my_init().
However, there is no need to explicitely add ws2_32 to
the list of TARGET_LINK_LIBRARIES multiple times.
Visual Studio comes with a handy pragma that tells linker
to add library. So patch replaces bunch of ws2_32 in
CMakeLists with single pragma comment(lib,"ws2_32")
in my_init.c
Additionally, reference to non-existing "debug" library
has been removed from TARGET_LINK_LIBRARIES. The correct
name of the library is "dbug".
Bug#44091: libmysqld gets stuck waiting on mutex on initialization
The problem was that libmysqld wasn't enforcing a certain
initialization and deinitialization order for the mysys
library. Another problem was that the global object used
for management of log event handlers (aka LOGGER) wasn't
being prepared for a possible reutilization.
What leads to the hang/crash reported is that a failure
to load the language file triggers a double call of the
cleanup functions, causing an already destroyed mutex to
be used.
The solution is enforce a order on the initialization and
deinitialization of the mysys library within the libmysqld
library and to ensure that the global LOGGER object reset
it's internal state during cleanup.
code backported from 6.0
per-file messages:
include/my_global.h
Remove SC_MAXWIDTH. This is unused and irrelevant nowadays.
include/my_sys.h
Remove errbuf declaration and unused definitions.
mysys/my_error.c
Remove errbuf definition and move and adjust ERRMSGSIZE.
mysys/my_init.c
Declare buffer on the stack and use my_snprintf.
mysys/safemalloc.c
Use size explicitly. It's more than enough for the message at hand.
sql/sql_error.cc
Use size explicitly. It's more than enough for the message at hand.
sql/sql_parse.cc
Declare buffer on the stack. Use my_snprintf as it will result in
less stack space being used than by a system provided sprintf --
this allows us to put the buffer on the stack without causing much
trouble. Also, the use of errbuff here was not thread-safe as the
function can be entered concurrently from multiple threads.
sql/sql_table.cc
Use MYSQL_ERRMSG_SIZE. Extra space is not needed as my_snprintf will
nul terminate strings.
storage/myisam/ha_myisam.cc
Use MYSQL_ERRMSG_SIZE.
sql/share/errmsg.txt
Error message truncation in test "innodb" in embedded mode
filename in the error message can safely take up to 210 symbols.
conflicts:
Text conflict in client/mysqltest.cc
Text conflict in mysql-test/include/wait_until_connected_again.inc
Text conflict in mysql-test/lib/mtr_report.pm
Text conflict in mysql-test/mysql-test-run.pl
Text conflict in mysql-test/r/events_bugs.result
Text conflict in mysql-test/r/log_state.result
Text conflict in mysql-test/r/myisam_data_pointer_size_func.result
Text conflict in mysql-test/r/mysqlcheck.result
Text conflict in mysql-test/r/query_cache.result
Text conflict in mysql-test/r/status.result
Text conflict in mysql-test/suite/binlog/r/binlog_index.result
Text conflict in mysql-test/suite/binlog/r/binlog_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_packet.result
Text conflict in mysql-test/suite/rpl/t/rpl_packet.test
Text conflict in mysql-test/t/disabled.def
Text conflict in mysql-test/t/events_bugs.test
Text conflict in mysql-test/t/log_state.test
Text conflict in mysql-test/t/myisam_data_pointer_size_func.test
Text conflict in mysql-test/t/mysqlcheck.test
Text conflict in mysql-test/t/query_cache.test
Text conflict in mysql-test/t/rpl_init_slave_func.test
Text conflict in mysql-test/t/status.test
Bug#36428: MY_MUTEX_INIT_FAST is used before initialization
On some thread implementations, we need a fake mutex attri-
bute as a placeholder, which we define as a global variable,
"my_fast_mutexattr". Well. that must be initialized before
used in any mutexes, and the ordering of initializations in
the API function my_init() was wrong.
Now, put my_thread_global_init(), which initializes the attri-
butes that mutexes require.
japanese characters.
Fix - removed obsolvete setlocale from my_init.c . In MBCS
environments it caused unwanted character-to-byte translations
in fputc() in client code and wrong output as result.
- Reserver namespace and place in frm for TABLE_CHECKSUM and PAGE_CHECKSUM create options
- Added syncing of directory when creating .frm files
- Portability fixes
- Added missing cast that could cause bugs
- Code cleanups
- Made some bit functions inline
- Moved things out of myisam.h to my_handler.h to make them more accessable
- Renamed some myisam variables and defines to make them more globaly usable (as they are used outside of MyISAM)
- Fixed bugs in error conditions
- Use compiler time asserts instead of run time
- Fixed indentation
HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP as the old name was wrong
(Added a define for old value to ensure we don't break any old code)
Added HA_EXTRA_PREPARE_FOR_RENAME as a signal for rename (before we used a DROP signal which is wrong)
- Initialize error messages early to get better errors when mysqld or an engine fails to start
- Fix windows bug that query_performance_frequency was not initialized if registry code failed
- thread_stack -> my_thread_stack_size
When locking a "fast" mutex a static variable cpu_count
was used as a flag to initialize itself on the first usage
by calling sysconf() and setting non-zero value.
This is not thread and optimization safe on some
platforms. That's why the global initialization needs
to be done once in a designated function.
This will also speed up the usage (by a small bit)
because it won't have to check if it's initialized on
every call.
Fixed by moving the fast mutexes initialization out of
my_pthread_fastmutex_lock() to fastmutex_global_init()
and call it from my_init()
Fixed bug in query cache that made it impossible to run mysqld with --debug
Fixed memory leaks in mysqldump and mysqltest
Memory leaks associated with wrong usage of mysqltest is not fixed. To find these, run
mysql-test-run --debug mysqltest
Fixed compiler warnings, errors and link errors
Fixed new bug on Solaris with gethrtime()
Added --debug-check option to all mysql clients to print errors and memory leaks
Added --debug-info to all clients. This now works as --debug-check but also prints memory and cpu usage
--long-query-time is now given in seconds with microseconds as decimals
--min_examined_row_limit added for slow query log
long_query_time user variable is now double with 6 decimals
Added functions to get time in microseconds
Added faster time() functions for system that has gethrtime() (Solaris)
We now do less time() calls.
Added field->in_read_set() and field->in_write_set() for easier field manipulation by handlers
set_var.cc and my_getopt() can now handle DOUBLE variables.
All time() calls changed to my_time()
my_time() now does retry's if time() call fails.
Added debug function for stopping in mysql_admin_table() when tables are locked
Some trivial function and struct variable renames to avoid merge errors.
Fixed compiler warnings
Initialization of some time variables on windows moved to my_init()
