One of the reported problems manifested itself in the scenario when one
thread tried to to get statistics on a key cache while the second thread
had not finished initialization of the key cache structure yet.
The problem was resolved by forcing serialization of such operations
on key caches.
To serialize function calls to perform certain operations over a key cache
a new mutex associated with the key cache now is used. It is stored in the
field op_lock of the KEY_CACHE structure. It is locked when the operation
is performed. Some of the serialized key cache operations utilize calls
for other key cache operations. To avoid recursive locking of op_lock
the new functions that perform the operations of key cache initialization,
destruction and re-partitioning with an additional parameter were introduced.
The parameter says whether the operation over op_lock are to be performed or
are to be omitted. The old functions for the operations of key cache
initialization, destruction,and re-partitioning now just call the
corresponding new functions with the additional parameter set to true
requesting to use op_lock while all other calls of these new function
have this parameter set to false.
Another problem reported in the bug entry concerned the operation of
assigning an index to a key cache. This operation can be called
while the key cache structures are not initialized yet. In this
case any call of flush_key_blocks() should return without any actions.
No test case is provided with this patch.
The field key_cache_mem_size of the KEY_CACHE structure must be
initialized in the function init_key_cache() and updated in the
function resize_key_cache().
and 'THREAD_SAFE_CLIENT'.
As of MySQL 5.5, we no longer support non-threaded
builds. This patch removes all references to the
obsolete THREAD and THREAD_SAFE_CLIENT preprocessor
symbols. These were used to distinguish between
threaded and non-threaded builds.
Bug#57995: Compiler flag change build error on OSX 10.4: my_getncpus.c
Bug#57996: Compiler flag change build error on OSX 10.5 : bind.c
Bug#57994: Compiler flag change build error : my_redel.c
Bug#57993: Compiler flag change build error on FreeBsd 7.0 : regexec.c
Bug#57992: Compiler flag change build error on FreeBsd : mf_keycache.c
Bug#57997: Compiler flag change build error on OSX 10.6: debug_sync.cc
Fix assorted compiler generated warnings.
cmd-line-utils/readline/bind.c:
Bug#57996: Compiler flag change build error on OSX 10.5 : bind.c
Initialize variable to work around a false positive warning.
include/m_string.h:
Bug#57994: Compiler flag change build error : my_redel.c
The expansion of stpcpy (in glibc) causes warnings if the
return value of strmov is not being used. Since stpcpy is
a GNU extension and the expansion ends up using a built-in
provided by GCC, use the compiler provided built-in directly
when possible.
include/my_compiler.h:
Define a dummy MY_GNUC_PREREQ when not compiling with GCC.
libmysql/libmysql.c:
Bug#58057: 5.1 libmysql/libmysql.c unused variable/compile failure
Variable might not be used in some cases. So, tag it as unused.
mysys/mf_keycache.c:
Bug#57992: Compiler flag change build error on FreeBsd : mf_keycache.c
Use UNINIT_VAR to work around a false positive warning.
mysys/my_getncpus.c:
Bug#57995: Compiler flag change build error on OSX 10.4: my_getncpus.c
Declare variable in the same block where it is used.
regex/regexec.c:
Bug#57993: Compiler flag change build error on FreeBsd 7.0 : regexec.c
Work around a compiler bug which causes the cast to not be enforced.
sql/debug_sync.cc:
Bug#57997: Compiler flag change build error on OSX 10.6: debug_sync.cc
Use UNINIT_VAR to work around a false positive warning.
sql/handler.cc:
Use UNINIT_VAR to work around a false positive warning.
sql/slave.cc:
Use UNINIT_VAR to work around a false positive warning.
sql/sql_partition.cc:
Use UNINIT_VAR to work around a false positive warning.
storage/myisam/ft_nlq_search.c:
Use UNINIT_VAR to work around a false positive warning.
storage/myisam/mi_create.c:
Use UNINIT_VAR to work around a false positive warning.
storage/myisammrg/myrg_open.c:
Use UNINIT_VAR to work around a false positive warning.
tests/mysql_client_test.c:
Change function to take a pointer to const, no need for a cast.
mysys/mf_keycache.c:
warning on windows
storage/example/ha_example.cc:
fighting the warnings don't ifdef too much.
a function must return a value!
storage/xtradb/Makefile.am:
add -lmysqlservices for ha_xtradb.so plugin to load
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Disable pbxt for test cases not using pbxt (speeds up test suite)
extra/comp_err.c:
Added cast to get rid of compiler warning
extra/libevent/kqueue.c:
Added cast to get rid of compiler warning
mysql-test/lib/mtr_cases.pm:
Use --skip-pbxt for test cases that doesn't need pbxt
Collect default-storage-engine from suite.opt file (should actually be my.cnf file, but that wasn't easy to do)
mysql-test/suite/pbxt/t/suite.opt:
Added marker for mysql-test-run.pl that this suite require pbxt
mysys/mf_keycache.c:
Use LINT_INIT() to remove compiler warnings
mysys/my_gethostbyname.c:
Remove compiler warnings
sql/handler.cc:
Reset variable that may be used uninitialized
sql/item.h:
Remove compiler warning
sql/mysqld.cc:
Use LINT_INIT() to remove compiler warnings
sql/sql_class.h:
Remove compiler warning
sql/sql_table.cc:
Ensure variable is always set (to remove compiler warning)
sql/sql_view.cc:
Use LINT_INIT() to remove compiler warnings
storage/maria/ma_loghandler.c:
Use LINT_INIT() to remove compiler warnings
storage/myisammrg/ha_myisammrg.cc:
Fixed wrong type to printf
storage/myisammrg/myrg_open.c:
Use LINT_INIT() to remove compiler warnings
storage/xtradb/include/ut0lst.h:
Trivial change of macro to remove compiler warning
strings/ctype-ucs2.c:
Use LINT_INIT() to remove compiler warnings
strings/ctype-utf8.c:
Use LINT_INIT() to remove compiler warnings
support-files/compiler_warnings.supp:
Suppress some not relevant warnings
unittest/mysys/waiting_threads-t.c:
Don't use ftruncate() as this gives warning about ignored return value
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
The bug is a compilation issue:
Function "find_key_block()" had thread operations
which were not guarded by "#if THREAD", add that now.
mysys/mf_keycache.c:
Without thread-support, we are in a single-threaded world,
so there is no parallelism, and the keycache cannot be
within a resize operation while this request is being processed.
can crash under load
Merge from 5.0, after backport from 5.1/5.4 to 5.0.
This makes the fixes for
Bug 44068 (RESTORE can disable the MyISAM Key Cache)
Bug 40944 (Backup: crash after myisampack)
available to 5.1.
* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
can crash under load
Backport from 5.1.
Does also include key cache fixes from:
Bug 44068 (RESTORE can disable the MyISAM Key Cache)
Bug 40944 (Backup: crash after myisampack)
include/keycache.h:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Added KEY_CACHE components in_resize and waiting_for_resize_cnt.
myisam/mi_preload.c:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Added code to allow LOAD INDEX to load indexes of different block size.
mysys/mf_keycache.c:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
.
Changed resize_key_cache() to not disable the key cache
after the flush phase. Changed queue handling to use
standard functions. Wake all threads waiting on resize_queue.
We can now have read/write threads waiting there (see below).
.
Combined add_to_queue() and the wait loops that were always
following it to the new function wait_on_queue().
Combined release_queue() and the condition that was always
preceding it to the new function release_whole_queue().
.
Added code to flag and respect the exceptional situation
BLOCK_IN_EVICTION.
.
Rewrote the resize branch of find_key_block().
.
Added code to the eviction handling in find_key_block()
to catch more exceptional cases.
.
Changed key_cache_read(), key_cache_insert() and key_cache_write()
so that they lock keycache->cache_lock whenever the key cache is
initialized. Checking for a disabled cache and incrementing and
decrementing the "resize counter" is always done within the lock.
Locking and unlocking as well as counting the "resize counter" is
now done once outside the loop. All three functions can now handle
a NULL return from find_key_block. This happens in the flush phase
of a resize and demands direct file I/O. Care is taken for
secondary requests (PAGE_WAIT_TO_BE_READ) to wait in any case.
Moved block status changes behind the copying of buffer data.
key_cache_insert() does now read the block if the caller did
supply less data than a full cache block.
key_cache_write() does now take care of parallel running flushes
(BLOCK_FOR_UPDATE, BLOCK_IN_FLUSHWRITE).
.
Changed free_block() to un-initialize block variables in the
correct order and respect an exceptional BLOCK_IN_EVICTION state.
.
Changed flushing to take care for parallel running writes.
Changed flushing to avoid freeing blocks in eviction.
Changed flushing to consider that parallel writes can move blocks
from the file_blocks hash to the changed_blocks hash.
Changed flushing to take care for other parallel flushes.
Changed flushing to assure that it ends with everything flushed.
Optimized normal flush at end of statement (FLUSH_KEEP),
but let other flush types be stringent.
.
Added some comments and debugging statements.
mysys/my_static.c:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Removed an unused global variable.
sql/ha_myisam.cc:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Moved an automatic (stack) variable to the scope where it is used.
sql/sql_table.cc:
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Changed TL_READ to TL_READ_NO_INSERT in mysql_preload_keys.
key_buffer_size.
The cause of corruption was number overflow when multiplying
two ulong values, number of used keycache blocks with size
of a single block. The result of multiplication exceeded ulong
range (4G) and this lead to incorrectly calculated buffer offset
in the key cache.
The fix is to use size_t for multiplication result.
This patch also fixes pointless cast in safemalloc
(size of allocated block to uint), that creates lot of false
alarm warnings when using big keycache (> 4GB) in debug mode.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
into mysql.com:/home/my/mysql-new
BitKeeper/etc/ignore:
auto-union
BUILD/SETUP.sh:
Auto merged
CMakeLists.txt:
Auto merged
client/get_password.c:
Auto merged
client/mysqldump.c:
Auto merged
client/mysqltest.c:
Auto merged
cmd-line-utils/readline/bind.c:
Auto merged
cmd-line-utils/readline/display.c:
Auto merged
cmd-line-utils/readline/histexpand.c:
Auto merged
cmd-line-utils/readline/history.c:
Auto merged
cmd-line-utils/readline/readline.c:
Auto merged
cmd-line-utils/readline/text.c:
Auto merged
dbug/user.r:
Auto merged
extra/yassl/src/handshake.cpp:
Auto merged
include/config-win.h:
Auto merged
include/m_string.h:
Auto merged
include/my_global.h:
Auto merged
include/my_pthread.h:
Auto merged
include/mysql/plugin.h:
Auto merged
include/mysql_com.h:
Auto merged
include/thr_alarm.h:
Auto merged
libmysql/CMakeLists.txt:
Auto merged
libmysql/Makefile.shared:
Auto merged
libmysql/dll.c:
Auto merged
libmysql/get_password.c:
Auto merged
libmysql/libmysql.c:
Auto merged
libmysqld/Makefile.am:
Auto merged
mysql-test/lib/mtr_cases.pl:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/alter_table.result:
Auto merged
mysql-test/r/change_user.result:
Auto merged
mysql-test/r/create.result:
Auto merged
mysql-test/r/innodb.result:
Auto merged
mysql-test/r/merge.result:
Auto merged
mysql-test/r/mix2_myisam.result:
Auto merged
mysql-test/r/mysqldump.result:
Auto merged
mysql-test/r/query_cache.result:
Auto merged
mysql-test/r/subselect.result:
Auto merged
mysql-test/valgrind.supp:
Auto merged
mysql-test/r/view.result:
Auto merged
mysql-test/suite/rpl/r/rpl_events.result:
Auto merged
mysql-test/suite/rpl/t/rpl_switch_stm_row_mixed.test:
Auto merged
mysql-test/t/create.test:
Auto merged
mysql-test/t/mysqldump.test:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/variables.test:
Auto merged
mysql-test/t/view.test:
Auto merged
mysys/mf_iocache.c:
Auto merged
mysys/mf_tempfile.c:
Auto merged
mysys/my_atomic.c:
Auto merged
mysys/my_bit.c:
Auto merged
mysys/my_bitmap.c:
Auto merged
mysys/my_compress.c:
Auto merged
mysys/my_create.c:
Auto merged
mysys/my_delete.c:
Auto merged
mysys/my_error.c:
Auto merged
mysys/my_init.c:
Auto merged
mysys/my_open.c:
Auto merged
mysys/my_realloc.c:
Auto merged
mysys/my_rename.c:
Auto merged
mysys/my_symlink.c:
Auto merged
mysys/my_sync.c:
Auto merged
mysys/my_thr_init.c:
Auto merged
mysys/thr_alarm.c:
Auto merged
mysys/thr_lock.c:
Auto merged
scripts/make_binary_distribution.sh:
Auto merged
server-tools/instance-manager/mysql_connection.cc:
Auto merged
sql/CMakeLists.txt:
Auto merged
sql/Makefile.am:
Auto merged
sql/events.cc:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/filesort.cc:
Auto merged
sql/gen_lex_hash.cc:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/handler.h:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_strfunc.cc:
Auto merged
sql/item_strfunc.h:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/lock.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/net_serv.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/partition_info.cc:
Auto merged
sql/rpl_injector.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/slave.cc:
Auto merged
sql/slave.h:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_cache.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_load.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_prepare.cc:
Auto merged
sql/sql_repl.cc:
Auto merged
sql/sql_test.cc:
Auto merged
sql/sql_union.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.cc:
Auto merged
sql/table.h:
Auto merged
sql/unireg.cc:
Auto merged
sql/share/errmsg.txt:
Auto merged
storage/csv/ha_tina.cc:
Auto merged
storage/csv/ha_tina.h:
Auto merged
storage/myisam/CMakeLists.txt:
Auto merged
storage/myisam/ft_boolean_search.c:
Auto merged
storage/myisam/ft_eval.c:
Auto merged
storage/myisam/ft_nlq_search.c:
Auto merged
storage/myisam/ft_parser.c:
Auto merged
storage/myisam/ft_static.c:
Auto merged
storage/myisam/ft_stopwords.c:
Auto merged
storage/myisam/ft_test1.c:
Auto merged
storage/myisam/ft_update.c:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
storage/myisam/mi_check.c:
Auto merged
storage/myisam/mi_create.c:
Auto merged
storage/myisam/mi_delete.c:
Auto merged
storage/myisam/mi_delete_all.c:
Auto merged
storage/myisam/mi_dynrec.c:
Auto merged
storage/myisam/mi_key.c:
Auto merged
storage/myisam/mi_packrec.c:
Auto merged
storage/myisam/mi_range.c:
Auto merged
storage/myisam/mi_search.c:
Auto merged
storage/myisam/mi_test1.c:
Auto merged
storage/myisam/mi_test2.c:
Auto merged
storage/myisam/mi_test3.c:
Auto merged
storage/myisam/mi_unique.c:
Auto merged
storage/myisam/mi_write.c:
Auto merged
storage/myisam/myisamchk.c:
Auto merged
storage/myisam/myisamdef.h:
Auto merged
storage/myisam/myisampack.c:
Auto merged
storage/myisam/sort.c:
Auto merged
storage/myisam/sp_test.c:
Auto merged
support-files/mysql.spec.sh:
Auto merged
tests/mysql_client_test.c:
Auto merged
configure.in:
Manual merge
dbug/dbug.c:
Restore to original state in Maria tree
The big diff comes from a wrong pull from 5.0 -> 5.1 after backporting dbug to 5.0 from 5.1
include/Makefile.am:
Manual merge
include/my_atomic.h:
Ignore changes
include/my_base.h:
Manual merge
include/my_dbug.h:
Use orginal my_dbug.h from maria tree
include/my_handler.h:
Manual merge
include/my_sys.h:
Manual merge
include/myisam.h:
Manual merge
mysql-test/lib/mtr_report.pl:
Manual merge
mysql-test/r/myisam.result:
Manual merge
mysql-test/suite/binlog/r/binlog_unsafe.result:
Manual merge
mysql-test/suite/binlog/t/binlog_unsafe.test:
Manual merge
mysql-test/suite/rpl/r/rpl_row_flsh_tbls.result:
Manual merge
mysql-test/suite/rpl/r/rpl_switch_stm_row_mixed.result:
No changes
mysql-test/suite/rpl/t/rpl_row_flsh_tbls.test:
Manual merge
mysql-test/t/change_user.test:
Manual merge
mysql-test/t/disabled.def:
Manual merge
mysql-test/t/merge.test:
No changes
mysql-test/t/myisam.test:
Manual merge
mysys/Makefile.am:
Manual merge
mysys/array.c:
Manual merge
mysys/mf_keycache.c:
Manual merge
mysys/my_getsystime.c:
Manual merge
mysys/my_handler.c:
Manual merge
mysys/my_pread.c:
Manual merge
mysys/safemalloc.c:
Manual merge
sql/ha_partition.cc:
Manual merge
sql/handler.cc:
Manual merge
sql/lex.h:
Manual merge
sql/mysql_priv.h:
Manual merge
sql/mysqld.cc:
Manual merge
sql/set_var.h:
Manual merge
sql/sql_class.cc:
Manual merge
sql/sql_insert.cc:
Manual merge
sql/sql_parse.cc:
Manual merge
sql/sql_select.cc:
Manual merge
sql/sql_show.cc:
Manual merge
sql/sql_table.cc:
Manual merge
storage/myisam/mi_checksum.c:
No changes
storage/myisam/mi_extra.c:
Manual merge
storage/myisam/mi_open.c:
Manual merge
storage/myisammrg/ha_myisammrg.cc:
Manual merge
strings/strmake.c:
No changes
