Base code and idea from a patch from by plinux at Taobao.
The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.
Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.
The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.
Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)
client/completion_hash.cc:
Updated call to init_alloc_root()
client/mysql.cc:
Updated call to init_alloc_root()
client/mysqlbinlog.cc:
init_dynamic_array() -> my_init_dynamic_array()
Updated call to init_alloc_root()
client/mysqlcheck.c:
Updated call to my_init_dynamic_array()
client/mysqldump.c:
Updated call to init_alloc_root()
client/mysqltest.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Fixed compiler warnings
extra/comp_err.c:
Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
Updated call to my_init_dynamic_array()
include/hash.h:
Added HASH_THREAD_SPECIFIC
include/heap.h:
Added flag is internal temporary table.
include/my_dir.h:
Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
Added malloc_flags to DYNAMIC_ARRAY
Added extra mysys flag argument to my_init_dynamic_array()
Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
Updated paramaters for init_alloc_root()
include/my_tree.h:
Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
Updated parameters to init_tree()
include/myisamchk.h:
Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
Added MYSQL_THREAD_SPECIFIC_MALLOC
Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
Updated file
include/mysql_com.h:
Used 'unused1' to mark memory as thread specific.
Updated parameters for my_net_init()
libmysql/libmysql.c:
Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
Updated call to init_alloc_root()
mysql-test/r/create.result:
Updated results
mysql-test/r/user_var.result:
Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
Updated results
mysql-test/t/show_explain.test:
Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
Added test of memory_usage variable.
mysys/array.c:
Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
All allocated memory is marked with the given my_flags.
Removed obsolete function init_dynamic_array()
mysys/default.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
mysys/hash.c:
Updated call to my_init_dynamic_array_ci().
Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
init_dynamic_array() -> my_init_dynamic_array()
Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
Extend MEM_ROOT with a flag if memory is thread specific.
This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
Store at start of each allocated memory block the size of the block and if the block is thread specific.
Call malloc_size_cb_func, if set, with the memory allocated/freed.
Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
Added flags to init_tree() so that one can mark tree to be thread specific.
Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
Updated call to init_alloc_root()
All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
Updated call to my_init_dynamic_array()
sql-common/client.c:
Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
Updated call to my_init_dynamic_array().
Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
Updated call to init_alloc_root()
sql/debug_sync.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
Removed calls to net_end() as this is now done in ~THD()
Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
Updated call to init_alloc_root()
Updated call to my_net_init()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
Updated call to my_net_init()
Updated call to init_sql_alloc()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
Updated call to init_alloc_root()
sql/handler.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
Added missing call to my_dir_end()
sql/item_func.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
More DBUG
Updated call to init_alloc_root()
sql/mdl.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
Added total_memory_used
Updated call to init_alloc_root()
Move mysql_cond_broadcast() before my_thread_end()
Added mariadb_dbug_id() to count memory per THD instead of per thread.
Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
Updated call to my_init_dynamic_array().
Updated call to my_net_init().
Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
Added status variable 'memory_used'.
Updated call to init_alloc_root()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
Added set_current_thd()
sql/net_serv.cc:
Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
Store in net->thread_specific_malloc if memory is thread specific.
Mark memory to be thread specific if requested.
sql/opt_range.cc:
Updated call to my_init_dynamic_array()
Updated call to init_sql_alloc()
Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
Fixed compiler warning
sql/records.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
Updated call to init_sql_alloc()
sql/rpl_mi.cc:
Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
Updated call to init_alloc_root()
sql/rpl_utility.cc:
Updated call to my_init_dynamic_array()
sql/slave.cc:
Initialize things properly before calling functions that allocate memory.
Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
Updated call to my_init_dynamic_array()
sql/sql_base.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
Updated comment
sql/sql_class.cc:
Added parameter to warning_info() that not initialize it until THD is fully created.
Updated call to init_sql_alloc()
Mark THD::user_vars has to be thread specific.
Updated call to my_init_dynamic_array()
Ensure that memory allocated by THD is assigned to the THD.
More DBUG
Always acll net_end() in ~THD()
Assert that all memory signed to this THD is really deleted at ~THD.
Fixed set_status_var_init() to not reset memory_used.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
Added MY_THREAD_SPECIFIC to allocated memory.
Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
This is used by THD::THD() to not allocate memory until THD is ready.
Added Warning_info::free_memory()
sql/sql_error.h:
Updated Warning_info() class.
sql/sql_handler.cc:
Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
More DBUG
sql/sql_join_cache.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
Updated call to my_init_dynamic_array()
sql/sql_lex.h:
Updated call to my_init_dynamic_array()
sql/sql_load.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
Ensure that examined_row_count() is reset before query.
Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
Added parameter to warning_info() that it should be fully initialized.
Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
Added MY_THREAD_SPECIFIC to allocated memory.
More DBUG
sql/sql_servers.cc:
Updated call to init_sql_alloc() to mark memory some memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
Updated call to my_init_dynamic_array()
Mark my_dir() memory thread specific.
Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
More DBUG.
Added malloc_size and examined_row_count to SHOW PROCESSLIST.
Added MY_THREAD_SPECIFIC to allocated memory.
Updated call to init_sql_alloc()
Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
Fixed compiler warning
sql/sql_string.cc:
String elements can now be marked as thread specific.
sql/sql_string.h:
String elements can now be marked as thread specific.
sql/sql_table.cc:
Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Fixed compiler warning
sql/sql_test.cc:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
Updated call to init_sql_alloc()
sql/sql_udf.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
Updated call to init_sql_alloc().
Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
Updated call to init_sql_alloc()
sql/thr_malloc.cc:
Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
Updated prototype for init_sql_alloc()
sql/tztime.cc:
Updated call to init_sql_alloc()
Updated call to init_alloc_root() to mark memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
Updated call to init_tree()
storage/maria/ma_ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
Updated call to init_tree()
storage/myisam/ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
Updated call to init_tree()
storage/myisam/mi_check.c:
Updated call to init_alloc_root()
storage/myisam/mi_write.c:
Updated call to my_init_dynamic_array()
Updated call to init_tree()
storage/myisam/myisamlog.c:
Updated call to init_tree()
storage/myisam/myisampack.c:
Updated call to init_tree()
storage/myisam/sort.c:
Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
Rest current_thd
storage/perfschema/pfs_instr.cc:
Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
Disable compiler warning from offsetof macro.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
CMakeLists.txt:
1. add -DSAFEMALLOC -DSAFE_MUTEX in the top-level CMakeLists.txt
don't force plugins to copy-paste these lines in their CMakeLists.txt
2.1 search plugin/* for plugins (not only storage/*),
2.2 recognize MYSQL_PLUGIN (not only MYSQL_STORAGE_ENGINE),
2.3 extract library names from the plug.in (don't force library names to
be ha_<engine>.dll and <engine>.lib)
include/mysql/plugin.h:
define MYSQL_PLUGIN_EXPORT appropriately
(backport from 5.5)
libmysqld/CMakeLists.txt:
remove unnecessary workaround
plugin/fulltext/CMakeLists.txt:
build fulltext example plugin on windows
storage/maria/CMakeLists.txt:
The library is called libmaria_s.lib, not maria.lib
storage/maria/unittest/CMakeLists.txt:
The library is called libmaria_s.lib, not maria.lib
storage/myisam/CMakeLists.txt:
The library is called libmyisam_s.lib, not myisam.lib
storage/mysql_storage_engine.cmake:
introduce MYSQL_PLUGIN macro.
don't force library names to be ha_<engine>.dll and <engine>.lib
storage/xtradb/CMakeLists.txt:
remove a condition from include
win/README:
don't use deprecated syntax
win/configure-mariadb.sh:
don't use deprecated syntax
win/configure.js:
1. support MYSQL_PLUGIN in addition to MYSQL_STORAGE_ENGINE.
2. support plugin/* in addition to storage/*
A lot of small fixes and new test cases.
client/mysqlbinlog.cc:
Cast removed
client/mysqltest.cc:
Added missing DBUG_RETURN
include/my_pthread.h:
set_timespec_time_nsec() now only takes one argument
mysql-test/t/date_formats.test:
Remove --disable_ps_protocl as now also ps supports microseconds
mysys/my_uuid.c:
Changed to use my_interval_timer() instead of my_getsystime()
mysys/waiting_threads.c:
Changed to use my_hrtime()
sql/field.h:
Added bool special_const_compare() for fields that may convert values before compare (like year)
sql/field_conv.cc:
Added test to get optimal copying of identical temporal values.
sql/item.cc:
Return that item_int is equal if it's positive, even if unsigned flag is different.
Fixed Item_cache_str::save_in_field() to have identical null check as other similar functions
Added proper NULL check to Item_cache_int::save_in_field()
sql/item_cmpfunc.cc:
Don't call convert_constant_item() if there is nothing that is worth converting.
Simplified test when years should be converted
sql/item_sum.cc:
Mark cache values in Item_sum_hybrid as not constants to ensure they are not replaced by other cache values in compare_datetime()
sql/item_timefunc.cc:
Changed sec_to_time() to take a my_decimal argument to ensure we don't loose any sub seconds.
Added Item_temporal_func::get_time() (This simplifies some things)
sql/mysql_priv.h:
Added Lazy_string_decimal()
sql/mysqld.cc:
Added my_decimal constants max_seconds_for_time_type, time_second_part_factor
sql/table.cc:
Changed expr_arena to be of type CONVENTIONAL_EXECUTION to ensure that we don't loose any items that are created by fix_fields()
sql/tztime.cc:
TIME_to_gmt_sec() now sets *in_dst_time_gap in case of errors
This is needed to be able to detect if timestamp is 0
storage/maria/lockman.c:
Changed from my_getsystime() to set_timespec_time_nsec()
storage/maria/ma_loghandler.c:
Changed from my_getsystime() to my_hrtime()
storage/maria/ma_recovery.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/maria/unittest/trnman-t.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/xtradb/handler/ha_innodb.cc:
Added support for new time,datetime and timestamp
unittest/mysys/thr_template.c:
my_getsystime() -> my_interval_timer()
unittest/mysys/waiting_threads-t.c:
my_getsystime() -> my_interval_timer()
storage/maria/ma_loghandler.c:
Added assert if we give wrong type to translog_write_record().
(This should be impossible, but the failure record in buildbot also looks impossible so we need more information...)
storage/maria/ma_loghandler.h:
Added prototype for check_translog_description_table()
storage/maria/unittest/ma_loghandler_examples.c:
Call check_translog_description_table() to register number of log_record_type_descriptor[] events.
aria_chk now returns 1 if one got any warnings during check and 2 if one got errors.
lp:728919 maria_chk should fail on all detected corruptions
mysql-test/lib/My/SafeProcess.pm:
Removed ^M lines from the file
storage/maria/ma_check.c:
Fixed indentation
storage/maria/maria_chk.c:
aria_chk now returns 1 if one got any warnings during check and 2 if one got errors.
storage/maria/unittest/ma_control_file-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_maria_log_cleanup.c:
Added help function create_tmpdir()
storage/maria/unittest/ma_pagecache_consist.c:
Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_rwconsist.c:
Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_rwconsist2.c:
Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_single.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_all-t:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Use unique sub directory for test results
remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Use unique sub directory for test results
remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Use unique sub directory for test results
remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Use unique sub directory for test results
support-files/compiler_warnings.supp:
Supress some warnings
unittest/unit.pl:
Run 4 unittests in parallel
storage/maria/unittest/ma_test_loghandler-t.c:
Don't sync during test
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Don't sync during test
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Don't sync during test
unittest/mysys/bitmap-t.c:
Don't test all bit combinations (not needed)
unittest/mysys/thr_template.c:
Remove sleep as old bug should be fixed nowadays
unittest/mysys/waiting_threads-t.c:
Only run test with --big
unittest/mytap/tap.c:
Print total time at end of test.
unittest/unit.pl:
Use TAP::Harness instead of Test::Harness (recommended according to manual)
Add times to tests.
- Fixed mysql-test-run failures on window
- Fixed compiler warnings from my last push (sorry about that)
- Fixed that maria_chk --repair --extended works again
- Fixed compiler warnings about using not unitialized data
mysql-test/mysql-test-run.pl:
Better output
mysql-test/suite/parts/inc/partition_check_drop.inc:
Use remove_files_wildcard instead of rm
mysys/safemalloc.c:
Fixed argument to printf
storage/maria/ma_cache.c:
Don't give errors when running maria_chk
storage/maria/ma_dynrec.c:
Don't give errors when running maria_chk
storage/maria/ma_rt_test.c:
Added option --datadir for where to put logs and test data
storage/maria/ma_test1.c:
Added option --datadir for where to put logs and test data
storage/maria/ma_test2.c:
Added option --datadir for where to put logs and test data
storage/maria/maria_chk.c:
If --datadir is used but --logdir is not, set --logdir from --datadir
(this reflects how --help said how things should work)
storage/maria/maria_read_log.c:
Changed short option for 'maria-log-dir-path' from -l to -h to be same as mysqld, maria_chk, ma_test1 etc..
storage/maria/unittest/ma_test_all-t:
Allow one to specify --tmpdir for where to store logs and data
storage/xtradb/buf/buf0buf.c:
Fixed compiler warnings about using not unitialized data
storage/xtradb/row/row0upd.c:
Fixed compiler warnings about using not unitialized data
storage/xtradb/srv/srv0srv.c:
Fixed compiler warnings about using not unitialized data
Fixed error in Maria unittest
Fixes other issues found by test case for lp:700623 "Aria recovery: ma_blockrec.c:3930: _ma_update_at_original_place: Assertion `block->org_bitmap_value == .."
Fixes lp:670356 "Aria table "is marked as crashed and should be repaired"
mysql-test/lib/v1/mysql-test-run.pl:
Don't delete directory mysql-test/var if we use mysql-test-run --vardir=
mysql-test/mysql-test-run.pl:
Don't delete directory mysql-test/var if we use mysql-test-run --vardir=
(Needed to be able to run multiple concurrent mysql-test-run's/randgen on the same server)
Added extra supression
storage/maria/ma_pagecache.c:
Removed duplicate DBUG_PRINT information
Added KEYCACHE_PRINT for printing information that was already printed to DBUG log (to get rid of some duplicated output)
More comments
Fixed bug that caused page block to be used by two treads at the same time (with different page information)
Mark block->status with PCBLOCK_DIRECT_W independent of the block is changed or not. (Safety fix)
storage/maria/maria_chk.c:
Better --help
storage/maria/unittest/ma_test_recovery.pl:
Ignore differences in 'recover time'.
Fixed error in Maria unittest
This makes the keys smaller (no row pointer) and gives us proper errors if we
use the table wrongly.
sql/sql_select.cc:
Use NO_RECORD for tables that doesn't need row data.
storage/maria/Makefile.am:
Added ma_norec.c
storage/maria/ma_check.c:
Added support for NO_RECORD record format (don't store any row data)
storage/maria/ma_norec.c:
Added support for NO_RECORD record format
storage/maria/ma_open.c:
Added support for NO_RECORD record format
storage/maria/ma_search.c:
Added support for 0 size row pointers (used with NO_RECORD)
storage/maria/ma_test1.c:
Added testing of NO_RECORD record format.
storage/maria/maria_chk.c:
Added support for NO_RECORD
storage/maria/maria_def.h:
Added support for NO_RECORD
storage/maria/unittest/ma_test_all-t:
Added testing of NO_RECORD record format
Reason: inconsistent compilation, federatedx is compiled without SAFEMALLOC
flag, while anything else is compiled with SAFEMALLOC.
As a consequence, my_hash_init used inside federatedx initialization does not
provide correct caller info parameters (file, line) , so they are initialized with
whatever is on stack. When info about allocated memory is output in
COM_DEBUG command, the server crashes trying to output string starting at
0xcccccccccccccccc.
The fix is to remove SAFEMALLOC preprocessor flags
from every CMakeLists.txt, except the top-level one.
Also, SAFEMALLOC is not defined by default now, instead
there is WITH_DEBUG_FULL CMake option which adds
-DSAFEMALLOC to C and C++ flags in debug compilation.
This option is off by default, because
1) Debug C runtime already has heap debugging builtin with
overwrite and leak detection
2)safemalloc considerably slows down the tests.
Note also that
- SAFEMALLOC is gone in MySQL5.5
- On Windows, heap related overflows can also be found using free pageheap utility
(that is also part of application verifier). This is even more efficient if there are no other layers
on top of Windows heap allocator, e.g it is most efficient with release version.
- Removed SCCS rule from Makefile.am
- Made dummy rule in sql_yacc.yy to get rid of compiler warning about not used label.
Don't use maintainer mode with valgrind (as we don't want to initialize all variables)
config/ac-macros/maintainer.m4:
Don't use maintainer mode with valgrind (as we don't want to initialize all variables)
Force initialization of variables when using -Werror (To get rid of compiler warnings)
configure.in:
Don't use maintainer mode with valgrind (as we don't want to initialize all variables)
sql/sql_yacc.yy:
Made dummy rule in sql_yacc.yy to get rid of compiler warning about not used label.
Fix cleanup to really remove 'aria_log' files. Fixes failures in maria unit tests on some platforms.
Fixed compiler warnings
include/mysql/plugin.h:
Changed def_val back to const, to remove compiler warnings.
storage/maria/ma_init.c:
Rename control file and log files from maria_xxx to aria_xxx when upgrading from MariaDB 5.1
storage/maria/unittest/ma_maria_log_cleanup.c:
Fix cleanup to really remove 'aria_log' files. Fixes failures in maria unit tests on some platforms.
More DBUG_ASSERT() to discover errors earlier
More checking of BLOCK structures in Aria.
Fixed crashing bug in Aria when doing UPDATE of several records in same block when doing table scan.
storage/federatedx/ha_federatedx.cc:
Added missing delete_dynamic(); This fixes the valgrind warnings about lost memory discovered by buildbot.
storage/maria/ma_blockrec.c:
Added debugging helper function _ma_print_block_info() to print block structure.
Changed arguments to _ma_print_directory() so it can be called by _ma_print_block_info()
check_directory() now also checks that empty_space on block is calculated correctly.
Added some local variables to get more information about what happend when things crash.
Fixed crashing bug when doing UPDATE of several records in same block when doing table scan.
storage/maria/ma_key_recover.c:
Simple optimization (don't call bmove_upp() if not needed); This could happen during page split.
storage/maria/ma_recovery.c:
Fixed compiler warning
storage/maria/ma_test2.c:
Run test without syncing to disk. (Not needed for this test)
storage/maria/ma_write.c:
More comments.
Added DBUG_ASSERT() to find wrong log entires, like the one reported at LP#616344, during log file creation.
storage/maria/unittest/ma_test_recovery.pl:
Added extra -s to maria_chk to get it more silent.
This fixes that ma_test_all.sh works again.
storage/maria/unittest/ma_pagecache_single.c:
Check if we can flush write locked page for normal flush
Check that we can't flush read locked page with FLUSH_KEEP_LAZY (used by checkpoint)
- Added checking of return value for system(), freopen(), fgets() and chown()
- Ensure that calls that require a format strings gets a format string
- Other trivial things
Updated test suite results (especially for pbxt and embedded server)
Removed warning for "Invalid (old?) table or database name 'mysqld.1'" from pbxt tests
Speed up some pbxt tests by inserting begin ; commit; around "while loops with inserts"
Added mysqld startup option '--debug-flush'
Create maria_recovery.trace in data directory instead of current directory
client/mysql.cc:
Check return value from system()
client/mysql_upgrade.c:
Check return value from fgets()
client/mysqladmin.cc:
Check return value from fgets()
client/mysqlslap.c:
Check return value from system() (but ignore it, as it's not critical)
extra/yassl/src/crypto_wrapper.cpp:
Check return value from fgets() (but ignore it, as it's internal file)
extra/yassl/taocrypt/src/aes.cpp:
Added extra {} to remove compiler warning
extra/yassl/taocrypt/src/blowfish.cpp:
Added extra {} to remove compiler warning
extra/yassl/taocrypt/src/misc.cpp:
Ifdef not used code
include/mysys_err.h:
Added error message for failing chown()
mysql-test/mysql-test-run.pl:
Don't give warning for skipping ndbcluster (never enabled in MariaDB)
mysql-test/suite/funcs_1/r/is_columns_is_embedded.result:
Update with new information schema information
mysql-test/suite/funcs_1/r/is_tables_is_embedded.result:
New test
mysql-test/suite/funcs_1/r/is_tables_myisam_embedded.result:
Update test results (has not been tested for a long time)
mysql-test/suite/funcs_1/r/is_tables_mysql_embedded.result:
Update test results (has not been tested for a long time)
mysql-test/suite/funcs_1/t/is_tables_is.test:
Don't run with embedded server (as results differ)
I added a new test for embedded server
mysql-test/suite/funcs_1/t/is_tables_is_embedded.test:
New test
mysql-test/suite/pbxt/my.cnf:
Allow one to run pbxt tests without having to specify --mysqld=--default-storage-engine=pbxt
mysql-test/suite/pbxt/t/count_distinct3.test:
Speed up test by inserting begin; ... commit;
mysql-test/suite/pbxt/t/subselect.test:
Speed up test by inserting begin; ... commit;
mysys/errors.c:
Added error message for failing chown()
mysys/my_copy.c:
Added error message for failing chown()
mysys/my_redel.c:
Added error message for failing chown()
mysys/safemalloc.c:
Added cast to get rid of compiler warning
sql/ha_partition.cc:
Fixed wrong argument to sql_print_error() (it requires a format string)
sql/log.cc:
Test return value of freopen()
sql/mysqld.cc:
Test return value of freopen()
Added startup option '--debug-flush' to be used when one gets a core dump (easy to explain to people on IRC)
sql/rpl_rli.cc:
Fixed wrong argument to sql_print_error() (it requires a format string)
sql/set_var.cc:
Added {} to get rid of compiler warnings
sql/slave.cc:
Fixed wrong argument to mi->report() and sql_print...() (they require a format string)
sql/sql_cache.cc:
Fixed wrong argument to sql_printinformation() (it requires a format string)
sql/sql_parse.cc:
Test return value of fgets()
sql/sql_plugin.cc:
Fixed wrong argument to sql_print_error() (it requires a format string)
sql/sql_select.cc:
Use unique table name for internal temp tables instead of full path
(Simple speed & space optimization)
sql/udf_example.c:
Removed compiler warning about not used variable
storage/maria/ha_maria.cc:
Fixed wrong argument to sql_print_error() and ma_check_print_error() (they require a format string)
storage/maria/ma_recovery.c:
Create maria_recovery.trace in data directory instead of current directory
storage/maria/unittest/ma_test_loghandler-t.c:
Fixed wrong argument to ok(); Requires a format string
storage/pbxt/src/strutil_xt.cc:
Detect temporary tables by checking if that path for the table is in the mysql data directory.
The database for temporary tables is after this patch, from PBXT point of view, ""
This is needed to stop PBXT from calling filename_to_tablename() with the base directory as an argument, which caused
ERROR: Invalid (old?) table or database name 'mysqld.1'" in the log when running the test suite.
tests/mysql_client_test.c:
Fixed compiler warnings
unittest/mysys/base64-t.c:
Fixed wrong argument to diag() (it requires a format string)
Added a comment that the current 'print' of differing buffers doesn't print the right thing,
but didn't fix this as it's not important (unless we find a bug in the real code)
include/atomic/generic-msvc.h:
prevent possible compiler warnings
include/lf.h:
comments, better definition for LF_HASH_OVERHEAD
include/maria.h:
define MARIA_CANNOT_ROLLBACK here
include/my_pthread.h:
avoid possible name clash
include/waiting_threads.h:
comments, const, move WT_RESOURCE to waiting_threads.c
mysql-test/suite/maria/r/maria_notembedded.result:
new test
mysql-test/suite/maria/t/maria_notembedded.test:
new test - 5-way deadlock
mysys/lf_hash.c:
better definition for LF_HASH_OVERHEAD
mysys/my_static.c:
comment
mysys/my_thr_init.c:
casts
mysys/waiting_threads.c:
comments, asserts, etc
server-tools/instance-manager/parse.cc:
fix my_init_dynamic_array() to follow new calling conventions
sql/mysqld.cc:
call wt_init after set_proper_floating_point_mode
sql/sql_class.h:
comment
storage/maria/ha_maria.cc:
move MARIA_CANNOT_ROLLBACK to a common header
storage/maria/ma_commit.c:
comment
storage/maria/ma_write.c:
comments, check for HA_ERR_FOUND_DUPP_KEY
storage/maria/trnman.c:
comments, assert
storage/maria/trnman.h:
comments
storage/maria/unittest/trnman-t.c:
be paranoid
unittest/mysys/lf-t.c:
comments
unittest/mysys/waiting_threads-t.c:
comments, safety, memory leak
storage/maria/ma_pagecache.c:
Fixed ability to read without read lock acquiring.
storage/maria/unittest/CMakeLists.txt:
New unit test which tests simple read and prolonged writes consistency added.
storage/maria/unittest/Makefile.am:
New unit test which tests simple read and prolonged writes consistency added.
storage/maria/unittest/ma_pagecache_rwconsist2.c:
New unit test which tests simple read and prolonged writes consistency added.
in write_changed_bitmap(), and page cache forbids that. Here we make the page
cache more relaxed. Original patch by Sanja, simplified by me as limited to
not-locked. See comment of ma_bitmap.c.
With that, maria_stress.yy runs until hitting BUG 39665.
storage/maria/ma_bitmap.c:
A thread which unpins bitmap pages in _ma_bitmap_unpin_all() sometimes
hit an assertion in the page cache (info!=0 in remove_pin()) which states
that you can unpin/unlock only what *you* have pinned/locked.
Fixed by setting the new last parameter of pagecache_unlock_by_link()
to TRUE in _ma_bitmap_unpin_all().
storage/maria/ma_blockrec.c:
new prototype and splitting assertion in three (3rd one fires: BUG 39665)
storage/maria/ma_check.c:
new prototype
storage/maria/ma_key_recover.c:
new prototype
storage/maria/ma_loghandler.c:
new prototype
storage/maria/ma_pagecache.c:
Allow a thread to unpin, with pagecache_unlock_by_link(), a non-locked page pinned by others.
This is a hack for _ma_bitmap_unpin_all() which needs to unpin pages which were
pinned by other threads in write_changed_bitmap().
storage/maria/ma_pagecache.h:
new prototype
storage/maria/ma_preload.c:
new prototype
storage/maria/unittest/ma_pagecache_rwconsist.c:
new prototype
storage/maria/unittest/ma_pagecache_single.c:
new prototype
