Commit graph

611 commits

Author SHA1 Message Date
Sergei Golubchik
72ba95873a 10.0-base merge
(without InnoDB - all InnoDB changes were ignored)
2013-06-06 21:32:29 +02:00
Sergei Golubchik
4749d40c63 5.5 merge 2013-06-06 17:51:28 +02:00
Sergei Golubchik
b381cf843c mysql-5.5.31 merge 2013-05-07 13:05:09 +02:00
Sergei Golubchik
a9035be5b7 10.0-base merge 2013-04-15 15:09:22 +02:00
Sergei Golubchik
e06cb31719 CREATE TABLE and frm-less discovering engines.
Now CREATE TABLE does not write the frm file on disk,
if the engine can discover it
2013-04-09 15:57:09 +02:00
Sergei Golubchik
bbbd7cedf5 my_dir() cleanup
* replace pointer acrobatics with a struct
* make sorting explicit: MY_DONT_SORT -> MY_WANT_SORT
(if you want something to be done - say it. fixes all places where
my_dir() was used without thinking)
* typo s/number_off_files/number_of_files/
* directory_file_name() doesn't need to be extern
* remove #ifdef __BORLANDC__
* ignore '.' and '..' entries
2013-04-07 15:19:45 +02:00
Murthy Narkedimilli
8f73221035 Updated/added copyright headers 2013-02-26 06:35:17 +01:00
Murthy Narkedimilli
053d7e775c Updated/added copyright headers. 2013-02-25 15:26:00 +01:00
Elena Stepanova
a555ceb2fb Fix for MDEV-4149, backport from 10.0:
committer: Sergei Golubchik <sergii@pisem.net> 
    fix for a valgrind builds. 
    my_alloca() cannot have MY_THREAD_SPECIFIC, because can be used 
    outside of the THD context.
2013-02-08 02:21:23 +04:00
Sergei Golubchik
48061c32f1 fix for a valgrind builds.
my_alloca() cannot have MY_THREAD_SPECIFIC, because can be used
outside of the THD context.
2013-01-31 21:32:21 +01:00
Sergei Golubchik
ab83952f29 10.0-base merge 2013-01-31 09:48:19 +01:00
Sergei Golubchik
cf20de000b my_alloca() when it's mapped to malloc() works most certainly MY_THREAD_SPECIFIC 2013-01-28 15:06:36 +01:00
Sergei Golubchik
c71fad9983 cleanup:
* don't use 'myf flags', when 'my_bool is_thread_specific' is meant
* call set_malloc_size_cb() for embedded too
* warn in safemalloc if the memory is freed by a wrong thread

sql/mysqld.cc:
  move set_malloc_size_cb() to a function that is also called for embedded
sql/mysqld.h:
  gdb-friendly, one can put breakpoint on a function, but not on a macro
sql/sql_class.cc:
  initialize thread_id earlier
2013-01-23 16:20:39 +01:00
Sergei Golubchik
e3f08e7e0b cleanup: remove unused init_dynamic_array and init_dynamic_array2 symbols,
as only my_init_dynamic_array and my_init_dynamic_array2 are used everywhere.
fix ha_cassandra to compile.
2013-01-23 16:18:53 +01:00
Michael Widenius
a260b15554 MDEV-4011 Added per thread memory counting and usage
Base code and idea from a patch from by plinux at Taobao.

The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.

Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.

The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array()  has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.

Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)


client/completion_hash.cc:
  Updated call to init_alloc_root()
client/mysql.cc:
  Updated call to init_alloc_root()
client/mysqlbinlog.cc:
  init_dynamic_array() -> my_init_dynamic_array()
  Updated call to init_alloc_root()
client/mysqlcheck.c:
  Updated call to my_init_dynamic_array()
client/mysqldump.c:
  Updated call to init_alloc_root()
client/mysqltest.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Fixed compiler warnings
extra/comp_err.c:
  Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
  Updated call to my_init_dynamic_array()
include/hash.h:
  Added HASH_THREAD_SPECIFIC
include/heap.h:
  Added flag is internal temporary table.
include/my_dir.h:
  Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
  Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
  Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
  Added malloc_flags to DYNAMIC_ARRAY
  Added extra mysys flag argument to my_init_dynamic_array()
  Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
  Updated paramaters for init_alloc_root()
include/my_tree.h:
  Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
  Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
  Updated parameters to init_tree()
include/myisamchk.h:
  Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
  Added MYSQL_THREAD_SPECIFIC_MALLOC
  Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
  Updated file
include/mysql_com.h:
  Used 'unused1' to mark memory as thread specific.
  Updated parameters for my_net_init()
libmysql/libmysql.c:
  Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
  Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
  Updated call to init_alloc_root()
mysql-test/r/create.result:
  Updated results
mysql-test/r/user_var.result:
  Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
  Updated results
mysql-test/t/show_explain.test:
  Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
  Added test of memory_usage variable.
mysys/array.c:
  Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
  All allocated memory is marked with the given my_flags.
  Removed obsolete function init_dynamic_array()
mysys/default.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
mysys/hash.c:
  Updated call to my_init_dynamic_array_ci().
  Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
  init_dynamic_array() -> my_init_dynamic_array()
  Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
  Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
  Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
  Extend MEM_ROOT with a flag if memory is thread specific.
  This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
  Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
  Store at start of each allocated memory block the size of the block and if the block is thread specific.
  Call malloc_size_cb_func, if set, with the memory allocated/freed.
  Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
  Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
  Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
  Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
  Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
  Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
  Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
  Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
  Added flags to init_tree() so that one can mark tree to be thread specific.
  Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
  Updated call to init_alloc_root()
  All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
  Updated call to my_init_dynamic_array()
sql-common/client.c:
  Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
  Updated call to my_init_dynamic_array().
  Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
  Updated call to init_alloc_root()
sql/debug_sync.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
  Removed calls to net_end() as this is now done in ~THD()
  Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
  Updated call to init_alloc_root()
  Updated call to my_net_init()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
  Updated call to my_net_init()
  Updated call to init_sql_alloc()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
  Updated call to init_alloc_root()
sql/handler.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added missing call to my_dir_end()
sql/item_func.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
  More DBUG
  Updated call to init_alloc_root()
sql/mdl.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
  Added total_memory_used
  Updated call to init_alloc_root()
  Move mysql_cond_broadcast() before my_thread_end()
  Added mariadb_dbug_id() to count memory per THD instead of per thread.
  Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
  Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
  Updated call to my_init_dynamic_array().
  Updated call to my_net_init().
  Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
  Added status variable 'memory_used'.
  Updated call to init_alloc_root()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
  Added set_current_thd()
sql/net_serv.cc:
  Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
  Store in net->thread_specific_malloc if memory is thread specific.
  Mark memory to be thread specific if requested.
sql/opt_range.cc:
  Updated call to my_init_dynamic_array()
  Updated call to init_sql_alloc()
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
  Fixed compiler warning
sql/records.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
  Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
  Updated call to init_sql_alloc()
sql/rpl_mi.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
  Updated call to init_alloc_root()
sql/rpl_utility.cc:
  Updated call to my_init_dynamic_array()
sql/slave.cc:
  Initialize things properly before calling functions that allocate memory.
  Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
  Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
  Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
  Updated call to my_init_dynamic_array()
sql/sql_base.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
  Updated comment
sql/sql_class.cc:
  Added parameter to warning_info() that not initialize it until THD is fully created.
  Updated call to init_sql_alloc()
  Mark THD::user_vars has to be thread specific.
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by THD is assigned to the THD.
  More DBUG
  Always acll net_end() in ~THD()
  Assert that all memory signed to this THD is really deleted at ~THD.
  Fixed set_status_var_init() to not reset memory_used.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
  Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
  This is used by THD::THD() to not allocate memory until THD is ready.
  Added Warning_info::free_memory()
sql/sql_error.h:
  Updated Warning_info() class.
sql/sql_handler.cc:
  Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
  More DBUG
sql/sql_join_cache.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
  Updated call to my_init_dynamic_array()
sql/sql_lex.h:
  Updated call to my_init_dynamic_array()
sql/sql_load.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
  Ensure that examined_row_count() is reset before query.
  Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
  Added parameter to warning_info() that it should be fully initialized.
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
  Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
  Added MY_THREAD_SPECIFIC to allocated memory.
  More DBUG
sql/sql_servers.cc:
  Updated call to init_sql_alloc() to mark memory some memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
  Updated call to my_init_dynamic_array()
  Mark my_dir() memory thread specific.
  Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
  More DBUG.
  Added malloc_size and examined_row_count to SHOW PROCESSLIST.
  Added MY_THREAD_SPECIFIC to allocated memory.
  Updated call to init_sql_alloc()
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
  Fixed compiler warning
sql/sql_string.cc:
  String elements can now be marked as thread specific.
sql/sql_string.h:
  String elements can now be marked as thread specific.
sql/sql_table.cc:
  Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Fixed compiler warning
sql/sql_test.cc:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
  Updated call to init_sql_alloc()
sql/sql_udf.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
  Updated call to init_sql_alloc().
  Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
  Updated call to init_sql_alloc()
sql/thr_malloc.cc:
  Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
  Updated prototype for init_sql_alloc()
sql/tztime.cc:
  Updated call to init_sql_alloc()
  Updated call to init_alloc_root() to mark memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
  Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
  Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
  Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
  - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
  - Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
  Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
  Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
  Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
  Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
  Updated call to init_tree()
storage/maria/ma_ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
  Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
  Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
  Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
  Updated call to init_tree()
storage/myisam/ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
  Updated call to init_tree()
storage/myisam/mi_check.c:
  Updated call to init_alloc_root()
storage/myisam/mi_write.c:
  Updated call to my_init_dynamic_array()
  Updated call to init_tree()
storage/myisam/myisamlog.c:
  Updated call to init_tree()
storage/myisam/myisampack.c:
  Updated call to init_tree()
storage/myisam/sort.c:
  Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
  Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
  Rest current_thd
storage/perfschema/pfs_instr.cc:
  Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
  Disable compiler warning from offsetof macro.
2013-01-23 16:16:14 +01:00
Venkatesh Duggirala
ce7e7a99a0 BUG#11753923-SQL THREAD CRASHES ON DISK FULL
Merging fix from mysql-5.1
2013-01-02 18:32:38 +05:30
Venkatesh Duggirala
3932392030 BUG#11753923-SQL THREAD CRASHES ON DISK FULL
Problem:If Disk becomes full while writing into the binlog,
then the server instance hangs till someone frees the space.
After user frees up the disk space, mysql server crashes
with an assert (m_status != DA_EMPTY)

Analysis: wait_for_free_space is being called in an
infinite loop i.e., server instance will hang until
someone frees up the space. So there is no need to
set status bit in diagnostic area.

Fix: Replace my_error/my_printf_error with
sql_print_warning() which prints the warning in error log.

include/my_sys.h:
  Provision to call sql_print_warning from mysys files
mysys/errors.c:
  Replace my_error/my_printf_error with
  sql_print_warning() which prints the warning in error log.
mysys/my_error.c:
  implementation of my_printf_warning
mysys/my_write.c:
  Adding logic to break infinite loop in the simulation
sql/mysqld.cc:
  Provision to call sql_print_warning from mysys files
2013-01-02 16:31:58 +05:30
unknown
61412c0c31 pre-merge 2012-12-23 23:37:11 +02:00
unknown
40ae63dd65 backport to 5.5 dyncol changes and names support 2012-12-23 20:57:54 +02:00
Ravinder Thakur
2d16c5bd4b bug#11761752: DO NOT ALLOW USE OF ALTERNATE DATA STREAMS ON NTFS FILESYSTEM.
File names with colon are being disallowed because of the Alternate Data 
Stream (ADS) feature of NTFS that could be misused. ADS allows data to be 
written to alternate streams of a normal file. The data in alternate 
streams cannot be seen by normal tools on Windows (explorer, cmd.exe). As 
a result someone can use this feature to hide large amount of data in 
alternate streams and admins will have no easy way of figuring out the 
files that are using that disk space. The fix also disallows ADS in the 
scenarios where file name is passed as some dynamic variable.

An important thing about the fix is that it DOES NOT disallow ADS file 
names if they are not dynamic (i.e. if the file is created by using some 
option that needs local access to the MySQL server, for example error log
file). The reasoning is that if some MySQL option related to files 
requires access to the local machine (it is not dynamic), then user can very 
well create data in ADS by some other means. This fixes only those scenarios 
which can allow users to create data in ADS over the wire.

File names with colon are being disallowed only on Windows. UNIX 
(Linux in particular) supports NTFS, but it will not be a common 
scenario for someone to configure a NTFS file system to store MySQL 
data on Linux.

Changes in file bug11761752-master.opt are needed due to 
bug number 15937938.
2012-12-13 20:33:44 +05:30
Ravinder Thakur
e604bb8a03 Merging from 5.1 to 5.5 for bug#11761752 2012-12-13 20:58:09 +05:30
Sergei Golubchik
e1f681c99b 10.0-base -> 10.0-monty 2012-10-19 20:38:59 +02:00
Michael Widenius
8ac1b41cf3 Made max_relay_log_size depending on master connection.
Changed names of multi-source log files so that original suffixes are kept.


include/my_sys.h:
  Added fn_ext2(), which returns pointer to last '.' in file name
mysql-test/extra/rpl_tests/rpl_max_relay_size.test:
  Updated test
mysql-test/suite/multi_source/info_logs-master.opt:
  Test with strange file names
mysql-test/suite/multi_source/info_logs.result:
  Updated results
mysql-test/suite/multi_source/info_logs.test:
  Changed to test with complex names to be able to verify the filename generator code
mysql-test/suite/multi_source/relaylog_events.result:
  Updated results
mysql-test/suite/multi_source/reset_slave.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.test:
  Added testing of max_relay_log_size
mysql-test/suite/rpl/r/rpl_row_max_relay_size.result:
  Updated results
mysql-test/suite/rpl/r/rpl_stm_max_relay_size.result:
  Updated results
mysql-test/suite/sys_vars/r/max_relay_log_size_basic.result:
  Updated results
mysql-test/suite/sys_vars/t/max_relay_log_size_basic.test:
  Updated results
mysys/mf_fn_ext.c:
  Added fn_ext2(), which returns pointer to last '.' in file name
sql/log.cc:
  Removed some wrong casts
sql/log.h:
  Updated comment to reflect new code
sql/log_event.cc:
  Updated DBUG_PRINT
sql/mysqld.cc:
  Added that max_relay_log_size copies it's values from max_binlog_size
sql/mysqld.h:
  Removed max_relay_log_size
sql/rpl_mi.cc:
  Changed names of multi-source log files so that original suffixes are kept.
sql/rpl_mi.h:
  Updated prototype
sql/rpl_rli.cc:
  Updated comment to reflect new code
  Made max_relay_log_size depending on master connection.
sql/rpl_rli.h:
  Made max_relay_log_size depending on master connection.
sql/set_var.h:
  Made option global so that one can check and change min & max values (sorry Sergei)
sql/sql_class.h:
  Made max_relay_log_size depending on master connection.
sql/sql_repl.cc:
  Updated calls to create_signed_file_name()
sql/sys_vars.cc:
  Made max_relay_log_size depending on master connection.
  Made old code more reusable
sql/sys_vars.h:
  Changed Sys_var_multi_source_uint to ulong to be able to handle max_relay_log_size
  Made old code more reusable
2012-10-01 02:30:44 +03:00
unknown
7327cd9717 MDEV-377 Name support for dynamic columns
MDEV-127 Optimization of memory allocation
MDEV-483 Make column_check function which cheсks dynamic columns integrit
JSON conversion function
2012-09-28 14:01:17 +03:00
Sergei Golubchik
474fe6d9d9 fixes for test failures
and small collateral changes

mysql-test/lib/My/Test.pm:
  somehow with "print" we get truncated writes sometimes
mysql-test/suite/perfschema/r/digest_table_full.result:
  md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/dml_handler.result:
  host table is not ported over yet
mysql-test/suite/perfschema/r/information_schema.result:
  host table is not ported over yet
mysql-test/suite/perfschema/r/nesting.result:
  this differs, because we don't rewrite general log queries, and multi-statement
  packets are logged as a one entry. this result file is identical to what mysql-5.6.5
  produces with the --log-raw option.
mysql-test/suite/perfschema/r/relaylog.result:
  MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB.
mysql-test/suite/perfschema/r/server_init.result:
  MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup.
mysql-test/suite/perfschema/r/stage_mdl_global.result:
  this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not
  do that, and this causes useless mutex locks and waits.
mysql-test/suite/perfschema/r/statement_digest.result:
  md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_consumers.result:
  md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_long_query.result:
  md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result:
  will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
  will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged
2012-09-27 20:09:46 +02:00
Michael Widenius
3e83c4e8f9 Automatic merge 2012-09-22 15:30:24 +03:00
Igor Babaev
bc644de9a9 Merge MDEV-415 -> 10.0-base. 2012-09-01 19:41:38 -07:00
Igor Babaev
a6b88f1431 MDEV-415: Back-port of the WL task #1393 from the mysql-5.6 code line.
The task adds a more efficient handling of the queries with 
ORDER BY order LIMIT n, such that n is small enough and
no indexes are used for order.
2012-09-01 14:21:59 -07:00
Michael Widenius
1999be8d4e Automatic merge with 5.5 2012-09-01 00:54:54 +03:00
Sergei Golubchik
59408093d1 5.3 merge 2012-08-31 14:15:52 +02:00
unknown
10802c4d90 MDEV-381: fdatasync() does not correctly flush growing binlog file.
When we append data to the binlog file, we use fdatasync() to ensure
the data gets to disk so that crash recovery can work.

Unfortunately there seems to be a bug in ext3/ext4 on linux, so that
fdatasync() does not correctly sync all data when the size of a file
is increased. This causes crash recovery to not work correctly (it
loses transactions from the binlog).

As a work-around, use fsync() for the binlog, not fdatasync(). Since
we are increasing the file size, (correct) fdatasync() will most
likely not be faster than fsync() on any file system, and fsync()
does work correctly on ext3/ext4. This avoids the need to try to
detect if we are running on buggy ext3/ext4.
2012-08-30 10:53:49 +02:00
unknown
ced3907c02 Merge from 5.3 2012-08-24 15:29:01 +02:00
unknown
fc666a0df6 merge from 5.2 2012-08-24 14:02:32 +02:00
unknown
96703a63da Merge from 5.1. 2012-08-24 12:32:46 +02:00
unknown
cdeabcfd43 MDEV-382: Incorrect quoting
Various places in the server replication code was incorrectly quoting
strings, which could lead to incorrect SQL on the slave/mysqlbinlog.
2012-08-24 10:06:16 +02:00
Michael Widenius
60589aeee0 Next part of merge. See TODO for details 2012-08-14 17:23:34 +03:00
Michael Widenius
1d0f70c2f8 Temporary commit of merge of MariaDB 10.0-base and MySQL 5.6 2012-08-01 17:27:34 +03:00
Michael Widenius
aa81e025a8 Added text for errno in error messages by:
- Adding %M my_sprintf() modifier that prints error number - system-error-text
- Modified mysys, mysql_client and SQL error messages to use %M instead of %d
- Added my_strerror()
Updated handler errors to 5.6 error numbers
Updated text for a few error messages (to match 5.6)
Increased length of command name in error output

extra/comp_err.c:
  Added support for %M
include/my_base.h:
  Updated handler errors to 5.6 error numbers
include/my_sys.h:
  Added my_strerror()
libmysql/errmsg.c:
  Updated error messages to use %M
mysql-test/r/errors.result:
  Updated result as error message have changed
mysql-test/r/innodb_mysql_sync.result:
  Updated result with text for errno
mysql-test/r/myisam-system.result:
  Updated result with text for errno
mysql-test/r/myisam.result:
  Updated result as error message have changed
mysql-test/r/myisampack.result:
  Updated result with text for errno
mysql-test/r/mysql.result:
  Updated result with text for errno
mysql-test/r/mysql_upgrade.result:
  Updated result with text for errno
mysql-test/r/partition_datatype.result:
  Updated result as error message have changed
mysql-test/r/partition_innodb_plugin.result:
  Updated result with text for errno
mysql-test/r/ps_1general.result:
  Updated result with text for errno
mysql-test/r/trigger.result:
  Updated result with text for errno
mysql-test/r/type_bit.result:
  Updated result as error message have changed
mysql-test/r/type_bit_innodb.result:
  Updated result as error message have changed
mysql-test/r/type_blob.result:
  Updated result as error message have changed
mysql-test/suite/archive/archive.result:
  Updated result with text for errno
mysql-test/suite/binlog/r/binlog_index.result:
  Updated result with text for errno
mysql-test/suite/binlog/r/binlog_ioerr.result:
  Updated result with text for errno
mysql-test/suite/csv/csv.result:
  Updated result with text for errno
mysql-test/suite/federated/federated_bug_35333.result:
  Updated result with text for errno
mysql-test/suite/innodb/r/innodb-create-options.result:
  Updated result with text for errno
mysql-test/suite/innodb/r/innodb-index.result:
  Updated result with text for errno
mysql-test/suite/innodb/r/innodb-zip.result:
  Updated result as error message have changed
mysql-test/suite/innodb/r/innodb.result:
  Updated result with text for errno
mysql-test/suite/innodb/r/innodb_bug21704.result:
  Updated result with text for errno
mysql-test/suite/innodb/r/innodb_bug46000.result:
  Updated result with text for errno
mysql-test/suite/innodb/r/innodb_bug53591.result:
  Updated result as error message have changed
mysql-test/suite/innodb/r/innodb_corrupt_bit.result:
  New error numbers
mysql-test/suite/innodb/r/innodb_prefix_index_liftedlimit.result:
  Updated result as error message have changed
mysql-test/suite/innodb/t/innodb-create-options.test:
  Added regexp to avoid system error text
mysql-test/suite/innodb/t/innodb-zip.test:
  Added regexp to avoid system error text
mysql-test/suite/maria/maria-recovery2.result:
  Updated supression rule
mysql-test/suite/maria/maria-recovery2.test:
  Updated supression rule
mysql-test/suite/maria/maria.result:
  Updated result as error message have changed
mysql-test/suite/parts/r/partition_bit_innodb.result:
  
  Updated result as error message have changed
mysql-test/suite/parts/r/partition_bit_myisam.result:
  
  Updated result as error message have changed
mysql-test/suite/percona/percona_innodb_fake_changes.result:
  Updated result with text for errno
mysql-test/suite/perfschema/r/dml_cond_instances.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_events_waits_current.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_events_waits_history.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_events_waits_history_long.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_ews_by_instance.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_ews_by_thread_by_event_name.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_ews_global_by_event_name.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_file_instances.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_file_summary_by_event_name.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_file_summary_by_instance.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_mutex_instances.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_performance_timers.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_rwlock_instances.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/dml_threads.result:
  Updated result as error message have changed
mysql-test/suite/perfschema/r/misc.result:
  Updated result with text for errno
mysql-test/suite/perfschema/r/privilege.result:
  Updated result with text for errno
mysql-test/suite/rpl/r/rpl_EE_err.result:
  Updated result with text for errno
mysql-test/suite/rpl/r/rpl_binlog_errors.result:
  Updated result with text for errno
mysql-test/suite/rpl/r/rpl_drop_db.result:
  Updated result with text for errno
mysys/errors.c:
  Updated error messages to use %M
  Changed all errors to use Errcode: consistenly
mysys/my_handler_errors.h:
  Updated handler errors to 5.6 error numbers
sql/share/errmsg-utf8.txt:
  Updated error messages to use %M
sql/sys_vars.cc:
  Added error number to ER_EVENT_SET_VAR_ERROR
strings/my_vsnprintf.c:
  Added %M my_sprintf() modifier that prints error number - system-error-text
  Simplify code
  Movied common code to function
  Removed some casts that was not necessary when reading integer/unsigned int stored in longlong
  Added my_strerror()
unittest/mysys/my_vsnprintf-t.c:
  Added testing of %M
2012-05-30 00:37:55 +03:00
Sergei Golubchik
0d5adca0de debug_sync is now a service, available to dynamically loaded plugins.
new make target - abi_update

libservices/HOWTO:
  remove references to Makefile.am
  small tweaks
2012-03-28 19:26:00 +02:00
Alexey Botchkov
07a82c58a7 MDEV-15 Log all SQL errors.
Added the logger service that provides us with the rotating logs.
              The plugin SQL_ERROR_LOG added. It logs the errors using the 'logger service'
                      for the rotating log files.
              the example record from the log:
                2012-03-09 15:07:29 root[root] @ localhost [] ERROR 1146: Table 'test.xyz' doesn't exist : select * from test.xyz
2012-03-14 00:55:56 +04:00
Vladislav Vaintroub
74cac79218 - Fix rpl_checksum test. Use basename of file in error messages, not the o
nes prefixed with .\ or ./
- Add my_basename() to mysys.
- Do not compile files that are not needed on Windows (my_addr_resolve, an
d safemalloc related stuff it it is not used)
Avoids linker warnings about compilation of essentially empty files.
2012-01-20 01:57:58 +01:00
Sergei Golubchik
6cc9d0ffa0 move safemalloc out of dbug.
remeber a real backtrace for every allocation.
make safemalloc to tract C++ new/delete too.
collateral fixes to make the test suite pass.
2011-12-12 22:58:24 +01:00
Sergei Golubchik
c753bedcb1 win64 sysvar portability fixes 2011-12-02 19:49:05 +01:00
Sergei Golubchik
dfc1901e80 compilation fixes
cmake/maintainer.cmake:
  don't do -Werror just yet
config.h.cmake:
  according to MSDN PSAPI_VERSION should be 1 in a portable application
mysys/my_thr_init.c:
  first, reset THR_KEY_mysys, and then free dbug data,
  because dbug data are automacially created on the next dbug call,
  unless THR_KEY_mysys is null.
2011-11-27 17:50:08 +01:00
Sergei Golubchik
d2755a2c9c 5.3->5.5 merge 2011-11-22 18:04:38 +01:00
Sergei Golubchik
76f0b94bb0 merge with 5.3
sql/sql_insert.cc:
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
  ******
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
  small cleanup
  ******
  small cleanup
2011-10-19 21:45:18 +02:00
Michael Widenius
3fb22ac5e1 Merge with 5.2 2011-09-10 18:01:27 +03:00
unknown
b152c4c71d Merge 5.2->5.3 2011-09-02 15:10:10 +03:00
Michael Widenius
13e4d54795 Fixed lp:814238 "safe_mutex issues must be assertions in debug binary"
Added --debug-assert-on-error variable which, if set, will cause safe_mutex to assert if it founds an error.

include/my_sys.h:
  Added my_assert_on_error
mysys/my_static.c:
  Added my_assert_on_error
mysys/thr_mutex.c:
  Assert when found wrong mutex usage if my_assert_on_error is set
sql/mysqld.cc:
  Added setting of my_assert_on_error
2011-09-02 12:41:20 +03:00
Michael Widenius
31c8c95bb2 Added logging of all errors from my_read/my_write/my_pread/my_pwrite/my_open & my_malloc to mysqld error log if one sets log-warning to 10 or 11
The idea is that my_global_flags is ored to the MyFlags parameter for the above functions if the MY_WME flag is not set.
As the my_global_flags has ME_JUST_INFO (mark error as 'note') and possible ME_NOREFRESH (write error to log) this will force mysqld to log the not critical error to the log as a note.


 

include/my_sys.h:
  Moved MY_SYNC_DIR to ensure it never clashes with ME_JUST_INFO
  Added my_global_flags
mysql-test/Makefile.am:
  Removed not used bugs directory
mysys/my_init.c:
  Added my_global_flags, a variable that is ored to MyFlags in a those mysys functions we want extra logging.
mysys/my_malloc.c:
  Added support for my_global_flags
mysys/my_open.c:
  Added support for my_global_flags
mysys/my_pread.c:
  Added support for my_global_flags
mysys/my_read.c:
  Added support for my_global_flags
mysys/my_static.c:
  Added my_global_flags
mysys/my_write.c:
  Added support for my_global_flags
sql/mysqld.cc:
  Set my_global_flags for warning levels 10 & 11
sql/sql_base.cc:
  Don't increment unhandled errors for notes or warnings.
2011-09-02 01:22:34 +03:00