Commit graph

3355 commits

Author SHA1 Message Date
Michael Widenius
5cf5a9a1e8 Fixed timing failure in myisam-metadata.test
mysql-test/include/wait_show_condition.inc:
  Print failing statement if timeout
mysql-test/r/myisam-metadata.result:
  Updated DBUG_SYNC
mysql-test/t/myisam-metadata.test:
  Updated DBUG_SYNC.
  Removed wait_show_condtion, as this is not needed when we use DBUG_SYNC
  This should fix timing issues with the test
mysys/thr_mutex.c:
  Added comments
sql/sql_acl.cc:
  atoi -> atoll()  (Safety)
storage/myisam/ha_myisam.cc:
  Send signal before mi_repair_by_sort.
2013-06-06 15:51:36 +03:00
Sergei Golubchik
f57ecb7786 5.5 merge 2013-04-14 10:04:07 +02:00
Sergei Golubchik
02c712aa54 * frm extra2 segment.
* persistent table versions in the extra2
* ha_archive::frm_compare using TABLE_SHARE::tabledef_version
* distinguish between "important" and "optional" extra2 frm values
* write engine-defined attributes (aka "table options") to extra2, not to extra,
  but still read from the old location, if they're found there.
2013-04-09 16:18:33 +02:00
Sergei Golubchik
e06cb31719 CREATE TABLE and frm-less discovering engines.
Now CREATE TABLE does not write the frm file on disk,
if the engine can discover it
2013-04-09 15:57:09 +02:00
Sergei Golubchik
bbbd7cedf5 my_dir() cleanup
* replace pointer acrobatics with a struct
* make sorting explicit: MY_DONT_SORT -> MY_WANT_SORT
(if you want something to be done - say it. fixes all places where
my_dir() was used without thinking)
* typo s/number_off_files/number_of_files/
* directory_file_name() doesn't need to be extern
* remove #ifdef __BORLANDC__
* ignore '.' and '..' entries
2013-04-07 15:19:45 +02:00
Sergei Golubchik
e56cad4b15 small cleanup 2013-04-07 14:03:43 +02:00
Sergei Golubchik
4ec6fe10e5 remove ULL() and LL(), because they're totally unnecessary
and sometimes harmful (used with expressions)
2013-04-07 14:00:16 +02:00
Sergei Golubchik
2901497b18 MDEV-4243 Warnings/errors while compiling with clang 2013-03-28 20:04:14 +01:00
Sergei Golubchik
993ea79f2d 5.5 merge 2013-03-27 23:41:02 +01:00
Michael Widenius
d8986fd6c3 Speed up connection time:
-Change my_rnd() slightly to make it safer if two threads use it at the same time.
-Avoid some sprintf and strmov in vio.
-Changed thread_count to be automaticly incremented (instead of under LOCK_thread_count).
-Thread cache now uses LOCK_thread_cache instead of LOCK_thread_count.
-Moved delete thd out from LOCK_thread_count.
-Save some mysql_cond_broadcast(&COND_thread_count) calls.
-Removed call to getsockname() during connect.
-Initialize random generator without locks.

Other things:
-Fixed test cases that depends on changes for LOCK_grant
-Added thread_safe_decrement32() and thread_safe_increment32()
-Removed sql_rnd_with_mutex() and get_thread_running()
-In check_table_access() don't lock LOCK_grant if we can resolve the grant with user or db level grants (the normal case).
-Don't use a lock for setting THD->query_id.
-Fixed bug where thd->set_query_id() could be set to same value by multiple threads.

Thanks to Yoshinori Matsunobu for the benchmark of connection speed and to
Domas Mituzas for the inspiration for many of the fixes.


include/violite.h:
  Change desc to a string pointer
mysql-test/suite/perfschema/r/all_instances.result:
  Added new mutex
mysql-test/suite/perfschema/t/func_mutex.test:
  Test for LOCK_system_variables_hash instead of LOCK_grant, as LOCK_grant is not anymore always taken for SELECT's.
mysys/my_gethwaddr.c:
  More DBUG
mysys/my_rnd.c:
  Change my_rnd() slightly to make it safer if two threads use it at the same time.
sql/event_scheduler.cc:
  Changed thread_count to be automically incremented
  Moved some safe things out from LOCK_thread_count.
  Simplify deleting of THD for running thread.
sql/mysqld.cc:
  Changed thread_count to be automically incremented
  Thread cache now uses LOCK_thread_cache instead of LOCK_thread_count
  Added delete_running_thd()
  Moved delete thd out from LOCK_thread_count
  More DBUG
  Only call  mysql_cond_broadcast(&COND_thread_count) if thread_count is 0
  Removed call to getsockname() (old not anymore needed check)
sql/mysqld.h:
  Removed sql_rnd_with_mutex() (not needed anymore)
  Removed not used function get_thread_running()
  Added thread_safe_decrement32() and thread_safe_increment32()
  Simplified dec_thread_running() and inc_thread_running()
  next_query_id() should return the original value for global_query_id, not the next one.
  (Bug introduced with MySQL 5.5 merge).
sql/sql_acl.cc:
  In check_table_access() don't lock LOCK_grant if we can resolve the grant with user or db level grants (the normal case).
sql/sql_class.cc:
  Removed thd_lock_thread_count() and thd_unlock_thread_count()
  Initialize random generator without locks
  Don't use a lock for setting THD->query_id.
  (This is only accessed by thread owning the THD)
sql/sql_class.h:
  Don't use a lock for setting THD->query_id.
sql/sql_insert.cc:
  Changed thread_count to be automically incremented
sql/sql_parse.cc:
  Changed thread_count to be automically incremented
  Fixed bug where thd->set_query_id() could be set to same value by multiple threads.
vio/vio.c:
  Don't generate 'desc' with sprintf/strmov. Assign a pointer instead.
  (Good enough as this is just for debugging)
2013-03-25 11:13:42 +02:00
Sergei Golubchik
213f1c76a0 5.3->5.5 merge 2013-02-28 22:47:29 +01:00
Sergei Golubchik
c4341d5095 5.2 -> 5.3 2013-02-28 21:48:47 +01:00
Sergei Golubchik
5dec570d7c 5.1 -> 5.2 merge 2013-02-28 19:00:58 +01:00
Sergei Golubchik
8161c6772d merge with mysql-5.5.30 minus few incorrect or not applicable changesets 2013-02-28 18:42:49 +01:00
Sergei Golubchik
08ba257846 mysql-5.1 merge
mysys/errors.c:
  revert upstream's fix. use a much simpler one
mysys/my_write.c:
  revert upstream's fix. use a simpler one
sql/item_xmlfunc.cc:
  useless, but ok
sql/mysqld.cc:
  simplify upstream's fix
storage/heap/hp_delete.c:
  remove upstream's fix.
  we'll use a much less expensive approach.
2013-02-28 09:58:39 +01:00
Vladislav Vaintroub
b3ad9de0c0 MDEV-4194: Fix typo (missing comma) in mysys error messages 2013-02-21 23:20:26 +01:00
Sergei Golubchik
330d3a7d1f missing cast added 2013-02-04 11:47:57 +01:00
Sergei Golubchik
0af4b6c6ee 5.5 merge 2013-01-29 15:10:47 +01:00
Sergei Golubchik
eb809ec061 32-bit fix: first cast the value to a signed type, then subtract 2013-01-23 16:23:50 +01:00
Sergei Golubchik
c71fad9983 cleanup:
* don't use 'myf flags', when 'my_bool is_thread_specific' is meant
* call set_malloc_size_cb() for embedded too
* warn in safemalloc if the memory is freed by a wrong thread

sql/mysqld.cc:
  move set_malloc_size_cb() to a function that is also called for embedded
sql/mysqld.h:
  gdb-friendly, one can put breakpoint on a function, but not on a macro
sql/sql_class.cc:
  initialize thread_id earlier
2013-01-23 16:20:39 +01:00
Sergei Golubchik
5205188200 cleanup:
* remove unused mysql_option
* don't allocate 5GB of memory in the mtr tests
* restore the behavior in dynamic_column_offset_byte(), put the ifdef correctly
* prefer __attribute__((unused)) to #ifdef
* prefer UNINIT_VAR to LINT_INIT
* make most Warning_info users blissfully unaware of the postponed initialization
* use my_offsetof instead of offsetof where the compiler thinks the latter is incorrect
2013-01-23 16:19:37 +01:00
Sergei Golubchik
e3f08e7e0b cleanup: remove unused init_dynamic_array and init_dynamic_array2 symbols,
as only my_init_dynamic_array and my_init_dynamic_array2 are used everywhere.
fix ha_cassandra to compile.
2013-01-23 16:18:53 +01:00
Sergei Golubchik
682da0aa75 cleanup: use MYF() for mysys flags 2013-01-23 16:18:09 +01:00
Michael Widenius
a260b15554 MDEV-4011 Added per thread memory counting and usage
Base code and idea from a patch from by plinux at Taobao.

The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.

Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.

The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array()  has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.

Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)


client/completion_hash.cc:
  Updated call to init_alloc_root()
client/mysql.cc:
  Updated call to init_alloc_root()
client/mysqlbinlog.cc:
  init_dynamic_array() -> my_init_dynamic_array()
  Updated call to init_alloc_root()
client/mysqlcheck.c:
  Updated call to my_init_dynamic_array()
client/mysqldump.c:
  Updated call to init_alloc_root()
client/mysqltest.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Fixed compiler warnings
extra/comp_err.c:
  Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
  Updated call to my_init_dynamic_array()
include/hash.h:
  Added HASH_THREAD_SPECIFIC
include/heap.h:
  Added flag is internal temporary table.
include/my_dir.h:
  Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
  Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
  Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
  Added malloc_flags to DYNAMIC_ARRAY
  Added extra mysys flag argument to my_init_dynamic_array()
  Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
  Updated paramaters for init_alloc_root()
include/my_tree.h:
  Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
  Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
  Updated parameters to init_tree()
include/myisamchk.h:
  Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
  Added MYSQL_THREAD_SPECIFIC_MALLOC
  Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
  Updated file
include/mysql_com.h:
  Used 'unused1' to mark memory as thread specific.
  Updated parameters for my_net_init()
libmysql/libmysql.c:
  Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
  Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
  Updated call to init_alloc_root()
mysql-test/r/create.result:
  Updated results
mysql-test/r/user_var.result:
  Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
  Updated results
mysql-test/t/show_explain.test:
  Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
  Added test of memory_usage variable.
mysys/array.c:
  Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
  All allocated memory is marked with the given my_flags.
  Removed obsolete function init_dynamic_array()
mysys/default.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
mysys/hash.c:
  Updated call to my_init_dynamic_array_ci().
  Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
  init_dynamic_array() -> my_init_dynamic_array()
  Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
  Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
  Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
  Extend MEM_ROOT with a flag if memory is thread specific.
  This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
  Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
  Store at start of each allocated memory block the size of the block and if the block is thread specific.
  Call malloc_size_cb_func, if set, with the memory allocated/freed.
  Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
  Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
  Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
  Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
  Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
  Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
  Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
  Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
  Added flags to init_tree() so that one can mark tree to be thread specific.
  Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
  Updated call to init_alloc_root()
  All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
  Updated call to my_init_dynamic_array()
sql-common/client.c:
  Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
  Updated call to my_init_dynamic_array().
  Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
  Updated call to init_alloc_root()
sql/debug_sync.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
  Removed calls to net_end() as this is now done in ~THD()
  Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
  Updated call to init_alloc_root()
  Updated call to my_net_init()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
  Updated call to my_net_init()
  Updated call to init_sql_alloc()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
  Updated call to init_alloc_root()
sql/handler.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added missing call to my_dir_end()
sql/item_func.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
  More DBUG
  Updated call to init_alloc_root()
sql/mdl.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
  Added total_memory_used
  Updated call to init_alloc_root()
  Move mysql_cond_broadcast() before my_thread_end()
  Added mariadb_dbug_id() to count memory per THD instead of per thread.
  Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
  Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
  Updated call to my_init_dynamic_array().
  Updated call to my_net_init().
  Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
  Added status variable 'memory_used'.
  Updated call to init_alloc_root()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
  Added set_current_thd()
sql/net_serv.cc:
  Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
  Store in net->thread_specific_malloc if memory is thread specific.
  Mark memory to be thread specific if requested.
sql/opt_range.cc:
  Updated call to my_init_dynamic_array()
  Updated call to init_sql_alloc()
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
  Fixed compiler warning
sql/records.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
  Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
  Updated call to init_sql_alloc()
sql/rpl_mi.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
  Updated call to init_alloc_root()
sql/rpl_utility.cc:
  Updated call to my_init_dynamic_array()
sql/slave.cc:
  Initialize things properly before calling functions that allocate memory.
  Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
  Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
  Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
  Updated call to my_init_dynamic_array()
sql/sql_base.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
  Updated comment
sql/sql_class.cc:
  Added parameter to warning_info() that not initialize it until THD is fully created.
  Updated call to init_sql_alloc()
  Mark THD::user_vars has to be thread specific.
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by THD is assigned to the THD.
  More DBUG
  Always acll net_end() in ~THD()
  Assert that all memory signed to this THD is really deleted at ~THD.
  Fixed set_status_var_init() to not reset memory_used.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
  Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
  This is used by THD::THD() to not allocate memory until THD is ready.
  Added Warning_info::free_memory()
sql/sql_error.h:
  Updated Warning_info() class.
sql/sql_handler.cc:
  Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
  More DBUG
sql/sql_join_cache.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
  Updated call to my_init_dynamic_array()
sql/sql_lex.h:
  Updated call to my_init_dynamic_array()
sql/sql_load.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
  Ensure that examined_row_count() is reset before query.
  Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
  Added parameter to warning_info() that it should be fully initialized.
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
  Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
  Added MY_THREAD_SPECIFIC to allocated memory.
  More DBUG
sql/sql_servers.cc:
  Updated call to init_sql_alloc() to mark memory some memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
  Updated call to my_init_dynamic_array()
  Mark my_dir() memory thread specific.
  Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
  More DBUG.
  Added malloc_size and examined_row_count to SHOW PROCESSLIST.
  Added MY_THREAD_SPECIFIC to allocated memory.
  Updated call to init_sql_alloc()
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
  Fixed compiler warning
sql/sql_string.cc:
  String elements can now be marked as thread specific.
sql/sql_string.h:
  String elements can now be marked as thread specific.
sql/sql_table.cc:
  Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Fixed compiler warning
sql/sql_test.cc:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
  Updated call to init_sql_alloc()
sql/sql_udf.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
  Updated call to init_sql_alloc().
  Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
  Updated call to init_sql_alloc()
sql/thr_malloc.cc:
  Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
  Updated prototype for init_sql_alloc()
sql/tztime.cc:
  Updated call to init_sql_alloc()
  Updated call to init_alloc_root() to mark memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
  Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
  Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
  Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
  - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
  - Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
  Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
  Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
  Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
  Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
  Updated call to init_tree()
storage/maria/ma_ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
  Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
  Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
  Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
  Updated call to init_tree()
storage/myisam/ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
  Updated call to init_tree()
storage/myisam/mi_check.c:
  Updated call to init_alloc_root()
storage/myisam/mi_write.c:
  Updated call to my_init_dynamic_array()
  Updated call to init_tree()
storage/myisam/myisamlog.c:
  Updated call to init_tree()
storage/myisam/myisampack.c:
  Updated call to init_tree()
storage/myisam/sort.c:
  Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
  Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
  Rest current_thd
storage/perfschema/pfs_instr.cc:
  Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
  Disable compiler warning from offsetof macro.
2013-01-23 16:16:14 +01:00
Sergei Golubchik
aca8e7ed6b 5.3 merge 2013-01-15 19:07:46 +01:00
unknown
936b6fab60 Compiler warning fixed. 2013-01-14 13:36:28 +02:00
unknown
bc75820fb7 Windows compiler warnings fix. 2013-01-11 14:12:59 +02:00
unknown
31efe8e0cb Fix windows compiler warnings. 2013-01-11 13:27:19 +02:00
unknown
4d6e5b2fed fixed crossplatform double values representation. 2013-01-10 19:47:07 +02:00
Sergei Golubchik
4f67a14700 5.2->5.3 merge 2013-01-10 15:40:21 +01:00
unknown
26b5351cac 32 bit systems warnings fixed. 2013-01-10 17:12:31 +02:00
Nirbhay Choubey
62d7006f0b Merge of patch for Bug#16066243 from mysql-5.1. 2013-01-07 16:19:06 +05:30
Nirbhay Choubey
e7c2ae94dd Bug#16066243 PB2 FAILURES I_MAIN.BUG15912213 AND
I_MAIN.CTYPE_UTF8 FOR MACOSX10.6 FOR 5.1

Part 2: Fix for test failures on Windows.
2013-01-07 16:16:08 +05:30
Nirbhay Choubey
31a720abbd Merge of patch for bug#16066243 from mysql-5.1. 2013-01-04 16:42:49 +05:30
Nirbhay Choubey
1ef420b8d4 Bug#16066243 PB2 FAILURES I_MAIN.BUG15912213 AND
I_MAIN.CTYPE_UTF8 FOR MACOSX10.6 FOR 5.1

While converting directory name to filename, a
file separator (FN_LIBCHAR) might get appended
to the resulting file name. This can result in
off-by-one error when length of the input string
is equal to FN_REFLEN. In this case, the terminating
'\0' gets written beyond the buffer allocated to store
the result.

Fixed by incrementing the dst buffer size by 1. As
extra safety, switched to strnmov() and added a debug
assert to check the length of the input file name.

No test case added as the scenario is already
covered by the test cases added for bugs in
the description.
2013-01-04 16:38:12 +05:30
Venkatesh Duggirala
ce7e7a99a0 BUG#11753923-SQL THREAD CRASHES ON DISK FULL
Merging fix from mysql-5.1
2013-01-02 18:32:38 +05:30
Venkatesh Duggirala
3932392030 BUG#11753923-SQL THREAD CRASHES ON DISK FULL
Problem:If Disk becomes full while writing into the binlog,
then the server instance hangs till someone frees the space.
After user frees up the disk space, mysql server crashes
with an assert (m_status != DA_EMPTY)

Analysis: wait_for_free_space is being called in an
infinite loop i.e., server instance will hang until
someone frees up the space. So there is no need to
set status bit in diagnostic area.

Fix: Replace my_error/my_printf_error with
sql_print_warning() which prints the warning in error log.

include/my_sys.h:
  Provision to call sql_print_warning from mysys files
mysys/errors.c:
  Replace my_error/my_printf_error with
  sql_print_warning() which prints the warning in error log.
mysys/my_error.c:
  implementation of my_printf_warning
mysys/my_write.c:
  Adding logic to break infinite loop in the simulation
sql/mysqld.cc:
  Provision to call sql_print_warning from mysys files
2013-01-02 16:31:58 +05:30
unknown
61412c0c31 pre-merge 2012-12-23 23:37:11 +02:00
unknown
082ff59317 Post-post review fixes. 2012-12-23 22:17:22 +02:00
unknown
40ae63dd65 backport to 5.5 dyncol changes and names support 2012-12-23 20:57:54 +02:00
Vladislav Vaintroub
77b4290b28 merge 2012-12-21 14:04:25 +01:00
Ahmad Abdullateef
b12fe50226 BUG#14727815 - CRASH IN PTHREAD_RWLOCK_WRLOCK/SRW_UNLOCK
IN QUERY CACHE CODE

DESCRIPTION:
MySQL Server crashes sporadically when Query Caching is on and
the server has high contention among clients. 


ANALYSIS :

Scenario 1:
In Query_cache::move_by_type() when handling RESULT or its related blocks,
Write Lock is acquired on its parent Query block. However the next and prev
pointers are cached in local variables before lock acquisition. In an extremely
high contention scenario there exists a possibility that
Query_cache::append_result_data() is operating on the same query block
and as a consequence might append a new Result block to the end of Result
blocks Linked List of the Query. This would manipulate the next, prev pointers
of the Block being processed in move_by_type(), however the local pointers
still point to previous nodes there by causing Data Corruption leading to crash.

Scenario 2:
In Windows SDK "BOOL" is typedefed as "int" and BOOLEAN is typedefed as
"usigned char". The function pointer definition "srw_bool_func" mistakenly uses 
BOOL instead of BOOLEAN thereby virtually making the function 
my_TryAcquireSRWLockExclusive() always succeed because only the LSB of EAX
has the actual result of the call, however due to type mismatch all bytes of EAX
are used for evaluation. Again during high contention scenarios in 
Query_cache::free_old_query() calls try_lock_writing() on a Query, this call 
always succeeds and the query is freed, even though it is used by some other
thread, in this case Query_cache::send_result_to_client() was using it and the
code causes a crash because it accessed free or reallocated memory.

FIX :

Scenario 1:
The next, prev pointers are now accessed only after Lock acquisition in 
Query_cache::move_by_type().

Scenario 2:
In the definition of "srw_bool_func" BOOL has been replaced with "BOOLEAN"
2012-12-18 22:16:12 +05:30
Igor Babaev
7760efad74 Merge mariadb-5.5 -> 10.0-base. 2012-12-16 16:49:19 -08:00
Ravinder Thakur
2d16c5bd4b bug#11761752: DO NOT ALLOW USE OF ALTERNATE DATA STREAMS ON NTFS FILESYSTEM.
File names with colon are being disallowed because of the Alternate Data 
Stream (ADS) feature of NTFS that could be misused. ADS allows data to be 
written to alternate streams of a normal file. The data in alternate 
streams cannot be seen by normal tools on Windows (explorer, cmd.exe). As 
a result someone can use this feature to hide large amount of data in 
alternate streams and admins will have no easy way of figuring out the 
files that are using that disk space. The fix also disallows ADS in the 
scenarios where file name is passed as some dynamic variable.

An important thing about the fix is that it DOES NOT disallow ADS file 
names if they are not dynamic (i.e. if the file is created by using some 
option that needs local access to the MySQL server, for example error log
file). The reasoning is that if some MySQL option related to files 
requires access to the local machine (it is not dynamic), then user can very 
well create data in ADS by some other means. This fixes only those scenarios 
which can allow users to create data in ADS over the wire.

File names with colon are being disallowed only on Windows. UNIX 
(Linux in particular) supports NTFS, but it will not be a common 
scenario for someone to configure a NTFS file system to store MySQL 
data on Linux.

Changes in file bug11761752-master.opt are needed due to 
bug number 15937938.
2012-12-13 20:33:44 +05:30
Ravinder Thakur
e604bb8a03 Merging from 5.1 to 5.5 for bug#11761752 2012-12-13 20:58:09 +05:30
Vladislav Vaintroub
c4b35f9279 MDEV-3918: myisamchk bogus error for files larger than 4GB.
The failure is caused by failing stat() call . C Runtime function stat() uses old struct with 32bit st_size member,
and since Visual Studio 2010 , it returns an error on st_size overflow (i.e on files larger than 4GB)

Fix replaces stat() by my_stat(), the later is backed by 64bit-able stat64().
2012-12-06 00:37:06 +01:00
Sergei Golubchik
a48a91d90f 5.3->5.5 merge 2012-11-22 10:19:31 +01:00
unknown
cbae294396 Merge MariaDB 5.1->5.2. 2012-11-20 13:40:13 +01:00
unknown
ecf04668a2 Merge MariaDB 5.2->5.3 2012-11-20 13:57:49 +01:00
Vladislav Vaintroub
353130204d MDEV-3849 - 1 bytes stack overwrite in normalize_dirname().
Take into account that length of strings passed down to this function can be up to FN_REFLEN+1 bytes. including terminating zero.
The overwrite was caused by incomplete fix to MySQL Bug # 44834
2012-11-10 20:36:18 +01:00