Pre-MDL versions had direct relationship between LOCK_open and
LOCK_global_system_variables, e.g.:
intern_sys_var_ptr // locks LOCK_global_system_variable
mysql_sys_var_char
create_options_are_valid
ha_innobase::create
handler::ha_create
ha_create_table
rea_create_table
mysql_create_table_no_lock // locks LOCK_open
mysql_create_table
With MDL this relationship was removed, but mutex order was still
recorded. In fact there is indirect relationship between LOCK_open
and LOCK_global_system_variables via rwlocks in reverse order.
Removed LOCK_open and LOCK_global_system_variables order recording,
instead assert that LOCK_open is never held in intern_sys_var_ptr().
This solves only one of many problems detected with MDEV-5089.
INSTALL PLUGIN
There was mixed lock order between LOCK_plugin, LOCK_global_system_variables
and LOCK_system_variables_hash. This patch ensures that write-lock on
LOCK_system_variables_hash doesn't intersect with LOCK_plugin.
Fixed by moving initialization/deinitialization of plugin options from
plugin_add()/plugin_del() to plugin_initialize()/plugin_deinitalize().
So that plugin options are handled without protection of LOCK_plugin.
When a DSO is loaded we rewrite service pointers to point to the actual service structures.
But when a DSO is unloaded, we have to restore their original values, in case this DSO
wasn't removed from memory on dlclose() and is later loaded again.
* add TokuDB, together with the ft-index library
* cmake support, auto-detecting whether tokudb can be built
* fix packaging - tokudb-engine.rpm, deb
* remove PBXT
* add jemalloc
* the server is built with jemalloc by default even if TokuDB is not built
* documentation files in RPM are installed in the correct location
* support for optional deb packages (tokudb has specific build requirements)
* move plugins from mariadb-server deb to appropriate debs (server/test/libmariadbclient)
* correct mariadb-test.deb to be not architecture-independent
* fix out-of-tree builds to never modify in-tree files
* new handler::prepare_index_scan() method
file_logger became the service.
Data like query_id now are sent to the audit plugin.
Fix for MDEV-4770 ported from 10.0.
Fix added for the read_maria_plugin_info().
Log rotation can be disabled with 'set rotations=0'.
innodb_ft_user_stopword_table to DEFAULT
Setting plugin string variable with PLUGIN_VAR_MEMALLOC flag
to NULL causes server crash.
mysql-test/suite/sys_vars/r/innodb_ft_user_stopword_table_basic.result:
Reset innodb_ft_user_stopword_table. Also tests MDEV-4801.
mysql-test/suite/sys_vars/t/innodb_ft_user_stopword_table_basic.test:
Reset innodb_ft_user_stopword_table. Also tests MDEV-4801.
sql/sql_plugin.cc:
When we got NULL value, do not strdup(NULL).
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
(it worked fine with the modern gcc, but failed on some other compilers).
modified:
include/my_global.h
include/mysql/plugin.h
include/mysql/plugin_audit.h.pp
include/mysql/plugin_auth.h.pp
include/mysql/plugin_ftparser.h.pp
include/mysql/service_thd_timezone.h
sql/item_func.cc
sql/mysqld.cc
sql/set_var.cc
sql/sql_plugin.cc
sql/sql_plugin.h
sql/sql_show.cc
sql/sys_vars.cc
storage/perfschema/ha_perfschema.cc
USING THE PLUGIN INTERFACE.
ISSUE: No support for floating-point plugin
system variables.
SOLUTION: Allowing plugins to define and expose floating-point
system variables of type double. MYSQL_SYSVAR_DOUBLE
and MYSQL_THDVAR_DOUBLE are added.
ISSUE: Fractional part of the def, min, max values of system
variables are ignored.
SOLUTION: Adding functions that are used to store the raw
representation of a double in the raw bits of unsigned
longlong in a way that the binary representation
remains the same.
Base code and idea from a patch from by plinux at Taobao.
The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.
Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.
The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.
Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)
client/completion_hash.cc:
Updated call to init_alloc_root()
client/mysql.cc:
Updated call to init_alloc_root()
client/mysqlbinlog.cc:
init_dynamic_array() -> my_init_dynamic_array()
Updated call to init_alloc_root()
client/mysqlcheck.c:
Updated call to my_init_dynamic_array()
client/mysqldump.c:
Updated call to init_alloc_root()
client/mysqltest.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Fixed compiler warnings
extra/comp_err.c:
Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
Updated call to my_init_dynamic_array()
include/hash.h:
Added HASH_THREAD_SPECIFIC
include/heap.h:
Added flag is internal temporary table.
include/my_dir.h:
Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
Added malloc_flags to DYNAMIC_ARRAY
Added extra mysys flag argument to my_init_dynamic_array()
Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
Updated paramaters for init_alloc_root()
include/my_tree.h:
Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
Updated parameters to init_tree()
include/myisamchk.h:
Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
Added MYSQL_THREAD_SPECIFIC_MALLOC
Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
Updated file
include/mysql_com.h:
Used 'unused1' to mark memory as thread specific.
Updated parameters for my_net_init()
libmysql/libmysql.c:
Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
Updated call to init_alloc_root()
mysql-test/r/create.result:
Updated results
mysql-test/r/user_var.result:
Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
Updated results
mysql-test/t/show_explain.test:
Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
Added test of memory_usage variable.
mysys/array.c:
Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
All allocated memory is marked with the given my_flags.
Removed obsolete function init_dynamic_array()
mysys/default.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
mysys/hash.c:
Updated call to my_init_dynamic_array_ci().
Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
init_dynamic_array() -> my_init_dynamic_array()
Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
Extend MEM_ROOT with a flag if memory is thread specific.
This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
Store at start of each allocated memory block the size of the block and if the block is thread specific.
Call malloc_size_cb_func, if set, with the memory allocated/freed.
Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
Added flags to init_tree() so that one can mark tree to be thread specific.
Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
Updated call to init_alloc_root()
All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
Updated call to my_init_dynamic_array()
sql-common/client.c:
Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
Updated call to my_init_dynamic_array().
Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
Updated call to init_alloc_root()
sql/debug_sync.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
Removed calls to net_end() as this is now done in ~THD()
Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
Updated call to init_alloc_root()
Updated call to my_net_init()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
Updated call to my_net_init()
Updated call to init_sql_alloc()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
Updated call to init_alloc_root()
sql/handler.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
Added missing call to my_dir_end()
sql/item_func.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
More DBUG
Updated call to init_alloc_root()
sql/mdl.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
Added total_memory_used
Updated call to init_alloc_root()
Move mysql_cond_broadcast() before my_thread_end()
Added mariadb_dbug_id() to count memory per THD instead of per thread.
Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
Updated call to my_init_dynamic_array().
Updated call to my_net_init().
Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
Added status variable 'memory_used'.
Updated call to init_alloc_root()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
Added set_current_thd()
sql/net_serv.cc:
Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
Store in net->thread_specific_malloc if memory is thread specific.
Mark memory to be thread specific if requested.
sql/opt_range.cc:
Updated call to my_init_dynamic_array()
Updated call to init_sql_alloc()
Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
Fixed compiler warning
sql/records.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
Updated call to init_sql_alloc()
sql/rpl_mi.cc:
Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
Updated call to init_alloc_root()
sql/rpl_utility.cc:
Updated call to my_init_dynamic_array()
sql/slave.cc:
Initialize things properly before calling functions that allocate memory.
Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
Updated call to my_init_dynamic_array()
sql/sql_base.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
Updated comment
sql/sql_class.cc:
Added parameter to warning_info() that not initialize it until THD is fully created.
Updated call to init_sql_alloc()
Mark THD::user_vars has to be thread specific.
Updated call to my_init_dynamic_array()
Ensure that memory allocated by THD is assigned to the THD.
More DBUG
Always acll net_end() in ~THD()
Assert that all memory signed to this THD is really deleted at ~THD.
Fixed set_status_var_init() to not reset memory_used.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
Added MY_THREAD_SPECIFIC to allocated memory.
Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
This is used by THD::THD() to not allocate memory until THD is ready.
Added Warning_info::free_memory()
sql/sql_error.h:
Updated Warning_info() class.
sql/sql_handler.cc:
Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
More DBUG
sql/sql_join_cache.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
Updated call to my_init_dynamic_array()
sql/sql_lex.h:
Updated call to my_init_dynamic_array()
sql/sql_load.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
Ensure that examined_row_count() is reset before query.
Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
Added parameter to warning_info() that it should be fully initialized.
Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
Added MY_THREAD_SPECIFIC to allocated memory.
More DBUG
sql/sql_servers.cc:
Updated call to init_sql_alloc() to mark memory some memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
Updated call to my_init_dynamic_array()
Mark my_dir() memory thread specific.
Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
More DBUG.
Added malloc_size and examined_row_count to SHOW PROCESSLIST.
Added MY_THREAD_SPECIFIC to allocated memory.
Updated call to init_sql_alloc()
Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
Fixed compiler warning
sql/sql_string.cc:
String elements can now be marked as thread specific.
sql/sql_string.h:
String elements can now be marked as thread specific.
sql/sql_table.cc:
Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Fixed compiler warning
sql/sql_test.cc:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
Updated call to init_sql_alloc()
sql/sql_udf.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
Updated call to init_sql_alloc().
Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
Updated call to init_sql_alloc()
sql/thr_malloc.cc:
Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
Updated prototype for init_sql_alloc()
sql/tztime.cc:
Updated call to init_sql_alloc()
Updated call to init_alloc_root() to mark memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
Updated call to init_tree()
storage/maria/ma_ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
Updated call to init_tree()
storage/myisam/ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
Updated call to init_tree()
storage/myisam/mi_check.c:
Updated call to init_alloc_root()
storage/myisam/mi_write.c:
Updated call to my_init_dynamic_array()
Updated call to init_tree()
storage/myisam/myisamlog.c:
Updated call to init_tree()
storage/myisam/myisampack.c:
Updated call to init_tree()
storage/myisam/sort.c:
Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
Rest current_thd
storage/perfschema/pfs_instr.cc:
Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
Disable compiler warning from offsetof macro.
revno: 3383
revision-id: georgi.kodinov@oracle.com-20110818083108-qa3h3ufqu4zne80a
committer: Georgi Kodinov <Georgi.Kodinov@Oracle.com>
timestamp: Thu 2011-08-18 11:31:08 +0300
message:
Bug #11766001: 59026: ALLOW MULTIPLE --PLUGIN-LOAD OPTIONS
Implemented support for a new command line option :
--plugin-load-add=<comma-separated-name-equals-value-list>
This option takes the same type of arguments that --plugin-load does
and complements --plugin-load (that continues to operate as before) by
appending its argument to the list specified by --plugin-load.
So --plugin-load can be considered a composite option consisting of
resetting the plugin load list and then calling --plugin-load-add to process
the argument.
Note that the order in which you specify --plugin-load and --plugin-load-add
is important : "--plugin-load=x --plugin-load-add=y" will be equivalent to
"--plugin-load=x,y" whereas "--plugin-load-add=y --plugin-load=x" will be
equivalent to "plugin-load=x".
Incompatible change : the --help --verbose command will no longer print the
--plugin-load variable's values (as it doesn't have one). Otherwise both --plugin-load
and --plugin-load-add are mentioned in it.
DISABLE AND ENABLED DURING DDL OPERATION
PROBLEM: Same thread trying to acquire the same mutex
second time leads to hang/server crash.
While [un]installing audit_log plugin
a thread acquires the LOCK_plugin mutex
and after successful initialization tries
to write in mysql.plugin table. It holds
this mutex for a long time. If some how
plugin table is corrupted then a write to
plugin table will throw an error, thread try
to log this error in the audit_log plugin,
doing so it tries to acquire the mutex
again and results is server hang/crash.
SOLUTION: Releasing the LOCK_plugin mutex before
writing in mysql.plugin table. We dont
need to hold this mutex as thread already
acquired a TL_WRITE lock on mysql.plugin
table.
ENABLE AUDI PLUGIN WHEN DDL
OPERATION HAPPENING
PROBLEM: While unloading the plugin, state is
not checked before it is to be reaped.
This can lead to simultaneous free of
plugin memory by more than one thread.
Multiple deallocation leads to server
crash. In the present bug two threads
deallocate the alog_log plugin.
SOLUTION: A check is added to ensure that only
one thread is unloading the plugin.
NOTE: No mtr test is added as it requires
multiple threads to access critical
section. debug_sync cannot be used in
the current senario because we dont
have access to thread pointer in
some of the plugin functions. IMHO no
test case in the current time frame.
NUMBERS
If a system variable was declared as deprecated without mention of an
alternative, the message would look funny, e.g. for @@delayed_insert_limit:
Warning 1287 '@@delayed_insert_limit' is deprecated and
will be removed in MySQL .
The message was meant to display the version number, but it's not
possible to give one when declaring a system variable.
The fix does two things:
1) The definition of the message
ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT is changed so that it does
not display a version number. I.e. in English the message now reads:
Warning 1287 The syntax '@@delayed_insert_limit' is deprecated and
will be removed in a future version.
2) The message ER_WARN_DEPRECATED_SYNTAX_WITH_VER is discontinued in
favor of ER_WARN_DEPRECATED_SYNTAX for system variables. This change
was already done in versions 5.6 and above as part of wl#5265. This
part is simply back-ported from the worklog.
client/mysqldump.c:
Added LINT_INIT
mysql-test/mysql-test-run.pl:
Disable warning if example engine is not found
mysql-test/suite/rpl/t/rpl_semi_sync.test:
Rpl_semi_sync_master_yes_tx may be different on Windows
sql/sql_plugin.cc:
More DBUG_PRINT
support-files/compiler_warnings.supp:
Disable some innobase warnings
unittest/mysys/my_vsnprintf-t.c:
Fixed test failure on Solaris (typo)
Fixed some mtr test problems
dbug/tests.c:
Fixed compiler warnings
mysql-test/r/handlersocket.result:
Fixed that plugin_license is written
mysql-test/suite/innodb/t/innodb_bug60196.test:
Force sorted results as it was sometimes different on windows
mysql-test/suite/rpl/t/rpl_heartbeat_basic.test:
Prolong test as this failed on windows
mysql-test/t/handlersocket.test:
Fixed that plugin_license is written
plugin/handler_socket/handlersocket/handlersocket.cpp:
Use maria_declare_plugin
plugin/handler_socket/handlersocket/mysql_incl.hpp:
Fixed compiler warning
plugin/handler_socket/libhsclient/auto_addrinfo.hpp:
Fixed compiler warning
sql/handler.h:
Fixed typo
sql/sql_plugin.cc:
Fixed bug that caused plugin library name twice in error message
storage/maria/ma_checkpoint.c:
Fixed compiler warning
storage/maria/ma_loghandler.c:
Fixed compiler warning
unittest/mysys/base64-t.c:
Fixed compiler warning
unittest/mysys/bitmap-t.c:
Fixed compiler warning
unittest/mysys/my_malloc-t.c:
Fixed compiler warning
Fixed failing tests in sys_vars as we have now stricter checking of setting of variables.
mysql-test/suite/sys_vars/r/innodb_adaptive_flushing_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_adaptive_hash_index_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_large_prefix_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_random_read_ahead_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_stats_on_metadata_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_strict_mode_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_support_xa_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_table_locks_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/rpl_semi_sync_master_enabled_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/rpl_semi_sync_slave_enabled_basic.result:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_adaptive_flushing_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_adaptive_hash_index_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_large_prefix_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_random_read_ahead_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_stats_on_metadata_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_strict_mode_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_support_xa_basic.test:
One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_table_locks_basic.test:
One can now only assign 0 or 1 to boolean variables
mysys/my_getsystime.c:
Merge + fixed bug that __NR_clock_gettime didn't work in 5.5
Thanks to Maarten Vanraes for the patch
sql/sql_plugin.cc:
Fix plugin boolean variables to receive the value "1", not "-1", when they are set to 1.
Aside from being bizarre, the existing behavior is unportable: machines where char is unsigned print "255" instead.
mysql-test/suite/innodb/t/group_commit_crash.test:
remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
my_vsnprintf() is ok here, in 5.5
and cryptic error 1126 message
The problem was that dlopen() related code was using just a subset
of the path normalization routines used in other places.
Fixed the expansion of the pre-dlopen() behavior for plugins and UDFs
to use a platform-dependent consistent encoding of the paths.
Fixed the error dlopen() error handling to take the correct error message
and strip off the trailing newline character(s).
Fixed tests to do a platform independent replace of directories and to
account for the traling slash.
ALTER TABLE AND/OR PLUGIN/SEMISYNC
If a plugin was uninstalled, thread local values for plugin
variables of string type with PLUGIN_VAR_MEMALLOC flag were
not freed.
With this patch these variables are freed when thread is
done (like all other variables).
sql/sql_class.h:
Added variable which stores memory hunks allocated
for PLUGIN_VAR_MEMALLOC values.
sql/sql_plugin.cc:
Normally all memory allocated for dynamic variables values must
be freed by cleanup_variables().
But if a plugin was uninstalled, descriptors of it's system
variables are lost. Still some memory may be occupied for thread
local values. It is ok for most kinds of variables, as they're
stored on dynamic_variables_ptr and freed when thread is done.
Values for PLUGIN_VAR_MEMALLOC variables are stored separately.
These lost values are handled by plugin_var_memalloc_free().
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
bugfix: garbage in PLUGIN_VAR_STR variables when INSTALL'ing a plugin
mysql-test/include/default_mysqld.cnf:
disable feedback plugin by default.
when enabled - tag is as a test run
client/mysql_upgrade.c:
missing DBUG_RETURN
client/mysqladmin.cc:
client plugin memory wasn't freed
client/mysqlcheck.c:
client plugin memory, defaults, a result set, a command-line option value were not freed.
missing DBUG_RETURN.
client/mysqldump.c:
client plugin memory wasn't freed
client/mysqlslap.c:
client plugin memory wasn't freed
client/mysqltest.cc:
hopeless. cannot be fixed.
mysql-test/valgrind.supp:
Bug#56666 is now fixed.
mysys/array.c:
really, don't allocate if the caller didn't ask to.
mysys/my_init.c:
safemalloc checks must be done at the very end
mysys/my_thr_init.c:
not needed anymore
sql-common/client.c:
memory leak
sql/log.cc:
log_file was not closed, memory leak.
sql/mysqld.cc:
fix bug#56666 (causing many P_S related memory leaks).
close_active_mi() not called for --bootstrap, memory leak.
sql/sql_lex.cc:
redo Lex->mi handling
sql/sql_lex.h:
redo Lex->mi handling
sql/sql_plugin.cc:
plugins having PLUGIN_VAR_MEMALLOC string variables have this variables allocated in every THD.
The memory was freed in ~THD but only if plugin was still active. If plugin was unloaded the
variable was not found and the memory was lost. By loading and unloading plugins an arbitrary
amount of memory can be lost.
sql/sql_repl.cc:
redo Lex->mi handling
sql/sql_yacc.yy:
completely wrong handling of Lex->mi - run-time memory leak, by repeating the statement
arbitrary amount of memory can be lost.
Lex->mi.repl_ignore_server_ids_opt was allocated when parsing CHANGE MASTER,
and freed after executing the statement. if parser failed on syntax (or another)
error the statement was never executed. Lex->mi was simply bzero-ed for the next
CHANGE MASTER statement.
sql/table.cc:
didn't compile
storage/perfschema/pfs_lock.h:
Bug#56666 is fixed
sql/mysql_priv.h:
Make opt_help global
sql/mysqld.cc:
Don't give (double) warnings about lower case file systems if --help is given.
Don't give warning about non existing data directory if --help is given.
sql/sql_plugin.cc:
Give a better warning if --help is used and plugin table doesn't exists
In sql_class.cc, 'row_count', of type 'ha_rows', was used as last argument for
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD which is
"Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld".
So 'ha_rows' was used as 'long'.
On SPARC32 Solaris builds, 'long' is 4 bytes and 'ha_rows' is 'longlong' i.e. 8 bytes.
So the printf-like code was reading only the first 4 bytes.
Because the CPU is big-endian, 1LL is 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01
so the first four bytes yield 0. So the warning message had "row 0" instead of
"row 1" in test outfile_loaddata.test:
-Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1
+Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 0
All error-messaging functions which internally invoke some printf-life function
are potential candidate for such mistakes.
One apparently easy way to catch such mistakes is to use
ATTRIBUTE_FORMAT (from my_attribute.h).
But this works only when call site has both:
a) the format as a string literal
b) the types of arguments.
So:
func(ER(ER_BLAH), 10);
will silently not be checked, because ER(ER_BLAH) is not known at
compile time (it is known at run-time, and depends on the chosen
language).
And
func("%s", a va_list argument);
has the same problem, as the *real* type of arguments is not
known at this site at compile time (it's known in some caller).
Moreover,
func(ER(ER_BLAH));
though possibly correct (if ER(ER_BLAH) has no '%' markers), will not
compile (gcc says "error: format not a string literal and no format
arguments").
Consequences:
1) ATTRIBUTE_FORMAT is here added only to functions which in practice
take "string literal" formats: "my_error_reporter" and "print_admin_msg".
2) it cannot be added to the other functions: my_error(),
push_warning_printf(), Table_check_intact::report_error(),
general_log_print().
To do a one-time check of functions listed in (2), the following
"static code analysis" has been done:
1) replace
my_error(ER_xxx, arguments for substitution in format)
with the equivalent
my_printf_error(ER_xxx,ER(ER_xxx), arguments for substitution in
format),
so that we have ER(ER_xxx) and the arguments *in the same call site*
2) add ATTRIBUTE_FORMAT to push_warning_printf(),
Table_check_intact::report_error(), general_log_print()
3) replace ER(xxx) with the hard-coded English text found in
errmsg.txt (like: ER(ER_UNKNOWN_ERROR) is replaced with
"Unknown error"), so that a call site has the format as string literal
4) this way, ATTRIBUTE_FORMAT can effectively do its job
5) compile, fix errors detected by ATTRIBUTE_FORMAT
6) revert steps 1-2-3.
The present patch has no compiler error when submitted again to the
static code analysis above.
It cannot catch all problems though: see Field::set_warning(), in
which a call to push_warning_printf() has a variable error
(thus, not replacable by a string literal); I checked set_warning() calls
by hand though.
See also WL 5883 for one proposal to avoid such bugs from appearing
again in the future.
The issues fixed in the patch are:
a) mismatch in types (like 'int' passed to '%ld')
b) more arguments passed than specified in the format.
This patch resolves mismatches by changing the type/number of arguments,
not by changing error messages of sql/share/errmsg.txt. The latter would be wrong,
per the following old rule: errmsg.txt must be as stable as possible; no insertions
or deletions of messages, no changes of type or number of printf-like format specifiers,
are allowed, as long as the change impacts a message already released in a GA version.
If this rule is not followed:
- Connectors, which use error message numbers, will be confused (by insertions/deletions
of messages)
- using errmsg.sys of MySQL 5.1.n with mysqld of MySQL 5.1.(n+1)
could produce wrong messages or crash; such usage can easily happen if
installing 5.1.(n+1) while /etc/my.cnf still has --language=/path/to/5.1.n/xxx;
or if copying mysqld from 5.1.(n+1) into a 5.1.n installation.
When fixing b), I have verified that the superfluous arguments were not used in the format
in the first 5.1 GA (5.1.30 'bteam@astra04-20081114162938-z8mctjp6st27uobm').
Had they been used, then passing them today, even if the message doesn't use them
anymore, would have been necessary, as explained above.
include/my_getopt.h:
this function pointer is used only with "string literal" formats, so we can add
ATTRIBUTE_FORMAT.
mysql-test/collections/default.experimental:
test should pass now
sql/derror.cc:
by having a format as string literal, ATTRIBUTE_FORMAT check becomes effective.
sql/events.cc:
Change justified by the following excerpt from sql/share/errmsg.txt:
ER_EVENT_SAME_NAME
eng "Same old and new event name"
ER_EVENT_SET_VAR_ERROR
eng "Error during starting/stopping of the scheduler. Error code %u"
sql/field.cc:
ER_TOO_BIG_SCALE 42000 S1009
eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu."
ER_TOO_BIG_PRECISION 42000 S1009
eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu."
ER_TOO_BIG_DISPLAYWIDTH 42000 S1009
eng "Display width out of range for column '%-.192s' (max = %lu)"
sql/ha_ndbcluster.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
(sizeof() returns size_t)
sql/ha_ndbcluster_binlog.cc:
Too many arguments for:
ER_GET_ERRMSG
eng "Got error %d '%-.100s' from %s"
Patch by Jonas Oreland.
sql/ha_partition.cc:
print_admin_msg() is used only with a literal as format, so ATTRIBUTE_FORMAT
works.
sql/handler.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
(sizeof() returns size_t)
sql/item_create.cc:
ER_TOO_BIG_SCALE 42000 S1009
eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu."
ER_TOO_BIG_PRECISION 42000 S1009
eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu."
'c_len' and 'c_dec' are char*, passed as %d !! We don't know their value
(as strtoul() failed), but they are likely big, so we use INT_MAX.
'len' is ulong.
sql/item_func.cc:
ER_WARN_DATA_OUT_OF_RANGE 22003
eng "Out of range value for column '%s' at row %ld"
ER_CANT_FIND_UDF
eng "Can't load function '%-.192s'"
sql/item_strfunc.cc:
ER_TOO_BIG_FOR_UNCOMPRESS
eng "Uncompressed data size too large; the maximum size is %d (probably, length of uncompressed data was corrupted)"
max_allowed_packet is ulong.
sql/mysql_priv.h:
sql_print_message_func is a function _pointer_.
sql/sp_head.cc:
ER_SP_RECURSION_LIMIT
eng "Recursive limit %d (as set by the max_sp_recursion_depth variable) was exceeded for routine %.192s"
max_sp_recursion_depth is ulong
sql/sql_acl.cc:
ER_PASSWORD_NO_MATCH 42000
eng "Can't find any matching row in the user table"
ER_CANT_CREATE_USER_WITH_GRANT 42000
eng "You are not allowed to create a user with GRANT"
sql/sql_base.cc:
ER_NOT_KEYFILE
eng "Incorrect key file for table '%-.200s'; try to repair it"
ER_TOO_MANY_TABLES
eng "Too many tables; MySQL can only use %d tables in a join"
MAX_TABLES is size_t.
sql/sql_binlog.cc:
ER_UNKNOWN_ERROR
eng "Unknown error"
sql/sql_class.cc:
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD
eng "Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld"
WARN_DATA_TRUNCATED 01000
eng "Data truncated for column '%s' at row %ld"
sql/sql_connect.cc:
ER_HANDSHAKE_ERROR 08S01
eng "Bad handshake"
ER_BAD_HOST_ERROR 08S01
eng "Can't get hostname for your address"
sql/sql_insert.cc:
ER_WRONG_VALUE_COUNT_ON_ROW 21S01
eng "Column count doesn't match value count at row %ld"
sql/sql_parse.cc:
ER_WARN_HOSTNAME_WONT_WORK
eng "MySQL is started in --skip-name-resolve mode; you must restart it without this switch for this grant to work"
ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT
eng "Too high level of nesting for select"
ER_UNKNOWN_ERROR
eng "Unknown error"
sql/sql_partition.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_plugin.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_prepare.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
ER_UNKNOWN_STMT_HANDLER
eng "Unknown prepared statement handler (%.*s) given to %s"
length value (for '%.*s') must be 'int', per the doc of printf()
and the code of my_vsnprintf().
sql/sql_show.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_table.cc:
ER_TOO_BIG_FIELDLENGTH 42000 S1009
eng "Column length too big for column '%-.192s' (max = %lu); use BLOB or TEXT instead"
sql/table.cc:
ER_NOT_FORM_FILE
eng "Incorrect information in file: '%-.200s'"
ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE
eng "Column count of mysql.%s is wrong. Expected %d, found %d. Created with MySQL %d, now running %d. Please use mysql_upgrade to fix this error."
table->s->mysql_version is ulong.
sql/unireg.cc:
ER_TOO_LONG_TABLE_COMMENT
eng "Comment for table '%-.64s' is too long (max = %lu)"
ER_TOO_LONG_FIELD_COMMENT
eng "Comment for field '%-.64s' is too long (max = %lu)"
ER_TOO_BIG_ROWSIZE 42000
eng "Row size too large. The maximum row size for the used table type, not counting BLOBs, is %ld. You have to change some columns to TEXT or BLOBs"
Changed some String.ptr() -> String.c_ptr() for String that are not guaranteed to end with \0
Removed some c_ptr() usage from parameters to functions that takes ptr & length
Use preallocate buffers to avoid calling malloc() for most operations.
sql/event_db_repository.cc:
alias is now a String
sql/event_scheduler.cc:
c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
sql/events.cc:
c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
c_ptr -> ptr() as function takes ptr & length
sql/field.cc:
alias is now a String
sql/field.h:
alias is now a String
sql/ha_partition.cc:
alias is now a String
sql/handler.cc:
alias is now a String
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/item.cc:
Store error parameter in separarte buffer to ensure correct error message
sql/item_func.cc:
ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/item_sum.h:
Use my_strtod() instead of my_atof() to not have to make string \0 terminated
sql/lock.cc:
alias is now a String
sql/log.cc:
c_ptr() -> ptr() as function takes ptr & length
sql/log_event.cc:
c_ptr_quick() -> ptr() as we only want to get the pointer to String buffer
sql/opt_range.cc:
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/opt_table_elimination.cc:
alias is now a String
sql/set_var.cc:
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
c_ptr() -> ptr() as function takes ptr & length
Simplify some code.
sql/sp.cc:
c_ptr() -> ptr() as function takes ptr & length
sql/sp_rcontext.cc:
alias is now a String
sql/sql_base.cc:
alias is now a String.
Here we win a realloc() for most alias usage.
sql/sql_class.cc:
Use size descriptor for printf() to avoid accessing bytes outside of buffer
sql/sql_insert.cc:
Change allocation of TABLE as it's now contains a String
_ptr() -> ptr() as function takes ptr & length
sql/sql_load.cc:
Use preallocate buffers to avoid calling malloc() for most operations.
sql/sql_parse.cc:
Use c_ptr_safe() to ensure string is \0 terminated.
sql/sql_plugin.cc:
c_ptr_quick() -> ptr() as function takes ptr & length
sql/sql_select.cc:
alias is now a String
sql/sql_show.cc:
alias is now a String
sql/sql_string.h:
Added move() function to change who owns the string (owner does the free)
sql/sql_table.cc:
alias is now a String
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
sql/sql_test.cc:
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
alias is now a String
sql/sql_trigger.cc:
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
Use field->init() to setup pointers to alias.
sql/sql_update.cc:
alias is now a String
sql/sql_view.cc:
ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/sql_yacc.yy:
r() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/table.cc:
alias is now a String
sql/table.h:
alias is now a String
storage/federatedx/ha_federatedx.cc:
Remove extra 1 byte alloc that is automaticly done by strmake()
Ensure that error message ends with \0
storage/maria/ha_maria.cc:
alias is now a String
storage/myisam/ha_myisam.cc:
alias is now a String
- Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria"
- Fixed wrong assert in Aria
Now need to merge with latest xtradb before pushing
sql/ha_partition.cc:
Ensure that m_ordered_rec_buffer is not freed before close.
sql/mysqld.cc:
Changed to use opt_stack_trace instead of opt_pstack.
Removed references to pstack
sql/partition_element.h:
Ensure that connect_string is initialized
storage/maria/ma_key_recover.c:
Fixed wrong assert
When installing plugins, there is a missing check
for slash (/) in the path on Windows. Note that on
Windows, both / and \ can be used to separate
directories.
This patch fixes the issue by:
- Adding a FN_DIRSEP symbol for all platforms
consisting of a string of legal directory
separators.
- Adding a charset-aware version of strcspn().
- Adding a check_valid_path() function that uses
my_strcspn() to check if any FN_DIRSEP character
is in the supplied string.
- Using the check_valid_path() function in
sql_plugin.cc and sql_udf.cc (which means
replacing the existing test there).
include/config-netware.h:
Adding FN_DIRSEP
******
Adding FN_DIRSEP
include/config-win.h:
Adding FN_DIRSEP
******
Adding FN_DIRSEP
include/m_ctype.h:
Adding my_strspn() and my_strcspn().
******
Adding my_strspn() and my_strcspn().
include/my_global.h:
Adding FN_DIRSEP
******
Adding FN_DIRSEP
mysql-test/t/plugin_not_embedded.test:
Adding test that file names containing / is
disallowed on *all* platforms.
******
Adding test that file names containing / is
disallowed on *all* platforms.
sql/sql_plugin.cc:
Introducing check_if_path() function for
checking if filename is a path to include
/ on Windows.
******
Introducing check_if_path() function for
checking if filename is a path to include
/ on Windows.
sql/sql_udf.cc:
Switching to use check_if_path() function.
******
Switching to use check_if_path() function.
strings/my_strchr.c:
Adding my_strspn() and my_strcspn().
******
Adding my_strspn() and my_strcspn().
If a relative path is supplied to option --defaults-file or
--defaults-extra-file, the server will crash when executing
an INSTALL PLUGIN command. The reason is that the defaults
file is initially read relative the current working directory
when the server is started, but when INSTALL PLUGIN is executed,
the server has changed working directory to the data directory.
Since there is no check that the call to my_load_defaults()
inside mysql_install_plugin(), the subsequence call to
free_defaults() will crash the server.
This patch fixes the problem by:
- Prepending the current working directory to the file name when
a relative path is given to the --defaults-file or --defaults-
extra-file option the first time my_load_defaults() is called,
which is just after the server has started in main().
- Adding a check of the return value of my_load_defaults() inside
mysql_install_plugin() and aborting command (with an error) if
an error is returned.
- It also adds a check of the return value for load_defaults in
lib_sql.cc for the embedded server since that was missing.
To test that the relative files for the options --defaults-file and
--defaults-extra-file is handled properly, mysql-test-run.pl is also
changed to not add a --defaults-file option if one is provided in the
tests *.opt file.
This patch implements "permanent" load option for
plugins as specified by WL#5341.
mysql-test/r/plugin_load_option.result:
A test case for WL#5341.
mysql-test/t/plugin_load_option-master.opt:
A test case for WL#5341.
mysql-test/t/plugin_load_option.test:
A test case for WL#5341.
sql/share/errmsg-utf8.txt:
An error message for WL#5341.
sql/sql_plugin.cc:
Added FORCE_PLUS_PERMANENT plugin load option.
sql/sql_plugin.h:
Expose and use plugin load option instead of
is_mandatory flag. This is a requirement for
to-be-implemented WL5496.
attempt to install a plugin twice
Server crashes when [UN]INSTALL PLUGIN fails (returns an
error) and general log is disabled and there are audit
plugins interested in MYSQL_AUDIT_GENERAL_CLASS.
When audit event is triggered, audit subsystem acquires interested
plugins by walking through plugin list. Evidently plugin list
iterator protects plugin list by acquiring LOCK_plugin, see
plugin_foreach_with_mask().
On the other hand [UN]INSTALL PLUGIN is acquiring LOCK_plugin
rather for a long time.
When audit event is triggered during [UN]INSTALL PLUGIN, plugin
list iterator acquires the same lock (within the same thread)
second time.
Repeatable only with general_log disabled, because general_log
triggers MYSQL_AUDIT_GENERAL_LOG event, which acquires audit
plugins before [UN]INSTALL PLUGIN acquired LOCK_plugin.
With this fix we pre-acquire audit plugins for events that
may potentially occur during [UN]INSTALL PLUGIN.
This hack should be removed when LOCK_plugin is fixed so it
protects only what it supposed to protect.
No test case for this fix - we do not have facility to test
audit plugins yet.
sql/sql_audit.cc:
Move "acquire audit plugin" logics to a separate
function.
sql/sql_audit.h:
Move "acquire audit plugin" logics to a separate
function.
sql/sql_plugin.cc:
Pre-acquire audit plugins for events that may potentially occur
during [UN]INSTALL PLUGIN.
check_if_table_exists()
This assert was triggered when the server tried to load plugins
while running in embedded server mode. In embedded server mode,
check_if_table_exists() was used to check if mysql.plugin existed
so that ER_NO_SUCH_TABLE could be silently ignored.
The problem was that this check was done without acquiring a metadata
lock on mysql.plugin first. This triggered the assert.
This patch fixes the problem by removing the call to
check_if_table_exists() from plugin_load(). Instead an error handler
which traps ER_NO_SUCH_TABLE is installed before trying to open
mysql.plugin when running in embedded server mode.
No test coverage added since this assert was triggered by
existing tests running in embedded server mode.
sql/sql_base.cc:
Renamed Prelock_error_handler to No_such_table_error_handler
and moved the declaration to sql_base.h to make it usable
in plugin_load().
sql/sql_base.h:
Renamed Prelock_error_handler to No_such_table_error_handler
and moved the declaration to sql_base.h to make it usable
in plugin_load().
sql/sql_plugin.cc:
Removed call to check_if_table_exists() used to check for mysql.plugin
in plugin_load() for embedded server. Instead install error handler
which traps ER_NO_SUCH_TABLE during open_and_lock_tables().
Reverted the ulong->uint diff
Re-applied the first diff.
The original commit message follows:
enum plugin system variables are ulong internally, not int.
On systems where long is not the same as an int it causes
problems.
Fixed by correct typecasting. Removed the test from the
experimental list.
The enum system variables were handled inconsistently
as ints, unsigned int and unsigned long on various places.
This caused problems on platforms on which
sizeof(int) != sizeof(long).
Fixed by homogenizing the type of the enum variables
to unsigned int, since it's size compatible with the C enum
type.
Removed the test from the experimental list.
This patch also fixes Bug#55452 "SET PASSWORD is
replicated twice in RBR mode".
The goal of this patch is to remove the release of
metadata locks from close_thread_tables().
This is necessary to not mistakenly release
the locks in the course of a multi-step
operation that involves multiple close_thread_tables()
or close_tables_for_reopen().
On the same token, move statement commit outside
close_thread_tables().
Other cleanups:
Cleanup COM_FIELD_LIST.
Don't call close_thread_tables() in COM_SHUTDOWN -- there
are no open tables there that can be closed (we leave
the locked tables mode in THD destructor, and this
close_thread_tables() won't leave it anyway).
Make open_and_lock_tables() and open_and_lock_tables_derived()
call close_thread_tables() upon failure.
Remove the calls to close_thread_tables() that are now
unnecessary.
Simplify the back off condition in Open_table_context.
Streamline metadata lock handling in LOCK TABLES
implementation.
Add asserts to ensure correct life cycle of
statement transaction in a session.
Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.
mysql-test/r/variables.result:
Update results: set @@autocommit and statement transaction/
prelocked mode.
mysql-test/r/view.result:
A harmless change in CHECK TABLE <view> status for a broken view.
If previously a failure to prelock all functions used in a view
would leave the connection in LTM_PRELOCKED mode, now we call
close_thread_tables() from open_and_lock_tables()
and leave prelocked mode, thus some check in mysql_admin_table() that
works only in prelocked/locked tables mode is no longer activated.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
Fixed Bug#55452 "SET PASSWORD is replicated twice in
RBR mode": extra binlog events are gone from the
binary log.
mysql-test/t/variables.test:
Add a test case: set autocommit and statement transaction/prelocked
mode.
sql/event_data_objects.cc:
Simplify code in Event_job_data::execute().
Move sp_head memory management to lex_end().
sql/event_db_repository.cc:
Move the release of metadata locks outside
close_thread_tables().
Make sure we call close_thread_tables() when
open_and_lock_tables() fails and remove extra
code from the events data dictionary.
Use close_mysql_tables(), a new internal
function to properly close mysql.* tables
in the data dictionary.
Contract Event_db_repository::drop_events_by_field,
drop_schema_events into one function.
When dropping all events in a schema,
make sure we don't mistakenly release all
locks acquired by DROP DATABASE. These
include locks on the database name
and the global intention exclusive
metadata lock.
sql/event_db_repository.h:
Function open_event_table() does not require an instance
of Event_db_repository.
sql/events.cc:
Use close_mysql_tables() instead of close_thread_tables()
to bootstrap events, since the latter no longer
releases metadata locks.
sql/ha_ndbcluster.cc:
- mysql_rm_table_part2 no longer releases
acquired metadata locks. Do it in the caller.
sql/ha_ndbcluster_binlog.cc:
Deploy the new protocol for closing thread
tables in run_query() and ndb_binlog_index
code.
sql/handler.cc:
Assert that we never call ha_commit_trans/
ha_rollback_trans in sub-statement, which
is now the case.
sql/handler.h:
Add an accessor to check whether THD_TRANS object
is empty (has no transaction started).
sql/log.cc:
Update a comment.
sql/log_event.cc:
Since now we commit/rollback statement transaction in
mysql_execute_command(), we need a mechanism to communicate
from Query_log_event::do_apply_event() to mysql_execute_command()
that the statement transaction should be rolled back, not committed.
Ideally it would be a virtual method of THD. I hesitate
to make THD a virtual base class in this already large patch.
Use a thd->variables.option_bits for now.
Remove a call to close_thread_tables() from the slave IO
thread. It doesn't open any tables, and the protocol
for closing thread tables is more complicated now.
Make sure we properly close thread tables, however,
in Load_data_log_event, which doesn't
follow the standard server execution procedure
with mysql_execute_command().
@todo: this piece should use Server_runnable
framework instead.
Remove an unnecessary call to mysql_unlock_tables().
sql/rpl_rli.cc:
Update Relay_log_info::slave_close_thread_tables()
to follow the new close protocol.
sql/set_var.cc:
Remove an unused header.
sql/slave.cc:
Remove an unnecessary call to
close_thread_tables().
sql/sp.cc:
Remove unnecessary calls to close_thread_tables()
from SP DDL implementation. The tables will
be closed by the caller, in mysql_execute_command().
When dropping all routines in a database, make sure
to not mistakenly drop all metadata locks acquired
so far, they include the scoped lock on the schema.
sql/sp_head.cc:
Correct the protocol that closes thread tables
in an SP instruction.
Clear lex->sphead before cleaning up lex
with lex_end to make sure that we don't
delete the sphead twice. It's considered
to be "cleaner" and more in line with
future changes than calling delete lex->sphead
in other places that cleanup the lex.
sql/sp_head.h:
When destroying m_lex_keeper of an instruction,
don't delete the sphead that all lex objects
share.
@todo: don't store a reference to routine's sp_head
instance in instruction's lex.
sql/sql_acl.cc:
Don't call close_thread_tables() where the caller will
do that for us.
Fix Bug#55452 "SET PASSWORD is replicated twice in RBR
mode" by disabling RBR replication in change_password()
function.
Use close_mysql_tables() in bootstrap and ACL reload
code to make sure we release all metadata locks.
sql/sql_base.cc:
This is the main part of the patch:
- remove manipulation with thd->transaction
and thd->mdl_context from close_thread_tables().
Now this function is only responsible for closing
tables, nothing else.
This is necessary to be able to easily use
close_thread_tables() in procedures, that
involve multiple open/close tables, which all
need to be protected continuously by metadata
locks.
Add asserts ensuring that TABLE object
is only used when is protected by a metadata lock.
Simplify the back off condition of Open_table_context,
we no longer need to look at the autocommit mode.
Make open_and_lock_tables() and open_normal_and_derived_tables()
close thread tables and release metadata locks acquired so-far
upon failure. This simplifies their usage.
Implement close_mysql_tables().
sql/sql_base.h:
Add declaration for close_mysql_tables().
sql/sql_class.cc:
Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.
The code became dead when my_eof() was made a non-protocol method,
but a method that merely modifies the diagnostics area.
The code became redundant with the fix for Bug#37521, when
we started to cal close_thread_tables() before
Protocol::end_statement().
sql/sql_do.cc:
Do nothing in DO if inside a substatement
(the assert moved out of trans_rollback_stmt).
sql/sql_handler.cc:
Add comments.
sql/sql_insert.cc:
Remove dead code.
Release metadata locks explicitly at the
end of the delayed insert thread.
sql/sql_lex.cc:
Add destruction of lex->sphead to lex_end(),
lex "reset" method called at the end of each statement.
sql/sql_parse.cc:
Move close_thread_tables() and other related
cleanups to mysql_execute_command()
from dispatch_command(). This has become
possible after the fix for Bug#37521.
Mark federated SERVER statements as DDL.
Next step: make sure that we don't store
eof packet in the query cache, and move
the query cache code outside mysql_parse.
Brush up the code of COM_FIELD_LIST.
Remove unnecessary calls to close_thread_tables().
When killing a query, don't report "OK"
if it was a suicide.
sql/sql_parse.h:
Remove declaration of a function that is now static.
sql/sql_partition.cc:
Remove an unnecessary call to close_thread_tables().
sql/sql_plugin.cc:
open_and_lock_tables() will clean up
after itself after a failure.
Move close_thread_tables() above
end: label, and replace with close_mysql_tables(),
which will also release the metadata lock
on mysql.plugin.
sql/sql_prepare.cc:
Now that we no longer release locks in close_thread_tables()
statement prepare code has become more straightforward.
Remove the now redundant check for thd->killed() (used
only by the backup project) from Execute_server_runnable.
Reorder code to take into account that now mysql_execute_command()
performs lex->unit.cleanup() and close_thread_tables().
sql/sql_priv.h:
Add a new option to server options to interact
between the slave SQL thread and execution
framework (hack). @todo: use a virtual
method of class THD instead.
sql/sql_servers.cc:
Due to Bug 25705 replication of
DROP/CREATE/ALTER SERVER is broken.
Make sure at least we do not attempt to
replicate these statements using RBR,
as this violates the assert in close_mysql_tables().
sql/sql_table.cc:
Do not release metadata locks in mysql_rm_table_part2,
this is done by the caller.
Do not call close_thread_tables() in mysql_create_table(),
this is done by the caller.
Fix a bug in DROP TABLE under LOCK TABLES when,
upon error in wait_while_table_is_used() we would mistakenly
release the metadata lock on a non-dropped table.
Explicitly release metadata locks when doing an implicit
commit.
sql/sql_trigger.cc:
Now that we delete lex->sphead in lex_end(),
zero the trigger's sphead in lex after loading
the trigger, to avoid double deletion.
sql/sql_udf.cc:
Use close_mysql_tables() instead of close_thread_tables().
sql/sys_vars.cc:
Remove code added in scope of WL#4284 which would
break when we perform set @@session.autocommit along
with setting other variables and using tables or functions.
A test case added to variables.test.
sql/transaction.cc:
Add asserts.
sql/tztime.cc:
Use close_mysql_tables() rather than close_thread_tables().
The issue was that we didn't always check result of ha_rnd_init() which caused a problem for handlers that returned an error in this code.
- Changed prototype of ha_rnd_init() to ensure that we get a compile warning if result is not checked.
- Added ha_rnd_init_with_error() that prints error on failure.
- Checked all usage of ha_rnd_init() and ensure we generate an error message on failures.
- Changed init_read_record() to return 1 on failure.
sql/create_options.cc:
Fixed wrong printf
sql/event_db_repository.cc:
Check result from init_read_record()
sql/events.cc:
Check result from init_read_record()
sql/filesort.cc:
Check result from ha_rnd_init()
sql/ha_partition.cc:
Check result from ha_rnd_init()
sql/ha_partition.h:
Fixed compiler warning
sql/handler.cc:
Added ha_rnd_init_with_error()
Check result from ha_rnd_init()
sql/handler.h:
Added ha_rnd_init_with_error()
Changed prototype of ha_rnd_init() to ensure that we get a compile warning if result is not checked
sql/item_subselect.cc:
Check result from ha_rnd_init()
sql/log.cc:
Check result from ha_rnd_init()
sql/log_event.cc:
Check result from ha_rnd_init()
sql/log_event_old.cc:
Check result from ha_rnd_init()
sql/mysql_priv.h:
init_read_record() now returns error code on failure
sql/opt_range.cc:
Check result from ha_rnd_init()
sql/records.cc:
init_read_record() now returns error code on failure
Check result from ha_rnd_init()
sql/sql_acl.cc:
Check result from init_read_record()
sql/sql_cursor.cc:
Print error if ha_rnd_init() fails
sql/sql_delete.cc:
Check result from init_read_record()
sql/sql_help.cc:
Check result from init_read_record()
sql/sql_plugin.cc:
Check result from init_read_record()
sql/sql_select.cc:
Check result from ha_rnd_init()
Print error if ha_rnd_init() fails.
sql/sql_servers.cc:
Check result from init_read_record()
sql/sql_table.cc:
Check result from init_read_record()
sql/sql_udf.cc:
Check result from init_read_record()
sql/sql_update.cc:
Check result from init_read_record()
storage/example/ha_example.cc:
Don't return error on rnd_init()
storage/ibmdb2i/ha_ibmdb2i.cc:
Removed not relevant comment
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
an atomic counter"
Split the large LOCK_open section in open_table().
Do not call open_table_from_share() under LOCK_open.
Remove thd->version.
This fixes
Bug#50589 "Server hang on a query evaluated using a temporary
table"
Bug#51557 "LOCK_open and kernel_mutex are not happy together"
Bug#49463 "LOCK_table and innodb are not nice when handler
instances are created".
This patch has effect on storage engines that rely on
ha_open() PSEA method being called under LOCK_open.
In particular:
1) NDB is broken and left unfixed. NDB relies on LOCK_open
being kept as part of ha_open(), since it uses auto-discovery.
While previously the NDB open code was race-prone, now
it simply fails on asserts.
2) HEAP engine had a race in ha_heap::open() when
a share for the same table could be added twice
to the list of shares, or a dangling reference to a share
stored in HEAP handler. This patch aims to address this
problem by 'pinning' the newly created share in the
internal HEAP engine share list until at least one
handler instance is created using that share.
include/heap.h:
Add members to HP_CREATE_INFO.
Declare heap_release_share().
sql/lock.cc:
Remove thd->version, use thd->open_tables->s->version instead.
sql/repl_failsafe.cc:
Remove thd->version.
sql/sql_base.cc:
- close_thread_table(): move handler cleanup code outside the critical section protected by LOCK_open.
- remove thd->version
- split the large critical section in open_table() that
opens a new table from share and is protected by LOCK_open
into 2 critical sections, thus reducing the critical path.
- make check_if_table_exists() acquire LOCK_open internally.
- use thd->open_tables->s->version instead of thd->refresh_version to make sure that all tables in
thd->open_tables are in the same refresh series.
sql/sql_base.h:
Add declaration for check_if_table_exists().
sql/sql_class.cc:
Remove init_open_tables_state(), it's now equal to
reset_open_tables_state().
sql/sql_class.h:
Remove thd->version, THD::init_open_tables_state().
sql/sql_plugin.cc:
Use table->m_needs_reopen to mark the table as stale
rather than manipulate with thd->version, which is no more.
sql/sql_udf.cc:
Use table->m_needs_reopen to mark the table as stale
rather than manipulate with thd->version, which is no more.
sql/table.h:
Remove an unused variable.
sql/tztime.cc:
Use table->m_needs_reopen to mark the table as stale
rather than manipulate with thd->version, which is no more.
storage/heap/CMakeLists.txt:
Add heap tests to cmake build files.
storage/heap/ha_heap.cc:
Fix a race when ha_heap::ha_open() could insert two
HP_SHARE objects into the internal share list or store
a dangling reference to a share in ha_heap instance,
or wrongly set implicit_emptied.
storage/heap/hp_create.c:
Optionally pin a newly created share in the list of shares
by increasing its open_count. This is necessary to
make sure that a newly created share doesn't disappear while
a HP_INFO object is being created to reference it.
storage/heap/hp_open.c:
When adding a new HP_INFO object to the list of objects
in the heap share, make sure the open_count is not increased
twice.
storage/heap/hp_test1.c:
Adjust the test to new function signatures.
storage/heap/hp_test2.c:
Adjust the test to new function signatures.
strict aliasing violations.
Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.
As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.
The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.
client/mysql.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysql_upgrade.c:
Remove now-unnecessary casts.
client/mysqladmin.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysqlbinlog.cc:
Remove now-unnecessary casts.
client/mysqlcheck.c:
Remove now-unnecessary casts.
client/mysqldump.c:
Remove now-unnecessary casts.
client/mysqlimport.c:
Remove now-unnecessary casts.
client/mysqlshow.c:
Remove now-unnecessary casts.
client/mysqlslap.c:
Remove now-unnecessary casts.
client/mysqltest.cc:
Remove now-unnecessary casts.
extra/comp_err.c:
Remove now-unnecessary casts.
extra/my_print_defaults.c:
Remove now-unnecessary casts.
Break up large strings.
extra/mysql_waitpid.c:
Remove now-unnecessary casts.
extra/perror.c:
Remove now-unnecessary casts.
extra/resolve_stack_dump.c:
Remove now-unnecessary casts.
extra/resolveip.c:
Remove now-unnecessary casts.
include/my_getopt.h:
Use a void pointer type as the opaque type to avoid problems with type
incompatibility -- GCC issues warnings when the type name is not type
compatible with a operand. As a side bonus, a explicit cast won't be
necessary anymore.
include/sslopt-longopts.h:
Remove now-unnecessary casts.
Break up large strings.
mysys/my_getopt.c:
Update opaque type and introduce a type definition for the
argument to my_getopt_register_get_addr.
server-tools/instance-manager/options.cc:
Remove now-unnecessary casts.
sql/mysqld.cc:
Remove now-unnecessary casts.
Break up large strings.
Update mysql_getopt_value prototype (the old prototype
was different from the definition anyway).
sql/sql_plugin.cc:
The type of a pointer to a function must be compatible with the
pointed-to function type, otherwise the behavior is undefined.
sql/table.cc:
The variable buf pointer to pointer to pointer to constant char
could improperly alias a incompatible type in call to fix_type_
pointers. Since this was actually dead code, it is simply removed.
sql/unireg.cc:
Remove call to get_form_pos. The code creates a new FRM file which
is always truncated and writes the form position as 0. Hence, no
need to retrieve it, we now for sure it is 0.
storage/archive/archive_reader.c:
Remove now-unnecessary casts.
storage/myisam/ft_nlq_search.c:
Read weight directly from the buffer.
storage/myisam/fulltext.h:
Add explanation about the type duality of a key buffer.
Add accessor macro to retrieve a FT float value.
storage/myisam/mi_test1.c:
Remove now-unnecessary casts.
storage/myisam/myisam_ftdump.c:
Read weight directly from the buffer.
storage/myisam/myisamchk.c:
Remove now-unnecessary casts.
storage/myisam/myisamlog.c:
A pointer to char was used to alias a pointer to pointer to
unsigned char, thus violating strict aliasing rules.
storage/myisam/myisampack.c:
Remove now-unnecessary casts.
strings/decimal.c:
Remove aliasing violation, printing the value is enough for
debugging purposes.
tests/mysql_client_test.c:
Remove now-unnecessary casts.
Fixed some bugs introduced in 5.1.47
Disabled some tests until we have merged with latest Xtradb
configure.in:
Added testing if valgrind/memcheck.h exists
storage/pbxt/src/ha_pbxt.cc:
LOCK_plugin is not anymore locked in init
Conflicts:
Text conflict in configure.in
Text conflict in dbug/dbug.c
Text conflict in mysql-test/r/ps.result
Text conflict in mysql-test/t/ps.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/ha_ndbcluster.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_table.cc
Previously installed dynamic plugins are explicitly not loaded
on startup with --skip-grant-tables enabled. However, INSTALL
PLUGIN/UNINSTALL PLUGIN commands are allowed, and result in
inconsistent error messages (reporting duplicate plugin or
plugin does not exist).
This patch adds a check for --skip-grant-tables mode, and
returns error ER_OPTION_PREVENTS_STATEMENT to the user when
the above commands are attempted.
Potential deadlock situation involving LOCK_plugin,
LOCK_global_system_variables and LOCK_status.
This patch backports the fix from next-mr, unlocking
LOCK_plugin before calling plugin->init() and
add_status_vars().
update test results
fix compilation failure on windows
fix for ps-protocol
mysql-test/r/plugin.result:
make sure the PS handle is closed
mysql-test/t/plugin.test:
make sure the PS handle is closed
sql/mysqld.cc:
compilation failure: lable without a statement.
sql/sql_plugin.cc:
make plugin lock and plugin unlock use the same format of the dbug info string.
print the ref_count *after* the lock and *after* the unlock.
Removed random failures from test suite
mysql-test/extra/rpl_tests/rpl_insert_id_pk.test:
Make test predicatable.
mysql-test/include/maria_empty_logs.inc:
We can't use 'Threads_connected' for syncronization, as the 'check_warnings' thread that just quit may still be counted in 'Threads_connected'
Now we just wait until MySQLD answers again, which should be good enough for our purposes
mysql-test/suite/binlog/r/binlog_index.result:
Updated results file
mysql-test/suite/binlog/t/binlog_index-master.opt:
Added option file to not get stack traces in .err file.
mysql-test/suite/binlog/t/binlog_index.test:
Added 'flush tables' to remove warning about crashed suppression file from logs
mysql-test/suite/pbxt/r/multi_statement.result:
Updated results
mysql-test/suite/pbxt/t/multi_statement-master.opt:
Added options so that slow query testing makes sense
sql/events.cc:
Don't write Event Scheduler startup message if warnings are turned off.
sql/handler.cc:
Removed compiler warning
sql/log.cc:
Removed compiler warning
sql/mysqld.cc:
Added option 'test-expect-abort'; If this is set, we don't write message to log in case of 'DBUG_ABORT'.
(Gives us smaller, easier to read log files)
sql/set_var.cc:
Removed compiler warning
sql/slave.cc:
Removed compiler warning
sql/sql_plugin.cc:
Don't write warnings about disabled plugin if using --log_warnings=0
storage/xtradb/include/ut0lst.h:
Removed compiler warning
support-files/compiler_warnings.supp:
Supress warning from xtradb
Interface for maria extensions.
Alternative plugin interface with additional info (maturity and string version).
CMakeLists.txt:
Maria plugin interface used.
config/ac-macros/plugins.m4:
Maria plugin interface used.
configure.in:
Maria plugin interface used.
include/mysql/plugin.h:
Maria plugin interface added.
include/mysql/plugin_auth.h.pp:
Maria plugin interface added.
plugin/auth/auth_socket.c:
Maria plugin interface added.
plugin/auth/dialog.c:
Maria plugin interface added.
plugin/daemon_example/daemon_example.cc:
Maria plugin interface added.
plugin/fulltext/plugin_example.c:
Maria plugin interface added.
sql/ha_ndbcluster.cc:
Maria plugin interface added.
sql/ha_partition.cc:
Maria plugin interface added.
sql/log.cc:
Maria plugin interface added.
sql/sql_acl.cc:
Maria plugin interface added.
sql/sql_builtin.cc.in:
Maria plugin interface used.
sql/sql_plugin.cc:
Maria plugin interface added.
sql/sql_plugin.h:
Maria plugin interface used.
sql/sql_show.cc:
Maria plugin interface added.
storage/archive/ha_archive.cc:
Maria plugin interface added.
storage/blackhole/ha_blackhole.cc:
Maria plugin interface added.
storage/csv/ha_tina.cc:
Maria plugin interface added.
storage/example/ha_example.cc:
Maria plugin interface added.
storage/federated/ha_federated.cc:
Maria plugin interface added.
storage/federatedx/ha_federatedx.cc:
Maria plugin interface added.
storage/heap/ha_heap.cc:
Maria plugin interface added.
storage/ibmdb2i/ha_ibmdb2i.cc:
Maria plugin interface added.
storage/innobase/handler/ha_innodb.cc:
Maria plugin interface added.
storage/innodb_plugin/handler/i_s.cc:
Maria plugin interface added.
storage/maria/ha_maria.cc:
Maria plugin interface added.
storage/myisam/ha_myisam.cc:
Maria plugin interface added.
storage/myisammrg/ha_myisammrg.cc:
Maria plugin interface added.
storage/pbxt/src/ha_pbxt.cc:
Maria plugin interface added.
storage/xtradb/handler/ha_innodb.cc:
Maria plugin interface added.
storage/xtradb/handler/i_s.cc:
Maria plugin interface added.
storage/xtradb/handler/i_s.h:
Maria plugin interface added.
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
Makefile.am:
add new API files to the check_abi rule,
remove duplicates
client/CMakeLists.txt:
now a client can use dlopen too
client/Makefile.am:
be csh-friendly
include/my_global.h:
add dummy plugs for dlopen and co.
for the code that needs them to work in static builds
mysys/Makefile.am:
be csh-friendly
plugin/auth/dialog.c:
typo fixed
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/Makefile.am
Text conflict in mysql-test/collections/default.daily
Text conflict in mysql-test/r/mysqlbinlog_row_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_typeconv_innodb.result
Text conflict in mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test
Text conflict in mysql-test/suite/rpl/t/rpl_row_create_table.test
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in mysql-test/suite/rpl/t/rpl_typeconv_innodb.test
Text conflict in mysys/charset.c
Text conflict in sql/field.cc
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_func.cc
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/rpl_utility.cc
Text conflict in sql/rpl_utility.h
Text conflict in sql/set_var.cc
Text conflict in sql/share/Makefile.am
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_table.cc
Text conflict in storage/example/ha_example.h
Text conflict in storage/federated/ha_federated.cc
Text conflict in storage/myisammrg/ha_myisammrg.cc
Text conflict in storage/myisammrg/myrg_open.c
Conflicts:
Text conflict in mysql-test/r/partition_innodb.result
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/item_sum.h
Text conflict in sql/log_event_old.cc
Text conflict in sql/protocol.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_yacc.yy
The problem was that UNINSTALL PLUGIN wasn't performing privilege
checks before removing a plugin. Any user (including users without
any kind of privileges) could uninstall any plugin.
The solution is to verify if the user has the DELETE privilege for
the mysql.plugin table before uninstalling a plugin.
mysql-test/r/plugin_not_embedded.result:
Add test case result for Bug#51770.
mysql-test/t/plugin_not_embedded-master.opt:
Add example plugin path.
mysql-test/t/plugin_not_embedded.test:
Add test case for Bug#51770.
Skip embedded as test relies on privileges checks.
This patch prevents system threads and system table accesses from
using user-specified values for "lock_wait_timeout". Instead all
such accesses are done using the default value (1 year).
This prevents background tasks (such as replication, events,
accessing stored function definitions, logging, reading time-zone
information, etc.) from failing in cases where the global value
of "lock_wait_timeout" is set very low.
The patch also simplifies the open tables API. Rather than adding
another convenience function for opening and locking system tables,
this patch removes most of the existing convenience functions for
open_and_lock_tables_derived(). Before, open_and_lock_tables() was
a convenience function that enforced derived tables handling, while
open_and_lock_tables_derived() was the main function where derived
tables handling was optional. Now, this convencience function is
gone and the main function is renamed to open_and_lock_tables().
No test case added as it would have required the use of --sleep to
check that system threads and system tables have a different timeout
value from the user-specified "lock_wait_timeout" system variable.
mysql-test/t/disabled.def:
Restore disabled ssl tests: SSL certificates were updated.
Disable sp_sync.test, the test case can't work in next-4284.
mysql-test/t/partition_innodb.test:
Disable parsing of the test case for Bug#47343,
the test can not work in next-4284.
mysql-test/t/ps_ddl.test:
Update results (CREATE TABLE IF NOT EXISTS takes
into account existence of the temporary table).
The number for MYSQL_REPLICATION_PLUGIN was shifted when backporting
because MYSQL_AUDIT_PLUGIN was not backported.
This problem is fixed by backporting only the number of audit plugin
and print an error when trying to load audit plugins. Note that
replication plugins compiled against old MYSQL_REPLICATION_PLUGIN
number will also be recognized as audit plugin and be rejected.
include/mysql/plugin.h:
backporting the number of audit plugin (MYSQL_AUDIT_PLUGIN)
sql/sql_plugin.cc:
backporting the number of audit plugin (MYSQL_AUDIT_PLUGIN)
print an error when trying to load audit plugins
Conflicts:
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/instance_map.cc
Contents conflict in server-tools/instance-manager/listener.cc
Contents conflict in server-tools/instance-manager/options.cc
Contents conflict in server-tools/instance-manager/user_map.cc
Also some small fixes to make the PBXT testsuite work in --embedded.
config/ac-macros/plugins.m4:
MYSQL_PLUGIN_DEPENDS_ON_MYSQL_INTERNALS macro extended to support multiple files.
mysql-test/std_data/pbxt_load_unique_error1.inc:
Move file to be accessible also for testing embedded server.
mysql-test/suite/pbxt/r/pbxt_bugs.result:
Fix LOAD DATA LOCAL INFILE path so it works also for testing embedded server.
mysql-test/suite/pbxt/t/pbxt_bugs.test:
Fix LOAD DATA LOCAL INFILE path so it works also for testing embedded server.
mysql-test/suite/pbxt/t/pbxt_locking.test:
Disable for embedded, as it needs SHOW PROCESSLIST functionality not available there.
mysql-test/suite/pbxt/t/pbxt_transactions.test:
Disable test for embedded, as it needs ability to connect from outside (mysqldump).
mysql-test/suite/pbxt/t/ps_1general.test:
Fix replace_result for new mysql-test-run.pl.
sql/sql_plugin.cc:
Remove hack that disables PBXT in embedded.
storage/pbxt/plug.in:
Fix crash in PBXT in embedded server.
storage/pbxt/src/Makefile.am:
Remove not needed CFLAGS/CXXFLAGS (they cause autotools to generate different object names, which in turn cause the MYSQL_PLUGIN_DEPENDS_ON_MYSQL_INTERNALS replacement of object files inside library files not to work).
storage/pbxt/src/ha_pbxt.cc:
Ugly hack to allow more threads in embedded server (need a better fix I think).
storage/pbxt/src/table_xt.cc:
Use stderr for logging not stdout (prevent spamming --embedded test suite output with stray messages).
storage/pbxt/src/thread_xt.cc:
Use stderr for logging not stdout (prevent spamming --embedded test suite output with stray messages).
storage/pbxt/src/trace_xt.cc:
Use stderr for logging not stdout (prevent spamming --embedded test suite output with stray messages).
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies
CMakeLists.txt:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
client/mysql.cc:
don't crash with --show-warnings when mysqld dies
config/ac-macros/plugins.m4:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
include/my_getopt.h:
comments
include/my_pthread.h:
fix double #define
mysql-test/mysql-test-run.pl:
run sys_vars suite by default
properly recognize envirinment variables (e.g. MTR_MAX_SAVE_CORE) set to 0
escape gdb command line arguments
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
init_slave+utf8 bug
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
init_slave+utf8 bug
mysys/my_getopt.c:
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
mysys/typelib.c:
support for flagset
sql/ha_ndbcluster.cc:
backport from telco tree
sql/item_func.cc:
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
sql/sql_builtin.cc.in:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
sql/sql_plugin.cc:
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#32902 plugin variables don't know their names
Bug#44797 plugins w/o command-line options have no disabling option in --help
sql/sys_vars.cc:
all server variables are defined here
storage/myisam/ft_parser.c:
remove unnecessary updates of param->quot
storage/myisam/ha_myisam.cc:
myisam_* variables belong here
strings/my_vsnprintf.c:
%o and %llx
unittest/mysys/my_vsnprintf-t.c:
%o and %llx tests
vio/viosocket.c:
bugfix: fix @@wait_timeout to work with socket timeouts (vs. alarm thread)
Bug #36098 Audit plugin (wl 3771) feature disabled in 6.0
avoid recusrive locking of LOCK_plugin
include/mysql/plugin_audit.h:
fix incorrect version
sql/log.cc:
move the common code to a shared header
sql/mysqld.cc:
restore the deleted functionality
sql/set_var.cc:
remove unused parameter
sql/sql_audit.h:
two inline convenience functions
sql/sql_parse.cc:
use a simplified convenience call
sql/sql_plugin.cc:
unlock LOCK_plugin for plugin->init() call, add missing OOM check,
issue "unknown variable" error in find_sys_var, not down the stack
WL#3771
"Audit Plugin Interface"
Implement new plug-in type - AUDIT
New plug-in: audit_null
simply increments counter for how many times it was called.
include/Makefile.am:
wl3771
add new headers to distribution
include/mysql/plugin.h:
wl3771
define new AUDIT plugin type
Split out fulltext plugin type into its own header
include/mysql/plugin.h.pp:
wl3771
no real API change, just re-arranged some code
include/mysql/plugin_audit.h:
wl3771
pluggable audit interface
include/mysql/plugin_ftparser.h:
wl3771
Split out fulltext plugin type into its own header
libmysqld/CMakeLists.txt:
wl3771
add sql_audit.cc to build
libmysqld/Makefile.am:
wl3771
add sql_audit.cc to build
plugin/audit_null:
wl3771
an example plugin for testing pluggable audit interface
plugin/audit_null/Makefile.am:
wl3771
an example plugin for testing pluggable audit interface
plugin/audit_null/audit_null.c:
wl3771
an example plugin for testing pluggable audit interface
plugin/audit_null/plug.in:
wl3771
an example plugin for testing pluggable audit interface
sql/CMakeLists.txt:
wl3771
add sql_audit.cc to build
sql/Makefile.am:
wl3771
add sql_audit.cc to build
sql/event_queue.cc:
wl3771
release audit resources before waiting
sql/log.cc:
wl3771
add general audit call for log
sql/mysqld.cc:
wl3771
add audit initialize/finalize
add general audit call for error
sql/sql_audit.cc:
wl3771
pluggable audit interface implementation
sql/sql_audit.h:
wl3771
pluggable audit interface implementation
sql/sql_class.cc:
wl3771
add thd audit init/deinit calls
sql/sql_class.h:
wl3771
add required data structures for audit to THD
sql/sql_connect.cc:
wl3771
release audit resources before waiting
sql/sql_insert.cc:
wl3771
release audit plugins before waiting
sql/sql_parse.cc:
wl3771
add general audit call for results
sql/sql_plugin.cc:
wl3771
add declarations for audit plugin type
----------------------------------------------------------
revno: 2617.69.20
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-4284-1-assert
timestamp: Thu 2009-08-13 18:29:55 +0400
message:
WL#4284 "Transactional DDL locking"
A review fix.
Since WL#4284 implementation separated MDL_request and MDL_ticket,
MDL_request becamse a utility object necessary only to get a ticket.
Store it by-value in TABLE_LIST with the intent to merge
MDL_request::key with table_list->table_name and table_list->db
in future.
Change the MDL subsystem to not require MDL_requests to
stay around till close_thread_tables().
Remove the list of requests from the MDL context.
Requests for shared metadata locks acquired in open_tables()
are only used as a list in recover_from_failed_open_table_attempt(),
which calls mdl_context.wait_for_locks() for this list.
To keep such list for recover_from_failed_open_table_attempt(),
introduce a context class (Open_table_context), that collects
all requests.
A lot of minor cleanups and simplications that became possible
with this change.
sql/event_db_repository.cc:
Remove alloc_mdl_requests(). Now MDL_request instance is a member
of TABLE_LIST, and init_one_table() initializes it.
sql/ha_ndbcluster_binlog.cc:
Remove now unnecessary declaration and initialization
of binlog_mdl_request.
sql/lock.cc:
No need to allocate MDL requests in lock_table_names() now.
sql/log.cc:
Use init_one_table() method, remove alloc_mdl_requests(),
which is now unnecessary.
sql/log_event.cc:
No need to allocate mdl_request separately now.
Use init_one_table() method.
sql/log_event_old.cc:
Update to the new signature of close_tables_for_reopen().
sql/mdl.cc:
Update try_acquire_exclusive_lock() to be more easy to use.
Function lock_table_name_if_not_cached() has been removed.
Make acquire_shared_lock() signature consistent with
try_acquire_exclusive_lock() signature.
Remove methods that are no longer used.
Update comments.
sql/mdl.h:
Implement an assignment operator that doesn't
copy MDL_key (MDL_key::operator= is private and
should remain private).
This is a hack to work-around assignment of TABLE_LIST
by value in several places. Such assignments violate
encapsulation, since only perform a shallow copy.
In most cases these assignments are a hack on their own.
sql/mysql_priv.h:
Update signatures of close_thread_tables() and close_tables_for_reopen().
sql/sp.cc:
Allocate TABLE_LIST in thd->mem_root.
Use init_one_table().
sql/sp_head.cc:
Use init_one_table().
Remove thd->locked_tables_root, it's no longer needed.
sql/sql_acl.cc:
Use init_mdl_requests() and init_one_table().
sql/sql_base.cc:
Update to new signatures of try_acquire_shared_lock() and
try_acquire_exclusive_lock().
Remove lock_table_name_if_not_cached().
Fix a bug in open_ltable() that would not return ER_LOCK_DEADLOCK
in case of a failed lock_tables() and a multi-statement
transaction.
Fix a bug in open_and_lock_tables_derived() that would
not return ER_LOCK_DEADLOCK in case of a multi-statement
transaction and a failure of lock_tables().
Move assignment of enum_open_table_action to a method of Open_table_context, a new class that maintains information
for backoff actions.
Minor rearrangements of the code.
Remove alloc_mdl_requests() in functions that work with system
tables: instead the patch ensures that callers always initialize
TABLE_LIST argument.
sql/sql_class.cc:
THD::locked_tables_root is no more.
sql/sql_class.h:
THD::locked_tables_root is no more.
Add a declaration for Open_table_context class.
sql/sql_delete.cc:
Update to use the simplified MDL API.
sql/sql_handler.cc:
TABLE_LIST::mdl_request is stored by-value now.
Ensure that mdl_request.ticket is NULL for every request
that is passed into MDL, to satisfy MDL asserts.
@ sql/sql_help.cc
Function open_system_tables_for_read() no longer initializes
mdl_requests.
Move TABLE_LIST::mdl_request initialization closer to
TABLE_LIST initialization.
sql/sql_help.cc:
Function open_system_tables_for_read() no longer initializes
mdl_requests.
Move TABLE_LIST::mdl_request initialization closer to
TABLE_LIST initialization.
sql/sql_insert.cc:
Remove assignment by-value of TABLE_LIST in
TABLEOP_HOOKS. We can't carry over a granted
MDL ticket from one table list to another.
sql/sql_parse.cc:
Change alloc_mdl_requests() -> init_mdl_requests().
@todo We can remove init_mdl_requests() altogether
in some places: all places that call add_table_to_list()
already have mdl requests initialized.
sql/sql_plugin.cc:
Use init_one_table().
THD::locked_tables_root is no more.
sql/sql_servers.cc:
Use init_one_table().
sql/sql_show.cc:
Update acquire_high_priority_shared_lock() to use
TABLE_LIST::mdl_request rather than allocate an own.
Fix get_trigger_table_impl() to use init_one_table(),
check for out of memory, follow the coding style.
sql/sql_table.cc:
Update to work with TABLE_LIST::mdl_request by-value.
Remove lock_table_name_if_not_cached().
The code that used to delegate to it is quite simple and
concise without it now.
sql/sql_udf.cc:
Use init_one_table().
sql/sql_update.cc:
Update to use the new signature of close_tables_for_reopen().
sql/table.cc:
Move re-setting of mdl_requests for prepared statements
and stored procedures from close_thread_tables() to
reinit_stmt_before_use().
Change alloc_mdl_requests() to init_mdl_requests().
init_mdl_requests() is a hack that can't be deleted
until we don't have a list-aware TABLE_LIST constructor.
Hopefully its use will be minimal
sql/table.h:
Change alloc_mdl_requests() to init_mdl_requests()
TABLE_LIST::mdl_request is stored by value.
sql/tztime.cc:
We no longer initialize mdl requests in open_system_tables_for*()
functions. Move this initialization closer to initialization
of the rest of TABLE_LIST members.
storage/myisammrg/ha_myisammrg.cc:
Simplify mdl_request initialization.
2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1
- initial changeset that introduced the fix for
Bug#989 and follow up fixes for all test suite failures
introduced in the initial changeset.
------------------------------------------------------------
revno: 2617.31.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Fri 2009-03-06 19:17:00 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
Currently the MySQL server does not keep metadata locks on
schema objects for the duration of a transaction, thus failing
to guarantee the integrity of the schema objects being used
during the transaction and to protect then from concurrent
DDL operations. This also poses a problem for replication as
a DDL operation might be replicated even thought there are
active transactions using the object being modified.
The solution is to defer the release of metadata locks until
a active transaction is either committed or rolled back. This
prevents other statements from modifying the table for the
entire duration of the transaction. This provides commitment
ordering for guaranteeing serializability across multiple
transactions.
- Incompatible change:
If MySQL's metadata locking system encounters a lock conflict,
the usual schema is to use the try and back-off technique to
avoid deadlocks -- this schema consists in releasing all locks
and trying to acquire them all in one go.
But in a transactional context this algorithm can't be utilized
as its not possible to release locks acquired during the course
of the transaction without breaking the transaction commitments.
To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be
returned if a lock conflict is encountered during a transaction.
Let's consider an example:
A transaction has two statements that modify table t1, then table
t2, and then commits. The first statement of the transaction will
acquire a shared metadata lock on table t1, and it will be kept
utill COMMIT to ensure serializability.
At the moment when the second statement attempts to acquire a
shared metadata lock on t2, a concurrent ALTER or DROP statement
might have locked t2 exclusively. The prescription of the current
locking protocol is that the acquirer of the shared lock backs off
-- gives up all his current locks and retries. This implies that
the entire multi-statement transaction has to be rolled back.
- Incompatible change:
FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ
LOCK won't cause locked tables to be implicitly unlocked anymore.
mysql-test/extra/binlog_tests/drop_table.test:
Add test case for Bug#989.
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/include/mix1.inc:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/include/mix2.inc:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/r/flush_block_commit.result:
Update test case result (WL#4284).
mysql-test/r/flush_block_commit_notembedded.result:
Update test case result (WL#4284).
mysql-test/r/innodb.result:
Update test case result (WL#4284).
mysql-test/r/innodb_mysql.result:
Update test case result (WL#4284).
mysql-test/r/lock.result:
Add test case result for an effect of WL#4284/Bug#989
(all locks should be released when a connection terminates).
mysql-test/r/mix2_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/not_embedded_server.result:
Update test case result (effects of WL#4284/Bug#989).
Add a test case for interaction of WL#4284 and FLUSH PRIVILEGES.
mysql-test/r/partition_innodb_semi_consistent.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/partition_sync.result:
Temporarily disable the test case for Bug#43867,
which will be fixed by a subsequent backport.
mysql-test/r/ps.result:
Add a test case for effect of PREPARE on transactional
locks: we take a savepoint at beginning of PREAPRE
and release it at the end. Thus PREPARE does not
accumulate metadata locks (Bug#989/WL#4284).
mysql-test/r/read_only_innodb.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_drop_tbl.result:
Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_drop_tbl.result:
Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_unsafe.result:
A side effect of Bug#989 -- slightly different table map ids.
mysql-test/suite/binlog/t/binlog_row_drop_tbl.test:
Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_drop_tbl.test:
Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_row.test:
Update to the new state name. This
is actually a follow up to another patch for WL#4284,
that changes Locked thread state to Table lock.
mysql-test/suite/ndb/r/ndb_index_ordered.result:
Remove result for disabled part of the test case.
mysql-test/suite/ndb/t/disabled.def:
Temporarily disable a test case (Bug#45621).
mysql-test/suite/ndb/t/ndb_index_ordered.test:
Disable a part of a test case (needs update to
reflect semantics of Bug#989).
mysql-test/suite/rpl/t/disabled.def:
Disable tests made meaningless by transactional metadata
locking.
mysql-test/suite/sys_vars/r/autocommit_func.result:
Add a commit (Bug#989).
mysql-test/suite/sys_vars/t/autocommit_func.test:
Add a commit (Bug#989).
mysql-test/t/flush_block_commit.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/t/flush_block_commit_notembedded.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
Add a test case for transaction-scope locks and the global
read lock (Bug#989/WL#4284).
mysql-test/t/innodb.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(effects of Bug#989/WL#4284).
mysql-test/t/lock.test:
Add a test case for Bug#989/WL#4284.
mysql-test/t/not_embedded_server.test:
Add a test case for Bug#989/WL#4284.
mysql-test/t/partition_innodb_semi_consistent.test:
Replace TRUNCATE with DELETE, to not issue
an implicit commit of a transaction, and not depend
on metadata locks.
mysql-test/t/partition_sync.test:
Temporarily disable the test case for Bug#43867,
which needs a fix to be backported from 6.0.
mysql-test/t/ps.test:
Add a test case for semantics of PREPARE and transaction-scope
locks: metadata locks on tables used in PREPARE are enclosed into a
temporary savepoint, taken at the beginning of PREPARE,
and released at the end. Thus PREPARE does not effect
what locks a transaction owns.
mysql-test/t/read_only_innodb.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(Bug#989/WL#4284).
Wait for the read_only statement to actually flush tables before
sending other concurrent statements that depend on its state.
mysql-test/t/xa.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(Bug#989/WL#4284).
sql/ha_ndbcluster_binlog.cc:
Backport bits of changes of ha_ndbcluster_binlog.cc
from 6.0, to fix the failing binlog test suite with
WL#4284. WL#4284 implementation does not work
with 5.1 implementation of ndbcluster binlog index.
sql/log_event.cc:
Release metadata locks after issuing a commit.
sql/mdl.cc:
Style changes (WL#4284).
sql/mysql_priv.h:
Rename parameter to match the name used in the definition (WL#4284).
sql/rpl_injector.cc:
Release metadata locks on commit (WL#4284).
sql/rpl_rli.cc:
Remove assert made meaningless, metadata locks are released
at the end of the transaction.
sql/set_var.cc:
Close tables and release locks if autocommit mode is set.
sql/slave.cc:
Release metadata locks after a rollback.
sql/sql_acl.cc:
Don't implicitly unlock locked tables. Issue a implicit commit
at the end and unlock tables.
sql/sql_base.cc:
Defer the release of metadata locks when closing tables
if not required to.
Issue a deadlock error if the locking protocol requires
that a transaction re-acquire its locks.
Release metadata locks when closing tables for reopen.
sql/sql_class.cc:
Release metadata locks if the thread is killed.
sql/sql_parse.cc:
Release metadata locks after implicitly committing a active
transaction, or after explicit commits or rollbacks.
sql/sql_plugin.cc:
Allocate MDL request on the stack as the use of the table
is contained within the function. It will be removed from
the context once close_thread_tables is called at the end
of the function.
sql/sql_prepare.cc:
The problem is that the prepare phase of the CREATE TABLE
statement takes a exclusive metadata lock lock and this can
cause a self-deadlock the thread already holds a shared lock
on the table being that should be created.
The solution is to make the prepare phase take a shared
metadata lock when preparing a CREATE TABLE statement. The
execution of the statement will still acquire a exclusive
lock, but won't cause any problem as it issues a implicit
commit.
After some discussions with stakeholders it has been decided that
metadata locks acquired during a PREPARE statement must be released
once the statement is prepared even if it is prepared within a multi
statement transaction.
sql/sql_servers.cc:
Don't implicitly unlock locked tables. Issue a implicit commit
at the end and unlock tables.
sql/sql_table.cc:
Close table and release metadata locks after a admin operation.
sql/table.h:
The problem is that the prepare phase of the CREATE TABLE
statement takes a exclusive metadata lock lock and this can
cause a self-deadlock the thread already holds a shared lock
on the table being that should be created.
The solution is to make the prepare phase take a shared
metadata lock when preparing a CREATE TABLE statement. The
execution of the statement will still acquire a exclusive
lock, but won't cause any problem as it issues a implicit
commit.
sql/transaction.cc:
Release metadata locks after the implicitly committed due
to a new transaction being started. Also, release metadata
locks acquired after a savepoint if the transaction is rolled
back to the save point.
The problem is that in some cases transaction-long metadata locks
could be released before the transaction was committed. This could
happen when a active transaction was ended by a "START TRANSACTION"
or "BEGIN" statement, in which case the metadata locks would be
released before the actual commit of the active transaction.
The solution is to defer the release of metadata locks to after the
transaction has been implicitly committed. No test case is provided
as the effort to provide one is too disproportional to the size of
the fix.
------------------------------------------------------------
revno: 2617.23.18
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Mon 2009-03-02 18:18:26 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
This is a prerequisite patch:
These changes are intended to split lock requests from granted
locks and to allow the memory and lifetime of granted locks to
be managed within the MDL subsystem. Furthermore, tickets can
now be shared and therefore are used to satisfy multiple lock
requests, but only shared locks can be recursive.
The problem is that the MDL subsystem morphs lock requests into
granted locks locks but does not manage the memory and lifetime
of lock requests, and hence, does not manage the memory of
granted locks either. This can be problematic because it puts the
burden of tracking references on the users of the subsystem and
it can't be easily done in transactional contexts where the locks
have to be kept around for the duration of a transaction.
Another issue is that recursive locks (when the context trying to
acquire a lock already holds a lock on the same object) requires
that each time the lock is granted, a unique lock request/granted
lock structure structure must be kept around until the lock is
released. This can lead to memory leaks in transactional contexts
as locks taken during the transaction should only be released at
the end of the transaction. This also leads to unnecessary wake
ups (broadcasts) in the MDL subsystem if the context still holds
a equivalent of the lock being released.
These issues are exacerbated due to the fact that WL#4284 low-level
design says that the implementation should "2) Store metadata locks
in transaction memory root, rather than statement memory root" but
this is not possible because a memory root, as implemented in mysys,
requires all objects allocated from it to be freed all at once.
This patch combines review input and significant code contributions
from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev).
mysql-test/r/mdl_sync.result:
Add test case result.
mysql-test/t/mdl_sync.test:
Add test case for shared lock upgrade case.
sql/event_db_repository.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/ha_ndbcluster_binlog.cc:
Use new function names to initialize MDL lock requests.
sql/lock.cc:
Rename MDL functions.
sql/log_event.cc:
The MDL request now holds the table and database name data (MDL_KEY).
sql/mdl.cc:
Move the MDL key to the MDL_LOCK structure in order to make the
object suitable for allocation from a fixed-size allocator. This
allows the simplification of the lists in the MDL_LOCK object,
which now are just two, one for granted tickets and other for
waiting (upgraders) tickets.
Recursive requests for a shared lock on the same object can now
be granted using the same lock ticket. This schema is only used
for shared locks because that the only case that matters. This
is used to avoid waste of resources in case a context (connection)
already holds a shared lock on a object.
sql/mdl.h:
Introduce a metadata lock object key which is used to uniquely
identify lock objects.
Separate the structure used to represent pending lock requests
from the structure used to represent granted metadata locks.
Rename functions used to manipulate locks requests in order to
have a more consistent function naming schema.
sql/sp_head.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/sql_acl.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_base.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_class.h:
Last acquired lock before the savepoint was set.
sql/sql_delete.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_handler.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_insert.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_parse.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_plist.h:
Typedef for iterator type.
sql/sql_plugin.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_servers.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_show.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_table.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_trigger.cc:
Save reference to the lock ticket so it can be downgraded later.
sql/sql_udf.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/table.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/table.h:
Separate MDL lock requests from lock tickets (granted locks).
storage/myisammrg/ha_myisammrg.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
Backport of:
------------------------------------------------------------
revno: 2630.4.1
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Fri 2008-05-23 17:54:03 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
------------------------------------------------------------
This is the first patch in series. It transforms the metadata
locking subsystem to use a dedicated module (mdl.h,cc). No
significant changes in the locking protocol.
The import passes the test suite with the exception of
deprecated/removed 6.0 features, and MERGE tables. The latter
are subject to a fix by WL#4144.
Unfortunately, the original changeset comments got lost in a merge,
thus this import has its own (largely insufficient) comments.
This patch fixes Bug#25144 "replication / binlog with view breaks".
Warning: this patch introduces an incompatible change:
Under LOCK TABLES, it's no longer possible to FLUSH a table that
was not locked for WRITE.
Under LOCK TABLES, it's no longer possible to DROP a table or
VIEW that was not locked for WRITE.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.2
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:03:45 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.3
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:08:51 +0400
message:
WL#3726 "DDL locking for all metadata objects"
Fixed failing Windows builds by adding mdl.cc to the lists
of files needed to build server/libmysqld on Windows.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.4
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 21:57:58 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Fix for assert failures in kill.test which occured when one
tried to kill ALTER TABLE statement on merge table while it
was waiting in wait_while_table_is_used() for other connections
to close this table.
These assert failures stemmed from the fact that cleanup code
in this case assumed that temporary table representing new
version of table was open with adding to THD::temporary_tables
list while code which were opening this temporary table wasn't
always fulfilling this.
This patch changes code that opens new version of table to
always do this linking in. It also streamlines cleanup process
for cases when error occurs while we have new version of table
open.
******
WL#3726 "DDL locking for all metadata objects"
Add libmysqld/mdl.cc to .bzrignore.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.6
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sun 2008-05-25 00:33:22 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Addition to the fix of assert failures in kill.test caused by
changes for this worklog.
Make sure we close the new table only once.
.bzrignore:
Add libmysqld/mdl.cc
libmysqld/CMakeLists.txt:
Added mdl.cc to the list of files needed for building of libmysqld.
libmysqld/Makefile.am:
Added files implementing new meta-data locking subsystem to the server.
mysql-test/include/handler.inc:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/create.result:
Adjusted test case after change in implementation of CREATE TABLE
... SELECT. We no longer have special check in open_table() which
catches the case when we select from the table created. Instead we
rely on unique_table() call which happens after opening and
locking all tables.
mysql-test/r/flush.result:
FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
TABLES. Updated test accordingly.
mysql-test/r/flush_table.result:
Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
locked for read. Updated test accordingly.
mysql-test/r/handler_innodb.result:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/handler_myisam.result:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/information_schema.result:
Additional test for WL#3726 "DDL locking for all metadata
objects". Check that we use high-priority metadata lock requests
when filling I_S tables.
Rearrange tests to match 6.0 better (fewer merge conflicts).
mysql-test/r/kill.result:
Added tests checking that DDL and DML statements waiting for
metadata locks can be interrupted by KILL command.
mysql-test/r/lock.result:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation write locks on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/r/partition_column_prune.result:
Update results (same results in 6.0), WL#3726
mysql-test/r/partition_pruning.result:
Update results (same results in 6.0), WL#3726
mysql-test/r/ps_ddl.result:
We no longer invalidate prepared CREATE TABLE ... SELECT statement
if target table changes. This is OK since it is not strictly
necessary.
The first change is wrong, is caused by FLUSH TABLE
now flushing all unused tables. This is a regression that
Dmitri fixed in 6.0 in a follow up patch.
mysql-test/r/sp.result:
Under LOCK TABLES we no longer allow accessing views which were
not explicitly locked. To access view we need to obtain metadata
lock on it and doing this under LOCK TABLES may lead to deadlocks.
mysql-test/r/view.result:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation even "write locks" on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock
mysql-test/r/view_grant.result:
ALTER VIEW implementation was changed to open a view only after
checking that user which does alter has appropriate privileges on
it. This means that in case when user's privileges are
insufficient for this we won't check that new view definer is the
same as original one or user performing alter has SUPER privilege.
Adjusted test case accordingly.
mysql-test/r/view_multi.result:
Added test case for bug#25144 "replication / binlog with view
breaks".
mysql-test/suite/rpl/t/disabled.def:
Disable test for deprecated features (they don't work with new MDL).
mysql-test/t/create.test:
Adjusted test case after change in implementation of CREATE TABLE
... SELECT. We no longer have special check in open_table() which
catches the case when we select from the table created. Instead we
rely on unique_table() call which happens after opening and
locking all tables.
mysql-test/t/disabled.def:
Disable merge.test, subject of WL#4144
mysql-test/t/flush.test:
FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
TABLES. Updated test accordingly.
mysql-test/t/flush_table.test:
Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
locked for read. Updated test accordingly.
mysql-test/t/information_schema.test:
Additional test for WL#3726 "DDL locking for all metadata
objects". Check that we use high-priority metadata lock requests
when filling I_S tables.
Rearrange the results for easier merges with 6.0.
mysql-test/t/kill.test:
Added tests checking that DDL and DML statements waiting for
metadata locks can be interrupted by KILL command.
mysql-test/t/lock.test:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation write locks on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/t/lock_multi.test:
Adjusted test case to the changes of status in various places
caused by change in implementation FLUSH TABLES WITH READ LOCK,
which is now takes global metadata lock before flushing tables and
therefore waits on at these places.
mysql-test/t/ps_ddl.test:
We no longer invalidate prepared CREATE TABLE ... SELECT statement
if target table changes. This is OK since it is not strictly
necessary.
The first change is wrong, is caused by FLUSH TABLE
now flushing all unused tables. This is a regression that
Dmitri fixed in 6.0 in a follow up patch.
mysql-test/t/sp.test:
Under LOCK TABLES we no longer allow accessing views which were
not explicitly locked. To access view we need to obtain metadata
lock on it and doing this under LOCK TABLES may lead to deadlocks.
mysql-test/t/trigger_notembedded.test:
Adjusted test case to the changes of status in various places
caused by change in implementation FLUSH TABLES WITH READ LOCK,
which is now takes global metadata lock before flushing tables and
therefore waits on at these places.
mysql-test/t/view.test:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation even "write locks" on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/t/view_grant.test:
ALTER VIEW implementation was changed to open a view only after
checking that user which does alter has appropriate privileges on
it. This means that in case when user's privileges are
insufficient for this we won't check that new view definer is the
same as original one or user performing alter has SUPER privilege.
Adjusted test case accordingly.
mysql-test/t/view_multi.test:
Added test case for bug#25144 "replication / binlog with view
breaks".
sql/CMakeLists.txt:
Added mdl.cc to the list of files needed for building of server.
sql/Makefile.am:
Added files implementing new meta-data locking subsystem to the
server.
sql/event_db_repository.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when TABLE_LIST objects is also allocated
there or on stack.
sql/ha_ndbcluster.cc:
Adjusted code to work nicely with new metadata locking subsystem.
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/ha_ndbcluster_binlog.cc:
Adjusted code to work with new metadata locking subsystem.
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/handler.cc:
update_frm_version():
Directly update TABLE_SHARE::mysql_version member instead of
going through all TABLE instances for this table (old code was a
legacy from pre-table-definition-cache days).
sql/lock.cc:
Use new metadata locking subsystem. Threw away most of functions
related to name locking as now one is supposed to use metadata
locking API instead. In lock_global_read_lock() and
unlock_global_read_lock() in order to avoid problems with global
read lock sneaking in at the moment when we perform FLUSH TABLES
or ALTER TABLE under LOCK TABLES and when tables being reopened
are protected only by metadata locks we also have to take global
shared meta data lock.
sql/log_event.cc:
Adjusted code to work with new metadata locking subsystem. For
tables open by slave thread for applying RBR events allocate
memory for lock request object in the same chunk of memory as
TABLE_LIST objects for them. In order to ensure that we keep these
objects around until tables are open always close tables before
calling Relay_log_info::clear_tables_to_lock(). Use new auxiliary
Relay_log_info::slave_close_thread_tables() method to enforce
this.
sql/log_event_old.cc:
Adjusted code to work with new metadata locking subsystem. Since
for tables open by slave thread for applying RBR events memory for
lock request object is allocated in the same chunk of memory as
TABLE_LIST objects for them we have to ensure that we keep these
objects around until tables are open. To ensure this we always
close tables before calling
Relay_log_info::clear_tables_to_lock(). To enfore this we use
new auxiliary Relay_log_info::slave_close_thread_tables()
method.
sql/mdl.cc:
Implemented new metadata locking subsystem and API described in
WL3726 "DDL locking for all metadata objects".
sql/mdl.h:
Implemented new metadata locking subsystem and API described in
WL3726 "DDL locking for all metadata objects".
sql/mysql_priv.h:
- close_thread_tables()/close_tables_for_reopen() now has one more
argument which indicates that metadata locks should be released
but not removed from the context in order to be used later in
mdl_wait_for_locks() and tdc_wait_for_old_version().
- close_cached_table() routine is no longer public.
- Thread waiting in wait_while_table_is_used() can be now killed
so this function returns boolean to make caller aware of such
situation.
- We no longer have table cache as separate entity instead used
and unused TABLE instances are linked to TABLE_SHARE objects in
table definition cache.
- Now third argument of open_table() is also used for requesting
table repair or auto-discovery of table's new definition. So its
type was changed from bool to enum.
- Added tdc_open_view() function for opening view by getting its
definition from disk (and table cache in future).
- reopen_name_locked_table() no longer needs "link_in" argument as
now we have exclusive metadata locks instead of dummy TABLE
instances when this function is called.
- find_locked_table() now takes head of list of TABLE instances
instead of always scanning through THD::open_tables list. Also
added find_write_locked_table() auxiliary.
- reopen_tables(), close_cached_tables() no longer have
mark_share_as_old and wait_for_placeholder arguments. Instead of
relying on this parameters and related behavior FLUSH TABLES
WITH READ LOCK now takes global shared metadata lock.
- We no longer need drop_locked_tables() and
abort_locked_tables().
- mysql_ha_rm_tables() now always assume that LOCK_open is not
acquired by caller.
- Added notify_thread_having_shared_lock() callback invoked by
metadata locking subsystem when acquiring an exclusive lock, for
each thread that has a conflicting shared metadata lock.
- Introduced expel_table_from_cache() as replacement for
remove_table_from_cache() (the main difference is that this new
function assumes that caller follows metadata locking protocol
and never waits).
- Threw away most of functions related to name locking. One should
use new metadata locking subsystem and API instead.
sql/mysqld.cc:
Got rid of call initializing/deinitializing table cache since now
it is embedded into table definition cache. Added calls for
initializing/ deinitializing metadata locking subsystem.
sql/rpl_rli.cc:
Introduced auxiliary Relay_log_info::slave_close_thread_tables()
method which is used for enforcing that we always close tables
open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
objects for them.
sql/rpl_rli.h:
Introduced auxiliary Relay_log_info::slave_close_thread_tables()
method which is used for enforcing that we always close tables
open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
objects for them.
sql/set_var.cc:
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/sp_head.cc:
For tables added to the statement's table list by prelocking
algorithm we allocate these objects either on the same memory as
corresponding table list elements or on THD::locked_tables_root
(if we are building table list for LOCK TABLES).
sql/sql_acl.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables. Got rid of redundant code by using unlock_locked_tables()
function.
sql/sql_base.cc:
Changed code to use new MDL subsystem. Got rid of separate table
cache. Now used and unused TABLE instances are linked to the
TABLE_SHAREs in table definition cache.
check_unused():
Adjusted code to the fact that we no longer have separate table
cache. Removed dead code.
table_def_free():
Free TABLE instances referenced from TABLE_SHARE objects before
destroying table definition cache.
get_table_share():
Added assert which ensures that noone will be able to access
table (and its share) without acquiring some kind of metadata
lock first.
close_handle_and_leave_table_as_lock():
Adjusted code to the fact that TABLE instances now are linked to
list in TABLE_SHARE.
list_open_tables():
Changed this function to use table definition cache instead of
table cache.
free_cache_entry():
Unlink freed TABLE elements from the list of all TABLE instances
for the table in TABLE_SHARE.
kill_delayed_thread_for_table():
Added auxiliary for killing delayed insert threads for
particular table.
close_cached_tables():
Got rid of wait_for_refresh argument as we now rely on global
shared metadata lock to prevent FLUSH WITH READ LOCK sneaking in
when we are reopening tables. Heavily reworked this function to
use new MDL code and not to rely on separate table cache entity.
close_open_tables():
We no longer have separate table cache.
close_thread_tables():
Release metadata locks after closing all tables. Added skip_mdl
argument which allows us not to remove metadata lock requests
from the context in case when we are going to use this requests
later in mdl_wait_for_locks() and tdc_wait_for_old_versions().
close_thread_table()/close_table_for_reopen():
Since we no longer have separate table cache and all TABLE
instances are linked to TABLE_SHARE objects in table definition
cache we have to link/unlink TABLE object to/from appropriate
lists in the share.
name_lock_locked_table():
Moved redundant code to find_write_locked_table() function and
adjusted code to the fact that wait_while_table_is_used() can
now return with an error if our thread is killed.
reopen_table_entry():
We no longer need "link_in" argument as with MDL we no longer
call this function with dummy TABLE object pre-allocated and
added to the THD::open_tables. Also now we add newly-open TABLE
instance to the list of share's used TABLE instances.
table_cache_insert_placeholder():
Got rid of name-locking legacy.
lock_table_name_if_not_cached():
Moved to sql_table.cc the only place where it is used. It was
also reimplemented using new MDL API.
open_table():
- Reworked this function to use new MDL subsystem.
- Changed code to deal with table definition cache directly
instead of going through separate table cache.
- Now third argument is also used for requesting table repair
or auto-discovery of table's new definition. So its type was
changed from bool to enum.
find_locked_table()/find_write_locked_table():
Accept head of list of TABLE objects as first argument and use
this list instead of always searching in THD::open_tables list.
Also added auxiliary for finding write-locked locked tables.
reopen_table():
Adjusted function to work with new MDL subsystem and to properly
manuipulate with lists of used/unused TABLE instaces in
TABLE_SHARE.
reopen_tables():
Removed mark_share_as_old parameter. Instead of relying on it
and related behavior FLUSH TABLES WITH READ LOCK now takes
global shared metadata lock. Changed code after removing
separate table cache.
drop_locked_tables()/abort_locked_tables():
Got rid of functions which are no longer needed.
unlock_locked_tables():
Moved this function from sql_parse.cc and changed it to release
memory which was used for allocating metadata lock requests for
tables open and locked by LOCK TABLES.
tdc_open_view():
Intoduced function for opening a view by getting its definition
from disk (and table cache in future).
reopen_table_entry():
Introduced function for opening table definitions while holding
exclusive metatadata lock on it.
open_unireg_entry():
Got rid of this function. Most of its functionality is relocated
to open_table() and open_table_fini() functions, and some of it
to reopen_table_entry() and tdc_open_view(). Also code
resposible for auto-repair and auto-discovery of tables was
moved to separate function.
open_table_entry_fini():
Introduced function which contains common actions which finalize
process of TABLE object creation.
auto_repair_table():
Moved code responsible for auto-repair of table being opened
here.
handle_failed_open_table_attempt()
Moved code responsible for handling failing attempt to open
table to one place (retry due to lock conflict/old version,
auto-discovery and repair).
open_tables():
- Flush open HANDLER tables if they have old version of if there
is conflicting metadata lock against them (before this moment
we had this code in open_table()).
- When we open view which should be processed via derived table
on the second execution of prepared statement or stored
routine we still should call open_table() for it in order to
obtain metadata lock on it and prepare its security context.
- In cases when we discover that some special handling of
failure to open table is needed call
handle_failed_open_table_attempt() which handles all such
scenarios.
open_ltable():
Handling of various special scenarios of failure to open a table
was moved to separate handle_failed_open_table_attempt()
function.
remove_db_from_cache():
Removed this function as it is no longer used.
notify_thread_having_shared_lock():
Added callback which is invoked by MDL subsystem when acquiring
an exclusive lock, for each thread that has a conflicting shared
metadata lock.
expel_table_from_cache():
Introduced function for removing unused TABLE instances. Unlike
remove_table_from_cache() it relies on caller following MDL
protocol and having appropriate locks when calling it and thus
does not do any waiting if table is still in use.
tdc_wait_for_old_version():
Added function which allows open_tables() to wait in cases when
we discover that we should back-off due to presence of old
version of table.
abort_and_upgrade_lock():
Use new MDL calls.
mysql_wait_completed_table():
Got rid of unused function.
open_system_tables_for_read/for_update()/performance_schema_table():
Allocate MDL_LOCK objects on execution memory root in cases when
TABLE_LIST objects for corresponding tables is allocated on
stack.
close_performance_schema_table():
Release metadata locks after closing tables.
******
Use I_P_List for free/used tables list in the table share.
sql/sql_binlog.cc:
Use Relay_log_info::slave_close_thread_tables() method to enforce
that we always close tables open for RBR before deallocating
TABLE_LIST elements and MDL_LOCK objects for them.
sql/sql_class.cc:
Added meta-data locking contexts as part of Open_tables_state
context. Also introduced THD::locked_tables_root memory root
which is to be used for allocating MDL_LOCK objects for tables in
LOCK TABLES statement (end of lifetime for such objects is UNLOCK
TABLES so we can't use statement or execution root for them).
sql/sql_class.h:
Added meta-data locking contexts as part of Open_tables_state
context. Also introduced THD::locked_tables_root memory root
which is to be used for allocating MDL_LOCK objects for tables in
LOCK TABLES statement (end of lifetime for such objects is UNLOCK
TABLES so we can't use statement or execution root for them).
Note: handler_mdl_context and locked_tables_root and
mdl_el_root will be removed by subsequent patches.
sql/sql_db.cc:
mysql_rm_db() does not really need to call remove_db_from_cache()
as it drops each table in the database using
mysql_rm_table_part2(), which performs all necessary operations on
table (definition) cache.
sql/sql_delete.cc:
Use the new metadata locking API for TRUNCATE.
sql/sql_handler.cc:
Changed HANDLER implementation to use new metadata locking
subsystem. Note that MDL_LOCK objects for HANDLER tables are
allocated in the same chunk of heap memory as TABLE_LIST object
for those tables.
sql/sql_insert.cc:
mysql_insert():
find_locked_table() now takes head of list of TABLE object as
its argument instead of always scanning through THD::open_tables
list.
handle_delayed_insert():
Allocate metadata lock request object for table open by delayed
insert thread on execution memroot. create_table_from_items():
We no longer allocate dummy TABLE objects for tables being
created if they don't exist. As consequence
reopen_name_locked_table() no longer has link_in argument.
open_table() now has one more argument which is not relevant for
temporary tables.
sql/sql_parse.cc:
- Moved unlock_locked_tables() routine to sql_base.cc and made
available it in other files. Got rid of some redundant code by
using this function.
- Replaced boolean TABLE_LIST::create member with enum
open_table_type member.
- Use special memory root for allocating MDL_LOCK objects for
tables open and locked by LOCK TABLES (these object should live
till UNLOCK TABLES so we can't allocate them on statement nor
execution memory root). Also properly set metadata lock
upgradability attribure for those tables.
- Under LOCK TABLES it is no longer allowed to flush tables which
are not write-locked as this breaks metadata locking protocol
and thus potentially might lead to deadlock.
- Added auxiliary adjust_mdl_locks_upgradability() function.
sql/sql_partition.cc:
Adjusted code to the fact that reopen_tables() no longer has
"mark_share_as_old" argument. Got rid of comments which are no
longer true.
sql/sql_plist.h:
Added I_P_List template class for parametrized intrusive doubly
linked lists and I_P_List_iterator for corresponding iterator.
Unlike for I_List<> list elements of such list can participate in
several lists. Unlike List<> such lists are doubly-linked and
intrusive.
sql/sql_plugin.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables.
sql/sql_prepare.cc:
Replaced boolean TABLE_LIST::create member with enum
open_table_type member. This allows easily handle situation in
which instead of opening the table we want only to take exclusive
metadata lock on it.
sql/sql_rename.cc:
Use new metadata locking subsystem in implementation of RENAME
TABLE.
sql/sql_servers.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables. Got rid of redundant code by using unlock_locked_tables()
function.
sql/sql_show.cc:
Acquire shared metadata lock when we are getting information for
I_S table directly from TABLE_SHARE without doing full-blown table
open. We use high priority lock request in this situation in
order to avoid deadlocks.
Also allocate metadata lock requests objects (MDL_LOCK) on
execution memory root in cases when TABLE_LIST objects are also
allocated there
sql/sql_table.cc:
mysql_rm_table():
Removed comment which is no longer relevant.
mysql_rm_table_part2():
Now caller of mysql_ha_rm_tables() should not own LOCK_open.
Adjusted code to use new metadata locking subsystem instead of
name-locks.
lock_table_name_if_not_cached():
Moved this function from sql_base.cc to this file and
reimplemented it using metadata locking API.
mysql_create_table():
Adjusted code to use new MDL API.
wait_while_table_is_used():
Changed function to use new MDL subsystem. Made thread waiting
in it killable (this also led to introduction of return value so
caller can distinguish successful executions from situations
when waiting was aborted).
close_cached_tables():
Thread waiting in this function is killable now. As result it
has return value for distinguishing between succes and failure.
Got rid of redundant boradcast_refresh() call.
prepare_for_repair():
Use MDL subsystem instead of name-locks.
mysql_admin_table():
mysql_ha_rm_tables() now always assumes that caller doesn't own
LOCK_open.
mysql_repair_table():
We should mark all elements of table list as requiring
upgradable metadata locks.
mysql_create_table_like():
Use new MDL subsystem instead of name-locks.
create_temporary_tables():
We don't need to obtain metadata locks when creating temporary
table.
mysql_fast_or_online_alter_table():
Thread waiting in wait_while_table_is_used() is now killable.
mysql_alter_table():
Adjusted code to work with new MDL subsystem and to the fact
that threads waiting in what_while_table_is_used() and
close_cached_table() are now killable.
sql/sql_test.cc:
We no longer have separate table cache. TABLE instances are now
associated with/linked to TABLE_SHARE objects in table definition
cache.
sql/sql_trigger.cc:
Adjusted code to work with new metadata locking subsystem. Also
reopen_tables() no longer has mark_share_as_old argument (Instead
of relying on this parameter and related behavior FLUSH TABLES
WITH READ LOCK now takes global shared metadata lock).
sql/sql_udf.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables.
sql/sql_update.cc:
Adjusted code to work with new meta-data locking subsystem.
sql/sql_view.cc:
Added proper meta-data locking to implementations of
CREATE/ALTER/DROP VIEW statements. Now we obtain exclusive
meta-data lock on a view before creating/ changing/dropping it.
This ensures that all concurrent statements that use this view
will finish before our statement will proceed and therefore we
will get correct order of statements in the binary log.
Also ensure that TABLE_LIST::mdl_upgradable attribute is properly
propagated for underlying tables of view.
sql/table.cc:
Added auxiliary alloc_mdl_locks() function for allocating metadata
lock request objects for all elements of table list.
sql/table.h:
TABLE_SHARE:
Got rid of unused members. Introduced members for storing lists
of used and unused TABLE objects for this share.
TABLE:
Added members for linking TABLE objects into per-share lists of
used and unused TABLE instances. Added member for holding
pointer to metadata lock for this table.
TABLE_LIST:
Replaced boolean TABLE_LIST::create member with enum
open_table_type member. This allows easily handle situation in
which instead of opening the table we want only to take
exclusive meta-data lock on it (we need this in order to handle
ALTER VIEW and CREATE VIEW statements).
Introduced new mdl_upgradable member for marking elements of
table list for which we need to take upgradable shared metadata
lock instead of plain shared metadata lock. Added pointer for
holding pointer to MDL_LOCK for the table.
Added auxiliary alloc_mdl_locks() function for allocating metadata
lock requests objects for all elements of table list. Added
auxiliary set_all_mdl_upgradable() function for marking all
elements in table list as requiring upgradable metadata locks.
storage/myisammrg/ha_myisammrg.cc:
Allocate MDL_LOCK objects for underlying tables of MERGE table.
To be reworked once Ingo pushes his patch for WL4144.
port from mysql-next (5.4) to mysql-next-mr-bugfixing (5.5/5.6?)
2755 Konstantin Osipov 2008-11-27
Bug#32115 will remove the pre-requisite to initialize LEX to open tables.
This dependency was added in 5.1 and was supposed to be removed in 6.0.
Remove asserts and initialization of LEX in places where we don't deal
with partitioned tables.
sql/sp_head.cc:
Fix a typo.
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
