mysqldump --routine fails to dump databases containing backslash ("\")
character. This happened because escaped database name was being used as an
identifier while changing current database. Such identifers are not supposed
to be escaped, they must be properly quoted instead.
This includes fixing all utilities to not have any memory leaks,
as safemalloc warnings stopped tests from passing on MacOSX.
- Ensure that all clients takes character-set-dir, as the
libmysqlclient library will use it.
- mysql-test-run now passes character-set-dir to all external clients.
- Changed dynstr_free() so that it can be called twice (made freeing code easier)
- Changed rpl_global_gtid_slave_state to be allocated dynamicly as it
includes a mutex that needs to be initizlied/destroyed before my_end() is called.
- Removed rpl_slave_state::init() and rpl_slave_stage::deinit() as
their job are better handling by constructor and delete.
- Print alias instead of table_name in check_duplicate_key as
table_name may have been converted to lower case.
Other things:
- Fixed a case in time_to_datetime_with_warn() where we where
using && instead of & in tests
While querying INFORMATION SCHEMA, check for a table's engine
only used table name, but not schema name; so, if there were different
rows with the same table name, a wrong one could be retrieved.
The result of the check affected the decision whether the contents
of the table should be dumped, and whether a DELAYED option can be used.
Fixed by adding a clause for table_schema to the query.
IMPLEMENTED IN ALL CLIENT PROGRAMS
Description: Option "enable-cleartext-plugin" is not
available for the following client utilities:-
mysqldump
mysqlimport
mysqlshow
mysqlcheck
Analysis: The unavailability of this option limits the
features like PAM authentication from using the above
mentioned utilities.
Fix: Option "enable-cleartext-plugin" is implemented in the
above mentioned client utilities.
mysqldump now attempts to make use of the INFORMATION_SCHEMA tables.
If the table name is not found with a case sensitive search, it
fallbacks to a case insensitive search.
Removed 'galera-sst-mode' option from mysqldump and added logic
in wsrep_sst_mysqldump script to retrieve gtid_binlog_state from
donor node and send it to the joiner node.
Merged lp:maria/maria-10.0-galera up to revision 3880.
Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
Merged lp:maria/maria-10.0-galera up to revision 3879.
Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
mysqldump:
- Added --log-queries to allow one to disable logging for the dump
sql/log_event.cc:
- Removed setting of enable_slow_log as it's not required anymore.
sql/sql_parse.cc:
- Set enable_slow_log to value of thd->variables.sql_log_slow as this will speed up tests if slow log is disabled.
- opt_log_slow_admin_statements can now only disable slow log, not enable it.
sql/sql_explain.cc:
- Minor cleanup
Other things:
- Added sql_log_slow to system variables.
- Changed opt_slow_log to global_system_variables.sql_log_slow in all files
- Updated tests to reflect changes
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
A new command line option "galera-sst-mode" was introduced
in mysqldump as part of fix for MDEV-6316. But, since the
fix was pushed to maria-10.0-galera branch, the mysqldump
tool supplied with mariadb client deb/rpm packages, does not
have this new opion.
This fix contains the same patch along with a test case.
Mysqldump overflows stack buffer when copying table name from commandline arguments resulting in stack corruption and ability to execute arbitrary code.
Fix: Check length of all positional arguments passed to mysqldump is smaller than NAME_LEN.
Note: Mysqldump heavily depends on that database objects (databases, tablespaces, tables, etc) are limited to small size (now it is 64).
Mysqldump overflows stack buffer when copying table name from commandline arguments resulting in stack corruption and ability to execute arbitrary code.
Fix: Check length of all positional arguments passed to mysqldump is smaller than NAME_LEN.
Note: Mysqldump heavily depends on that database objects (databases, tablespaces, tables, etc) are limited to small size (now it is 64).
Mysqldump overflows stack buffer when copying table name from commandline arguments resulting in stack corruption and ability to execute arbitrary code.
Fix: Check length of all positional arguments passed to mysqldump is smaller than NAME_LEN.
Note: Mysqldump heavily depends on that database objects (databases, tablespaces, tables, etc) are limited to small size (now it is 64).
MDEV-6344: mysqldump issues FLUSH TABLES, which gets written into binlog and replicated
Add a --gtid option (for compatibility, the original behaviour is preserved
when --gtid is not used).
With --gtid, --master-data and --dump-slave output the GTID position (the
old-style file/offset position is still output, but commented out). Also, a
CHANGE MASTER TO master_use_gtid=slave_pos is output to ensure a provisioned
slave is configured in GTID, as requested.
Without --gtid, the GTID position is still output, if available, but commented
out.
Also fix MDEV-6344, to avoid FLUSH TABLES getting into the binlog. Otherwise a
mysqldump on a slave server will silently inject a GTID which does not exist
on the master, which is highly undesirable.
Also fix an incorrect error handling around obtaining binlog position with
--master-data (was probably unlikely to trigger in most cases).
do_start_slave_sql() is called from maybe_exit().
We should not recurse when maybe_exit() is called for an error during do_start_slave_sql().
Also remove a meaningless (but safe) "goto err".
- MariaDB-5.5 part of the fix: since we can't easily fix query optimization for I_S tables,
run the affected-tablespaces query with semijoin=off. It happens to have a good query plan
with that setting.
[This is a forward-port to MariaDB 10.0]
- MariaDB-5.5 part of the fix: since we can't easily fix query optimization for I_S tables,
run the affected-tablespaces query with semijoin=off. It happens to have a good query plan
with that setting.
When running with multi-source enabled master (MariaDB 10.0) all master will be recorded
mysql-test/r/rpl_mysqldump_slave.result:
Updated results to new syntax
mysql_upgrade --help now also prints out --default options and variable values.
mysql_upgrade now prints permission errors.
mysql_upgrade doesn't print some non essential info if --silent is used.
Added handler error message about incompatible versions
Fixed that mysqlbug and mysql_install_db have the executable flag set.
Removed executable flag for some non executable files.
Changed in mysql_install_db askmonty.org to mariadb.com.
Ensured that all client executables prints --default options the same way.
Allow REPAIR ... USE_FRM for old .frm files if the are still compatible.
Extended shown error for storage engine messages.
client/mysql.cc:
print_defaults() should be first (as in all other programs)
client/mysql_upgrade.c:
--help now also prints out --default options and variable values
Print out error if wrong permissions
Don't print info if --silent
client/mysqladmin.cc:
print_defaults() should be first (as in all other programs)
client/mysqlbinlog.cc:
Added print_defaults() to --help
client/mysqlcheck.c:
Added empty line in --help
client/mysqlimport.c:
Added empty line in --help
client/mysqlshow.c:
Made --help compatible
client/mysqlslap.c:
Made --help compatible
client/mysqltest.cc:
Added print_defaults() to --help
include/handler_ername.h:
Added handler error message
include/my_base.h:
Added handler error message
mysql-test/r/mysql_upgrade.result:
Updated results
mysql-test/r/repair.result:
Added test case for better error messages
mysql-test/std_data/host_old.MYD:
Added test case for better error messages
mysql-test/std_data/host_old.MYI:
Added test case for better error messages
mysql-test/std_data/host_old.frm:
Added test case for better error messages
mysql-test/t/repair.test:
Added test case for better error messages
mysys/my_handler_errors.h:
Added handler error message
scripts/CMakeLists.txt:
Fixed that mysqlbug and mysql_install_db have the executable flag set
scripts/mysql_install_db.sh:
askmonty.org -> mariadb.com
sql/ha_partition.cc:
Sometimes table_type() can be called for errors even if partition didn't manage to open any files
sql/handler.cc:
Write clear text for not handled, but defined error messages.
sql/share/errmsg-utf8.txt:
Extended shown error for storage engine messages
sql/sql_admin.cc:
Allow REPAIR ... USE_FRM for old .frm files if the are still compatible
storage/myisam/ha_myisam.cc:
Use new error message
TO DUMP DATA FROM MYSQL-5.6
Analysis
--------
Dumping mysql-5.6 data using mysql-5.1/mysql-5.5 'myqldump'
utility fails with a syntax error.
Server system variable 'sql_quote_show_create' which quotes the
identifiers is set in the mysqldump utility. The mysldump utility
of mysql-5.1/mysql-5.5 uses deprecated syntax 'SET OPTION' to set
the 'sql_quote_show_create' option. The support for the syntax is
removed in mysql-5.6. Hence syntax error is reported while taking
the dump.
Fix:
---
Changed the 'mysqldump' code to use the syntax
'SET SQL_QUOTE_SHOW_CREATE' to set the 'sql_quote_show_create'
option. That syntax is supported on mysql-5.1, mysql-5.5 and
mysql-5.6.
NOTE: I have not added an mtr test case since it is difficult
to simulate the condition. Also the syntax may not be further
simplified in the future.
TO DUMP DATA FROM MYSQL-5.6
Analysis
--------
Dumping mysql-5.6 data using mysql-5.1/mysql-5.5 'myqldump'
utility fails with a syntax error.
Server system variable 'sql_quote_show_create' which quotes the
identifiers is set in the mysqldump utility. The mysldump utility
of mysql-5.1/mysql-5.5 uses deprecated syntax 'SET OPTION' to set
the 'sql_quote_show_create' option. The support for the syntax is
removed in mysql-5.6. Hence syntax error is reported while taking
the dump.
Fix:
---
Changed the 'mysqldump' code to use the syntax
'SET SQL_QUOTE_SHOW_CREATE' to set the 'sql_quote_show_create'
option. That syntax is supported on mysql-5.1, mysql-5.5 and
mysql-5.6.
NOTE: I have not added an mtr test case since it is difficult
to simulate the condition. Also the syntax may not be further
simplified in the future.
MYSQL DB FROM REMOTE 5.0.96 SERVER
Problem: mysqldump tool assumes the existence of
general_log and slow_log tables in the server.
If mysqldump tool executes on a old server where
there are no log tables like these, mysqldump tool
fails.
Analysis: general_log and slow_log tables are added
in the ignore-table list as part of bug-26121 fix
causes bug-45740 (MYSQLDUMP DOESN'T DUMP GENERAL_LOG
AND SLOW_QUERY CAUSES RESTORE PROBLEM). As part of
the bug-45740 fix, mysqldump tool adds create table
queries for these two tables. But the fix assumes
that on all the servers, general_log and slow_log
will be there. If the new mysqldump tool is executed
against a old server where there are no general_log
and slow_log, the mysqldump tool fails with an error
that 'there is no general_log table'.
Fix: When mysqldump tool is trying to retrieve general_log
and slow_log table structures, first the tool should
check their existence of these tables in the server
instead of trying to dump it blindly.
MYSQL DB FROM REMOTE 5.0.96 SERVER
Problem: mysqldump tool assumes the existence of
general_log and slow_log tables in the server.
If mysqldump tool executes on a old server where
there are no log tables like these, mysqldump tool
fails.
Analysis: general_log and slow_log tables are added
in the ignore-table list as part of bug-26121 fix
causes bug-45740 (MYSQLDUMP DOESN'T DUMP GENERAL_LOG
AND SLOW_QUERY CAUSES RESTORE PROBLEM). As part of
the bug-45740 fix, mysqldump tool adds create table
queries for these two tables. But the fix assumes
that on all the servers, general_log and slow_log
will be there. If the new mysqldump tool is executed
against a old server where there are no general_log
and slow_log, the mysqldump tool fails with an error
that 'there is no general_log table'.
Fix: When mysqldump tool is trying to retrieve general_log
and slow_log table structures, first the tool should
check their existence of these tables in the server
instead of trying to dump it blindly.
WITH A PORT NUMBER ENCLOSED IN QUOTES
Problem: mysqldump --dump-slave --include-master-host-port
prints the CHANGE MASTER command in the generated logical
backup. The PORT number that is generated with this command
is a string and should be an integer.
Fix: Remove the Enclosed quotes for port number.
WITH A PORT NUMBER ENCLOSED IN QUOTES
Problem: mysqldump --dump-slave --include-master-host-port
prints the CHANGE MASTER command in the generated logical
backup. The PORT number that is generated with this command
is a string and should be an integer.
Fix: Remove the Enclosed quotes for port number.
Base code and idea from a patch from by plinux at Taobao.
The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.
Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.
The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.
Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)
client/completion_hash.cc:
Updated call to init_alloc_root()
client/mysql.cc:
Updated call to init_alloc_root()
client/mysqlbinlog.cc:
init_dynamic_array() -> my_init_dynamic_array()
Updated call to init_alloc_root()
client/mysqlcheck.c:
Updated call to my_init_dynamic_array()
client/mysqldump.c:
Updated call to init_alloc_root()
client/mysqltest.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Fixed compiler warnings
extra/comp_err.c:
Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
Updated call to my_init_dynamic_array()
include/hash.h:
Added HASH_THREAD_SPECIFIC
include/heap.h:
Added flag is internal temporary table.
include/my_dir.h:
Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
Added malloc_flags to DYNAMIC_ARRAY
Added extra mysys flag argument to my_init_dynamic_array()
Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
Updated paramaters for init_alloc_root()
include/my_tree.h:
Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
Updated parameters to init_tree()
include/myisamchk.h:
Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
Added MYSQL_THREAD_SPECIFIC_MALLOC
Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
Updated file
include/mysql_com.h:
Used 'unused1' to mark memory as thread specific.
Updated parameters for my_net_init()
libmysql/libmysql.c:
Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
Updated call to init_alloc_root()
mysql-test/r/create.result:
Updated results
mysql-test/r/user_var.result:
Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
Updated results
mysql-test/t/show_explain.test:
Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
Added test of memory_usage variable.
mysys/array.c:
Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
All allocated memory is marked with the given my_flags.
Removed obsolete function init_dynamic_array()
mysys/default.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
mysys/hash.c:
Updated call to my_init_dynamic_array_ci().
Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
init_dynamic_array() -> my_init_dynamic_array()
Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
Extend MEM_ROOT with a flag if memory is thread specific.
This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
Store at start of each allocated memory block the size of the block and if the block is thread specific.
Call malloc_size_cb_func, if set, with the memory allocated/freed.
Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
Added flags to init_tree() so that one can mark tree to be thread specific.
Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
Updated call to init_alloc_root()
All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
Updated call to my_init_dynamic_array()
sql-common/client.c:
Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
Updated call to my_init_dynamic_array().
Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
Updated call to init_alloc_root()
sql/debug_sync.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
Removed calls to net_end() as this is now done in ~THD()
Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
Updated call to init_alloc_root()
Updated call to my_net_init()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
Updated call to my_net_init()
Updated call to init_sql_alloc()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
Updated call to init_alloc_root()
sql/handler.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
Added missing call to my_dir_end()
sql/item_func.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
More DBUG
Updated call to init_alloc_root()
sql/mdl.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
Added total_memory_used
Updated call to init_alloc_root()
Move mysql_cond_broadcast() before my_thread_end()
Added mariadb_dbug_id() to count memory per THD instead of per thread.
Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
Updated call to my_init_dynamic_array().
Updated call to my_net_init().
Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
Added status variable 'memory_used'.
Updated call to init_alloc_root()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
Added set_current_thd()
sql/net_serv.cc:
Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
Store in net->thread_specific_malloc if memory is thread specific.
Mark memory to be thread specific if requested.
sql/opt_range.cc:
Updated call to my_init_dynamic_array()
Updated call to init_sql_alloc()
Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
Fixed compiler warning
sql/records.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
Updated call to init_sql_alloc()
sql/rpl_mi.cc:
Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
Updated call to init_alloc_root()
sql/rpl_utility.cc:
Updated call to my_init_dynamic_array()
sql/slave.cc:
Initialize things properly before calling functions that allocate memory.
Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
Updated call to my_init_dynamic_array()
sql/sql_base.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
Updated comment
sql/sql_class.cc:
Added parameter to warning_info() that not initialize it until THD is fully created.
Updated call to init_sql_alloc()
Mark THD::user_vars has to be thread specific.
Updated call to my_init_dynamic_array()
Ensure that memory allocated by THD is assigned to the THD.
More DBUG
Always acll net_end() in ~THD()
Assert that all memory signed to this THD is really deleted at ~THD.
Fixed set_status_var_init() to not reset memory_used.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
Added MY_THREAD_SPECIFIC to allocated memory.
Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
This is used by THD::THD() to not allocate memory until THD is ready.
Added Warning_info::free_memory()
sql/sql_error.h:
Updated Warning_info() class.
sql/sql_handler.cc:
Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
More DBUG
sql/sql_join_cache.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
Updated call to my_init_dynamic_array()
sql/sql_lex.h:
Updated call to my_init_dynamic_array()
sql/sql_load.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
Ensure that examined_row_count() is reset before query.
Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
Added parameter to warning_info() that it should be fully initialized.
Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
Added MY_THREAD_SPECIFIC to allocated memory.
More DBUG
sql/sql_servers.cc:
Updated call to init_sql_alloc() to mark memory some memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
Updated call to my_init_dynamic_array()
Mark my_dir() memory thread specific.
Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
More DBUG.
Added malloc_size and examined_row_count to SHOW PROCESSLIST.
Added MY_THREAD_SPECIFIC to allocated memory.
Updated call to init_sql_alloc()
Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
Fixed compiler warning
sql/sql_string.cc:
String elements can now be marked as thread specific.
sql/sql_string.h:
String elements can now be marked as thread specific.
sql/sql_table.cc:
Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Fixed compiler warning
sql/sql_test.cc:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
Updated call to init_sql_alloc()
sql/sql_udf.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
Updated call to init_sql_alloc().
Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
Updated call to init_sql_alloc()
sql/thr_malloc.cc:
Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
Updated prototype for init_sql_alloc()
sql/tztime.cc:
Updated call to init_sql_alloc()
Updated call to init_alloc_root() to mark memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
Updated call to init_tree()
storage/maria/ma_ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
Updated call to init_tree()
storage/myisam/ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
Updated call to init_tree()
storage/myisam/mi_check.c:
Updated call to init_alloc_root()
storage/myisam/mi_write.c:
Updated call to my_init_dynamic_array()
Updated call to init_tree()
storage/myisam/myisamlog.c:
Updated call to init_tree()
storage/myisam/myisampack.c:
Updated call to init_tree()
storage/myisam/sort.c:
Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
Rest current_thd
storage/perfschema/pfs_instr.cc:
Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
Disable compiler warning from offsetof macro.
Problem: When a view, with a specific character set and collation,
is created on another view with a different character set and collation the
dump restoration results in an illegal mix of collations error.
SOLUTION: To avoid this confusion of collations, the create table datatype
being used is hardcoded as "tinyint NOT NULL". This will not matter as the table
created will be dropped at runtime and specifically tinyint is used to
avoid hitting the row size conflicts.
Problem: When a view, with a specific character set and collation,
is created on another view with a different character set and collation the
dump restoration results in an illegal mix of collations error.
SOLUTION: To avoid this confusion of collations, the create table datatype
being used is hardcoded as "tinyint NOT NULL". This will not matter as the table
created will be dropped at runtime and specifically tinyint is used to
avoid hitting the row size conflicts.
MYSQLDUMP OUTPUT
Problem: mysqldump when used with option --routines, dumps
all the routines of the specified database into
output. The statements in this output are written
in such a way that they are version safe using C
style version commenting (of the format
/*!<version num> <sql statement>*/). If a semicolon
is present right before closing of the comment in
dump output, it results in a syntax error while
importing.
Solution: Version comments for dumped routines are
specifically to protect the ones older than 5.0.
When the import is done on 5.0 or later versions,
entire create statement gets executed as all the
check conditions at the beginning of the comments
are cleared. Since the trade off is between the
performance of newer versions which are more in
use and protection of very old versions which are
no longer supported, it is proposed that these
comments be removed altogether to maintain
stability of the versions supported.
client/mysqldump.c:
Bug#14463669 FAILURE TO CORRECTLY PARSE ROUTINES IN
MYSQLDUMP OUTPUT
Output of mysqldump is derived by getting the queries from
show create and appending version comments to them.
query_str is the variable used to store the final string.
Since it is no longer required, its declaration and
manipulations made on it are deleted. At the step where
output is printed, query_str is replaced with the original
query string derived from 'show create'.
MYSQLDUMP OUTPUT
Problem: mysqldump when used with option --routines, dumps
all the routines of the specified database into
output. The statements in this output are written
in such a way that they are version safe using C
style version commenting (of the format
/*!<version num> <sql statement>*/). If a semicolon
is present right before closing of the comment in
dump output, it results in a syntax error while
importing.
Solution: Version comments for dumped routines are
specifically to protect the ones older than 5.0.
When the import is done on 5.0 or later versions,
entire create statement gets executed as all the
check conditions at the beginning of the comments
are cleared. Since the trade off is between the
performance of newer versions which are more in
use and protection of very old versions which are
no longer supported, it is proposed that these
comments be removed altogether to maintain
stability of the versions supported.
TABLE DATA IF DUMPS MYSQL DATABA
Problem: If mysqldump is run without --events (or with --skip-events)
it will not dump the mysql.event table's data. This behaviour is inconsistent
with that of --routines option, which does not affect the dumping of
mysql.proc table. According to the Manual, --events (--skip-events) defines,
if the Event Scheduler events for the dumped databases should be included
in the mysqldump output and this has nothing to do with the mysql.event table
itself.
Solution: A warning has been added when mysqldump is used without --events
(or with --skip-events) and a separate patch with the behavioral change
will be prepared for 5.6/trunk.
TABLE DATA IF DUMPS MYSQL DATABA
Problem: If mysqldump is run without --events (or with --skip-events)
it will not dump the mysql.event table's data. This behaviour is inconsistent
with that of --routines option, which does not affect the dumping of
mysql.proc table. According to the Manual, --events (--skip-events) defines,
if the Event Scheduler events for the dumped databases should be included
in the mysqldump output and this has nothing to do with the mysql.event table
itself.
Solution: A warning has been added when mysqldump is used without --events
(or with --skip-events) and a separate patch with the behavioral change
will be prepared for 5.6/trunk.
TABLE DATA IF DUMPS MYSQL DATABA
Problem: If mysqldump is run without --events (or with --skip-events)
it will not dump the mysql.event table's data. This behaviour is inconsistent
with that of --routines option, which does not affect the dumping of
mysql.proc table. According to the Manual, --events (--skip-events) defines,
if the Event Scheduler events for the dumped databases should be included
in the mysqldump output and this has nothing to do with the mysql.event table
itself.
Solution: A warning has been added when mysqldump is used without --events
(or with --skip-events) and a separate patch with the behavioral change
will be prepared for 5.6/trunk.
TABLE DATA IF DUMPS MYSQL DATABA
Problem: If mysqldump is run without --events (or with --skip-events)
it will not dump the mysql.event table's data. This behaviour is inconsistent
with that of --routines option, which does not affect the dumping of
mysql.proc table. According to the Manual, --events (--skip-events) defines,
if the Event Scheduler events for the dumped databases should be included
in the mysqldump output and this has nothing to do with the mysql.event table
itself.
Solution: A warning has been added when mysqldump is used without --events
(or with --skip-events) and a separate patch with the behavioral change
will be prepared for 5.6/trunk.
and small collateral changes
mysql-test/lib/My/Test.pm:
somehow with "print" we get truncated writes sometimes
mysql-test/suite/perfschema/r/digest_table_full.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/dml_handler.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/information_schema.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/nesting.result:
this differs, because we don't rewrite general log queries, and multi-statement
packets are logged as a one entry. this result file is identical to what mysql-5.6.5
produces with the --log-raw option.
mysql-test/suite/perfschema/r/relaylog.result:
MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB.
mysql-test/suite/perfschema/r/server_init.result:
MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup.
mysql-test/suite/perfschema/r/stage_mdl_global.result:
this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not
do that, and this causes useless mutex locks and waits.
mysql-test/suite/perfschema/r/statement_digest.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_consumers.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_long_query.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result:
will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged
two tests still fail:
main.innodb_icp and main.range_vs_index_merge_innodb
call records_in_range() with both range ends being open
(which triggers an assert)
client/mysqldump.c:
Slave needs to be initialized with 0
dbug/dbug.c:
Removed not existing function
plugin/semisync/semisync_master.cc:
Fixed compiler warning
sql/opt_range.cc:
thd needs to be set early as it's used in some error conditions.
sql/sql_table.cc:
Changed to use uchar* to make array indexing portable
storage/innobase/handler/ha_innodb.cc:
Removed not used variable
storage/maria/ma_delete.c:
Fixed compiler warning
storage/maria/ma_write.c:
Fixed compiler warning
SHOW 2012 INSTEAD OF 2011
* Added a new macro to hold the current year :
COPYRIGHT_NOTICE_CURRENT_YEAR
* Modified ORACLE_WELCOME_COPYRIGHT_NOTICE macro
to take the initial year as parameter and pick
current year from the above mentioned macro.
SHOW 2012 INSTEAD OF 2011
* Added a new macro to hold the current year :
COPYRIGHT_NOTICE_CURRENT_YEAR
* Modified ORACLE_WELCOME_COPYRIGHT_NOTICE macro
to take the initial year as parameter and pick
current year from the above mentioned macro.
executing
The problem is that mysql lacks information about the objects a view
depends on so it can't dump views and tables in the proper order.
Thus it needs to create "stand-in" myisam tables for each view while
dumping the tables that it later drops and replaces with the actual view
view definition.
But since views can have much more columns than an actual table creating
these stand-in tables may be problematic.
There's no way to portably find out how many columns an mysiam table
can have. It's a complicated formula depending on internal server constants.
Thus we can't have a reliable error check without repeating the logic and
the formula inside mysqldump.
1. Changed the type of the columns of the stand-in tables mysqldump
makes to satisfy view dependencies from the original type to smallint
to save on row space.
2. Added a warning on the mysqldump's standard error for a possible
problems replaying the dump file if the columns of a view exceed 1000.
3. Added a test case.
executing
The problem is that mysql lacks information about the objects a view
depends on so it can't dump views and tables in the proper order.
Thus it needs to create "stand-in" myisam tables for each view while
dumping the tables that it later drops and replaces with the actual view
view definition.
But since views can have much more columns than an actual table creating
these stand-in tables may be problematic.
There's no way to portably find out how many columns an mysiam table
can have. It's a complicated formula depending on internal server constants.
Thus we can't have a reliable error check without repeating the logic and
the formula inside mysqldump.
1. Changed the type of the columns of the stand-in tables mysqldump
makes to satisfy view dependencies from the original type to smallint
to save on row space.
2. Added a warning on the mysqldump's standard error for a possible
problems replaying the dump file if the columns of a view exceed 1000.
3. Added a test case.
mysqldump with --include-master-host-port putting quotes around port number
Patch from Stewart Smith
client/mysqldump.c:
Remove quotes from MASTER_PORT
client/mysqldump.c:
Added LINT_INIT
mysql-test/mysql-test-run.pl:
Disable warning if example engine is not found
mysql-test/suite/rpl/t/rpl_semi_sync.test:
Rpl_semi_sync_master_yes_tx may be different on Windows
sql/sql_plugin.cc:
More DBUG_PRINT
support-files/compiler_warnings.supp:
Disable some innobase warnings
unittest/mysys/my_vsnprintf-t.c:
Fixed test failure on Solaris (typo)
CAUSES RESTORE PROBLEM
Merging the fix from mysql-5.1 to mysql-5.5
mysql-test/t/mysqldump.test:
There is a difference in the testcase which is added as
part of this fix, when compared with mysql-5.1. In mysql-5.5
and mysql-5.6, "DROP mysql database" fails by enabling
logging, hence removed those lines.
CAUSES RESTORE PROBLEM
Problem Statement:
------------------
mysqldump is not having the dump stmts for general_log and slow_log
tables. That is because of the fix for Bug#26121. Hence, after
dropping the mysql database, and applying the dump by enabling the
logging, "'general_log' table not found" errors are logged into the
server log file.
Analysis:
---------
As part of the fix for Bug#26121, we skipped the dumping of tables
for general_log and slow_log, because the data dump of those tables
are taking LOCKS, which is not allowed for log tables.
Fix:
----
We came up with an approach that instead of taking both meta data
and data dump information for those tables, take only the meta data
dump which doesn't need LOCKS.
As part of fixing the issue we came up with below algorithm.
Design before fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
Design with the fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Explicitly call the 'show create table' for general_log and
slow_log
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
While taking the meta data dump for general_log and slow_log the
"CREATE TABLE" is replaced with "CREATE TABLE IF NOT EXISTS".
This is because we skipped "DROP TABLE" for those tables,
"DROP TABLE" fails for these tables if logging is enabled.
Customer is applying the dump by enabling logging so, if the dump
has "DROP TABLE" it will fail. Hence, removed the "DROP TABLE"
stmts for those tables.
After the fix we could observe "Table 'mysql.general_log'
doesn't exist" errors initially that is because in the customer
scenario they are dropping the mysql database by enabling the
logging, Hence, those errors are expected. Once we apply the
dump which is taken before the "drop database mysql", the errors
will not be there.
client/mysqldump.c:
In get_table_structure() added code to skip the DROP TABLE stmts for general_log
and slow_log tables, because when logging is enabled those stmts will fail. And
replaced CREATE TABLE with CREATE IF NOT EXISTS for those tables, just to make
sure CREATE stmt for those tables doesn't fail since we removed DROP stmts for
those tables.
In dump_all_tables_in_db() added code to call get_table_structure() for
general_log and slow_log tables.
mysql-test/r/mysqldump.result:
Added a test as part of fix for Bug #11754178
mysql-test/t/mysqldump.test:
Added a test as part of fix for Bug #11754178
CAUSES RESTORE PROBLEM
Problem Statement:
------------------
mysqldump is not having the dump stmts for general_log and slow_log
tables. That is because of the fix for Bug#26121. Hence, after
dropping the mysql database, and applying the dump by enabling the
logging, "'general_log' table not found" errors are logged into the
server log file.
Analysis:
---------
As part of the fix for Bug#26121, we skipped the dumping of tables
for general_log and slow_log, because the data dump of those tables
are taking LOCKS, which is not allowed for log tables.
Fix:
----
We came up with an approach that instead of taking both meta data
and data dump information for those tables, take only the meta data
dump which doesn't need LOCKS.
As part of fixing the issue we came up with below algorithm.
Design before fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
Design with the fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Explicitly call the 'show create table' for general_log and
slow_log
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
While taking the meta data dump for general_log and slow_log the
"CREATE TABLE" is replaced with "CREATE TABLE IF NOT EXISTS".
This is because we skipped "DROP TABLE" for those tables,
"DROP TABLE" fails for these tables if logging is enabled.
Customer is applying the dump by enabling logging so, if the dump
has "DROP TABLE" it will fail. Hence, removed the "DROP TABLE"
stmts for those tables.
After the fix we could observe "Table 'mysql.general_log'
doesn't exist" errors initially that is because in the customer
scenario they are dropping the mysql database by enabling the
logging, Hence, those errors are expected. Once we apply the
dump which is taken before the "drop database mysql", the errors
will not be there.
CAUSES RESTORE PROBLEM
Problem Statement:
------------------
mysqldump is not having the dump stmts for general_log and slow_log
tables. That is because of the fix for Bug#26121. Hence, after
dropping the mysql database, and applying the dump by enabling the
logging, "'general_log' table not found" errors are logged into the
server log file.
Analysis:
---------
As part of the fix for Bug#26121, we skipped the dumping of tables
for general_log and slow_log, because the data dump of those tables
are taking LOCKS, which is not allowed for log tables.
Fix:
----
We came up with an approach that instead of taking both meta data
and data dump information for those tables, take only the meta data
dump which doesn't need LOCKS.
As part of fixing the issue we came up with below algorithm.
Design before fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
Design with the fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Explicitly call the 'show create table' for general_log and
slow_log
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
While taking the meta data dump for general_log and slow_log the
"CREATE TABLE" is replaced with "CREATE TABLE IF NOT EXISTS".
This is because we skipped "DROP TABLE" for those tables,
"DROP TABLE" fails for these tables if logging is enabled.
Customer is applying the dump by enabling logging so, if the dump
has "DROP TABLE" it will fail. Hence, removed the "DROP TABLE"
stmts for those tables.
After the fix we could observe "Table 'mysql.general_log'
doesn't exist" errors initially that is because in the customer
scenario they are dropping the mysql database by enabling the
logging, Hence, those errors are expected. Once we apply the
dump which is taken before the "drop database mysql", the errors
will not be there.
client/mysqldump.c:
In get_table_structure() added code to skip the DROP TABLE stmts for general_log
and slow_log tables, because when logging is enabled those stmts will fail. And
replaced CREATE TABLE with CREATE IF NOT EXISTS for those tables, just to make
sure CREATE stmt for those tables doesn't fail since we removed DROP stmts for
those tables.
In dump_all_tables_in_db() added code to call get_table_structure() for
general_log and slow_log tables.
mysql-test/r/mysqldump.result:
Added a test as part of fix for Bug #11754178
mysql-test/t/mysqldump.test:
Added a test as part of fix for Bug #11754178
CAUSES RESTORE PROBLEM
Problem Statement:
------------------
mysqldump is not having the dump stmts for general_log and slow_log
tables. That is because of the fix for Bug#26121. Hence, after
dropping the mysql database, and applying the dump by enabling the
logging, "'general_log' table not found" errors are logged into the
server log file.
Analysis:
---------
As part of the fix for Bug#26121, we skipped the dumping of tables
for general_log and slow_log, because the data dump of those tables
are taking LOCKS, which is not allowed for log tables.
Fix:
----
We came up with an approach that instead of taking both meta data
and data dump information for those tables, take only the meta data
dump which doesn't need LOCKS.
As part of fixing the issue we came up with below algorithm.
Design before fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
Design with the fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Explicitly call the 'show create table' for general_log and
slow_log
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
While taking the meta data dump for general_log and slow_log the
"CREATE TABLE" is replaced with "CREATE TABLE IF NOT EXISTS".
This is because we skipped "DROP TABLE" for those tables,
"DROP TABLE" fails for these tables if logging is enabled.
Customer is applying the dump by enabling logging so, if the dump
has "DROP TABLE" it will fail. Hence, removed the "DROP TABLE"
stmts for those tables.
After the fix we could observe "Table 'mysql.general_log'
doesn't exist" errors initially that is because in the customer
scenario they are dropping the mysql database by enabling the
logging, Hence, those errors are expected. Once we apply the
dump which is taken before the "drop database mysql", the errors
will not be there.
mysql-test/suite/innodb/t/group_commit_crash.test:
remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
my_vsnprintf() is ok here, in 5.5
suppress these harmless but confusing warnings.
fix the program name (MY_INIT) in mysqldump
client/mysqldump.c:
for backward compatibility, prefix mysqldump error messages with "mysqldump", not with the full path of the executable
Fix memory leak in one error case in mysqldump.
Fix that HAVE_VALGRIND_VALGRIND_H is now HAVE_VALGRIND in 5.5.
Fix that @have_ssl should not be set in embedded (introduced when
removing #undef HAVE_OPENSSL from my_global.h).
routines.
mysqldump in xml mode did not dump routines, events or
triggers.
This patch fixes this issue by fixing the if conditions
that disallowed the dump of above mentioned objects in
xml mode, and added the required code to enable dump
in xml format.
client/mysqldump.c:
BUG#11760384 - 52792: mysqldump in XML mode does not dump
routines.
Fixed some if conditions to allow execution of dump methods
for xml and further added the relevant code at places to produce
the dump in xml format.
mysql-test/r/mysqldump.result:
Added a test case for Bug#11760384.
mysql-test/t/mysqldump.test:
Added a test case for Bug#11760384.
routines.
mysqldump in xml mode did not dump routines, events or
triggers.
This patch fixes this issue by fixing the if conditions
that disallowed the dump of above mentioned objects in
xml mode, and added the required code to enable dump
in xml format.
--FLUSH-LOG BREAKS CONSISTENCY
The transaction started by mysqldump gets committed
implicitly when flush-log is specified along with
single-transaction option, and hence can break
consistency.
This is because, COM_REFRESH is executed in order
to flush logs and starting from 5.5 this command
performs an implicit commit.
Fixed by making sure that COM_REFRESH is executed
before the transaction has started and not after it.
Note : This patch triggers following behavioral
changes in mysqldump :
1) After this patch we no longer flush logs before
dumping each database if --single-transaction
option is given like it was done before (in the
absence of --lock-all-tables and --master-data
options).
2) Also, after this patch, we start acquiring
FTWRL before flushing logs in cases when only
--single-transaction and --flush-logs are given.
It becomes safe to use mysqldump with these two
options and without --master-data parameter for
backups.
client/mysqldump.c:
Bug#12809202 61854: MYSQLDUMP --SINGLE-TRANSACTION
--FLUSH-LOG BREAKS CONSISTENCY
Added logic to make sure that, if flush-log option
is specified, mysql_refresh() is never executed after
the transaction has started.
Added verbose messages for all the executions of
mysql_refresh() in order to track its invocation.
mysql-test/r/mysqldump.result:
Added test case for Bug#12809202.
mysql-test/t/mysqldump.test:
Added test case for Bug#12809202.
--FLUSH-LOG BREAKS CONSISTENCY
The transaction started by mysqldump gets committed
implicitly when flush-log is specified along with
single-transaction option, and hence can break
consistency.
This is because, COM_REFRESH is executed in order
to flush logs and starting from 5.5 this command
performs an implicit commit.
Fixed by making sure that COM_REFRESH is executed
before the transaction has started and not after it.
Note : This patch triggers following behavioral
changes in mysqldump :
1) After this patch we no longer flush logs before
dumping each database if --single-transaction
option is given like it was done before (in the
absence of --lock-all-tables and --master-data
options).
2) Also, after this patch, we start acquiring
FTWRL before flushing logs in cases when only
--single-transaction and --flush-logs are given.
It becomes safe to use mysqldump with these two
options and without --master-data parameter for
backups.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
This fixes a bug that when you use mysqldump --no-create-info to generate a dump that you want to merge with an existing table,
you can get an innodb table with duplicated unique keys.
Patch orignally by Eric Bergen.
client/mysqldump.c:
Only use UNIQUE_CHECKS=0 for tables that are created.
This solves the issue that you can't get duplicate unique keys when merging two dumps.
mysql-test/r/mysqldump.result:
Test for mysqldump --no-create-info
TOOLS
Backport a fix for Bug 57094 from 5.5.
The following revision was backported:
# revision-id: alexander.nozdrin@oracle.com-20101006150613-ls60rb2tq5dpyb5c
# parent: bar@mysql.com-20101006121559-am1e05ykeicwnx48
# committer: Alexander Nozdrin <alexander.nozdrin@oracle.com>
# branch nick: mysql-5.5-bugteam-bug57094
# timestamp: Wed 2010-10-06 19:06:13 +0400
# message:
# Fix for Bug 57094 (Copyright notice incorrect?).
#
# The fix is to:
# - introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
# to specify copyright notice;
# - replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
# in programs.
TOOLS
Backport a fix for Bug 57094 from 5.5.
The following revision was backported:
# revision-id: alexander.nozdrin@oracle.com-20101006150613-ls60rb2tq5dpyb5c
# parent: bar@mysql.com-20101006121559-am1e05ykeicwnx48
# committer: Alexander Nozdrin <alexander.nozdrin@oracle.com>
# branch nick: mysql-5.5-bugteam-bug57094
# timestamp: Wed 2010-10-06 19:06:13 +0400
# message:
# Fix for Bug 57094 (Copyright notice incorrect?).
#
# The fix is to:
# - introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
# to specify copyright notice;
# - replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
# in programs.
client/mysql_upgrade.c:
missing DBUG_RETURN
client/mysqladmin.cc:
client plugin memory wasn't freed
client/mysqlcheck.c:
client plugin memory, defaults, a result set, a command-line option value were not freed.
missing DBUG_RETURN.
client/mysqldump.c:
client plugin memory wasn't freed
client/mysqlslap.c:
client plugin memory wasn't freed
client/mysqltest.cc:
hopeless. cannot be fixed.
mysql-test/valgrind.supp:
Bug#56666 is now fixed.
mysys/array.c:
really, don't allocate if the caller didn't ask to.
mysys/my_init.c:
safemalloc checks must be done at the very end
mysys/my_thr_init.c:
not needed anymore
sql-common/client.c:
memory leak
sql/log.cc:
log_file was not closed, memory leak.
sql/mysqld.cc:
fix bug#56666 (causing many P_S related memory leaks).
close_active_mi() not called for --bootstrap, memory leak.
sql/sql_lex.cc:
redo Lex->mi handling
sql/sql_lex.h:
redo Lex->mi handling
sql/sql_plugin.cc:
plugins having PLUGIN_VAR_MEMALLOC string variables have this variables allocated in every THD.
The memory was freed in ~THD but only if plugin was still active. If plugin was unloaded the
variable was not found and the memory was lost. By loading and unloading plugins an arbitrary
amount of memory can be lost.
sql/sql_repl.cc:
redo Lex->mi handling
sql/sql_yacc.yy:
completely wrong handling of Lex->mi - run-time memory leak, by repeating the statement
arbitrary amount of memory can be lost.
Lex->mi.repl_ignore_server_ids_opt was allocated when parsing CHANGE MASTER,
and freed after executing the statement. if parser failed on syntax (or another)
error the statement was never executed. Lex->mi was simply bzero-ed for the next
CHANGE MASTER statement.
sql/table.cc:
didn't compile
storage/perfschema/pfs_lock.h:
Bug#56666 is fixed
The mysqld server and all clients now reads the new client-server section
Fixed that mysqldumpslow supports new slow log formats and new mysqld --slow- options
client/mysql.cc:
Read also client-server and client-mariadb sections.
client/mysql_upgrade.c:
Read also client-server and client-mariadb sections.
client/mysqladmin.cc:
Read also client-server and client-mariadb sections.
client/mysqlbinlog.cc:
Read also client-server and client-mariadb sections.
client/mysqlcheck.c:
Read also client-server and client-mariadb sections.
client/mysqldump.c:
Read also client-server and client-mariadb sections.
client/mysqlimport.c:
Read also client-server and client-mariadb sections.
client/mysqlshow.c:
Read also client-server and client-mariadb sections.
client/mysqltest.cc:
Read also client-server and client-mariadb sections.
extra/my_print_defaults.c:
Updated help text
scripts/mysql_fix_privilege_tables.sh:
Read also sections client client-server client-mariadb
scripts/mysql_install_db.pl.in:
Also allow --data=* option
Read also groups mariadb, server and client-server.
scripts/mysql_install_db.sh:
Also allow --data=* option
Read also groups mariadb, server and client-server.
Added --lose-skip-pbxt to bootstrap
scripts/mysql_secure_installation.sh:
Read also groups client-server and client-mariadb
scripts/mysqld_multi.sh:
Read also group mariadb
scripts/mysqld_safe.sh:
Read also groups mariadb server and client-server
scripts/mysqldumpslow.sh:
Fixed to support new slow log formats
Added sorting on -ae (aggregated number of retreived rows) and e (retrieved rows)
Read also group 'mariadb'
If there is many instances of same option, use last one.
Get slow log file from options log-slow-queries=filename or query-log-file=filename
Added support for future --log-basename option
sql-common/client.c:
Read also groups 'client-server' and 'client-mariadb'
tests/mysql_client_test.c:
Read also groups 'client-server' and 'client-mariadb'
tests/thread_test.c:
Read also groups 'client-server' and 'client-mariadb'
("-") IN DATABASE NAMES IN ALTER DATABASE.
mysqldump did not quote database name in 'ALTER DATABASE'
statements in its output. This can further cause a failure
while loading if database name contains a hyphen '-'.
This happened as, while printing the 'ALTER DATABASE'
statements, the database name was not quoted.
Fixed by quoting the database name.
client/mysqldump.c:
Bug#11766310 : 59398: MYSQLDUMP 5.1 CAN'T HANDLE A DASH
("-") IN DATABASE NAMES IN ALTER DATABASE.
Modified the print statement in order to print the quoted
database name for 'ALTER DATABASE' statements.
mysql-test/r/mysqldump.result:
Added a test case for bug#11766310.
mysql-test/t/mysqldump.test:
Added a test case for bug#11766310.
("-") IN DATABASE NAMES IN ALTER DATABASE.
mysqldump did not quote database name in 'ALTER DATABASE'
statements in its output. This can further cause a failure
while loading if database name contains a hyphen '-'.
This happened as, while printing the 'ALTER DATABASE'
statements, the database name was not quoted.
Fixed by quoting the database name.
"set optimizer_switch to e or d causes invalid memory writes/valgrind warnings":
due to prefix support, the argument "e" was overwritten with its full value
"engine_condition_pushdown", which caused a buffer overrun.
This was wrong usage of find_type(); other wrong usages are fixed here too.
Please start reading with the comment of typelib.c.
client/mysqldump.c:
A bug: find_type() expects a bitmap as 3rd argument
(each bit is a flag controlling a behaviour of the function);
here it was instead passed the length of the string to search!
That could give random behaviour of find_type()
depending on the string.
We rather need to pass a correct flag to find_type().
The correct flag is FIND_TYPE_BASIC (0).
Flag 8 is not needed as buff cannot have a comma (see how buff is filled).
Flag 1 looks like a superfluous restriction.
Flag 4 is not user-friendly (why use
--compatible=2 rather than --compatible=mysql40 ?, and
we probably not commit to "2" always meaning "mysql40"
until the end of times).
include/mysql.h.pp:
This isn't a problematic API change as we go from char* to const char*:
existing code will run unchanged.
include/typelib.h:
named constants. Not an enum to not significantly change
the declaration of find_type() which would be an API change
(typelib.h is included in mysql.h).
mysql-test/r/mysqldump.result:
correct result (see the two requested modes in SQL_MODE)
mysql-test/suite/sys_vars/t/optimizer_switch_basic.test:
test for BUG#59894. The second SET used to crash.
mysql-test/t/mysqldump.test:
we had no test for multiple modes in --compatible, which is
supported according to --help
mysys/typelib.c:
Fix for BUG#59894. parse_name() is asked to match "e" with a row
of the TYPELIB (the TYPELIB lists permitted flags of optimizer_switch;
and comes from optimizer_switch_names[] of sys_vars.cc).
find_type() is capable of supporting prefixes, but if it is not
passed flag 2 in third argument, it will overwrite its first
argument (the string to search for) with the complete name,
here overwriting "e" with "engine_condition_pushdown". But
as this "e" was a buffer allocated in an Item, it was not big
enough to host the longer name, thus the crash.
We don't need to know the complete flag's name; the output used
from find_type() is just the flag's number (== function's return
code). So we can pass flag 2 to find_type() in parse_name().
After doing this fix and the other fixes in this patch, all usages
of find_type() were using flag 2; in most usages the string to search for,
is not guaranteed to be long enough to host the complete name
(it is either directly from argv, or from alloc_root/my_malloc
done in an earlier call).
Thus, flag 2 is here made implicit: callers need not pass it anymore,
it is always automatically turned on.
This allows to eliminate an oddity: parse_name() took a const char**,
and then removed "const" before calling find_type(), which could
theoretically modify the pointed data, thus lying on constness.
Last, constants for find_type() are now named.
sql-common/client.c:
Two bugs:
1) The enum was not in sync with the array (due to a bad porting of WL 1054;
the extra OPT_ values are about options present in 5.1 and deleted in 5.5);
added a compile_time_assert() to make sure this doesn't happen again
2) find_type() was writing past the end of opt_arg; as opt_arg was allocated
with alloc_root() with no extra space, this was an overrun; it could be seen
when
** building with -DWITH_VALGRIND -DHAVE_purify -DEXTRA_DEBUG
** making execution go through the faulty code; this faulty
code is executed only if the client asks to read a configuration
file like this:
mysql_options(mysql, MYSQL_READ_DEFAULT_FILE, "/tmp/cnf.cnf");
so by adding such line to the start of mysql_client_test.c::client_connect(),
we could see the valgrind warning:
==30548== Invalid write of size 1
==30548== at 0x4C2624C: strcpy (mc_replace_strmem.c:303)
==30548== by 0x48DC29: find_type (typelib.c:120)
==30548== by 0x465686: mysql_read_default_options (client.c:1344)
==30548== by 0x46830F: mysql_real_connect (client.c:2971)
==30548== by 0x409339: client_connect (mysql_client_test.c:331)
==30548== by 0x463A7F: main (mysql_client_test.c:19902)
==30548== Address 0x61875ad is 0 bytes after a block of size 29 alloc'd
==30548== at 0x4C25153: malloc (vg_replace_malloc.c:195)
==30548== by 0x49BFF1: my_malloc (my_malloc.c:38)
==30548== by 0x49C65C: alloc_root (my_alloc.c:166)
==30548== by 0x48EF97: handle_default_option (default.c:381)
==30548== by 0x49068C: search_default_file_with_ext (default.c:992)
==30548== by 0x48F929: search_default_file (default.c:670)
==30548== by 0x48EDC4: my_search_option_files (default.c:312)
==30548== by 0x48F4B1: my_load_defaults (default.c:576)
==30548== by 0x46517A: mysql_read_default_options (client.c:1207)
==30548== by 0x46830F: mysql_real_connect (client.c:2971)
==30548== by 0x409339: client_connect (mysql_client_test.c:331)
==30548== by 0x463A7F: main (mysql_client_test.c:19902)
This is fixed by having find_type() not overwrite anymore.
sql/sql_help.cc:
cast not needed anymore.
sql/table.cc:
cast not needed anymore.
"set optimizer_switch to e or d causes invalid memory writes/valgrind warnings":
due to prefix support, the argument "e" was overwritten with its full value
"engine_condition_pushdown", which caused a buffer overrun.
This was wrong usage of find_type(); other wrong usages are fixed here too.
Please start reading with the comment of typelib.c.
When mysqldump tries to dump information in xml format,
the result does not contain field level comments.
In order to retrieve various informations for a field/column,
mysqldump currently uses 'show fields from <tab>' statement.
The attributes returned by the statement lacks the information
regarding field comments.
Fixed by changing the query to one that probes I_S to retrieve
required field informations, including the field comment.
client/mysqldump.c:
Bug#13618 : mysqldump --xml omits comment on table field.
Replaced the 'show fields' command by a statement that
queries I_S, in order to retrieve information on all the
attributes that 'show fields' returns along-with an additional
column_comment information.
mysql-test/r/client_xml.result:
Result modifications for bug#13618.
mysql-test/r/mysqldump.result:
Result modifications for bug#13618.
mysql-test/t/mysqldump.test:
Added a testcase for bug#13618.
When mysqldump tries to dump information in xml format,
the result does not contain field level comments.
In order to retrieve various informations for a field/column,
mysqldump currently uses 'show fields from <tab>' statement.
The attributes returned by the statement lacks the information
regarding field comments.
Fixed by changing the query to one that probes I_S to retrieve
required field informations, including the field comment.
command line clients.
Postfix covering other mysql standard clients like mysql_upgrade,
mysqlbinlog, mysqlcheck, mysqlimport, mysqlshow and mysqlslap.
client/client_priv.h:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients
Added an entry for 'default-auth' option.
client/mysql.cc:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients
Updated the id entry for default_auth option.
client/mysql_upgrade.c:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
client/mysqladmin.cc:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients
Updated the id entry for default_auth option.
client/mysqlbinlog.cc:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
client/mysqlcheck.c:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
client/mysqldump.c:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients
Updated the id entry for default_auth option.
client/mysqlimport.c:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
client/mysqlshow.c:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
client/mysqlslap.c:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
mysql-test/r/plugin_auth.result:
Added test case for Bug#58139 for mysql_upgrade.
mysql-test/t/plugin_auth.test:
Added test case for Bug#58139 for mysql_upgrade.
command line clients.
Command line tools like mysqladmin and mysqldump did not recognize
default-auth and plugin-dir options.
Support for these options was found missing in these command line
tools.
Fixed by adding support for the same.
client/mysqladmin.cc:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
client/mysqldump.c:
Bug#58139 : default-auth option not recognized in MySQL standard
command line clients.
Introduced two new variables to hold values from default-auth and
plugin-dir options and further pushed them to client's st_mysql
instance.
mysql-test/r/plugin_auth.result:
Added test case for Bug#58139.
mysql-test/t/plugin_auth.test:
Added test case for Bug#58139.
command line clients.
Command line tools like mysqladmin and mysqldump did not recognize
default-auth and plugin-dir options.
Support for these options was found missing in these command line
tools.
Fixed by adding support for the same.
Make the binlog handlerton participate in START TRANSACTION WITH CONSISTENT
SNAPSHOT, recording the binlog position corresponding to the snapshot taken
in other MVCC storage engines.
Expose this consistent binlog position as the new status variables
binlog_trx_file and binlog_trx_position. This enables to get a fully
non-locking snapshot of the database (including binlog position for
slave provisioning), avoiding the need for FLUSH TABLES WITH READ LOCK.
Modify mysqldump to detect if the server supports this new feature, and
if so, avoid FLUSH TABLES WITH READ LOCK for --single-transaction
--master-data snapshot backups.
The fix is to:
- introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
to specify copyright notice;
- replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
in programs.
The fix is to:
- introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
to specify copyright notice;
- replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
in programs.
client/mysqlcheck.c:
Added missing casts
client/mysqldump.c:
Added missing casts
client/mysqlimport.c:
Added missing casts
extra/my_print_defaults.c:
Added missing casts
mysql-test/mysql-test-run.pl:
Added suppression for non-critical warning on windows
storage/maria/maria_pack.c:
Added missing casts
storage/xtradb/buf/buf0lru.c:
Added missing casts
storage/xtradb/fil/fil0fil.c:
Added missing casts
storage/xtradb/handler/i_s.cc:
Added extra argument to call store() function for longlong.
storage/xtradb/srv/srv0srv.c:
Added cast to suppress compiler warning
support-files/compiler_warnings.supp:
Added suppression for some non critical compiler warnings on Windows
unittest/mytap/tap.h:
Fixed prototypes to be same as the actual functions
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
- Fixed memory leaks in mysqldump
- Fixed printf of NULL which caused crashes on OpenSolaris when using --debug
- Fixed realloc() problem that caused out of memory when running mysqldump.test on OpenSolaris
client/mysqldump.c:
Fixed memory leaks
Fixed printf of NULL which caused crashes on OpenSolaris when using --debug
client/mysqltest.cc:
Fixed printf of NULL which caused crashes on OpenSolaris when using --debug
include/my_global.h:
Added simple macro val_or_null() to simplify detecting of NULL strings for printf
sql/handler.cc:
Fixed printf of NULL which caused crashes on OpenSolaris when using --debug
sql/sql_db.cc:
Fixed printf of NULL which caused crashes on OpenSolaris when using --debug
Removed testing of 'new_db_name' as this is guranteed never NULL
sql/sql_show.cc:
Fixed printf of NULL which caused crashes on OpenSolaris when using --debug
storage/csv/ha_tina.cc:
Fixed realloc() problem that caused out of memory when running mysqldump.test on OpenSolaris
(OpenSolaris default malloc() can't handle a lot of reallocs() of strings that are growing one byte at a time)
This did speed up logging to cvs with a magnitude for large strings.
- Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
- Fixed a rase condition when two threads calls external_lock and thr_lock() in different order. When this happend the transaction that called external lock first
and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
- Fixed that one can run maria_chk on an automatcally recovered tables without warnings about too small transaction id
- Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
- Fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.
client/mysqldump.c:
Add "" around error message to make it more readable
client/mysqltest.cc:
Free environment variables
mysql-test/r/mysqldump.result:
Updated results
mysql-test/r/openssl_1.result:
Updated results
mysql-test/suite/maria/r/maria-recover.result:
Updated results
mysql-test/suite/maria/r/maria3.result:
Updated results
mysql-test/suite/maria/t/maria3.test:
Added more test of temporary tables
storage/maria/ha_maria.cc:
Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
Start transaction in ma_block_get_status() instead of in ha_maria::external_lock().
- This fixes a rase condition when two threads calls external lock and thr_lock() in different order. When this happend the transaction that called external lock first and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
Store latest transaction id in controll file if recovery was done.
- This allows one to run maria_chk on an automatcally recovered tables without warnings about too small transaction id
storage/maria/ha_maria.h:
Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
storage/maria/ma_blockrec.h:
Added new function "_ma_block_get_status_no_versioning()"
storage/maria/ma_init.c:
Added hook to create trn in ma_block_get_status() if we are using MariaDB
storage/maria/ma_open.c:
Ensure we call _ma_block_get_status_no_versioning() for transactional tables without versioning (like tables with fulltext)
storage/maria/ma_pagecache.c:
Allow one to flush blocks that are pinned for read.
This fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.
storage/maria/ma_recovery.c:
Set maria_recovery_changed_data to 1 if recover changed something.
Set max_trid_in_control_file to max found trn if we found a bigger trn.
The allows will ensure that the control file is up to date after recovery which allows one to run maria_chk on the tables without warnings about too big trn
storage/maria/ma_state.c:
Call maria_create_trn_hook() in _ma_setup_live_state() instead of ha_maria::external_lock()
This ensures that 'state' and trn are in sync and thus fixes the race condition mentioned for ha_maria.cc
storage/maria/ma_static.c:
Added maria_create_trn_hook() and maria_recovery_changed_data
storage/maria/maria_def.h:
Added MARIA_HANDLER->external_ptr, which is used to hold MariaDB thd.
Added some new external variables
Removed reference to non existing function: maria_concurrent_inserts()
strict aliasing violations.
Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.
As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.
The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.
client/mysql.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysql_upgrade.c:
Remove now-unnecessary casts.
client/mysqladmin.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysqlbinlog.cc:
Remove now-unnecessary casts.
client/mysqlcheck.c:
Remove now-unnecessary casts.
client/mysqldump.c:
Remove now-unnecessary casts.
client/mysqlimport.c:
Remove now-unnecessary casts.
client/mysqlshow.c:
Remove now-unnecessary casts.
client/mysqlslap.c:
Remove now-unnecessary casts.
client/mysqltest.cc:
Remove now-unnecessary casts.
extra/comp_err.c:
Remove now-unnecessary casts.
extra/my_print_defaults.c:
Remove now-unnecessary casts.
Break up large strings.
extra/mysql_waitpid.c:
Remove now-unnecessary casts.
extra/perror.c:
Remove now-unnecessary casts.
extra/resolve_stack_dump.c:
Remove now-unnecessary casts.
extra/resolveip.c:
Remove now-unnecessary casts.
include/my_getopt.h:
Use a void pointer type as the opaque type to avoid problems with type
incompatibility -- GCC issues warnings when the type name is not type
compatible with a operand. As a side bonus, a explicit cast won't be
necessary anymore.
include/sslopt-longopts.h:
Remove now-unnecessary casts.
Break up large strings.
mysys/my_getopt.c:
Update opaque type and introduce a type definition for the
argument to my_getopt_register_get_addr.
server-tools/instance-manager/options.cc:
Remove now-unnecessary casts.
sql/mysqld.cc:
Remove now-unnecessary casts.
Break up large strings.
Update mysql_getopt_value prototype (the old prototype
was different from the definition anyway).
sql/sql_plugin.cc:
The type of a pointer to a function must be compatible with the
pointed-to function type, otherwise the behavior is undefined.
sql/table.cc:
The variable buf pointer to pointer to pointer to constant char
could improperly alias a incompatible type in call to fix_type_
pointers. Since this was actually dead code, it is simply removed.
sql/unireg.cc:
Remove call to get_form_pos. The code creates a new FRM file which
is always truncated and writes the form position as 0. Hence, no
need to retrieve it, we now for sure it is 0.
storage/archive/archive_reader.c:
Remove now-unnecessary casts.
storage/myisam/ft_nlq_search.c:
Read weight directly from the buffer.
storage/myisam/fulltext.h:
Add explanation about the type duality of a key buffer.
Add accessor macro to retrieve a FT float value.
storage/myisam/mi_test1.c:
Remove now-unnecessary casts.
storage/myisam/myisam_ftdump.c:
Read weight directly from the buffer.
storage/myisam/myisamchk.c:
Remove now-unnecessary casts.
storage/myisam/myisamlog.c:
A pointer to char was used to alias a pointer to pointer to
unsigned char, thus violating strict aliasing rules.
storage/myisam/myisampack.c:
Remove now-unnecessary casts.
strings/decimal.c:
Remove aliasing violation, printing the value is enough for
debugging purposes.
tests/mysql_client_test.c:
Remove now-unnecessary casts.
strict aliasing violations.
Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.
As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.
The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.
WL#5154 was a task for formally deprecating and removing items that
were mentioned in the manual as having been deprecated since MySQL
4.1 or 5.0, but that had never been removed.
Since WL#5154 was created, examination of mysqld.cc, mysql.cc, and
mysqldump.c reveals additional deprecations not mentioned in the
manual. (In some cases, the items are simply not mentioned in the
5.1+ manuals.)
This is a follow-on task to deprecate and remove these additional
items.
The deprecation happened in MySQL 5.1, and the options/variables
are now removed from the code.
client/mysql.cc:
--no-tee is now removed
client/mysqldump.c:
--all is now removed
-a now points to --create-options
sql/mysqld.cc:
delay-key-write-for-all-tables is removed
--enable-locking is removed
--log-update is removed
--skip-locking is removed
--skip-symlink is removed
--sql-bin-update-same is removed
--warnings is removed
--record-buffer is removed
WL#5154 was a task for formally deprecating and removing items that
were mentioned in the manual as having been deprecated since MySQL
4.1 or 5.0, but that had never been removed.
Since WL#5154 was created, examination of mysqld.cc, mysql.cc, and
mysqldump.c reveals additional deprecations not mentioned in the
manual. (In some cases, the items are simply not mentioned in the
5.1+ manuals.)
This is a follow-on task to deprecate and remove these additional
items.
The deprecation happened in MySQL 5.1, and the options/variables
are now removed from the code.
A set of program options and variables was deprecated in
MySQL 5.1, and is hereby removed.
client/mysql.cc:
--no-auto-rehash (-A) is no longer deprecated
--no-named-commands (-g) is now removed
--skip-line-numbers (-L) is no longer deprecated
--set-variable (-O) is now removed
--no-pager is now removed
client/mysqlbinlog.cc:
--position is now removed (use --start-position)
-j is now equivalent with --start-position
client/mysqldump.c:
--first-slave is now removed
--no-set-names (-N) is now removed
--set-variable (-O) is now removed
mysql-test/include/default_mysqld.cnf:
default-character-set is removed as an option
character-set-server is equivalent.
mysql-test/t/bug47671-master.opt:
default-character-set option is removed
character-set-server is equivalent
mysql-test/t/ctype_latin1_de-master.opt:
default-character-set option is removed
character-set-server is equivalent
mysql-test/t/ctype_ucs2_def-master.opt:
default-collation is removed
collation-server is equicalent
scripts/mysqld_multi.sh:
--config-file has been superseded by
--defaults-extra-file
sql/mysql_priv.h:
Removed the version number in the deprecation
warning text, as decided by ServerPT.
sql/mysqld.cc:
--default-character-set (-C) is removed
--default-collation is removed
--log-long-format (-0) is removed
--safe-show-database is removed
--set-variable (-O) is removed
sql/sql_yacc.yy:
The FRAC_SECOND keyword is removed
sql/sys_vars.cc:
The sql_log_update system variable is removed
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_slow_query_log.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/options.cc
Text conflict in sql/mysqld.cc
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_slow_query_log.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/options.cc
Text conflict in sql/mysqld.cc
