* only copy args[0] to args[2] after fix_fields (when all item
substitutions have already happened)
* change QT_ITEM_FUNC_NULLIF_TO_CASE (that allows to print NULLIF
as CASE) to QT_ITEM_ORIGINAL_FUNC_NULLIF (that prohibits it).
So that NULLIF-to-CASE is allowed by default and only disabled
explicitly for SHOW VIEW|FUNCTION|PROCEDURE and mysql_make_view.
By default it is allowed (in particular in error messages and
debug output, that can happen anytime before or after optimizer).
This fixes
MDEV-9538 Server crashes in check_show_access on SHOW STATISTICS
MDEV-9539 Server crashes in make_columns_old_format on SHOW GEOMETRY_COLUMNS
MDEV-9540 SHOW SPATIAL_REF_SYS and SHOW SYSTEM_VARIABLES return empty results with numerous warnings
Problem:
========
1) Drop table queries are re-generated by server
before writing the events(queries) into binlog
for various reasons. If table name/db name contains
a non regular characters (like latin characters),
the generated query is wrong. Hence it breaks the
replication.
2) In the edge case, when table name/db name contains
64 characters, server is throwing an assert
assert(M_TBLLEN < 128)
3) In the edge case, when db name contains 64 latin
characters, binlog content is interpreted badly
which is leading replication failure.
Analysis & Fix :
================
1) Parser reads the table name from the query and converts
it to standard charset(utf8) and stores it in table_name variable.
When drop table query is regenerated with the same table_name
variable, it should be converted back to the original charset
from standard charset(utf8).
2) Latin character takes two bytes for each character. Limit
of the identifier is 64. SYSTEM_CHARSET_MBMAXLEN is set to '3'.
So there is a possiblity that tablename/dbname contains 3 * 64.
Hence assert is changed to
(M_TBLLEN <= NAME_CHAR_LEN*SYSTEM_CHARSET_MBMAXLEN)
3) db_len in the binlog event header is taking 1 byte.
db_len is ranged from 0 to 192 bytes (3 * 64).
While reading the db_len from the event, server
is casting to uint instead of uchar which is leading
to bad db_len. This problem is fixed by changing the
cast type to uchar.
The following left in semi-improved state to keep patch size reasonable:
- Field operator new: left thd_alloc(current_thd)
- Sql_alloc operator new: left thd_alloc(thd_get_current_thd())
- Item_args constructors: left thd_alloc(thd)
- Item_func_interval::fix_length_and_dec(): no THD arg, have to call current_thd
- Item_func_dyncol_exists::val_int(): same
- Item_dyncol_get::val_str(): same
- Item_dyncol_get::val_int(): same
- Item_dyncol_get::val_real(): same
- Item_dyncol_get::val_decimal(): same
- Item_singlerow_subselect::fix_length_and_dec(): same
Fixed race condition in code filling INFORMATION_SCHEMA.PROCESSLIST.INFO_BINARY.
When loading query string/length of another connection one must have
LOCK_thd_data locked.
- Part 4: Removing calls to sql_alloc() and sql_calloc()
Other things:
- Added current_thd in some places to make it clear that it's called (easier to remove later)
- Move memory allocation from Item_func_case::fix_length_and_dec() to Item_func_case::fix_fields()
- Added mem_root to some new calls
- Fixed some wrong UNINIT_VAR() calls
- Fixed a bug in generate_partition_syntax() in case of errors
- Added mem_root to argument to new thread_info
- Simplified my_parse_error() call in sql_yacc.yy
- Part 3: Adding mem_root to push_back() and push_front()
Other things:
- Added THD as an argument to some partition functions.
- Added memory overflow checking for XML tag's in read_xml()
- Added mem_root to all calls to new Item
- Added private method operator new(size_t size) to Item to ensure that
we always use a mem_root when creating an item.
This saves use once call to current_thd per Item creation
Added mandatory thd parameter to Item (and all derivative classes) constructor.
Added thd parameter to all routines that may create items.
Also removed "current_thd" from Item::Item. This reduced number of
pthread_getspecific() calls from 290 to 177 per OLTP RO transaction.
In galera, like other DDLs, CREATE/ALTER VIEW commands are recreated
and replicated during parsing. The ALGORITHM clause is internally set
to VIEW_ALGORITHM_INHERIT if its not explicitly specified by the user.
But since its not a valid type to be used in a command, it leads to an
assertion failure. The solution is to not include the ALGORITHM clause
in the command if its not explicitly specified (or INHERIT).
- Changed ER(ER_...) to ER_THD(thd, ER_...) when thd was known or if there was many calls to current_thd in the same function.
- Changed ER(ER_..) to ER_THD_OR_DEFAULT(current_thd, ER...) in some places where current_thd is not necessary defined.
- Removing calls to current_thd when we have access to thd
Part of this is optimization (not calling current_thd when not needed),
but part is bug fixing for error condition when current_thd is not defined
(For example on startup and end of mysqld)
Notable renames done as otherwise a lot of functions would have to be changed:
- In JOIN structure renamed:
examined_rows -> join_examined_rows
record_count -> join_record_count
- In Field, renamed new_field() to make_new_field()
Other things:
- Added DBUG_ASSERT(thd == tmp_thd) in Item_singlerow_subselect() just to be safe.
- Removed old 'tab' prefix in JOIN_TAB::save_explain_data() and use members directly
- Added 'thd' as argument to a few functions to avoid calling current_thd.
Initialize abs_timeout when it is about to be used. This saves one my_hrtime()
call on hot path (when we acquire MDL lock without waiting).
When filling I_S.PROCESSLIST use THD::start_utime/THD::utime_after_query instead
of THD::start_time. This allows us to save 2 clock_gettime() calls.
Overhead change:
__clock_gettime 0.13% -> 0.11% (122 -> 76 calls per OLTP RO transaction)
my_interval_timer 0.07% -> 0.06%
my_hrtime 0.04% -> 0.01%
THD::enter_stage() optimizations:
- stage backup code moved to THD::backup_stage(), saves one condition
- moved check for "new_stage" out to callers that actually need it
- remnants of enter_stage() moved to sql_class.h so it can be inlined
THD::enter_stage() overhead dropped 0.48% -> 0.07%.
PROFILING::status_change() optimizations:
- "status_arg" is now checked by QUERY_PROFILE::new_status()
- no need to check "enabled": !enabled && current is impossible
- remnants of status_change() moved to sql_profile.h so it can be inlined
PROFILING::status_change() overhead dropped 0.1% -> out of radar.
The bug was that open_tables() returned error in case of
thd->killed() without properly calling thd->send_kill_message()
to set the correct error. This was fixed already in get_schema_column_record,
so the code was just copied to get_geometry_column_record.
Description: On an example MySQL instance with 28k empty
InnoDB tables, a specific query to information_schema.tables
and information_schema.columns leads to memory consumption
over 38GB RSS.
Analysis: In get_all_tables() call, we fill the I_S tables
from frm files and storage engine. As part of that process
we call make_table_name_list() and allocate memory for all
the 28k frm file names in the THD mem_root through
make_lex_string_root(). Since it has been called around
28k * 28k times there is a huge memory getting hogged in
THD mem_root. This causes the RSS to grow to 38GB.
Fix: As part of fix we are creating a temporary mem_root
in get_all_tables and passing it to fill_fiels(). There we
replace the THD mem_root with the temporary mem_root and
allocates the file names in temporary mem_root and frees
it once we fill the I_S tables in get_all_tables and
re-assign the original mem_root back to THD mem_root.
Note: Checked the massif out put with the fix now the memory growth is just around 580MB at peak.
BINLOGGED INCORRECTLY - BREAKS A SLAVE
Submitted a incomplete patch with my previous push,
re submitting the extra changes the required to make
the patch complete.
Stage "Filling schema table" is now properly shown in 'show processlist'
mysys/mf_keycache.c:
Simple cleanup with more comments
sql/lock.cc:
Return to original stage after mysql_lock_tables
Made 'Table lock' as a true stage
sql/sql_show.cc:
Restore original stage after get_schema_tables_result
The GEOMETRY field metadata is stored in the FRM file.
SRID for a spatial column now can be stored, it was added to the CREATE TABLE syntax,
so the AddGeometryData() stored procedure is now possible. Script adding the required Add/DropGeometryColumn sp-s added.
The real bug was that open_tables() returned error in case of
thd->killed() without properly calling thd->send_kill_message()
to set the correct error. This was fixed some time ago.
So remove the, now redundant, extra checks for thd->is_error(),
possibly allowing to catch in debug builds more incorrect
error handling cases.
Extend existing plugins to support
* SHOW QUERY_RESPONSE_TIME
* FLUSH QUERY_RESPONSE_TIME
* SHOW LOCALE
move userstat tables to use the new API instead of
hand-coded syntax
