- Added MALLOC_LIBRARY variable to hold name of malloc library
- Back ported valgrind related fixes from jemalloc 3.4.1 to the included jemalloc 3.3.1
- Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free() to avoid clash with jemalloc 3.4.1
- Use option --soname-synonyms=somalloc=NON to valgrind when using jemalloc
- Show version related variables in mysqld --help
-- Added SHOW_VALUE_IN_HELP marker
Increased back_log to 150 as the original value was a bit too small
CMakeLists.txt:
Added MALLOC_LIBRARY variable to hold name of malloc library
cmake/jemalloc.cmake:
Added MALLOC_LIBRARY variable to hold name of malloc library
config.h.cmake:
Added MALLOC_LIBRARY variable to hold name of malloc library
extra/jemalloc/ChangeLog:
Updates changelog
extra/jemalloc/include/jemalloc/internal/arena.h:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/include/jemalloc/internal/jemalloc_internal.h.in:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/include/jemalloc/internal/private_namespace.h:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/include/jemalloc/internal/tcache.h:
Backported valgrind fixes from jemalloc 3.4.1
extra/jemalloc/src/arena.c:
Backported valgrind fixes from jemalloc 3.4.1
include/my_bitmap.h:
Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free() to avoid clash with jemalloc 3.4.1
mysql-test/mysql-test-run.pl:
Use option --soname-synonyms=somalloc=NON to valgrind when using jemalloc
mysql-test/valgrind.supp:
Supression of memory leak in OpenSuse 12.3
mysys/my_bitmap.c:
Renamed bitmap_init() and bitmap_free() to my_bitmap_init() and my_bitmap_free()
sql/ha_ndbcluster_binlog.cc:
Renames
sql/ha_ndbcluster_cond.h:
Renames
sql/ha_partition.cc:
Renames
sql/handler.cc:
Renames
sql/item_subselect.cc:
Renames
sql/log_event.cc:
Renames
sql/log_event_old.cc:
Renames
sql/mysqld.cc:
Renames
Show version related variables in mysqld --help
sql/opt_range.cc:
Renames
sql/opt_table_elimination.cc:
Renames
sql/partition_info.cc:
Renames
sql/rpl_injector.h:
Renames
sql/set_var.h:
Renames
sql/slave.cc:
Renames
sql/sql_bitmap.h:
Renames
sql/sql_insert.cc:
Renames
sql/sql_lex.h:
Renames
sql/sql_parse.cc:
Renames
sql/sql_partition.cc:
Renames
sql/sql_select.cc:
Renames
sql/sql_show.cc:
Renames
sql/sql_update.cc:
Renames
sql/sys_vars.cc:
Show version related variables in mysqld --help
sql/sys_vars.h:
Added SHOW_VALUE_IN_HELP marker for variables that should be shown in --help
sql/table.cc:
Renames
sql/table.h:
Removed not used bitmap_init_value
storage/connect/ha_connect.cc:
Removed compiler warning
storage/maria/ma_open.c:
Renames
unittest/mysys/bitmap-t.c:
Renames
- Query plan footprint (in new terms, "EXPLAIN structure") should always keep
a copy of key_name. This is because the table might be a temporary table which
may be already freed by the time we use query plan footprint.
- Make QPF structures store data members, not strings.
This is not fully possible, because table names (and hence
key names, etc) can be deleted, and we have to store strings.
For queries like
update t1 set ... where <unique key> order by ... limit ...
we need to handle the fact that unique keys allow NULL
values, and hence can return more than one row.
sql/opt_range.cc:
When the unique key has multiple key parts,
check each key_part for nullability, rather than the first key part.
(s/key->part ==/key_tree->part ==/)
Also: revert the if() test, for better readability.
For queries like
update t1 set ... where <unique key> order by ... limit ...
we need to handle the fact that unique keys allow NULL
values, and hence can return more than one row.
Currently the loose scan code in opt_range.cc considers all indexes as
possible for the access method. Due to inexact statistics it may happen
that a loose scan is selected over a unique index.
This is clearly wrong since a "loose scan" over a unique index will read
the same keys as a direct index scan, but the loose scan has more overhead.
This task adds a rule to skip unique indexes for loose scan.
Added forgotten partition_explicit_prune from mysql-5.6.
Backported the fix for
Bug#13559657: PARTITION SELECTION DOES NOT WORK WITH VIEWS
from mysql-5.6
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
In the case of loose scan used as input for order by, end_send()
didn't detect correctly that a loose scan was used, and didn't copy
the non-aggregated fields from the temp table used for ORDER BY.
The fix uses the fact that the quick select used for sorting is
attached to JOIN::pre_sort_join_tab instead of JOIN::join_tab.
- merge_same_index_scans() may put the same SEL_ARG tree in multiple result plans.
make it call incr_refs() on the SEL_ARG trees that it does key_or() on, because
key_or(sel_arg_tree_1, sel_arg_tree_2) call may invalidate SEL_ARG trees pointed
by sel_arg_tree_1 and sel_arg_tree_2.
-Added test and extra code to ensure we don't leave keyread on for a handler table.
-Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
mysql-test/include/default_mysqld.cnf:
Run test suite with smaller aria keybuffer size
mysql-test/suite/maria/maria3.result:
Run test suite with smaller aria keybuffer size
mysql-test/suite/sys_vars/r/aria_pagecache_buffer_size_basic.result:
Run test suite with smaller aria keybuffer size
sql/handler.cc:
Disable key read (extra safety if something went wrong)
sql/multi_range_read.cc:
Ensure we have don't leave keyread on for secondary_file
sql/opt_range.cc:
Simplify code with mark_columns_used_by_index_no_reset()
Ensure that read_keys_and_merge() disableds keyread if it enables it
sql/opt_subselect.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_derived.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_select.cc:
Use 'enable_keyread()' instead of calling HA_EXTRA_RESET. (Makes debugging easier)
Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
Remove not anymore used argument for create_internal_tmp_table()
More DBUG
sql/sql_select.h:
Remove not anymore used argument for create_internal_tmp_table()
get_mm_leaf function can store all sorts of spatial features in
one type of field it receives from an Item_field.
So we just allow that by setting the type of this field to GEOMETRY.
per-file comments:
mysql-test/r/gis-rtree.result
result updated
mysql-test/t/gis-rtree.test
test case added.
sql/opt_range.cc
set geom_type=GEOMETRY if we got Field_geom.
Solaris fixes:
- Fixed that wait_timeout_func and wait_timeout tests works on solaris
- We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO).
- Fixed that compile-solaris-amd64-debug works (before that we got a wrong ELF class: ELFCLASS64 on linkage)
- Added missing sync_with_master
Other bug fixes:
- Free memory for rpl_global_gtid_binlog_state before exit() to avoid 'accessing uninitalized mutex' error.
BUILD/FINISH.sh:
Fixed issues on Solaris with ksh
BUILD/compile-solaris-amd64-debug:
Added missing -m64 flag
configure.cmake:
We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO)
mysql-test/suite/rpl/t/rpl_gtid_mdev4473.test:
- Added missing sync_with_master (fix by knielsen)
sql-common/client.c:
Added () to get rid of compiler warning
sql/item_strfunc.cc:
Fixed compiler warning
sql/log.cc:
Free memory for static variable rpl_global_gtid_binlog_state before exit()
- If we are compiling with safemalloc, we would try to call sf_free() for some members after sf_terminate() was called, which would result of trying to access the uninitalized mutex 'sf_mutex'
sql/multi_range_read.cc:
Fixed compiler warnings of converting double to ulong.
sql/opt_range.cc:
Fixed compiler warnings of converting double to ulong or uint
- Better to have all variables that can be number of rows as 'ha_rows'
sql/rpl_gtid.cc:
Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/rpl_gtid.h:
Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/set_var.cc:
Fixed compiler warning
sql/sql_join_cache.cc:
Fixed compiler warnings of converting double to uint
sql/sql_show.cc:
Added cast to get rid of compiler warning
sql/sql_statistics.cc:
Remove code that didn't do anything.
(store_record() with record[0] is a no-op)
storage/xtradb/os/os0file.c:
Added __attribute__ ((unused))
support-files/compiler_warnings.supp:
Ignore warnings from atomic_add_64_nv
(was not able to fix this with a cast as the macro is a bit different between systems)
vio/viosocket.c:
Added more DBUG_PRINT
GROUP BY, MYISAM
Problem:-
In a query, where we are using loose index scan optimization and
we have MIN() causes segmentation fault(where table row length
is less then key_length).
Analysis:
While using loose index scan for MIN(), we call key_copy(), to copy
the key data from record.
This function is using temporary record buffer to store key data
from the record buffer.But in case where the key length is greater
then the buffer length, this will cause a segmentation fault.
Solution:
Give a proper buffer to store a key record.
sql/opt_range.cc:
We can't use record buffer to store key data.So, give a proper buffer to store a key record.
GROUP BY, MYISAM
Problem:-
In a query, where we are using loose index scan optimization and
we have MIN() causes segmentation fault(where table row length
is less then key_length).
Analysis:
While using loose index scan for MIN(), we call key_copy(), to copy
the key data from record.
This function is using temporary record buffer to store key data
from the record buffer.But in case where the key length is greater
then the buffer length, this will cause a segmentation fault.
Solution:
Give a proper buffer to store a key record.
PARTIAL INDEX
Consider the following table definition:
CREATE TABLE t (
my_col CHAR(10),
...
INDEX my_idx (my_col(1))
)
The my_idx index is not able to distinguish between rows with
equal first-character my_col-values (e.g. "f", "foo", "fee").
Prior to this CS, the range optimizer would translate
"WHERE my_col NOT IN ('f', 'h')" into (optimizer trace syntax)
"ranges": [
"NULL < my_col < f",
"f < my_col"
]
But this was not correct because the rows with values "foo"
and "fee" would not belong to any of those ranges. However, the
predicate "my_col != 'f' AND my_col != 'h'" would translate
to
"ranges": [
"NULL < my_col"
]
because get_mm_leaf() changes from "<" to "<=" for partial
keyparts. This CS changes the range optimizer implementation
for NOT IN to behave like a conjunction of NOT EQUAL: it
replaces "<" with "<=" for all but the first range when the
keypart is partial.
PARTIAL INDEX
Consider the following table definition:
CREATE TABLE t (
my_col CHAR(10),
...
INDEX my_idx (my_col(1))
)
The my_idx index is not able to distinguish between rows with
equal first-character my_col-values (e.g. "f", "foo", "fee").
Prior to this CS, the range optimizer would translate
"WHERE my_col NOT IN ('f', 'h')" into (optimizer trace syntax)
"ranges": [
"NULL < my_col < f",
"f < my_col"
]
But this was not correct because the rows with values "foo"
and "fee" would not belong to any of those ranges. However, the
predicate "my_col != 'f' AND my_col != 'h'" would translate
to
"ranges": [
"NULL < my_col"
]
because get_mm_leaf() changes from "<" to "<=" for partial
keyparts. This CS changes the range optimizer implementation
for NOT IN to behave like a conjunction of NOT EQUAL: it
replaces "<" with "<=" for all but the first range when the
keypart is partial.
When calculating the selectivity of a range in the function
get_column_range_cardinality a check whether NULL values are
included into into the range must be done.
currently get_mm_tree skipped the evaluation of this constant
and icorrectly proceeded. The correct behavior is to return a
NULL subtree, according to the IF branch being fixed - when it
evaluates the constant it returns a value, and doesn't continue
further.
When performing the range analysis for a conjunction the function
calculate_cond_selectivity_for_table should take in to account that
the analysis of some conjuncts may return SEL_ARG::IMPOSSIBLE.
- Let index_merge allocate table handlers on quick select's MEM_ROOT,
not on statement's MEM_ROOT.
This is crucial for big "range checked for each record" queries, where
index_merge can be created and deleted many times during query exection.
We should not make O(#rows) allocations on statement's MEM_ROOT.
The range optimizer uses 'save_in_field_no_warnings()' to verify properties of
'value <cmp> field' expressions.
If this execution yields an error, it should abort.
The range optimizer uses 'save_in_field_no_warnings()' to verify properties of
'value <cmp> field' expressions.
If this execution yields an error, it should abort.
Analysis:
Range analysis detects that the subquery is expensive and doesn't
build a range access method. Later, the applicability test for loose
scan doesn't take that into account, and builds a loose scan method
without a range scan on the min/max column. As a result loose scan
fetches the first key in each group, rather than the first key that
satisfies the condition on the min/max column.
Solution:
Since there is no SEL_ARG tree to be used for the min/max column,
it is not possible to use loose scan if the min/max column is compared
with an expensive scalar subquery. Make the test for loose scan
applicability to be in sync with the range analysis code by testing if
the min/max argument is compared with an expensive predicate.
Analys:
The cause for the wrong result was that the optimizer
incorrectly chose min/max loose scan when it is not
applicable. The applicability test missed the case when
a condition on the MIN/MAX argument was OR-ed with a
condition on some other field. In this case, the MIN/MAX
condition cannot be used for loose scan.
Solution:
Extend the test check_group_min_max_predicates() to check
that the WHERE clause is of the form: "cond1 AND cond2"
where
cond1 - does not use min_max_column at all.
cond2 - is an AND/OR tree with leaves in form "min_max_column $CMP$ const"
or $CMP$ is one of the functions between, is [not] null
Analysis:
Range analysis discoveres that the query can be executed via loose index scan for GROUP BY.
Later, GROUP BY analysis fails to confirm that the GROUP operation can be computed via an
index because there is no logic to handle duplicate field references in the GROUP clause.
As a result the optimizer produces an inconsistent plan. It constructs a temporary table,
but on the other hand the group fields are not set to point there.
Solution:
Make loose scan analysis work in sync with order by analysis. In the case of duplicate
columns loose scan will not be applicable. This limitation will be lifted in 10.0 by
removing duplicate columns.
ON COL WITH COMPOSITE INDEX
This problem is caused by the patch for the bug#11751794.
While checking for the keypart covering non grouping attribute. we are not
checking whether the root node of the SEL_ARG* tree for the index have any
cvalue or not.
sql/opt_range.cc:
check whether the keeypart_tree has any range tree.
ON COL WITH COMPOSITE INDEX
This problem is caused by the patch for the bug#11751794.
While checking for the keypart covering non grouping attribute. we are not
checking whether the root node of the SEL_ARG* tree for the index have any
cvalue or not.
ON COL WITH COMPOSITE INDEX
This problem is caused by the patch for the bug#11751794.
While checking for the keypart covering non grouping attribute. we are not
checking whether the root node of the SEL_ARG* tree for the index have any
cvalue or not.
ON COL WITH COMPOSITE INDEX
This problem is caused by the patch for the bug#11751794.
While checking for the keypart covering non grouping attribute. we are not
checking whether the root node of the SEL_ARG* tree for the index have any
cvalue or not.
from MariaDB 10.0.
The bug in mdev-3948 was an instance of the problem fixed by Sergey's patch
in 10.0 - namely that the range optimizer could change table->[read | write]_set,
and not restore it.
revno: 3471
committer: Sergey Petrunya <psergey@askmonty.org>
branch nick: 10.0-serg-fix-imerge
timestamp: Sat 2012-11-03 12:24:36 +0400
message:
# MDEV-3817: Wrong result with index_merge+index_merge_intersection, InnoDB table, join, AND and OR conditions
Reconcile the fixes from:
#
# guilhem.bichot@oracle.com-20110805143029-ywrzuz15uzgontr0
# Fix for BUG#12698916 - "JOIN QUERY GIVES WRONG RESULT AT 2ND EXEC. OR
# AFTER FLUSH TABLES [-INT VS NULL]"
#
# guilhem.bichot@oracle.com-20111209150650-tzx3ldzxe1yfwji6
# Fix for BUG#12912171 - ASSERTION FAILED: QUICK->HEAD->READ_SET == SAVE_READ_SET
# and
#
and related fixes from: BUG#1006164, MDEV-376:
Now, ROR-merged QUICK_RANGE_SELECT objects make no assumptions about the values
of table->read_set and table->write_set.
Each QUICK_ROR_SELECT has (and had before) its own column bitmap, but now, all
QUICK_ROR_SELECT's functions that care: reset(), init_ror_merged_scan(), and
get_next() will set table->read_set when invoked and restore it back to what
it was before the call before they return.
This allows to avoid the mess when somebody else modifies table->read_set for
some reason.
Base code and idea from a patch from by plinux at Taobao.
The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.
Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.
The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.
Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)
client/completion_hash.cc:
Updated call to init_alloc_root()
client/mysql.cc:
Updated call to init_alloc_root()
client/mysqlbinlog.cc:
init_dynamic_array() -> my_init_dynamic_array()
Updated call to init_alloc_root()
client/mysqlcheck.c:
Updated call to my_init_dynamic_array()
client/mysqldump.c:
Updated call to init_alloc_root()
client/mysqltest.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Fixed compiler warnings
extra/comp_err.c:
Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
Updated call to my_init_dynamic_array()
include/hash.h:
Added HASH_THREAD_SPECIFIC
include/heap.h:
Added flag is internal temporary table.
include/my_dir.h:
Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
Added malloc_flags to DYNAMIC_ARRAY
Added extra mysys flag argument to my_init_dynamic_array()
Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
Updated paramaters for init_alloc_root()
include/my_tree.h:
Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
Updated parameters to init_tree()
include/myisamchk.h:
Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
Added MYSQL_THREAD_SPECIFIC_MALLOC
Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
Updated file
include/mysql_com.h:
Used 'unused1' to mark memory as thread specific.
Updated parameters for my_net_init()
libmysql/libmysql.c:
Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
Updated call to init_alloc_root()
mysql-test/r/create.result:
Updated results
mysql-test/r/user_var.result:
Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
Updated results
mysql-test/t/show_explain.test:
Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
Added test of memory_usage variable.
mysys/array.c:
Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
All allocated memory is marked with the given my_flags.
Removed obsolete function init_dynamic_array()
mysys/default.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
mysys/hash.c:
Updated call to my_init_dynamic_array_ci().
Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
init_dynamic_array() -> my_init_dynamic_array()
Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
Extend MEM_ROOT with a flag if memory is thread specific.
This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
Store at start of each allocated memory block the size of the block and if the block is thread specific.
Call malloc_size_cb_func, if set, with the memory allocated/freed.
Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
Added flags to init_tree() so that one can mark tree to be thread specific.
Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
Updated call to init_alloc_root()
All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
Updated call to my_init_dynamic_array()
sql-common/client.c:
Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
Updated call to my_init_dynamic_array().
Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
Updated call to init_alloc_root()
sql/debug_sync.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
Removed calls to net_end() as this is now done in ~THD()
Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
Updated call to init_alloc_root()
Updated call to my_net_init()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
Updated call to my_net_init()
Updated call to init_sql_alloc()
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
Updated call to init_alloc_root()
sql/handler.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
Added missing call to my_dir_end()
sql/item_func.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
More DBUG
Updated call to init_alloc_root()
sql/mdl.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
Added total_memory_used
Updated call to init_alloc_root()
Move mysql_cond_broadcast() before my_thread_end()
Added mariadb_dbug_id() to count memory per THD instead of per thread.
Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
Updated call to my_init_dynamic_array().
Updated call to my_net_init().
Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
Added status variable 'memory_used'.
Updated call to init_alloc_root()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
Added set_current_thd()
sql/net_serv.cc:
Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
Store in net->thread_specific_malloc if memory is thread specific.
Mark memory to be thread specific if requested.
sql/opt_range.cc:
Updated call to my_init_dynamic_array()
Updated call to init_sql_alloc()
Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
Fixed compiler warning
sql/records.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
Updated call to init_sql_alloc()
sql/rpl_mi.cc:
Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
Updated call to init_alloc_root()
sql/rpl_utility.cc:
Updated call to my_init_dynamic_array()
sql/slave.cc:
Initialize things properly before calling functions that allocate memory.
Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
Updated call to init_sql_alloc()
Updated call to my_init_dynamic_array()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
Updated call to my_init_dynamic_array()
sql/sql_base.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
Updated comment
sql/sql_class.cc:
Added parameter to warning_info() that not initialize it until THD is fully created.
Updated call to init_sql_alloc()
Mark THD::user_vars has to be thread specific.
Updated call to my_init_dynamic_array()
Ensure that memory allocated by THD is assigned to the THD.
More DBUG
Always acll net_end() in ~THD()
Assert that all memory signed to this THD is really deleted at ~THD.
Fixed set_status_var_init() to not reset memory_used.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
Added MY_THREAD_SPECIFIC to allocated memory.
Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
This is used by THD::THD() to not allocate memory until THD is ready.
Added Warning_info::free_memory()
sql/sql_error.h:
Updated Warning_info() class.
sql/sql_handler.cc:
Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
More DBUG
sql/sql_join_cache.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
Updated call to my_init_dynamic_array()
sql/sql_lex.h:
Updated call to my_init_dynamic_array()
sql/sql_load.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
Ensure that examined_row_count() is reset before query.
Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
Added parameter to warning_info() that it should be fully initialized.
Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
Added MY_THREAD_SPECIFIC to allocated memory.
More DBUG
sql/sql_servers.cc:
Updated call to init_sql_alloc() to mark memory some memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
Updated call to my_init_dynamic_array()
Mark my_dir() memory thread specific.
Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
More DBUG.
Added malloc_size and examined_row_count to SHOW PROCESSLIST.
Added MY_THREAD_SPECIFIC to allocated memory.
Updated call to init_sql_alloc()
Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
Fixed compiler warning
sql/sql_string.cc:
String elements can now be marked as thread specific.
sql/sql_string.h:
String elements can now be marked as thread specific.
sql/sql_table.cc:
Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
Fixed compiler warning
sql/sql_test.cc:
Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
Updated call to init_sql_alloc()
sql/sql_udf.cc:
Updated call to init_sql_alloc()
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
Updated call to init_sql_alloc().
Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
Updated call to init_sql_alloc()
sql/thr_malloc.cc:
Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
Updated prototype for init_sql_alloc()
sql/tztime.cc:
Updated call to init_sql_alloc()
Updated call to init_alloc_root() to mark memory thread specific.
my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
Updated call to init_alloc_root()
Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
Updated call to init_tree()
storage/maria/ma_ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
Updated call to init_tree()
storage/myisam/ft_parser.c:
Updated call to init_tree()
Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
Updated call to init_tree()
storage/myisam/mi_check.c:
Updated call to init_alloc_root()
storage/myisam/mi_write.c:
Updated call to my_init_dynamic_array()
Updated call to init_tree()
storage/myisam/myisamlog.c:
Updated call to init_tree()
storage/myisam/myisampack.c:
Updated call to init_tree()
storage/myisam/sort.c:
Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
Rest current_thd
storage/perfschema/pfs_instr.cc:
Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
Disable compiler warning from offsetof macro.
Consider the following query:
SELECT f_1,..,f_m, AGGREGATE_FN(C)
FROM t1
WHERE ...
GROUP BY ...
Loose index scan ("Using index for group-by") can be used for
this query if there is an index 'i' covering all fields in the
select list, and the GROUP BY clause makes up a prefix f1,...,fn
of 'i'. Furthermore, according to rule NGA2 of
get_best_group_min_max(), the WHERE clause must contain a
conjunction of equality predicates for all fields fn+1,...,fm.
The problem in this bug was that a query with WHERE clause that
broke NGA2(NGA: Non Group Attribuite) was not detected and therefore
used loose index scan.
This lead to wrong result. The query had an index
covering (c1,c2) and had:
"WHERE (c1 = 1 AND c2 = 'a') OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
or
"WHERE (c1 = 1 ) OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
This WHERE clause cannot be transformed to a conjunction of
equality predicates.
The solution is to introduce another rule, NGA3, that complements
NGA2. NGA3 says that if a gap field (field between those
listed in GROUP BY and C in the index) has a predicate, then
there can only be one range in the query. This requirement is
more strict than it has to be in theory. BUG 15947433 will deal
with that.
sql/opt_range.cc:
check for the repetition of non group field.
Consider the following query:
SELECT f_1,..,f_m, AGGREGATE_FN(C)
FROM t1
WHERE ...
GROUP BY ...
Loose index scan ("Using index for group-by") can be used for
this query if there is an index 'i' covering all fields in the
select list, and the GROUP BY clause makes up a prefix f1,...,fn
of 'i'. Furthermore, according to rule NGA2 of
get_best_group_min_max(), the WHERE clause must contain a
conjunction of equality predicates for all fields fn+1,...,fm.
The problem in this bug was that a query with WHERE clause that
broke NGA2(NGA: Non Group Attribuite) was not detected and therefore
used loose index scan.
This lead to wrong result. The query had an index
covering (c1,c2) and had:
"WHERE (c1 = 1 AND c2 = 'a') OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
or
"WHERE (c1 = 1 ) OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
This WHERE clause cannot be transformed to a conjunction of
equality predicates.
The solution is to introduce another rule, NGA3, that complements
NGA2. NGA3 says that if a gap field (field between those
listed in GROUP BY and C in the index) has a predicate, then
there can only be one range in the query. This requirement is
more strict than it has to be in theory. BUG 15947433 will deal
with that.
Consider the following query:
SELECT f_1,..,f_m, AGGREGATE_FN(C)
FROM t1
WHERE ...
GROUP BY ...
Loose index scan ("Using index for group-by") can be used for
this query if there is an index 'i' covering all fields in the
select list, and the GROUP BY clause makes up a prefix f1,...,fn
of 'i'. Furthermore, according to rule NGA2 of
get_best_group_min_max(), the WHERE clause must contain a
conjunction of equality predicates for all fields fn+1,...,fm.
The problem in this bug was that a query with WHERE clause that
broke NGA2 was not detected and therefore used loose index scan.
This lead to wrong result. The query had an index
covering (c1,c2) and had:
"WHERE (c1 = 1 AND c2 = 'a') OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
or
"WHERE (c1 = 1 ) OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
This WHERE clause cannot be transformed to a conjunction of
equality predicates.
The solution is to introduce another rule, NGA3, that complements
NGA2. NGA3 says that if a gap field (field between those
listed in GROUP BY and C in the index) has a predicate, then
there can only be one range in the query. This requirement is
more strict than it has to be in theory. BUG 15947433 will deal
with that.
sql/opt_range.cc:
check for the repetition of non group field.
Consider the following query:
SELECT f_1,..,f_m, AGGREGATE_FN(C)
FROM t1
WHERE ...
GROUP BY ...
Loose index scan ("Using index for group-by") can be used for
this query if there is an index 'i' covering all fields in the
select list, and the GROUP BY clause makes up a prefix f1,...,fn
of 'i'. Furthermore, according to rule NGA2 of
get_best_group_min_max(), the WHERE clause must contain a
conjunction of equality predicates for all fields fn+1,...,fm.
The problem in this bug was that a query with WHERE clause that
broke NGA2 was not detected and therefore used loose index scan.
This lead to wrong result. The query had an index
covering (c1,c2) and had:
"WHERE (c1 = 1 AND c2 = 'a') OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
or
"WHERE (c1 = 1 ) OR (c1 = 2 AND c2 = 'b')
GROUP BY c1"
This WHERE clause cannot be transformed to a conjunction of
equality predicates.
The solution is to introduce another rule, NGA3, that complements
NGA2. NGA3 says that if a gap field (field between those
listed in GROUP BY and C in the index) has a predicate, then
there can only be one range in the query. This requirement is
more strict than it has to be in theory. BUG 15947433 will deal
with that.
Problem:
During the index intersect access method, the SQL layer will access one row,
that satisfies a set of conditions, using an index i1. And then it will try to
access the same row, with other set of conditions using the next index i2. If
the fetch from i2 fails (we are talking about an error situation here and not
simply an unmatched row situation), then it will unlock the row accessed via
i1. This will work in all situations except deadlock error.
When a deadlock happens, InnoDB will rollback the transaction. InnoDB intimates
the SQL layer about this through the THD::transaction_rollback_request member.
But this is not currently used by the SQL layer.
Solution:
When an error happens, the SQL layer must check the
THD::transaction_rollback_request member, before calling handler::unlock_row().
We have also added a debug assert in ha_innobase::unlock_row() checking that
it must be called only when the transaction is in active state.
rb#1773 approved by Marko and Sunny.
Problem:
During the index intersect access method, the SQL layer will access one row,
that satisfies a set of conditions, using an index i1. And then it will try to
access the same row, with other set of conditions using the next index i2. If
the fetch from i2 fails (we are talking about an error situation here and not
simply an unmatched row situation), then it will unlock the row accessed via
i1. This will work in all situations except deadlock error.
When a deadlock happens, InnoDB will rollback the transaction. InnoDB intimates
the SQL layer about this through the THD::transaction_rollback_request member.
But this is not currently used by the SQL layer.
Solution:
When an error happens, the SQL layer must check the
THD::transaction_rollback_request member, before calling handler::unlock_row().
We have also added a debug assert in ha_innobase::unlock_row() checking that
it must be called only when the transaction is in active state.
rb#1773 approved by Marko and Sunny.
The problem is a shift operation that is not 64-bit safe.
The consequence is that used tables information for a join with 32 tables
or more will be incorrect.
Fixed by adding a type cast in Item_sum::update_used_tables().
Also used the opportunity to fix some other potential bugs by adding an
explicit type-cast to an integer in a left-shift operation.
Some of them were quite harmless, but was fixed in order to get the same
signed-ness as the other operand of the operation it was used in.
sql/item_cmpfunc.cc
Adjusted signed-ness for some integers in left-shift.
sql/item_subselect.cc
Added type-cast to nesting_map (which is a 32/64 bit type, so
potential bug for deeply nested queries).
sql/item_sum.cc
Added type-cast to nesting_map (32/64-bit type) and table_map
(64-bit type).
sql/opt_range.cc
Added type-cast to ulonglong (which is a 64-bit type).
sql/sql_base.cc
Added type-cast to nesting_map (which is a 32/64-bit type).
sql/sql_select.cc
Added type-cast to nesting_map (32/64-bit type) and key_part_map
(64-bit type).
sql/strfunc.cc
Changed type-cast from longlong to ulonglong, to preserve signed-ness.
The problem is a shift operation that is not 64-bit safe.
The consequence is that used tables information for a join with 32 tables
or more will be incorrect.
Fixed by adding a type cast in Item_sum::update_used_tables().
Also used the opportunity to fix some other potential bugs by adding an
explicit type-cast to an integer in a left-shift operation.
Some of them were quite harmless, but was fixed in order to get the same
signed-ness as the other operand of the operation it was used in.
sql/item_cmpfunc.cc
Adjusted signed-ness for some integers in left-shift.
sql/item_subselect.cc
Added type-cast to nesting_map (which is a 32/64 bit type, so
potential bug for deeply nested queries).
sql/item_sum.cc
Added type-cast to nesting_map (32/64-bit type) and table_map
(64-bit type).
sql/opt_range.cc
Added type-cast to ulonglong (which is a 64-bit type).
sql/sql_base.cc
Added type-cast to nesting_map (which is a 32/64-bit type).
sql/sql_select.cc
Added type-cast to nesting_map (32/64-bit type) and key_part_map
(64-bit type).
sql/strfunc.cc
Changed type-cast from longlong to ulonglong, to preserve signed-ness.
that introduced engine independent persistent statistics.
In particular:
- added an enumeration type for possible values of the system
variable use_stat_tables
- renamed KEY::real_rec_per_key to KEY::actual_rec_per_key
- optimized the collection of statistical data for any primary
key defined only on one column.
MDEV-567: Wrong result from a query with correlated subquery if ICP is allowed:
backport the fix developed for SHOW EXPLAIN:
revision-id: psergey@askmonty.org-20120719115219-212cxmm6qvf0wlrb
branch nick: 5.5-show-explain-r21
timestamp: Thu 2012-07-19 15:52:19 +0400
BUG#992942 & MDEV-325: Pre-liminary commit for testing
and adjust it so that it handles DS-MRR scans correctly.
Reconcile the fixes from:
#
# guilhem.bichot@oracle.com-20110805143029-ywrzuz15uzgontr0
# Fix for BUG#12698916 - "JOIN QUERY GIVES WRONG RESULT AT 2ND EXEC. OR
# AFTER FLUSH TABLES [-INT VS NULL]"
#
# guilhem.bichot@oracle.com-20111209150650-tzx3ldzxe1yfwji6
# Fix for BUG#12912171 - ASSERTION FAILED: QUICK->HEAD->READ_SET == SAVE_READ_SET
# and
#
and related fixes from: BUG#1006164, MDEV-376:
Now, ROR-merged QUICK_RANGE_SELECT objects make no assumptions about the values
of table->read_set and table->write_set.
Each QUICK_ROR_SELECT has (and had before) its own column bitmap, but now, all
QUICK_ROR_SELECT's functions that care: reset(), init_ror_merged_scan(), and
get_next() will set table->read_set when invoked and restore it back to what
it was before the call before they return.
This allows to avoid the mess when somebody else modifies table->read_set for
some reason.
When a DML statement is issued, and if the index merge
access method is chosen, then many rows from the
storage engine will be locked because of the way the
algorithm works. Many rows will be locked, but they
will not be part of the final result set.
To reduce the excessive locking, the locks of unmatched
rows are released by this patch. This patch will
affect only transactions with isolation level
equal to or less stricter than READ COMMITTED. This is
because of the behaviour of ha_innobase::unlock_row().
rb://1296 approved by jorgen and olav.
When a DML statement is issued, and if the index merge
access method is chosen, then many rows from the
storage engine will be locked because of the way the
algorithm works. Many rows will be locked, but they
will not be part of the final result set.
To reduce the excessive locking, the locks of unmatched
rows are released by this patch. This patch will
affect only transactions with isolation level
equal to or less stricter than READ COMMITTED. This is
because of the behaviour of ha_innobase::unlock_row().
rb://1296 approved by jorgen and olav.
CONSISTENT SNAPSHOT OPTION
A transaction is started with a consistent snapshot. After
the transaction is started new indexes are added to the
table. Now when we issue an update statement, the optimizer
chooses an index. When the index scan is being initialized
via ha_innobase::change_active_index(), InnoDB reports
the error code HA_ERR_TABLE_DEF_CHANGED, with message
stating that "insufficient history for index".
This error message is propagated up to the SQL layer. But
the my_error() api is never called. The statement level
diagnostics area is not updated with the correct error
status (it remains in Diagnostics_area::DA_EMPTY).
Hence the following check in the Protocol::end_statement()
fails.
516 case Diagnostics_area::DA_EMPTY:
517 default:
518 DBUG_ASSERT(0);
519 error= send_ok(thd->server_status, 0, 0, 0, NULL);
520 break;
The fix is to backport the fix of bugs 14365043, 11761652
and 11746399.
14365043 PROTOCOL::END_STATEMENT(): ASSERTION `0' FAILED
11761652 HA_RND_INIT() RESULT CODE NOT CHECKED
11746399 RETURN VALUES OF HA_INDEX_INIT() AND INDEX_INIT() IGNORED
rb://1227 approved by guilhem and mattiasj.
CONSISTENT SNAPSHOT OPTION
A transaction is started with a consistent snapshot. After
the transaction is started new indexes are added to the
table. Now when we issue an update statement, the optimizer
chooses an index. When the index scan is being initialized
via ha_innobase::change_active_index(), InnoDB reports
the error code HA_ERR_TABLE_DEF_CHANGED, with message
stating that "insufficient history for index".
This error message is propagated up to the SQL layer. But
the my_error() api is never called. The statement level
diagnostics area is not updated with the correct error
status (it remains in Diagnostics_area::DA_EMPTY).
Hence the following check in the Protocol::end_statement()
fails.
516 case Diagnostics_area::DA_EMPTY:
517 default:
518 DBUG_ASSERT(0);
519 error= send_ok(thd->server_status, 0, 0, 0, NULL);
520 break;
The fix is to backport the fix of bugs 14365043, 11761652
and 11746399.
14365043 PROTOCOL::END_STATEMENT(): ASSERTION `0' FAILED
11761652 HA_RND_INIT() RESULT CODE NOT CHECKED
11746399 RETURN VALUES OF HA_INDEX_INIT() AND INDEX_INIT() IGNORED
rb://1227 approved by guilhem and mattiasj.
Bug#14530242 CRASH / MEMORY CORRUPTION IN FILESORT_BUFFER::GET_RECORD_BUFFER WITH MYISAM
This is a backport of
Bug#12694872 - VALGRIND: 18,816 BYTES IN 196 BLOCKS ARE DEFINITELY LOST
Bug#13340270: assertion table->sort.record_pointers == __null
Bug#14536113 CRASH IN CLOSEFRM (TABLE.CC) OR UNPACK (FIELD.H) ON SUBQUERY WITH MYISAM TABLES
Also:
removed and re-added test files with file-ids from trunk.
Bug#14530242 CRASH / MEMORY CORRUPTION IN FILESORT_BUFFER::GET_RECORD_BUFFER WITH MYISAM
This is a backport of
Bug#12694872 - VALGRIND: 18,816 BYTES IN 196 BLOCKS ARE DEFINITELY LOST
Bug#13340270: assertion table->sort.record_pointers == __null
Bug#14536113 CRASH IN CLOSEFRM (TABLE.CC) OR UNPACK (FIELD.H) ON SUBQUERY WITH MYISAM TABLES
Also:
removed and re-added test files with file-ids from trunk.
client/mysqldump.c:
Slave needs to be initialized with 0
dbug/dbug.c:
Removed not existing function
plugin/semisync/semisync_master.cc:
Fixed compiler warning
sql/opt_range.cc:
thd needs to be set early as it's used in some error conditions.
sql/sql_table.cc:
Changed to use uchar* to make array indexing portable
storage/innobase/handler/ha_innodb.cc:
Removed not used variable
storage/maria/ma_delete.c:
Fixed compiler warning
storage/maria/ma_write.c:
Fixed compiler warning
ROWS THAT ARE EXPECTED
For non range/list partitioned tables (i.e. HASH/KEY):
When prune_partitions finds a multi-range list
(or in this test '<>') for a field of the partition index,
even if it cannot make any use of the multi-range,
it will continue with the next field of the partition index
and use that for pruning (even if it the previous
field could not be used). This results in partitions is
pruned away, leaving partitions that only matches
the last field in the partition index, and will exclude
partitions which might match any previous fields.
Fixed by skipping rest of partitioning key fields/parts
if current key field/part could not be used.
Also notice it is the order of the fields in the CREATE TABLE
statement that triggers this bug, not the order of fields in
primary/unique key or PARTITION BY KEY ().
It must not be the last field in the partitioning expression that
is not equal (or have a non single point range).
I.e. the partitioning index is created with the same field order
as in the CREATE TABLE. And for the bug to appear
the last field must be a single point and some previous field
must be a multi-point range.
ROWS THAT ARE EXPECTED
For non range/list partitioned tables (i.e. HASH/KEY):
When prune_partitions finds a multi-range list
(or in this test '<>') for a field of the partition index,
even if it cannot make any use of the multi-range,
it will continue with the next field of the partition index
and use that for pruning (even if it the previous
field could not be used). This results in partitions is
pruned away, leaving partitions that only matches
the last field in the partition index, and will exclude
partitions which might match any previous fields.
Fixed by skipping rest of partitioning key fields/parts
if current key field/part could not be used.
Also notice it is the order of the fields in the CREATE TABLE
statement that triggers this bug, not the order of fields in
primary/unique key or PARTITION BY KEY ().
It must not be the last field in the partitioning expression that
is not equal (or have a non single point range).
I.e. the partitioning index is created with the same field order
as in the CREATE TABLE. And for the bug to appear
the last field must be a single point and some previous field
must be a multi-point range.
COUNT DISTINCT GROUP BY
PROBLEM:
To calculate the final result of the count(distinct(select 1))
we call 'end_send' function instead of 'end_send_group'.
'end_send' cannot be called if we have aggregate functions
that need to be evaluated.
ANALYSIS:
While evaluating for a possible loose_index_scan option for
the query, the variable 'is_agg_distinct' is set to 'false'
as the item in the distinct clause is not a field. But, we
choose loose_index_scan by not taking this into
consideration.
So, while setting the final 'select_function' to evaluate
the result, 'precomputed_group_by' is set to TRUE as in
this case loose_index_scan is chosen and we do not have
agg_distinct in the query (which is clearly wrong as we
have one).
As a result, 'end_send' function is chosen as the final
select_function instead of 'end_send_group'. The difference
between the two being, 'end_send_group' evaluates the
aggregates while 'end_send' does not. Hence the wrong result.
FIX:
The variable 'is_agg_distinct' always represents if
'loose_idnex_scan' can be chosen for aggregate_distinct
functions present in the select.
So, we check for this variable to continue with
loose_index_scan option.
sql/opt_range.cc:
Do not continue if is_agg_distinct is not set in case
of agg_distinct functions.
COUNT DISTINCT GROUP BY
PROBLEM:
To calculate the final result of the count(distinct(select 1))
we call 'end_send' function instead of 'end_send_group'.
'end_send' cannot be called if we have aggregate functions
that need to be evaluated.
ANALYSIS:
While evaluating for a possible loose_index_scan option for
the query, the variable 'is_agg_distinct' is set to 'false'
as the item in the distinct clause is not a field. But, we
choose loose_index_scan by not taking this into
consideration.
So, while setting the final 'select_function' to evaluate
the result, 'precomputed_group_by' is set to TRUE as in
this case loose_index_scan is chosen and we do not have
agg_distinct in the query (which is clearly wrong as we
have one).
As a result, 'end_send' function is chosen as the final
select_function instead of 'end_send_group'. The difference
between the two being, 'end_send_group' evaluates the
aggregates while 'end_send' does not. Hence the wrong result.
FIX:
The variable 'is_agg_distinct' always represents if
'loose_idnex_scan' can be chosen for aggregate_distinct
functions present in the select.
So, we check for this variable to continue with
loose_index_scan option.
This is a backport of the fix for MySQL bug #13723054 in 5.6.
Original comment:
The crash is caused by arbitrary memory area owerwriting in case of
BLOB fields during attempt to copy BLOB field key image into record
buffer(record buffer is too small to get BLOB key part image).
note:
QUICK_GROUP_MIN_MAX_SELECT can not work with BLOB fields
because it uses record buffer as temporary buffer for key values
however this case is filtered out by covering_keys() check
in get_best_group_min_max() as BLOBs always require key length
modificator in the key declaration and if the key has a BLOB
then it can not be covered key.
The fix is to use 'max_used_key_length' key length instead of 0.
Analysis:
Spcifically the crash in this bug was a result of the call to key_copy()
that copied the whole key, inlcuding the BLOB field which is not used
for index access. Copying the blob field overwrote memory as far as the
function parameter 'key_info'. As a result the contents of key_info was
all 0, which resulted in a crash when this key_info was accessed few
lines below in key_cmp().
