Analysis:
The following call stack shows that it is possible to set Item_cache::value_cached, and the relevant value
without setting Item_cache::example.
#0 Item_cache_temporal::store_packed at item.cc:8395
#1 get_datetime_value at item_cmpfunc.cc:915
#2 resolve_const_item at item.cc:7987
#3 propagate_cond_constants at sql_select.cc:12264
#4 propagate_cond_constants at sql_select.cc:12227
#5 optimize_cond at sql_select.cc:13026
#6 JOIN::optimize at sql_select.cc:1016
#7 st_select_lex::optimize_unflattened_subqueries at sql_lex.cc:3161
#8 JOIN::optimize_unflattened_subqueries at opt_subselect.cc:4880
#9 JOIN::optimize at sql_select.cc:1554
The fix is to set Item_cache_temporal::example even when the value is
set directly by Item_cache_temporal::store_packed. This makes the
Item_cache_temporal object consistent.
two tests still fail:
main.innodb_icp and main.range_vs_index_merge_innodb
call records_in_range() with both range ends being open
(which triggers an assert)
We set correct cmp_context during preparation to avoid changing it later by Item_field::equal_fields_propagator.
(see mysql bugs #57135#57692 during merging)
Analysis:
When a subquery that needs a temp table is executed during
the prepare or optimize phase of the outer query, at the end
of the subquery execution all the JOIN_TABs of the subquery
are replaced by a new JOIN_TAB that selects from the temp table.
However that temp table has no corresponding TABLE_LIST.
Once EXPLAIN execution reaches its last phase, it tries to print
the names of the subquery tables through its TABLE_LISTs, but in
the case of this bug there is no such TABLE_LIST (it is NULL),
hence a crash.
Solution:
The fix is to block subquery evaluation inside
Item_func_like::fix_fields and Item_func_like::select_optimize()
using the Item::is_expensive() test.
- Fix equality propagation to work with SJM nests and OR clauses (full descirption of problem and
solution in the comment in the patch)
(The second commit with post-review fixes)
make sure that stored routines are evaluated (that is, de facto - cached) in convert_const_to_int().
revert the fix for lp:806943 because it cannot be repeated anymore.
add few tests for convert_const_to_int()
Fixed Item* Item_equal::get_first(JOIN_TAB *context, Item *field_item) to work correctly in the case where:
- context!= NO_PARTICULAR_TAB, it points to a table within SJ-Materialization nest
- field_item points to an item_equal that has a constant Item_field but does not have any fields
from tables that are within semi-join nests.
- equality substitution code was geared towards processing WHERE/ON clauses.
that is, it assumed that it was doing substitions on the code that
= wasn't attached to any particular join_tab yet
= was going to be fed to make_join_select() which would take the condition
apart and attach various parts of it to tables inside/outside semi-joins.
- However, somebody added equality substition for ref access. That is, if
we have a ref access on TBL.key=expr, they would do equality substition in
'expr'. This possibility wasn't accounted for.
- Fixed equality substition code by adding a mode that does equality
substition under assumption that the processed expression will be
attached to a certain particular table TBL.
- Provide fix_after_pullout() function for Item_in_optimizer and other Item_XXX classes (basically, all of them
that have eval_not_null_tables, which means they have special rules for calculating not_null_tables_cache value)
ALL subquery should return TRUE if subquery rowa set is empty independently
of left part. The problem was that Item_func_(eq,ne,gt,ge,lt,le) do not
call execution of second argument if first is NULL no in this case subquery
will not be executed and when Item_func_not_all calls any_value() of the
subquery or aggregation function which report that there was rows. So for
NULL < ALL (SELECT...) result was FALSE instead of TRUE.
Fix is just swapping of arguments of Item_func_(eq,ne,gt,ge,lt,le) (with
changing the operation if it is needed) so that result will be the same
(for examole a < b is equal to b > a). This fix exploit the fact that
first argument will be executed in any case.
- The problem was that the code that made the check whether the subquery is an AND-part of the WHERE
clause didn't work correctly for nested subqueries. In particular, grand-child subquery in HAVING was
treated as if it was in the WHERE, which eventually caused an assert when replace_where_subcondition
looked for the subquery predicate in the WHERE and couldn't find it there.
- The fix: Removed implementation of "thd_marker approach". thd->thd_marker was used to determine the
location of subquery predicate: setup_conds() would set accordingly it when making the
{where|on_expr}->fix_fields(...)
call so that AND-parts of the WHERE/ON clauses can determine they are the AND-parts.
Item_cond_or::fix_fields(), Item_func::fix_fields(), Item_subselect::fix_fields (this one was missed),
and all other items-that-contain-items had to reset thd->thd_marker before calling fix_fields() for
their children items, so that the children can see they are not AND-parts of WHERE/ON.
- The "thd_marker approach" required that a lot of code in different locations maintains correct value of
thd->thd_marker, so it was replaced with:
- The new approach with mark_as_condition_AND_part does not keep context in thd->thd_marker. Instead,
setup_conds() now calls
{where|on_expr}->mark_as_condition_AND_part()
and implementations of that function make sure that:
- parts of AND-expressions get the mark_as_condition_AND_part() call
- Item_in_subselect objects record that they are AND-parts of WHERE/ON
Analysis:
During the first execution of the query through the stored
procedure, the optimization phase calls
substitute_for_best_equal_field(), which calls
Item_in_optimizer::transform(). The latter replaces
Item_in_subselect::left_expr with args[0] via assignment.
In this test case args[0] is an Item_outer_ref which is
created/deallocated for each re-execution. As a result,
during the second execution Item_in_subselect::left_expr
pointed to freed memory, which resulted in a crash.
Solution:
The solution is to use change_item_tree(), so that the
origianal left expression is restored after each execution.
The bug is a duplicate of MySQL's Bug#11764086,
however MySQL's fix is incomplete for MariaDB, so
this fix is slightly different.
In addition, this patch renames
Item_func_not_all::top_level() to is_top_level_item()
to make it in line with the analogous methods of
Item_in_optimizer, and Item_subselect.
Analysis:
It is possible to determine whether a predicate is
NULL-rejecting only if it is a top-level one. However,
this was not taken into account for Item_in_optimizer.
As a result, a NOT IN predicate was erroneously
considered as NULL-rejecting, and the NULL-complemented
rows generated by the outer join were rejected before
being checked by the NOT IN predicate.
Solution:
Change Item_in_optimizer to be considered as
NULL-rejecting only if it a top-level predicate.
ALL subquery should return TRUE if subquery rowa set is empty independently
of left part. The problem was that Item_func_(eq,ne,gt,ge,lt,le) do not
call execution of second argument if first is NULL no in this case subquery
will not be executed and when Item_func_not_all calls any_value() of the
subquery or aggregation function which report that there was rows. So for
NULL < ALL (SELECT...) result was FALSE instead of TRUE.
Fix is just swapping of arguments of Item_func_(eq,ne,gt,ge,lt,le) (with
changing the operation if it is needed) so that result will be the same
(for examole a < b is equal to b > a). This fix exploit the fact that
first argument will be executed in any case.
mysql-test/r/subselect.result:
The test suite added.
mysql-test/r/subselect_no_mat.result:
The test suite added.
mysql-test/r/subselect_no_opts.result:
The test suite added.
mysql-test/r/subselect_no_semijoin.result:
The test suite added.
mysql-test/r/subselect_scache.result:
The test suite added.
mysql-test/t/subselect.test:
The test suite added.
sql/item_cmpfunc.cc:
Swap arguments creation methods added.
sql/item_cmpfunc.h:
Swap arguments creation methods added.
sql/item_subselect.cc:
Swap arguments of the comparison.
The problem was that optimizer removes some outer references (it they are
constant for example) and the list of outer items built during prepare phase is
not actual during execution phase when we need it as the cache parameters.
First solution was use pointer on pointer on outer reference Item and
initialize temporary table on demand. This solved most problem except case
when optimiser also reduce Item which contains outer references ('OR' in
this bug test suite).
The solution is to build the list of outer reference items on execution
phase (after optimization) on demand (just before temporary table creation)
by walking Item tree and finding outer references among Item_ident
(Item_field/Item_ref) and Item_sum items.
Removed depends_on list (because it is not neede any mnore for the cache, in the place where it was used it replaced with upper_refs).
Added processor (collect_outer_ref_processor) and get_cache_parameters() methods to collect outer references (or other expression parameters in future).
mysql-test/r/subselect_cache.result:
A new test added.
mysql-test/r/subselect_scache.result:
Changes in creating the cache and its paremeters order or adding arguments of aggregate function (which is a parameter also, but this has no influence on the result).
mysql-test/t/subselect_cache.test:
Added a new test.
sql/item.cc:
depends_on removed.
Added processor (collect_outer_ref_processor) and get_cache_parameters() methods to collect outer references.
Item_cache_wrapper collect parameters befor initialization of its cache.
sql/item.h:
depends_on removed.
Added processor (collect_outer_ref_processor) and get_cache_parameters() methods to collect outer references.
sql/item_cmpfunc.cc:
depends_on removed.
Added processor (collect_outer_ref_processor) to collect outer references.
sql/item_cmpfunc.h:
Added processor (collect_outer_ref_processor) to collect outer references.
sql/item_subselect.cc:
depends_on removed.
Added processor get_cache_parameters() method to collect outer references.
sql/item_subselect.h:
depends_on removed.
Added processor get_cache_parameters() method to collect outer references.
sql/item_sum.cc:
Added processor (collect_outer_ref_processor) method to collect outer references.
sql/item_sum.h:
Added processor (collect_outer_ref_processor) and get_cache_parameters() methods to collect outer references.
sql/opt_range.cc:
depends_on removed.
sql/sql_base.cc:
depends_on removed.
sql/sql_class.h:
New iterator added.
sql/sql_expression_cache.cc:
Build of list of items resolved in outer query done just before creating expression cache on the first execution of the subquery which removes influence of optimizer removing items (all optimization already done).
sql/sql_expression_cache.h:
Build of list of items resolved in outer query done just before creating expression cache on the first execution of the subquery which removes influence of optimizer removing items (all optimization already done).
sql/sql_lex.cc:
depends_on removed.
sql/sql_lex.h:
depends_on removed.
sql/sql_list.h:
Added add_unique method to add only unique elements to the list.
sql/sql_select.cc:
Support of new Item list added.
sql/sql_select.h:
Support of new Item list added.
Bug#11766642: crash in Item_field::register_field_in_read_map
with view
(Former 59793)
Prior to the refactoring in this patch, Item_cond_xor behaved
partially as an Item_cond and partially as an Item_func. The
reasoning behind this was that XOR is currently not optimized
(thus should be Item_func instead of Item_cond), but it was
planned optimize it in the future (thus, made Item_cond anyway
to ease optimization later).
Even though Item_cond inherits from Item_func, there are
differences between these two. One difference is that the
arguments are stored differently. Item_cond stores them in a
list while Item_func store them in an args[].
BUG no 45221 was caused by Item_cond_xor storing arguments in
the list while users of the objects would look for them in
args[]. The fix back then was to store the arguments in both
locations.
In this bug, Item_cond_xor initially gets two Item_field
arguments. These are stored in the list inherited from
Item_cond and in args[] inherited from Item_func. During
resolution, find_field_in_view() replaces the Item_fields
stored in the list with Item_direct_view_refs, but args[]
still points to the unresolved Item_fields. This shows that
the fix for 45221 was incorrect.
The refactoring performed in this patch removes the confusion
by making the XOR item an Item_func period. A neg_transformer()
is also implemented for Item_func_xor to improve performance
when negating XOR expressions. An XOR is negated by negating
one of the operands.
Analysis:
This bug is yet another incarnation of the generic problem
where optimization of the outer query triggers evaluation
of a subquery, and this evaluation performs a destructive
change to the subquery plan. Specifically a temp table is
created for the DISTINCT operation that replaces the
original subquery table. Later, select_describe() attempts
to print the table name, however, there is no corresponding
TABLE_LIST object to the internal temp table, so we get a
crash. Execution works fine because it is not interested in
the corresponding TABLE_LIST object (or its name).
Solution:
Similar to other such bugs, block the evaluation of expensive
Items in convert_const_to_int().
microsecond(TIME)
alter table datetime<->datetime(6)
max(TIME), mix(TIME)
mysql-test/t/func_if.test:
fix the test case of avoid overflow
sql/field.cc:
don't use make_date() and make_time()
sql/field.h:
correct eq_def() for temporal fields
sql/item.cc:
move datetime caching from Item_cache_int
to Item_cache_temporal
sql/item.h:
move datetime caching from Item_cache_int
to Item_cache_temporal
sql/item_func.cc:
use existing helper methods, don't duplicate
sql/item_sum.cc:
argument cache must use argument's cmp_type, not result_type.
sql/item_timefunc.cc:
use existing methods, don't tuplicate.
remove unused function.
fix micorseconds() to support TIME argument
sql/mysql_priv.h:
dead code
sql/time.cc:
dead code
compilation error in mysys/my_getsystime.c fixed
some redundant code removed
sec_to_time, time_to_sec, from_unixtime, unix_timestamp, @@timestamp now
use decimal, not double for numbers with a fractional part.
purge_master_logs_before_date() fixed
many bugs in corner cases fixed
mysys/my_getsystime.c:
compilation failure fixed
sql/sql_parse.cc:
don't cut corners. it backfires.
Safety check that could cause core dump when doing create table with virtual column.
mysql-test/mysql-test-run.pl:
Show also warnings from thr_lock (which starts with just Warning, not Warning:)
mysql-test/r/lock.result:
Added test that showed not relevant warning when using table locks.
mysql-test/t/lock.test:
Added test that showed not relevant warning when using table locks.
mysys/thr_lock.c:
Fixed sorting of locks.
(Old sort code didn't handle case where TL_WRITE_CONCURRENT_INSERT must be sorted before TL_WRITE)
Added more information to check_locks warning output.
Fixed wrong testing of multiple different write locks for same table.
sql/item_cmpfunc.cc:
Safety check that could cause core dump when doing create table with virtual column.
A lot of small fixes and new test cases.
client/mysqlbinlog.cc:
Cast removed
client/mysqltest.cc:
Added missing DBUG_RETURN
include/my_pthread.h:
set_timespec_time_nsec() now only takes one argument
mysql-test/t/date_formats.test:
Remove --disable_ps_protocl as now also ps supports microseconds
mysys/my_uuid.c:
Changed to use my_interval_timer() instead of my_getsystime()
mysys/waiting_threads.c:
Changed to use my_hrtime()
sql/field.h:
Added bool special_const_compare() for fields that may convert values before compare (like year)
sql/field_conv.cc:
Added test to get optimal copying of identical temporal values.
sql/item.cc:
Return that item_int is equal if it's positive, even if unsigned flag is different.
Fixed Item_cache_str::save_in_field() to have identical null check as other similar functions
Added proper NULL check to Item_cache_int::save_in_field()
sql/item_cmpfunc.cc:
Don't call convert_constant_item() if there is nothing that is worth converting.
Simplified test when years should be converted
sql/item_sum.cc:
Mark cache values in Item_sum_hybrid as not constants to ensure they are not replaced by other cache values in compare_datetime()
sql/item_timefunc.cc:
Changed sec_to_time() to take a my_decimal argument to ensure we don't loose any sub seconds.
Added Item_temporal_func::get_time() (This simplifies some things)
sql/mysql_priv.h:
Added Lazy_string_decimal()
sql/mysqld.cc:
Added my_decimal constants max_seconds_for_time_type, time_second_part_factor
sql/table.cc:
Changed expr_arena to be of type CONVENTIONAL_EXECUTION to ensure that we don't loose any items that are created by fix_fields()
sql/tztime.cc:
TIME_to_gmt_sec() now sets *in_dst_time_gap in case of errors
This is needed to be able to detect if timestamp is 0
storage/maria/lockman.c:
Changed from my_getsystime() to set_timespec_time_nsec()
storage/maria/ma_loghandler.c:
Changed from my_getsystime() to my_hrtime()
storage/maria/ma_recovery.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/maria/unittest/trnman-t.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/xtradb/handler/ha_innodb.cc:
Added support for new time,datetime and timestamp
unittest/mysys/thr_template.c:
my_getsystime() -> my_interval_timer()
unittest/mysys/waiting_threads-t.c:
my_getsystime() -> my_interval_timer()
mysql-test/r/subselect4.result:
Moved test case for LP BUG#718593 into the correct test file subselect_mat_cost_bugs.test.
mysql-test/t/subselect4.test:
Moved test case for LP BUG#718593 into the correct test file subselect_mat_cost_bugs.test.
sql/event_parse_data.cc:
don't use "not_used" variable
sql/item_timefunc.cc:
Item_temporal_func::fix_length_and_dec()
and other changes
sql/item_timefunc.h:
introducing Item_timefunc::fix_length_and_dec()
sql/share/errmsg.txt:
don't say "column X" in the error message that used not only for columns
include/my_time.h:
remove duplicate defines.
cast to ulonglong to avoid overflow
sql/field.cc:
perform sign extension when reading packed TIME values
sql/item_cmpfunc.cc:
when converting a string to a date for the purpose of comparing it with another date,
we should ignore strict sql mode.
sql/item_timefunc.cc:
better error message
sql/item_timefunc.h:
limit decimals appropriately
sql/share/errmsg.txt:
don't refer to an object as a "column" in error messages that are used not only for columns.
When a view is merged into a select all the depended_from fields
pointing to the select of the view should have been corrected to
point to the select where the view is used. It was not done yet.
This could lead to wrong results returned by queries such as
one from the test case for bug 33389.
Correction of outer references required walking through all items
of the proccesed qurery. To avoid this the following solution was
implemented.
Each select now contains a pointer to the select it is merged into
(if there is any). Such pointers allow to get the corrected value
of depended_from on the fly. The function Item_ident::get_depended_from
was introduced for this purpose.
Resolved all conflicts, bad merges and fixed a few minor bugs in the code.
Commented out the queries from multi_update, view, subselect_sj, func_str,
derived_view, view_grant that failed either with crashes in ps-protocol or
with wrong results.
The failures are clear indications of some bugs in the code and these bugs
are to be fixed.
Analysis:
The subquery is evaluated first during ref-optimization of the outer
query because the subquery is considered constant from the perspective
of the outer query. Thus an attempt is made to evaluate the MAX subquery
and use the new constant to drive an index nested loops join.
During this evaluation the inner-most subquery replaces the JOIN_TAB
with a new one that fetches the data from a temp table.
The function select_describe crashes at the lines:
TABLE_LIST *real_table= table->pos_in_table_list;
item_list.push_back(new Item_string(real_table->alias,
strlen(real_table->alias),
cs));
because 'table' is a temp table, and it has no corresponding table
reference. This 'real_table' is NULL, and real_table->alias results
in a crash.
Solution:
In the spirit of MWL#89 prevent the evaluation of expensive predicates
during optimization. This patch prevents the evaluation of expensive
predicates during ref optimization.
sql/item_subselect.h:
Remove unused class member. Not needed for the fix, but noticed now and removed.
Analysis:
During optimization of the subquery, in the call chain:
update_ref_and_keys -> add_key_fields ->
merge_key_fields -> Item_direct_ref::is_null -> Item_cache::is_null
The call to Item_cache::is_null() returns TRUE, which is wrong.
This results in Item_null replacing the field 'f3' in the KEY_FIELD,
then this Item_null is used for index access, producing a wrong result.
The reason why Item_cache::is_null returns wrong result is that
this Item_cache object is a cache of the left operand of IN, and was
updated in Item_in_optimizer::val_int. In MWL#89 the latter method is
called during the execution phase, which is after we optimize the subquery.
Therefore during the optization phase the left operand cache of IN was
not updated.
Solution:
Update the left operand cache during optimization if it is a constant.
This bug fix also discoveres and fixes a wrong IF statement in
convert_constant_item().
