Analysis:
The problem lies in filesort.cc:find_all_keys().
When find_all_keys() is called for the outer query, it resets all
of the tree sets of fields - [read,write,vcol]_set and recomputes
them with respect to sorting.
However, in the loop for each current record the procedure calls
select->skip_record(thd), which evaluates the where clause, which
in turns evaluates the subquery. The JOIN evaluation of the
subquery eventually calls Field_long::val_int to evaluate the field
alias1.f1. The assertion condition
"bitmap_is_set(table->read_set, field_index)"
fails, because the outer query changed the read_set of table "alias1".
Solution:
Restore the original read_set of the table before calling
SQL_SELECT::skip_record, then revert back to the read_set used in
find_all_keys.
Analysis:
JOIN::optimize performs constant optimization of GROUP by clauses
by calling remove_const():
group_list= remove_const(this, (old_group_list= group_list), conds,
rollup.state == ROLLUP::STATE_NONE,
&simple_group);
If it turns out that a GROUP clause references a field that is
computed by a single-row subquery, then the said optimization
performs premature execution of the subquery referenced by the
group clause.
Solution:
Block the evaluation of subqueries similarly to the approach
for the WHERE and JOIN..ON clauses.
Analysis:
Single-row subqueries are not considered expensive and are
evaluated both during EXPLAIN in to detect errors like
"Subquery returns more than 1 row", and during optimization to
perform constant optimization.
The cause for the failed ASSERT is in JOIN::join_free, where we set
bool full= (!select_lex->uncacheable && !thd->lex->describe);
Thus for EXPLAIN statements full == FALSE, and as a result the call to
JOIN::cleanup doesn't call JOIN_TAB::cleanup which should have
called table->disable_keyread().
Solution:
Consider all kinds of subquery predicates as expensive.
Analysis:
This another instance of the problem fixed in LP BUG#675981 -
evaluation of subqueries during EXPLAIN when the subquery plan
is incomplete because JOIN::optimize() generally doesn't create
complete execution plans for EXPLAIN statements.
In this case the call path is:
mysql_explain_union -> outer_join.exec -> outer_join.init_execution ->
create_sort_index -> filesort -> find_all_keys ->
SQL_SELECT::skip_record -> outer_where_clause.val_int -> ...
-> subselect_join.exec -> ... -> sub_select_cache
When calling sub_select_cache JOIN_TAB::cache is NULL because the cache
objects are not created for EXPLAIN statements.
Solution:
Delay the call to init_execution() after all EXPLAIN related processing
is completed. Thus init_execution() is not called at all during EXPLAIN.
Cause:
The optimize() phase for the subquery selected to use join buffering via setting
JOIN_TAB::next_select= sub_select_cache in make_join_readinfo, however, the call
to check_join_cache_usage() from make_join_readinfo didn't create the corresponding
JOIN_CACHE_BNL object because of the condition:
if ((options & SELECT_DESCRIBE) ||
(((tab->cache= new JOIN_CACHE_BNL(join, tab, prev_cache))) &&
!tab->cache->init()))
Since EXPLAIN for subqueries runs regular execution, the constant predicates that
were delayed to be evaluated at the exec() phase, were evaluated during EXPLAIN.
As a result the outer JOIN::exec called JOIN::exec for the subquery, while the
subquery execution plan was no properly created, which resulted in an failed ASSERT.
Fix:
The patch blocks evaluation of constant expensive conditions during EXPLAIN. Notice
that these conditions are "constant" with respect to the outer query, thus in
general they could be arbitrarily expensive, which may result in very slow EXPLAINs.
This is a backport of the fix for
MySQL BUG#52317: Assertion failing in Field_varstring::store () at field.cc:6833
The orginal comment by Oystein is:
In order for EXPLAIN to print const-refs, a Store_key_const_item object
is created. This is different for normal execution of subqueries where
a temporary store_key_item object is used instead. The problem is that
EXPLAIN will execute subqueries. This leads to a scenario where a
store_key_const_item object it told to write to its underlying field.
This results in a failing assert since the write set of the underlying
table does not reflect this.
The resolution is to do the same trick as for store_key_item::copy_inner().
That is, temporarily change the write set to allow writes to all columns.
This is only necessary in debug version since non-debug version does not
contain asserts on write_set.
sql/sql_select.h:
Temporarily change write_set in store_key_const_item::copy_inner() to
allow initialization of underlying field. This is necessary since
subqueries are executed for EXPLAIN. (For normal execution,
store_key_item::copy_inner is used.)
The cause for the bug was two-fold:
1. Incorrect detection of whether a table is the first one in a query plan -
"used_table & 1" actually checks if used_table is table with number "1".
2. Missing logic to delay the evaluation of (expensive) constant conditions
during the execution phase.
The fix adds/changes:
The patch:
- removes incorrect treatment of expensive predicates from make_cond_for_table,
and lets the caller decide when to evaluate expensive predicates.
- saves expensive constant conditions in JOIN::exec_const_cond,
which is evaluated once in the beginning of JOIN::exec.
The EXISTS transformation has additional switches to catch the known corner
cases that appear when transforming an IN predicate into EXISTS. Guarded
conditions are used which are deactivated when a NULL value is seen in the
outer expression's row. When the inner query block supplies NULL values,
however, they are filtered out because no distinction is made between the
guarded conditions; guarded NOT x IS NULL conditions in the HAVING clause that
filter out NULL values cannot be de-activated in isolation from those that
match values or from the outer expression or NULL's.
The above problem is handled by making the guarded conditions remember whether
they have rejected a NULL value or not, and index access methods are taking
this into account as well.
The bug consisted of
1) Not resetting the property for every nested loop iteration on the inner
query's result.
2) Not propagating the NULL result properly from inner query to IN optimizer.
3) A hack that may or may not have been needed at some point. According to a
comment it was aimed to fix#2 by returning NULL when FALSE was actually
the result. This caused failures when #2 was properly fixed. The hack is
now removed.
The fix resolves all three points.
mysql-test/r/group_by.result:
Added test that showed problems that no_rows_in_results() didn't work for expressions
mysql-test/r/subselect4.result:
Test case for LP#612894
mysql-test/t/group_by.test:
Added test that showed problems that no_rows_in_results() didn't work for expressions
mysql-test/t/subselect4.test:
Test case for LP#612894
sql/item.h:
Added restore_to_before_no_rows_in_result()
Added function processor for no_rows_in_results() and restore_to_before_no_rows_in_results() to ensure it works with functions
Fix that above functions are handled by Item_ref()
sql/item_func.h:
Ensure that no_rows_in_results() and restore_to_before_no_rows_in_result() are called for all function arguments
sql/item_sum.cc:
Added restore_to_before_no_rows_in_result() to restore settings after Item_sum_hybrid::no_rows_in_result() was called.
This is needed to handle the case where we have made 'make_const()' on the item in opt_sum(), but the item will be reused again in a sub query.
Ignore multiple calls to no_rows_in_result() as Item_ref is calling it twice.
sql/item_sum.h:
Added restore_to_before_no_rows_in_result();
sql/sql_select.cc:
Added reset of no_rows_in_result() for JOIN::reinit()
sql/sql_select.h:
Added marker if no_rows_in_result() is called.
file .\item_subselect.cc, line 836
IN quantified predicates are never executed directly. They are rather wrapped
inside nodes called IN Optimizers (Item_in_optimizer) which take care of the
execution. However, this is not done during query preparation. Unfortunately
the LIKE predicate pre-evaluates constant right-hand side arguments even
during name resolution. Likely this is meant as an optimization.
Fixed by not pre-evaluating LIKE arguments in view prepare mode.
Don't call member functions for a NIL pointer.
mysql-test/r/subselect4.result:
Add test case.
mysql-test/t/subselect4.test:
Add test case.
sql/sql_select.cc:
If the (virtual) member function clone_item() returns NULL,
there is no substitution to be made, and we don't need to set the collation.
The test was invoking Item_cache::clone_item()
- Unify EXPLAIN printout for <subqueryN> tables with regular tables
- Update test results for <subqueryN> tables:
s/unique_key/distinct_key/g
s/1.0/100.0/ for "filtered" column
- Change "SUBQUERY#n" to "<subquery{n}>" in EXPLAIN output. We need to it to be
lowercase so that EXPLAIN results do not differ in case between systems with
case-sensitive and case-insensitive filesystems.
- Remove garbage comments, add better comments.
- Add Item_in_subselect::get_identifier() that returns subquery's id
- Change select_describe() to produce output in new format
- Update test results (checked)
query
The fix for bug 46749 removed the check for OUTER_REF_TABLE_BIT
and substituted it for a check on the presence of
Item_ident::depended_from.
Removing it altogether was wrong : OUTER_REF_TABLE_BIT should
still be checked in addition to depended_from (because it's not
set in all cases and doesn't contradict to the check of depended_from).
Fixed by returning the old condition back as a compliment to the
new one.
function,file sql_base.cc
When uncacheable queries are written to a temp table the optimizer must
preserve the original JOIN structure, because it is re-using the JOIN
structure to read from the resulting temporary table.
This was done only for uncacheable sub-queries.
But top level queries can also benefit from this mechanism, specially if
they're using index access and need a reset.
Fixed by not limiting the saving of JOIN structure to subqueries
exclusively.
Added a new test file to extend the existing (large) subquery.test.
