Do not call, directly or indirectly, SQL_SELECT::test_quick_select()
for derived materialized tables / views when optimizing joins referring
to these tables / views to get cost estimates of materialization.
The current code does not create B-tree indexes for materialized
derived tables / views. So now it's not possible to get any estimates
for ranges conditions over the results of the materialization.
The function mysql_derived_create() must take into account the fact
that array of the KEY structures specifying the keys over a derived
table / view may be moved after the optimization phase if the
derived table / view is materialized.
This bug in the function JOIN::drop_unused_derived_keys() could
leave the internal structures for a materialized derived table
in an inconsistent state. This led to a not quite correct EXPLAIN
output when no additional key had been created to access the table.
It also may lead to more serious consequences: so, the test case
added with this fix caused a crash in mariadb-5.5.20.
The result of materialization of the right part of an IN subquery predicate
is placed into a temporary table. Each row of the materialized table is
distinct. A unique key over all fields of the temporary table is defined and
created. It allows to perform key look-ups into the table.
The table created for a materialized subquery can be accessed by key as
any other table. The function best_access-path search for the best access
to join a table to a given partial join. With some where conditions this
function considers a possibility of a ref_or_null access. If such access
employs the unique key on the temporary table then when estimating
the cost this access the function tries to use the array rec_per_key. Yet,
such array is not built for this unique key. This causes a crash of the server.
Rows returned by the subquery that contain nulls don't have to be placed
into temporary table, as they cannot be match any row produced by the
left part of the subquery predicate. So all fields of the temporary table
can be defined as non-nullable. In this case any ref_or_null access
to the temporary table does not make any sense and it does not make sense
to estimate such an access.
The fix makes sure that the temporary table for a materialized IN subquery
is defined with columns that are all non-nullable. The also ensures that
any row with nulls returned by the subquery is not placed into the
temporary table.
If the expression for a derived table of a query contained a LIMIT
clause the estimate of the number of rows in this derived table
returned by the EXPLAIN command could be badly off since the
optimizer ignored the limit number from the LIMIT clause when
getting the estimate.
The call of the method SELECT_LEX_UNIT->set_limit added in the code
of mysql_derived_optimize() will be needed also in maria-5.5 where
parameters in the LIMIT clause are supported.
fixed several defects in the greedy optimization:
1) The greedy optimizer calculated the 'compare-cost' (CPU-cost)
for iterating over the partial plan result at each level in
the query plan as 'record_count / (double) TIME_FOR_COMPARE'
This cost was only used locally for 'best' calculation at each
level, and *not* accumulated into the total cost for the query plan.
This fix added the 'CPU-cost' of processing 'current_record_count'
records at each level to 'current_read_time' *before* it is used as
'accumulated cost' argument to recursive
best_extension_by_limited_search() calls. This ensured that the
cost of a huge join-fanout early in the QEP was correctly
reflected in the cost of the final QEP.
To get identical cost for a 'best' optimized query and a
straight_join with the same join order, the same change was also
applied to optimize_straight_join() and get_partial_join_cost()
2) Furthermore to get equal cost for 'best' optimized query and a
straight_join the new code substrcated the same '0.001' in
optimize_straight_join() as it had been already done in
best_extension_by_limited_search()
3) When best_extension_by_limited_search() aggregated the 'best' plan a
plan was 'best' by the check :
'if ((search_depth == 1) || (current_read_time < join->best_read))'
The term '(search_depth == 1' incorrectly caused a new best plan to be
collected whenever the specified 'search_depth' was reached - even if
this partial query plan was more expensive than what we had already
found.
If the optimizer switch 'semijoin_with_cache' is set to 'off' then
join cache cannot be used to join inner tables of a semijoin.
Also fixed a bug in the function check_join_cache_usage() that led
to wrong output of the EXPLAIN commands for some test cases.
- The bug was caused by the following scenario:
= a quick select is created with get_quick_select_for_ref. The quick
select refers to temporary (derived) table. It saves table->file, which
refers to a ha_heap object.
= When temp table is populated, ha_heap reaches max. size and is converted
to a ha_myisam. However, quick->file remains pointing to where ha_heap
was.
= Attempt to use the quick select causes crash.
- Fixed by introducing QUICK_SELECT_I::replace_handler(). Note that it will
not work for index_merge quick selects. Which is fine, because these
quick selects are never created for derived tables.
The function Item_direct_view_ref::fix_fields erroneously did not correct
the value of the flag maybe_null when the view for which the item was
being fixed happened to be an inner table of an outer join.
The function SELECT_LEX::update_used_tables first must clean up
all bitmaps to be recalculated for all tables that require it
and only after this walk through on conditions attached to the
tables to update these bitmaps.
This bug happened for the queries over multi-table mergeable views
because the bitmap TABLE::read_set of the underlying tables were not
updated after the views had been merged into the query.
Now this bitmaps are updated properly.
Also the bitmap TABLE::merge_keys now is updated in prevention of
future bugs.
The function JOIN::drop_unused_derived_keys could erroneously set
the value of REF::key to 0 for a joined materialized view/derived table
in the case when no REF access to the table was used by the query
execution plan. This could cause a crash of the server.
This bug happened due to incompleteness of the fix for bug 872735:
the occurrences of the fields in the conditions of correlated
subqueries were not taken into account when recalculating
covering keys bit maps.
This bug manifested itself with queries containing non-correlated
IN subqueries over materialized views/derived tables.
The bug happened because the code of the function generate_derived_keys did
not take into account that the function could be called twice when the
optimizer was deciding whether in-exist transformation should be applied.
This bug happened because the maps of covering keys for mergeable derived
tables/views was not recalculated after the derived tables/vies had been
merged into the main query.
of the 5.3 code line after a merge with 5.2 on 2010-10-28
in order not to allow the cost to access a joined table to be equal
to 0 ever.
Expanded data sets for many test cases to get the same execution plans
as before.
When merging a view / derived table the function SELECT_LEX::merge_subquery
incorrectly updated the list SELECT_LEX::leaf_tables. Erroneously it
appended the leaf_tables list of the merged object L and then removed the
reference to the merged object T from the SELECT_LEX::leaf_tables list.
A correct implementation should insert the list L into the
SELECT_LEX::leaf_tables list in place of the element of the list that
refers to T.
The bug could lead to wrong results or even crashes for queries with
nested outer joins over views / derived tables.
The method Item_func_isnull::update_used_tables() erroneously did not
update cached values stored in the fields used_tables_cache and
const_item_cache of the Item_func_isnull objects. As a result the
Item_func_isnull::used_tables() returned wrong bitmaps and, as a
consequence, push-down predicates could be attached to wrong tables.
This bug could lead to wrong result sets for a query over a
materialized derived table or view accessed by a multi-component
key.
It happened because the function get_next_field_for_derived_key
was supposed to update its argument, and it did not do it.
Also:
1. simplified the code of the function mysql_derived_merge_for_insert.
2. moved merge of views/dt for multi-update/delete to the prepare stage.
3. the list of the references to the candidates for semi-join now is
allocated in the statement memory.
The attribute not_null_tables could be calculated incorrectly in the
function SELECT_LEX::update_used_tables for queries over views
with row items in the WHERE clause. It happened because no
implementation of the virtual callback function eval_not_null_tables
was provided for the class Item_row.
Also slightly optimized the code calculating the value of the maybe_null
flag for tables in the function SELECT_LEX::update_used_tables.
The bitmap of used tables must be evaluated for the select list of every
materialized derived table / view and saved in a dedicated field.
This is also applied to materialized subqueries.
Missing initialization of the bitmap not_null_tables_cache to 0
in the function Item_func::eval_not_null_tables caused this bug.
This function is called indirectly from the function
SELECT_LEX::update_used_tables after merging mergeable views and
derived tables into the main query. The leaf tables of resulting
query may change their bitmap numbers after this merge. That's why
the not_null_tables_cache bitmaps must be updated. Due to the bug
mentioned above the result of the re-evaluation of the
not_null_tables_cache turned out to be incorrect in some cases.
This could trigger an invalid conversion of outer joins into
inner joins leading to invalid query result sets.
Also removed an implicit conversion from int to bool in the function
SELECT_LEX::update_used_tables.
The value of THD::used tables should be re-evaluated after merges
of views and derived tables into the main query.
Now it's done in the function SELECT_LEX::update_used_tables.
The re-evaluation of the 'used_table' bitmaps for the items
in HAVING, GROUP BY and ORDER BY clauses has been added as well.
The bug was caused by an incorrect code of the function
Item_direct_view_ref::replace_equal_field introduced in the
patch for bugs 717577, 724942. The function erroneously
returned the wrapped field instead of the Item_direct_view_ref
object itself in the cases when no replacement happened.
The bug masked two other minor bugs that could result in not
quite correct output of the EXPLAIN command for some queries.
They were fixed in the patch as well.
The offending query returns a wrong result set because the optimizer
erroneously eliminated the where condition evaluated it to TRUE.
The cause of this wrong transformation was that the flag maybe_null
for an inner table of the outer join was not set to TRUE after the
table had replaced the wrapping view.
Now the function SELECT_LEX::update_used_tables resets the value
of the maybe_null flag for each leaf table of the query after all
merges of views have been done.
The function generate_derived_keys_for_table incorrectly handled
the cases when a materialized view or derived table could be accessed
by different keys on the same fields if these keys depended on the
same tables.
