fixed several defects in the greedy optimization:
1) The greedy optimizer calculated the 'compare-cost' (CPU-cost)
for iterating over the partial plan result at each level in
the query plan as 'record_count / (double) TIME_FOR_COMPARE'
This cost was only used locally for 'best' calculation at each
level, and *not* accumulated into the total cost for the query plan.
This fix added the 'CPU-cost' of processing 'current_record_count'
records at each level to 'current_read_time' *before* it is used as
'accumulated cost' argument to recursive
best_extension_by_limited_search() calls. This ensured that the
cost of a huge join-fanout early in the QEP was correctly
reflected in the cost of the final QEP.
To get identical cost for a 'best' optimized query and a
straight_join with the same join order, the same change was also
applied to optimize_straight_join() and get_partial_join_cost()
2) Furthermore to get equal cost for 'best' optimized query and a
straight_join the new code substrcated the same '0.001' in
optimize_straight_join() as it had been already done in
best_extension_by_limited_search()
3) When best_extension_by_limited_search() aggregated the 'best' plan a
plan was 'best' by the check :
'if ((search_depth == 1) || (current_read_time < join->best_read))'
The term '(search_depth == 1' incorrectly caused a new best plan to be
collected whenever the specified 'search_depth' was reached - even if
this partial query plan was more expensive than what we had already
found.
The patch differs from the original MySQL patch as follows:
- All test case differences have been reviewed one by one, and
care has been taken to restore the original plan so that each
test case executes the code path it was designed for.
- A bug was found and fixed in MariaDB 5.3 in
Item_allany_subselect::cleanup().
- ORDER BY is not removed because we are unsure of all effects,
and it would prevent enabling ORDER BY ... LIMIT subqueries.
- ref_pointer_array.m_size is not adjusted because we don't do
array bounds checking, and because it looks risky.
Original comment by Jorgen Loland:
-------------------------------------------------------------
WL#5953 - Optimize away useless subquery clauses
For IN/ALL/ANY/SOME/EXISTS subqueries, the following clauses are
meaningless:
* ORDER BY (since we don't support LIMIT in these subqueries)
* DISTINCT
* GROUP BY if there is no HAVING clause and no aggregate
functions
This WL detects and optimizes away these useless parts of the
query during JOIN::prepare()
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
For ANY subqueries NULLs should be ignored (if there is other values) when finding max min.
For ALL subqueries NULLs should be saved if they found.
Optimisation for ALL suqbueries if NULL is possible in the SELECT list with max/min aggregate function switched off.
Some test changed where NULL is not used but optimization with max/min aggregate function important so NOT NULL added.
mysql-test/r/explain.result:
Forced old optimization.
mysql-test/r/subselect.result:
Forced old optimization.
New test suite.
mysql-test/t/explain.test:
Forced old optimization.
mysql-test/t/subselect.test:
Forced old optimization.
New test suite.
sql/item_subselect.cc:
Store converted subquery type.
Switch off aggregate function optimisation for ALL and nulls.
sql/sql_class.cc:
Fixed NULL comparison.
sql/sql_class.h:
Store converted subquery type.
mysql-test/r/explain.result:
fixed results (new item)
mysql-test/r/subselect.result:
fixed results (new item)
mysql-test/r/subselect_no_mat.result:
fixed results (new item)
mysql-test/r/subselect_no_opts.result:
fixed results (new item)
mysql-test/r/subselect_no_semijoin.result:
Fixed results (new item)
mysql-test/suite/pbxt/r/subselect.result:
Fixed results (new item)
mysql-test/t/explain.test:
Fixed results (correct behaviour)
sql/item_cmpfunc.cc:
Pass through for max/min
sql/item_subselect.cc:
moving max/min
sql/item_subselect.h:
moving max/min
sql/mysql_priv.h:
new uncacheble flags added
sql/opt_subselect.cc:
maxmin moved.
sql/opt_subselect.h:
New function for maxmin.
sql/sql_class.h:
debug code
sql/sql_lex.cc:
Fixed flags.
Limit setting fixed.
sql/sql_lex.h:
2 new flags.
sql/sql_select.cc:
Prepare divided on 2 function to be able recollect some info after transformation.
sql/sql_select.h:
Prepare divided on 2 functions.
The query was re-written *after* we had tagged it with NON_AGG_FIELD_USED.
Remove the flag before continuing.
mysql-test/r/explain.result:
Update test case for Bug#48295.
mysql-test/r/subselect.result:
New test case.
mysql-test/t/explain.test:
Update test case for Bug#48295.
mysql-test/t/subselect.test:
New test case.
sql/item.cc:
Use accessor functions for non_agg_field_used/agg_func_used.
sql/item_subselect.cc:
Remove non_agg_field_used when we rewrite query '1 < some (...)' => '1 < max(...)'
sql/item_sum.cc:
Use accessor functions for non_agg_field_used/agg_func_used.
sql/mysql_priv.h:
Remove unused #defines.
sql/sql_lex.cc:
Initialize new member variables.
sql/sql_lex.h:
Replace full_group_by_flag with two boolean flags,
and itroduce accessors for manipulating them.
sql/sql_select.cc:
Use accessor functions for non_agg_field_used/agg_func_used.
This is a backport of
Bug #46860 Crash/segfault using EXPLAIN EXTENDED on query using UNION in subquery.
mysql-test/r/explain.result:
Remove the extended Note, since explain returns error.
New test case.
mysql-test/r/func_gconcat.result:
Remove the extended Note, since explain returns error.
mysql-test/r/subselect3.result:
Remove the extended Note, since explain returns error.
mysql-test/t/explain.test:
New test case.
sql/sql_parse.cc:
Skip the extended description if we have an error.
create_sort_index() function overwrites original JOIN_TAB::type field.
At re-execution of subquery overwritten JOIN_TAB::type(JT_ALL) is
used instead of JT_FT. It misleads test_if_skip_sort_order() and
the function tries to find suitable key for the order that should
not be allowed for FULLTEXT(JT_FT) table.
The fix is to restore JOIN_TAB strucures for subselect on re-execution
for EXPLAIN.
Additional fix:
Update TABLE::maybe_null field which
affects list_contains_unique_index() behaviour as it
could have the value(maybe_null==TRUE) based on the
assumption that this join is outer
(see setup_table_map() func).
mysql-test/r/explain.result:
test case
mysql-test/t/explain.test:
test case
sql/item_subselect.cc:
Make subquery uncacheable in case of EXPLAIN. It allows to keep
original JOIN_TAB::type(see JOIN::save_join_tab) and restore it
on re-execution.
sql/sql_select.cc:
-restore JOIN_TAB strucures for subselect on re-execution for EXPLAIN
-Update TABLE::maybe_null field as it could have
the value(maybe_null==TRUE) based on the assumption
that this join is outer(see setup_table_map() func).
This change is not related to the crash problem but
affects EXPLAIN results in the test case.
and .tar.gz, windows vs linux..
On Intel x86 machines index selection by the MySQL query
optimizer could sometimes depend on the compiler version and
optimization flags used to build the server binary.
The problem was a result of a known issue with floating point
calculations on x86: since internal FPU precision (80 bit)
differs from precision used by programs (32-bit float or 64-bit
double), the result of calculating a complex expression may
depend on how FPU registers are allocated by the compiler and
whether intermediate values are spilled from FPU to memory. In
this particular case compiler versions and optimization flags
had an effect on cost calculation when choosing the best index
in best_access_path().
A possible solution to this problem which has already been
implemented in mysql-trunk is to limit FPU internal precision
to 64 bits. So the fix is a backport of the relevant code to
5.1 from mysql-trunk.
configure.in:
Configure check for fpu_control.h
mysql-test/r/explain.result:
Test case for bug #48537.
mysql-test/t/explain.test:
Test case for bug #48537.
sql/mysqld.cc:
Backport of the code to switch FPU on x86 to 64-bit precision.
Conflicts:
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/t/explain.test
Text conflict in sql/net_serv.cc
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_priv.h
WHERE predicates containing references to empty tables in a
subquery were handled incorrectly by the optimizer when
executing EXPLAIN. As a result, the optimizer could try to
evaluate such predicates rather than just stop with
"Impossible WHERE noticed after reading const tables" as
it would do in a non-subquery case. This led to valgrind
errors and crashes.
Fixed the code checking the above condition so that subqueries
are not excluded and hence are handled in the same way as top
level SELECTs.
mysql-test/r/explain.result:
Added a test case for bug #48419.
mysql-test/r/ps.result:
Updated test results to take the new (and more correct)
"Extra" comments in execution plans.
mysql-test/t/explain.test:
Added a test case for bug #48419.
sql/sql_select.cc:
There is no point in excluding subqueries from checking
for identically false WHERE conditions.
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/r/having.result
Text conflict in mysql-test/suite/rpl/t/disabled.def
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in storage/federated/ha_federated.cc
If an outer query is broken, a subquery might not even get set up.
EXPLAIN EXTENDED did not expect this and merrily tried to de-ref all
of the half-setup info.
We now catch this case and print as much as we have, as it doesn't cost us
anything (doesn't make regular execution slower).
backport from 5.1
mysql-test/r/explain.result:
Show that EXPLAIN EXTENDED with subquery and illegal out query doesn't crash.
Show also that SHOW WARNINGS will render an additional Note in the hope of
being, well, helpful.
mysql-test/t/explain.test:
If we have only half a query for EXPLAIN EXTENDED to print (i.e.,
incomplete subquery info as outer query is illegal), we should
provide the user with as much info as we easily can if they ask
for it. What we should not do is crash when they come asking for
help, that violates etiquette in some countries.
sql/item_subselect.cc:
If the sub-query's actually set up, print it. Otherwise, elide.
Item_field::print method does not take into
account fields whose values may be null.
The fix is to print 'NULL' if field value is null.
mysql-test/r/explain.result:
test case
mysql-test/r/func_str.result:
result fix
mysql-test/r/having.result:
result fix
mysql-test/r/select.result:
result fix
mysql-test/r/subselect.result:
result fix
mysql-test/r/union.result:
result fix
mysql-test/t/explain.test:
test case
sql/item.cc:
print 'NULL' if field value is null.
If an outer query is broken, a subquery might not even get set up.
EXPLAIN EXTENDED did not expect this and merrily tried to de-ref all
of the half-setup info.
We now catch this case and print as much as we have, as it doesn't cost us
anything (doesn't make regular execution slower).
mysql-test/r/explain.result:
Show that EXPLAIN EXTENDED with subquery and illegal out query doesn't crash.
Show also that SHOW WARNINGS will render an additional Note in the hope of
being, well, helpful.
mysql-test/t/explain.test:
If we have only half a query for EXPLAIN EXTENDED to print (i.e.,
incomplete subquery info as outer query is illegal), we should
provide the user with as much info as we easily can if they ask
for it. What we should not do is crash when they come asking for
help, that violates etiquette in some countries.
sql/item_subselect.cc:
If the sub-query's actually set up, print it. Otherwise, elide.
EXPLAIN EXTENDED warning.
Query optimizer searches for the constant tables and optimizes them away. This
means that fields of such tables are substituted for their values and on later
phases they are treated as constants. After this constant tables are removed
from the query execution plan. Nevertheless constant tables were shown in
the EXPLAIN EXTENDED warning thus producing query that might be not an
equivalent of the original query.
Now the print_join function skips all tables that were optimized away from
printing to the EXPLAIN EXTENDED warning. If all tables were optimized away it
produces the 'FROM dual' clause.
mysql-test/r/explain.result:
A test case added for the bug#30302.
mysql-test/r/func_default.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/r/func_regexp.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/r/func_test.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/r/having.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/r/select.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/r/subselect.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/r/subselect3.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/r/type_datetime.result:
Adjusted test case result after fix for the bug#30302.
mysql-test/t/explain.test:
A test case added for the bug#30302.
sql/sql_select.cc:
Bug#30302: Tables that were optimized away are printed in the
EXPLAIN EXTENDED warning.
Now the print_join function skips all tables that were optimized away from
printing to the EXPLAIN EXTENDED warning. If all tables were optimized away it
produces the 'FROM dual' clause.
sql/table.h:
Adjusted test case result after fix for the bug#30302.
The optimized_away flag is added to the TABLE_LIST struct.
Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.
mysql-test/r/explain.result:
test result
mysql-test/t/explain.test:
test case
sql/sql_select.cc:
Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.
The copy of the original arguments of a aggregate function was not
initialized until after fix_fields().
Sometimes (e.g. when there's an error processing the statement)
the print() can be called with no corresponding fix_fields() call.
Fixed by adding a check if the Item is fixed before using the arguments
copy.
mysql-test/r/explain.result:
Bug #43354: test case
mysql-test/t/explain.test:
Bug #43354: test case
sql/item_sum.cc:
Bug #43354: use the argument list copy only if it's initialized
The convert_constant_item function converts a constant to integer using
field for condition like 'field = a_constant'. In some cases the
convert_constant_item is called for a subquery when outer select is already
being executed, so convert_constant_item saves field's value to prevent its
corruption. For EXPLAIN and at the prepare phase field's value isn't
initialized yet, thus when convert_constant_item tries to restore saved
value it fails assertion.
Now the convert_constant_item doesn't save/restore field's value if it's
haven't been read yet. Outer constant values are always saved.
mysql-test/r/explain.result:
Added a test case for the bug#37870.
mysql-test/t/explain.test:
Added a test case for the bug#37870.
sql/item_cmpfunc.cc:
Bug#37870: Usage of uninitialized value caused failed assertion.
Now the convert_constant_item doesn't save/restore field's value if it's
haven't been read yet. Outer constant values are always saved.
The convert_constant_item function converts a constant to integer using
field for condition like 'field = a_constant'. When the convert_constant_item
is called for a subquery the outer select is already being executed, so
convert_constant_item saves field's value to prevent its corruption.
For EXPLAIN field's value isn't initialized thus when convert_constant_item
tries to restore saved value it fails assertion.
Now the convert_constant_item doesn't save/restore field's value
for EXPLAIN.
mysql-test/r/explain.result:
Added a test case for the bug#37870.
mysql-test/t/explain.test:
Added a test case for the bug#37870.
sql/item_cmpfunc.cc:
Bug#37870: Usage of uninitialized value caused failed assertion.
Now the convert_constant_item doesn't save/restore field's value
for EXPLAIN.
crashes server
When creating temporary table that contains aggregate functions a
non-reversible source transformation was performed to redirect aggregate
function arguments towards temporary table columns.
This caused EXPLAIN EXTENDED to fail because it was trying to resolve
references to the (freed) temporary table.
Fixed by preserving the original aggregate function arguments and
using them (instead of the transformed ones) for EXPLAIN EXTENDED.
mysql-test/r/explain.result:
Bug#34773: test case
mysql-test/t/explain.test:
Bug#34773: test case
sql/item.cc:
Bug#34773: use accessor functions instead of public members
sql/item_sum.cc:
Bug#34773:
- Encapsulate the arguments into Item_sum and
provide accessor and mutator methods
- print the orginal arguments (if present)
in EXPLAIN EXTENDED
- preserve the original arguments list.
sql/item_sum.h:
Bug#34773:
- Encapsulate the arguments into Item_sum and
provide accessor and mutator methods
- print the orginal arguments (if present)
in EXPLAIN EXTENDED
- preserve the original arguments list.
sql/opt_range.cc:
Bug#34773: use accessor functions instead of public members
sql/opt_sum.cc:
Bug#34773: use accessor functions instead of public members
sql/sql_select.cc:
Bug#34773: use accessor functions instead of public members
checked for each record'
The problem was in incorrectly calculated length of the buffer used to
store a hexadecimal representation of an index map in
select_describe(). This could result in buffer overrun and stack
corruption under some circumstances.
Fixed by correcting the calculation.
mysql-test/r/explain.result:
Added a test case for bug #32241.
mysql-test/t/explain.test:
Added a test case for bug #32241.
sql/sql_select.cc:
Corrected the buffer length calculation. Count one hex digit as 4 bits,
not 8.
Postfix for bug#22331 for windows platform.
explain.test, explain.result:
Cleanup after bugfix#22331.
mysql-test/t/explain.test:
Cleanup after bugfix#22331.
mysql-test/r/explain.result:
Cleanup after bugfix#22331.
sql/sql_select.cc:
Postfix for bug#22331 for windows platform.
away.
During optimization stage the WHERE conditions can be changed or even
be removed at all if they know for sure to be true of false. Thus they aren't
showed in the EXPLAIN EXTENDED which prints conditions after optimization.
Now if all elements of an Item_cond were removed this Item_cond is substituted
for an Item_int with the int value of the Item_cond.
If there were conditions that were totally optimized away then values of the
saved cond_value and having_value will be printed instead.
mysql-test/t/explain.test:
Added a test case for the bug#22331: Wrong WHERE in EXPLAIN EXTENDED when all expressions were optimized away.
mysql-test/r/subselect.result:
Corrected test case result after fix for bug#22331.
mysql-test/r/func_test.result:
Corrected test case result after fix for bug#22331.
mysql-test/r/explain.result:
Added a test case for the bug#22331: Wrong WHERE in EXPLAIN EXTENDED when all expressions were optimized away.
sql/sql_select.cc:
Bug#22331: Wrong WHERE in EXPLAIN EXTENDED when all expressions were optimized
away.
Now if all elements of an Item_cond were removed this Item_cond is substituted
for an Item_int with the int value of the Item_cond.
If there were conditions that were totally optimized away then values of the
saved cond_value and having_value will be printed instead.
sql/sql_lex.h:
Bug#22331: Wrong WHERE in EXPLAIN EXTENDED when all expressions were optimized
away.
The cond_value and the having_value variables are
added to the SELECT_LEX class.
sql/sql_lex.cc:
Bug#22331: Wrong WHERE in EXPLAIN EXTENDED when all expressions were optimized
away.
The initialization of the cond_value and the having_value variables.
sql/sql_select.h:
Bug#22331: Wrong WHERE in EXPLAIN EXTENDED when all expressions were optimized
away.
Now having_value is also stored in the JOIN class.
refers to a column name.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
include/mysqld_error.h:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
include/sql_state.h:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
mysql-test/r/explain.result:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
mysql-test/r/key_cache.result:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
mysql-test/r/preload.result:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
mysql-test/r/select.result:
Added a test case for bug #17873.
mysql-test/t/explain.test:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
mysql-test/t/select.test:
Added a test case for bug #17873.
sql/share/czech/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/danish/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/dutch/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/english/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/estonian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/french/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/german/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/greek/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/hungarian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/italian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/japanese-sjis/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/japanese/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/korean/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/norwegian-ny/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/norwegian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/polish/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/portuguese/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/romanian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/russian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/serbian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/slovak/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/spanish/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/swedish/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
sql/share/ukrainian/errmsg.txt:
Fixed bug #17873.
Added a new error message ER_INDEX_DOES_NOT_EXIST.
There were two distict bugs: parse error was returned for valid
statement and that error wasn't reported to the client.
The fix ensures that EXPLAIN SELECT..INTO is accepted by parser and any
other parse error will be reported to the client.
mysql-test/r/explain.result:
Add result for bug#15463.
mysql-test/t/explain.test:
Add test case for bug#15463.
sql/sql_parse.cc:
Assert that if parsing error has occured then apropriate error message
has been pushed into error stack.
sql/sql_yacc.yy:
If there is no lex->result in select_var_ident rule, then we have
to be in DESCRIBE mode.
