Problem 1:
When the 'Using index' optimization is used, the optimizer may still - after
cost-based optimization - decide to use another index in order to avoid using
a temporary table. But when this happens, the flag to the storage engine to
read index only (not table) was still set. Fixed by resetting the flag in the
storage engine and TABLE structure in the above scenario, unless the new index
allows for the same optimization.
Problem 2:
When a 'ref' access method was employed by cost-based optimizer, (when the column
is non-NULLable), it was assumed that it needed no initialization if 'quick' access
methods (since they are based on range scan). When ORDER BY optimization overrides
the decision, however, it expects to have this initialized and hence crashes.
Fixed in 5.1 (was fixed in 6.0 already) by initializing 'quick' even when there's
'ref' access.
mysql-test/r/order_by.result:
Bug#46454: Test result.
mysql-test/t/order_by.test:
Bug#46454: Test case.
sql/sql_select.cc:
Bug#46454:
Problem 1 fixed in make_join_select()
Problem 2 fixed in test_if_skip_sort_order()
sql/table.h:
Bug#46454: Added comment to field.
In create_myisam_from_heap() mark all errors as fatal except
HA_ERR_RECORD_FILE_FULL for a HEAP table.
Not doing so could lead to problems, e.g. in a case when a
temporary MyISAM table gets overrun due to its MAX_ROWS limit
while executing INSERT/REPLACE IGNORE ... SELECT.
The SELECT execution was aborted, but the error was
converted to a warning due to IGNORE clause, so neither 'ok'
nor 'error' packet could be sent back to the client. This
condition led to hanging client when using 5.0 server, or
assertion failure in 5.1.
mysql-test/r/insert_select.result:
Added a test case for bug #46075.
mysql-test/t/insert_select.test:
Added a test case for bug #46075.
sql/sql_select.cc:
In create_myisam_from_heap() mark all errors as fatal except
HA_ERR_RECORD_FILE_FULL for a HEAP table.
use partial primary key if another index can prevent filesort
The fix for bug #28404 causes the covering ordering indexes to be
preferred unconditionally over non-covering and ref indexes.
Fixed by comparing the cost of using a covering index to the cost of
using a ref index even for covering ordering indexes.
Added an assertion to clarify the condition the local variables should
be in.
mysql-test/include/mix1.inc:
Bug #36259: fixed a non-stable test case
mysql-test/r/innodb_mysql.result:
Bug #36259 and #45828 : test case
mysql-test/t/innodb_mysql.test:
Bug #36259 and #45828 : test case
sql/sql_select.cc:
Bug #36259 and #45828 : don't consider covering indexes supperior to
ref keys.
Using DECIMAL constants with more than 65 digits in CREATE
TABLE ... SELECT led to bogus errors in release builds or
assertion failures in debug builds.
The problem was in inconsistency in how DECIMAL constants and
fields are handled internally. We allow arbitrarily long
DECIMAL constants, whereas DECIMAL(M,D) columns are limited to
M<=65 and D<=30. my_decimal_precision_to_length() was used in
both Item and Field code and truncated precision to
DECIMAL_MAX_PRECISION when calculating value length without
adjusting precision and decimals. As a result, a DECIMAL
constant with more than 65 digits ended up having length less
than precision or decimals which led to assertion failures.
Fixed by modifying my_decimal_precision_to_length() so that
precision is truncated to DECIMAL_MAX_PRECISION only for Field
object which is indicated by the new 'truncate' parameter.
Another inconsistency fixed by this patch is how DECIMAL
constants and expressions are handled for CREATE ... SELECT.
create_tmp_field_from_item() (which is used for constants) was
changed as a part of the bugfix for bug #24907 to handle long
DECIMAL constants gracefully. Item_func::tmp_table_field()
(which is used for expressions) on the other hand was still
using a simplistic approach when creating a Field_new_decimal
from a DECIMAL expression.
mysql-test/r/type_newdecimal.result:
Added a test case for bug #45262.
mysql-test/t/type_newdecimal.test:
Added a test case for bug #45262.
sql/item.cc:
Use the new 'truncate' parameter in
my_decimal_precision_to_length().
sql/item_cmpfunc.cc:
Use the new 'truncate' parameter in
my_decimal_precision_to_length().
sql/item_func.cc:
1. Use the new 'truncate' parameter in
my_decimal_precision_to_length().
2. Do not truncate decimal precision to DECIMAL_MAX_PRECISION
for additive expressions involving long DECIMAL constants.
3. Fixed an incosistency in how DECIMAL constants and
expressions are handled for CREATE ... SELECT.
sql/item_func.h:
Use the new 'truncate' parameter in
my_decimal_precision_to_length().
sql/item_sum.cc:
Use the new 'truncate' parameter in
my_decimal_precision_to_length().
sql/my_decimal.h:
Do not truncate precision to DECIMAL_MAX_PRECISION
when calculating length in
my_decimal_precision_to_length() if 'truncate' parameter
is FALSE.
sql/sql_select.cc:
1. Use the new 'truncate' parameter in
my_decimal_precision_to_length().
2. Use a more correct logic when adjusting value's length.
- Last fixes
sql/item.cc:
MWL#17: Table elimination
- Don't make multiple calls of ::walk(check_column_usage_processor),
call once and cache the value
sql/item.h:
MWL#17: Table elimination
- s/KEYUSE::usable/KEYUSE::type/, more comments
sql/opt_table_elimination.cc:
MWL#17: Table elimination
- Don't make multiple calls of ::walk(check_column_usage_processor),
call once and cache the value
sql/sql_select.cc:
MWL#17: Table elimination
- s/KEYUSE::usable/KEYUSE::type/, more comments
sql/sql_select.h:
MWL#17: Table elimination
- s/KEYUSE::usable/KEYUSE::type/, more comments
sql/table.h:
MWL#17: Table elimination
- Better comments
- Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
- Extended the maximun key parts for an index from 16 to 32
- Extended MyISAM and Maria engines to support up to 32 parts
Added checks for return value from ha_index_init()
include/my_handler.h:
Extended number of key parts for MyISAM and Maria from 16 to 32
include/my_pthread.h:
Ensure we always have 256M of stack.
(Required to be able to handle the current number of keys and key parts in MyISAM)
mysql-test/r/create.result:
Extended to test for 32 key parts
mysql-test/r/myisam.result:
Test that we can create 32 but not 33 key parts
mysql-test/r/ps_1general.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_2myisam.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_3innodb.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_4heap.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_5merge.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/suite/maria/r/maria.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/maria3.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/ps_maria.result:
Max key length is now 1208 byte
mysql-test/t/create.test:
Extended to test for 32 key parts
mysql-test/t/myisam.test:
Test that we can create 32 but not 33 key parts
sql/handler.cc:
Check return value from ha_index_init()
sql/handler.h:
Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
sql/sql_select.cc:
Checks all return values from ha_index_init()
Call prepare_index_scan()) to inform storage engines that an index scan is about to take place.
Fixed indentation
sql/table.cc:
Fixed wrong types for key_length (rest of code assumed this was 32 bit)
sql/unireg.h:
Extended the maximun key parts for an index from 16 to 32
storage/maria/ha_maria.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/ha_myisam.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/mi_check.c:
Fixed wrong check if value overflow
tests/mysql_client_test.c:
Added fflush() to fix output in case of error
Fixed wrong check of 'ref' length in EXPLAIN
The TABLE::reginfo.impossible_range is used by the optimizer to indicate
that the condition applied to the table is impossible. It wasn't initialized
at table opening and this might lead to an empty result on complex queries:
a query might set the impossible_range flag on a table and when the query finishes,
all tables are returned back to the table cache. The next query that uses the table
with the impossible_range flag set and an index over the table will see the flag
and thus return an empty result.
The open_table function now initializes the TABLE::reginfo.impossible_range
variable.
mysql-test/r/select.result:
A test case for the bug#45266: Uninitialized variable lead to an empty result.
mysql-test/t/select.test:
A test case for the bug#45266: Uninitialized variable lead to an empty result.
sql/sql_base.cc:
Bug#45266: Uninitialized variable lead to an empty result.
The open_table function now initializes the TABLE::reginfo.impossible_range
variable.
sql/sql_select.cc:
Bug#45266: Uninitialized variable lead to an empty result.
The open_table function now initializes the TABLE::reginfo.impossible_range
variable.
sql/structs.h:
Bug#45266: Uninitialized variable lead to an empty result.
A comment is added.
- More testcases
- Let add_ft_key() set keyuse->usable
mysql-test/r/table_elim.result:
MWL#17: Table elimination
- More testcases
mysql-test/t/table_elim.test:
MWL#17: Table elimination
- More testcases
sql/sql_select.cc:
MWL#17: Table elimination
- Let add_ft_key() set keyuse->usable
- Make elimination check to be able detect cases like t.primary_key_col1=othertbl.col AND t.primary_key_col2=func(t.primary_key_col1).
These are needed to handle e.g. the case of func() being a correlated subquery that selects the latest value.
- If we've removed a condition with subquery predicate, EXPLAIN [EXTENDED] won't show the subquery anymore
sql/item.cc:
MWL#17: Table elimination
- Add tem_field::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item.h:
MWL#17: Table elimination
- Add tem_field::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item_subselect.cc:
MWL#17: Table elimination
- Item_subselect got 'eliminated' attribute. It is used only to determine if the subselect should be printed by EXPLAIN.
- Item_subselect got List<Item> refers_to - a list of item in the current select that are referred to from within the subselect.
- Added Item_*::check_column_usage_processor(). it allows to check which key parts a condition depends on.
- Added a comment about possible problem in Item_subselect::walk
sql/item_subselect.h:
MWL#17: Table elimination
- Item_subselect got 'eliminated' attribute. It is used only to determine if the subselect should be printed by EXPLAIN.
- Item_subselect got List<Item> refers_to - a list of item in the current select that are referred to from within the subselect.
- Added Item_*::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item_sum.cc:
MWL#17: Table elimination
sql/sql_lex.cc:
MWL#17: Table elimination
sql/sql_lex.h:
MWL#17: Table elimination
sql/sql_select.h:
MWL#17: Table elimination
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
- Move eliminate_tables() to before constant table detection.
- First code for benchmark
sql-bench/test-table-elimination.sh:
MWL#17: Table elimination
- sql-bench "Benchmark", incomplete
sql/sql_select.cc:
MWL#17: Table elimination
- Move eliminate_tables() to before constant table detection, this will allow
to spare const table reads (at a cost of not being able to take advantage of
tables that are constant because they have no records, but this case is of
lesser importance)
crashes server!
The problem affects the scenario when index merge is followed by a filesort
and the sort buffer is not big enough for all the sort keys.
In this case the filesort function will read the data to the end through the
index merge quick access method (and thus closing the cursor etc),
but will leave the pointer to the quick select method in place.
It will then create a temporary file to hold the results of the filesort and
will add it as a sort output file (in sort.io_cache).
Note that filesort will copy the original 'sort' structure in an automatic
variable and restore it after it's done.
As a result at exiting filesort() we have a sort.io_cache filled in and
nothing else (as a result of close of the cursors at end of reading data
through index merge).
Now create_sort_index() will note that there is a select and will clean it up
(as it's been used already by filesort() reading the data in). While doing that
a special case in the index merge destructor will clean up the sort.io_cache,
assuming it's an output of the index merge method and is not needed anymore.
As a result the code that tries to read the data back from the filesort output
will get no data in both memory and disk and will crash.
Fixed similarly to how filesort() does it : by copying the sort.io_cache structure
to a local variable, removing the pointer to the io_cache (so that it's not freed
by QUICK_INDEX_MERGE_SELECT::~QUICK_INDEX_MERGE_SELECT) and restoring the original
structure (together with the valid pointer) after the cleanup is done.
This is a safe thing to do because all the structures are already cleaned up by
hitting the end of the index merge's read method (QUICK_INDEX_MERGE_SELECT::get_next())
and the cleanup code being written in a way that tolerates repeating cleanups.
mysql-test/r/index_merge.result:
Bug #44810: test case
mysql-test/t/index_merge.test:
Bug #44810: test case
sql/sql_select.cc:
Bug #44810: preserve the io_cache produced by filesort while cleaning up
the index merge quick access method (QUICK_INDEX_MERGE_SELECT).
- Fix print_join() to work both for EXPLAIN EXTENDED (after table elimination) and for
CREATE VIEW (after join->prepare() but without any optimization).
mysql-test/r/union.result:
MWL#17: Table elimination
- Adjust test results
- Fix the previous cset: take into account that select_lex may be printed when
1. There is no select_lex->join at all (in that case, assume that no tables were eliminated)
2. select_lex->join exists but there was no JOIN::optimize() call yet. handle this by initializing join->eliminated really early.
with gcc 4.3.2
Compiling MySQL with gcc 4.3.2 and later produces a number of
warnings, many of which are new with the recent compiler
versions.
This bug will be resolved in more than one patch to limit the
size of changesets. This is the second patch, fixing more
of the warnings.
uninitialized variable used as subscript
Grouping select from a "constant" InnoDB table (a table
of a single row) joined with other tables caused a crash.
mysql-test/r/innodb_mysql.result:
Added test case for bug bug #44886.
mysql-test/t/innodb_mysql.test:
Added test case for bug bug #44886.
sql/sql_select.cc:
Bug #44886: SIGSEGV in test_if_skip_sort_order() -
uninitialized variable used as subscript
1. The test_if_order_by_key function returned unitialized
used_key_parts parameter in case of a "constant" InnoDB
table. Calling function uses this parameter values as
an array index, thus sometimes it caused a crash.
The test_if_order_by_key function has been modified
to set used_key_parts to 0 (no need for ordering).
2. The test_if_skip_sort_order function has been
modified to accept zero used_key_parts value and
to prevent an array access by negative index.
Holding on to the temporary inno hash index latch is an optimization in
many cases, but a pessimization in some others.
Release temporary latches for those corner cases we (or rather, or customers,
thanks!) have identified, that is, when we are about to do something that
might take a really long time, like REPAIR or filesort.
sql/ha_myisam.cc:
Let go of (inno, for now) latch when doing MyISAM-repair.
(optimize passes through repair.) ("Stuck" in "Repair with
keycache".)
sql/sql_insert.cc:
Let go of (inno, for now) latch when doing CREATE...SELECT
in select_insert::send_data() -- it might take a while.
("stuck" in "Sending data")
sql/sql_select.cc:
Release temporary (inno, for now) latch on
- free_tmp_table() (this can take surprisingly long, "removing tmp table")
- create_myisam_from_heap() (HEAP table overflowing onto disk as MyISAM,
"converting HEAP to MyISAM")
- First code. Elimination works for simple cases, passes the testsuite.
- Known issues:
= No elimination is done for aggregate functions.
= EXPLAIN EXTENDED shows eliminated tables (I think it better not)
= No benchmark yet
= The code needs some polishing.
mysql-test/r/table_elim.result:
MWL#17: Table elimination
- Testcases
mysql-test/t/table_elim.test:
MWL#17: Table elimination
- Testcases
sql/sql_select.cc:
MWL#17: Table elimination
sql/sql_select.h:
MWL#17: Table elimination
- Added JOIN_TAB::eliminated (is JOIN_TAB the best place to store this flag?)
sql/table.h:
MWL#17: Table elimination
- ADded NESTED_JOIN::n_tables. We need to have the number of real tables remaining in an outer join nest.
HAVING
When calculating GROUP BY the server caches some expressions. It does
that by allocating a string slot (Item_copy_string) and assigning the
value of the expression to it. This effectively means that the result
type of the expression can be changed from whatever it was to a string.
As this substitution takes place after the compile-time result type
calculation for IN but before the run-time type calculations,
it causes the type calculations in the IN function done at run time
to get unexpected results different from what was prepared at compile time.
In the CASE ... WHEN ... THEN ... statement there was a similar problem
and it was solved by artificially adding a STRING argument to the set of
types of the IN/CASE arguments at compile time, so if any of the
arguments of the CASE function changes its type to a string it will
still be covered by the information prepared at compile time.
mysql-test/include/mix1.inc:
Bug #44399: extended the test to cover the different types
mysql-test/r/func_in.result:
Bug #44399: test case
mysql-test/r/innodb_mysql.result:
Bug #44399: extended the test to cover the different types
mysql-test/t/func_in.test:
Bug #44399: test case
sql/item.cc:
Bug #44399: Implement typed caching for GROUP BY
sql/item.h:
Bug #44399: Implement typed caching for GROUP BY
sql/item_cmpfunc.cc:
Bug #44399: remove the special case
sql/sql_select.cc:
Bug #44399: Implement typed caching for GROUP BY
mysql-test/r/information_schema.result:
Fixed a result file.
mysql-test/r/innodb-autoinc.result:
Fixed a result file.
mysql-test/t/connect.test:
Fixed a problem with merge, needed to close
connections and use the default
SQL_SELECT::test_quick_select
The crash was caused by an incomplete cleanup of JOIN_TAB::select
during the filesort of rows for GROUP BY clause inside a subquery.
Queries where a quick index access is replaced with filesort was
was affected. For example:
SELECT 1 FROM
(SELECT COUNT(DISTINCT c1) FROM t1
WHERE c2 IN (1, 1) AND c3 = 2 GROUP BY c2) x
Quick index access related data in the SQL_SELECT::test_quick_select
function was inconsistent after an incomplete cleanup.
This function has been completed to prevent crashes in the
SQL_SELECT::test_quick_select function.
mysql-test/include/mix1.inc:
Add test case for bug #44290.
mysql-test/r/innodb_mysql.result:
Add test case for bug #44290.
sql/sql_select.cc:
Bug #44290: explain crashes for subquery with distinct in
SQL_SELECT::test_quick_select
Quick index access related data in the SQL_SELECT::test_quick_select
function was inconsistent after an incomplete cleanup.
This function has been completed to prevent crashes in the
SQL_SELECT::test_quick_select function.
'INSERT ... SELECT' statements
The code that produces result rows expected that a duplicate row
error could not occur in INSERT ... SELECT statements with
unfulfilled WHERE conditions. This may happen, however, if the
SELECT list contains only aggregate functions.
Fixed by checking if an error occured before trying to send EOF
to the client.
mysql-test/r/insert_select.result:
Bug#44306: Test result
mysql-test/t/insert_select.test:
Bug#44306: Test case
sql/sql_select.cc:
Bug#44306: Fix
EXPLAIN EXTENDED of nested query containing a error:
1054 Unknown column '...' in 'field list'
may cause a server crash.
Parse error like described above forces a call to
JOIN::destroy() on malformed subquery.
That JOIN::destroy function closes and frees temporary
tables. However, temporary fields of these tables
may be listed in st_select_lex::group_list of outer
query, and that st_select_lex may not cleanup them
properly. So, after the JOIN::destroy call that
st_select_lex::group_list may have Item_field
objects with dangling pointers to freed temporary
table Field objects. That caused a crash.
mysql-test/r/subselect3.result:
Added test case for bug #37362.
mysql-test/t/subselect3.test:
Added test case for bug #37362.
sql/sql_select.cc:
Bug #37362: Crash in do_field_eq
The JOIN::destroy function has been modified to
cleanup temporary table column items.
