Both these two bugs happened due to the following problem.
When a view column is referenced in the query an Item_direct_view_ref
object is created that is refers to the Item_field for the column.
All references to the same view column refer to the same Item_field.
Different references can belong to different AND/OR levels and,
as a result, can be included in different Item_equal object.
These Item_equal objects may include different constant objects.
If these constant objects are substituted for the Item_field created
for a view column we have a conflict situation when the second
substitution annuls the first substitution. This leads to
wrong result sets returned by the query. Bug #724942 demonstrates
such an erroneous behaviour.
Test case of the bug #717577 produces wrong result sets because best
equal fields of the multiple equalities built for different OR levels
of the WHERE condition differs. The subsitution for the best equal field
in the second OR branch overwrites the the substitution made for the
first branch.
To avoid such conflicts we have to substitute for the references
to the view columns rather than for the underlying field items.
To make such substitutions possible we have to include into
multiple equalities references to view columns rather than
field items created for such columns.
This patch modifies the Item_equal class to include references
to view columns into multiple equality objects. It also performs
a clean up of the class methods and adds more comments. The methods
of the Item_direct_view_ref class that assist substitutions for
references to view columns has been also added by this patch.
Analysis:
The wrong result is a consquence of sorting the subquery
result and then selecting only the first row due to the
artificial LIMIT 1 introduced by the fix_fields phase.
Normally, if there is an ORDER BY in a subquery, the ORDER
is removed (Item_in_subselect::select_in_like_transformer),
however if a GROUP BY is transformed into ORDER, this happens
later, after the removal of the ORDER clause of subqueries, so
we end up with a subquery with an ORDER clause, and an artificially
added LIMIT 1.
The reason why the same works in the main 5.3 without MWL#89, is
that the 5.3 performs all subquery transformations, including
IN->EXISTS before JOIN::optimize(). The beginning of JOIN::optimize
does:
if (having || (select_options & OPTION_FOUND_ROWS))
select_limit= HA_POS_ERROR;
which sets the limit back to infinity, thus 5.3 sorts the whole
subquery result, and IN performs the lookup into all subquery result
rows.
Solution:
Sorting of subqueries without LIMIT is meaningless. Since LIMIT in
subqueries is not supported, the patch removes sorting by setting
join->skip_sort_order= true
for each subquery JOIN object. This improves a number of execution
plans to not perform unnecessary sorting at all.
Before sorting HAVING condition is split into two parts,
first part is a table related condition and the rest of is
HAVING part. Extraction of HAVING part does not take into account
the fact that some of conditions might be non-const but
have 'used_tables' == 0 (independent subqueries)
and because of that these conditions are cut off by
make_cond_for_table() function.
The fix is to use (table_map) 0 instead of used_tables in
third argument for make_cond_for_table() function.
It allows to extract elements which belong to sorted
table and in addition elements which are independend
subqueries.
mysql-test/r/having.result:
test case
mysql-test/t/having.test:
test case
sql/sql_select.cc:
The fix is to use (table_map) 0 instead of used_tables in
third argument for make_cond_for_table() function.
It allows to extract elements which belong to sorted
table and in addition elements which are independend
subqueries.
Valgrind warnings were caused by comparing index values to an un-initialized field.
mysql-test/r/subselect.result:
New test cases.
mysql-test/t/subselect.test:
New test cases.
sql/opt_sum.cc:
Add thd to opt_sum_query enabling it to test for errors.
If we have a non-nullable index, we cannot use it to match null values,
since set_null() will be ignored, and we might compare uninitialized data.
sql/sql_select.cc:
Add thd to opt_sum_query, enabling it to test for errors.
sql/sql_select.h:
Add thd to opt_sum_query, enabling it to test for errors.
There are two problems with ANALYSE():
1. Memory leak
it happens because do_select() can overwrite
JOIN::procedure field(with zero value in our case) and
JOIN destructor don't free the memory allocated for
JOIN::procedure. The fix is to save original JOIN::procedure
before do_select() call and restore it after do_select
execution.
2. Wrong result
If ANALYSE() procedure is used for the statement with LIMIT clause
it could retrun empty result set. It happens because of missing
analyse::end_of_records() call. First end_send() function call
returns NESTED_LOOP_QUERY_LIMIT and second call of end_send() with
end_of_records flag enabled does not happen. The fix is to return
NESTED_LOOP_OK from end_send() if procedure is active.
mysql-test/r/analyse.result:
test case
mysql-test/t/analyse.test:
test case
sql/sql_select.cc:
--save original JOIN::procedure before do_select() call and
restore it after do_select execution.
--return NESTED_LOOP_OK from end_send() if procedure is active
- "Using MRR" is no longer shown with range access.
- Instead, both range and BKA accesses will show one of the following:
= "Rowid-ordered scan"
= "Key-ordered scan"
= "Key-ordered Rowid-ordered scan"
depending on whether DS-MRR implementation will do scan keys in order, rowids in order,
or both.
- The patch also introduces a way for other storage engines/MRR implementations to
pass information to EXPLAIN output about the properties of employed MRR scans.
- Auto-merge with 5.3 main.
- Changed the test for LP BUG#719198 so that
an two more queries were added, and removed a
query that produces a wrong result due to an
unrelated problem. The wrong result is submitted
as a separate bug.
Fixed memory leak from HEAP tables that was not deleted properly
BUILD/compile-alpha-ccc:
Use g++ instead of gcc for linking
BUILD/compile-alpha-debug:
Use g++ instead of gcc for linking
BUILD/compile-pentium-pgcc:
Use g++ instead of gcc for linking
BUILD/compile-solaris-sparc:
Use g++ instead of gcc for linking
BUILD/compile-solaris-sparc-debug:
Use g++ instead of gcc for linking
BUILD/compile-solaris-sparc-purify:
Use g++ instead of gcc for linking
sql/item.cc:
Safety fixes for expr_cache
Call Item_result:field::cleanup() in Item_cache_wrapper::cleanup()
More DBUG_PRINT
sql/sql_base.cc:
Simple optimization for setup_wild
More DBUG_PRINT
sql/sql_expression_cache.cc:
Added header
Removed not needed initialization
sql/sql_lex.cc:
More DBUG_PRINT
sql/sql_select.cc:
More DBUG_PRINT
Fixed memory leak from HEAP tables that was not deleted properly
storage/heap/hp_create.c:
More DBUG_PRINT
Analysis:
There are two code paths through which JOIN::exec may produce
an all-NULL row for an empty result set. One goes via the
function return_zero_rows(), when query processing detectes
early that the where clause is false, the other one is via
do_select() in the case of join execution.
In the case of do_select(), the problem was that the executioner
didn't set TABLE::null_row to 1. As result when sending the only
result row, the evaluation of each field didn't detect that all
non-aggregated fields are NULL, because Field::is_null returned
true, after checking that field->table->null_row was false.
Given that the each non-aggregated field was not considered NULL,
select_result::send_data sent whatever was in the buffer of each
field. However, since there was no actual data in the field buffer,
send_data() accessed and sent whatever junk was in the field's
data buffer.
Solution:
Similar to the analogous case in return_zero_rows() mark all
tables that their current row is NULL before sending the
artificailly created NULL row.
Analysis:
A query with implicit grouping is one with aggregate functions and
no GROUP BY clause. MariaDB inherits from MySQL an SQL extenstion
that allows mixing aggregate functions with non-aggregate fields.
If a query with such mixed select clause produces an empty result
set, the meaning of aggregate functions is well defined - either
NULL (MIN, MAX, etc.), or 0 (count(*)). However the non-aggregated
fields must also have some value, and the only reasonable value in
the case of empty result is NULL.
The cause of the many wrong results was that if a field is declared
as non-nullable (e.g. because it is a PK or NOT NULL), the semantic
analysis and the optimization phases treat this field as non-nullable,
and generate all related query plan elements based on this assumption.
Later during execution, these incorrectly configured/generated query
plan elements result in a wrong result because the selected fields
are not null due to the not-null assumption during optimization.
Solution:
Detect before the context analysys phase that a query uses implicit
grouping with mixed aggregates/non-aggregates, and set all fields
as nullable. The parser already walks the SELECT clause, and
already sets Item::with_sum_func for Items that reference aggreagate
functions. The patch adds a symmetric Item::with_field so that all
Items that reference an Item_field are marked during their
construction at parse time in the same way as with aggregate function
use.
mysql-test/r/union.result:
Added test for lp:732124
mysql-test/t/union.test:
Added test for lp:732124
sql/sp_rcontext.cc:
Updated function definition for ::send_data()
sql/sp_rcontext.h:
Updated function definition for ::send_data()
sql/sql_analyse.cc:
Test if send_data() returned an error
sql/sql_class.cc:
Updated function definition for ::send_data()
sql/sql_class.h:
Changed select_result::send_data(List<Item> &items) to return -1 in case of duplicate row that should not be counted as part of LIMIT
sql/sql_cursor.cc:
Check if send_data returned error
sql/sql_delete.cc:
Updated function definition for ::send_data()
sql/sql_insert.cc:
Updated function definition for ::send_data()
sql/sql_select.cc:
Don't count rows which send_data() tells you to ignore
sql/sql_union.cc:
Inform caller that the row should be ignored. This is the real bug fix for lp:732124
sql/sql_update.cc:
Updated function definition for ::send_data()
This allows us to simplify and speed up some tests and also remove get_cached_item()
sql/item.h:
Added item.real_type()
Removed get_cached_item()
sql/opt_range.cc:
Simplify test
sql/sql_select.cc:
Simplify test
sql/sql_show.cc:
Simplify test
If join condition is of the form <t2.key>=<t1.no_key> then the server
performs no index look-ups when looking for matching rows of t2 for
the rows from t1 with t1.no_key=NULL. It happens because the function
add_not_null_conds() injects an additional condition of the form
IS NOT NULL(<t1.no_key>) into the WHERE condition.
However if the join condition was of the form <t.key>=<outer_ref> no
additional null rejecting predicate was generated. This could lead
to extra records in the result set if the value of <outer_ref> happened
to be NULL.
The new code injects null rejecting predicates of the form
IS NOT NULL(<outer_ref>) and evaluates them before the first row
the subquery is constructed.
