in combination with IS NULL'
As this bug is a duplicate of bug#49322, it also includes test cases
covering this bugreport
Qualifying an OUTER JOIN with the condition 'WHERE <column> IS NULL',
where <column> is declared as 'NOT NULL' causes the
'not_exists_optimize' to be enabled by the optimizer.
In evaluate_join_record() the 'not_exists_optimize' caused
'NESTED_LOOP_NO_MORE_ROWS' to be returned immediately
when a matching row was found.
However, as the 'not_exists_optimize' is derived from
'JOIN_TAB::select_cond', the usual rules for condition guards
also applies for 'not_exist_optimize'. It is therefore incorrect
to check 'not_exists_optimize' without ensuring that all guards
protecting it is 'open'.
This fix uses the fact that 'not_exists_optimize' is derived from
a 'is_null' predicate term in 'tab->select_cond'. Furthermore,
'is_null' will evaluate to 'false' for any 'non-null' rows
once all guards protecting the is_null is open.
We can use this knowledge as an implicit guard check for the
'not_exists_optimize' by moving 'if (...not_exists_optimize)'
inside the handling of 'select_cond==false'. It will then
not take effect before its guards are open.
We also add an assert which requires that a
'not_exists_optimize' always comes together with
a select_cond. (containing 'is_null').
Item_equal::val_int() checked for NULL-values by checking Item::null_value
*before* the respective ::store_value() and ::cmp(Item*) metods where called.
As Item::null_value is set by these metods, the value of 'null_value'
is not valid until *after* ::store_value() or ::cmp() has
been called for the Item object.
Fix is to swap order of ::store_value()/::cmp() and checking of Item::null_value.
This pattern is widely used other places inside item_cmpfunc.cc .
Lines below which were added in the patch for Bug#56814 cause this crash:
+ if (table->table)
+ table->table->maybe_null= FALSE;
Consider following test case:
--
CREATE TABLE t1(f1 INT NOT NULL);
INSERT INTO t1 VALUES (16777214),(0);
SELECT COUNT(*) FROM t1 LEFT JOIN t1 t2
ON 1 WHERE t2.f1 > 1 GROUP BY t2.f1;
DROP TABLE t1;
--
We set TABLE::maybe_null to FALSE for t2 table
and in create_tmp_field() we create appropriate tmp table field
using create_tmp_field_from_item() function instead of
create_tmp_field_from_field. As a result we have
LONGLONG field. As we have GROUP BY clause we calculate
group buffer length, see calc_group_buffer().
Item from group list which is used for calculation
refer to the field from real tables and have LONG type.
So group buffer length become insufficient for storing of
LONGLONG value. It leads to overwriting of wrong memory
area in do_field_int() function which is called from
end_update().
After some investigation I found out that
create_tmp_field_from_item() is used only for OLAP
grouping and can not be used for common grouping
as it could be an incompatibility between tmp
table fields and group buffer length.
We can not remove create_tmp_field_from_item() call from
create_tmp_field as OLAP needs it and we can not use this
function for common grouping. So we should remove setting
TABLE::maybe_null to FALSE from simplify_joins().
In this case we'll get wrong behaviour of
list_contains_unique_index() back. To fix it we
could use Field::real_maybe_null() check instead of
Field::maybe_null() and add addition check of
TABLE_LIST::outer_join.
Before this fix, the ha_read_last_count status variable was defined and
updated internally, for never exposed as a system variable.
This fix exposes the system variable as "Handler_read_last",
for completness of the Handler_read_* system variables interface.
Adjusted tests results accordingly.
There are two problems:
1. In simplify_joins function we calculate table dependencies. If STRAIGHT_JOIN hint
is used for whole SELECT we do not count it and as result some dependendecies
might be lost. It leads to incorrect table order which is returned by
join_tab_cmp_straight() function.
2. make_join_statistics() calculate the transitive closure for relations a particular
JOIN_TAB is 'dependent on'.
We aggregate the dependent table_map of a JOIN_TAB by adding dependencies from other
tables which we depend on. However, this may also cause new dependencies to be
available after we have completed processing a certain JOIN_TAB.
Both these problems affect condition pushdown and as result condition might be pushed
into wrong table which leads to crash or even omitted which leads to wrong result.
The fix:
1. Use modified 'transitive closure' algorithm provided by Ole John Aske
2. Update table dependences in simplify_joins according to
global STRAIGHT_JOIN hint.
Note: the patch also fixes bugs 46091 & 51492
Conflicts:
Text conflict in mysql-test/r/grant.result
Text conflict in mysql-test/t/grant.test
Text conflict in mysys/mf_loadpath.c
Text conflict in sql/slave.cc
Text conflict in sql/sql_priv.h
greedy_search optimizer_search_depth=0
The algorithm inside restore_prev_nj_state failed to
properly update the counters within the NESTED_JOIN
tree. The counter was decremented each time a table in the
node was removed from the QEP, the correct thing to do being
only to decrement it when the last table in the child node
was removed from the plan. This lead to node counters
getting negative values and the plan thus appeared
impossible. An assertion caught this.
Fixed by not recursing up the tree unless the last table in
the join nest node is removed from the plan
Conflicts:
Text conflict in mysql-test/r/partition_innodb.result
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/item_sum.h
Text conflict in sql/log_event_old.cc
Text conflict in sql/protocol.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_yacc.yy
Optimizer erroneously translated LEFT JOIN into INNER JOIN.
It leads to cutting rows with NULL right side. It happens
because Item_row uses not_null_tables() method form the
base(Item) class and does not calculate 'null tables'
properly. The fix is adding calculation of 'not null tables'
to Item_row.
returns incorrect results with where
An outer join of a const table (outer) and a normal table
(inner) with GROUP BY on a field from the outer table would
optimize away GROUP BY, and thus trigger the optimization to
do away with a temporary table if grouping was performed on
columns from the const table, hence executing the query with
filesort without temporary table. But this should not be
done if there is a non-indexed access to the inner table,
since filesort does not handle joins. It expects either ref
access, range ditto or table scan. The join condition will
thus not be applied.
Fixed by always forcing execution with temporary table in
the case of ROLLUP with a query involving an outer join. This
is a slightly broader class of queries than need fixing, but
it is hard to ascertain the position of a ROLLUP field wrt
outer join with current query representation.
constant outer tables did not return null complemented
rows when conditions were evaluated to FALSE.
Wrong results were returned because the conditions over constant
outer tables, when being pushed down, were erroneously enclosed
into the guard function used for WHERE conditions.
'not exists' optimization is applied.
In fact 'not exists' optimization did not work anymore after the patch
introducing the evaluate_join_record function had been applied.
Corrected the evaluate_join_record function to respect the 'not_exists'
optimization.
Objects of the class Item_equal contain an auxiliary member
eval_item of the type cmp_item that is used only for direct
evaluation of multiple equalities. Currently a multiple equality
is evaluated directly only in the cases when the equality holds
at most for one row in the result set.
The compare collation of eval_item was determined incorectly.
It could lead to returning incorrect results for some queries.
account by the optimizer.
Now all row equalities are converted into conjunctions of
equalities between row elements. They are taken into account
by the optimizer together with the original regular equality
predicates.
table in a join
The optimizer removes redundant columns in ORDER BY. It is considering
redundant every reference to const table column, e.g b in :
create table t1 (a int, b int, primary key(a));
select 1 from t1 order by b where a = 1
But it must not remove references to const table columns if the
const table is an outer table because there still can be 2 values :
the const value and NULL. e.g.:
create table t1 (a int, b int, primary key(a));
select t2.b c from t1 left join t1 t2 on (t1.a = t2.a and t2.a = 5)
order by c;
When a default of '' was specified for TEXT/BLOB columns, the specification
was silently ignored. This is presumably to be nice to applications (or
people) who generate their column definitions in a not-very-clever fashion.
For clarity, doing this now results in a warning, or an error in strict
mode.
This bug was introduced when the patch resolving the
performance problem 17164 was applied. As a result
of that modification the not_null_tables attributes
were calculated incorrectly for constant OR conditions.
This triggered invalid conversion of outer joins into
inner joins.
over two views when using syntax with curly braces.
Each outer join operation must be placed in a separate
nest. This was not done when the syntax with curly braces
was used. In some cases, in particular, for queries with outer
join operation over views it could cause a crash.
If the WHERE condition of a query contained an ORed FALSE term
then the set of tables whose rows cannot serve for null complements
in outer joins was determined incorrectly. It resulted in blocking
possible conversions of outer joins into joins for such queries.
Post merge changes
sql_yacc.yy:
Post merge changes
sql_select.cc:
Fixed bugs #12101, #12102: wrong calculation of not_null_tables()
for some expressions (post merge changes).
The function add_key_fields() is modified. There cannot be NOT before
BETWEEN/IN anymore. Rather Item_func_between/in objects can represent
now [NOT]BETWEEN/IN expressions.
Fixed bugs #12101, #12102: wrong calculation of not_null_tables()
for some expressions.
The classes Item_func_between, Item_func_if, Item_func_in are modified.
Item_func_between/in objects can represent now [NOT]BETWEEN/IN expressions.
The class Item_func_opt_neg is added to factor out the functionality
common for the modified classes Item_func_between and Item_func_in.
item_cmpfunc.cc:
Fixed bugs #12101, #12102: wrong calculation of not_null_tables()
for some expressions.
Added Item_func_between::fix_fields(), Item_func_if::fix_fields(),
Item_func_in::fix_fields(). They correct generic calculation of
the not_null_tables attribute when it is needed.
Modified Item_func_between::val_int(), Item_func_in::val_int().
opt_range.cc:
Fixed bugs #12101, #12102: wrong calculation of not_null_tables()
for some expressions.
The function get_mm_tree() is modified. There cannot be NOT before
BETWEEN/IN anymore. Rather Item_func_between/in objects can represent
now [NOT]BETWEEN/IN expressions.
sql_yacc.yy:
Fixed bugs #12101, #12102: wrong calculation of not_null_tables()
for some expressions.
Item_func_between/in objects can represent now [NOT]BETWEEN/IN expresions.
join_outer.result:
Fixed some testcases results (bugs #12101, #12102)
join_outer.test:
Added testcases for bugs #12101, #12102
* Provide backwards compatibility extension to name resolution of
coalesced columns. The patch allows such columns to be qualified
with a table (and db) name, as it is in 4.1.
Based on a patch from Monty.
* Adjusted tests accordingly to test both backwards compatible name
resolution of qualified columns, and ANSI-style resolution of
non-qualified columns.
For this, each affected test has two versions - one with qualified
columns, and one without.
"Process NATURAL and USING joins according to SQL:2003".
* Some of the main problems fixed by the patch:
- in "select *" queries the * expanded correctly according to
ANSI for arbitrary natural/using joins
- natural/using joins are correctly transformed into JOIN ... ON
for any number/nesting of the joins.
- column references are correctly resolved against natural joins
of any nesting and combined with arbitrary other joins.
* This patch also contains a fix for name resolution of items
inside the ON condition of JOIN ... ON - in this case items must
be resolved only against the JOIN operands. To support such
'local' name resolution, the patch introduces a stack of
name resolution contexts used at parse time.
NOTICE:
- This patch is not complete in the sense that
- there are 2 test cases that still do not pass -
one in join.test, one in select.test. Both are marked
with a comment "TODO: WL#2486".
- it does not include a new test specific for the task
Added a test case for bug #11285.
sql_select.cc:
Fixed bug #11285.
The problem occurred with Item_equal in an 'on expression'
that was evaluated to false.
