Main fix was to not cache derivied tables as they may be temporary tables that are deleted before the next query.
This was a bit tricky as Item_field::fix_fields depended on cached_tables to be set to resolve some columns.
mysql-test/r/sp-bugs.result:
Added test case
mysql-test/t/sp-bugs.test:
Added test case
sql/item.cc:
Fixed fix_outer_field to handle case where found field did not have in cached_table
Idea is that if cached_table is not avaliable, use from_field->table->pos_in_table_list instead
sql/records.cc:
Also accept INTERNAL_TMP_TABLE for memmap
sql/sql_base.cc:
More DBUG_PRINT
Fixed that setup_natural_join_row_types() is not run twice.
Original code modified context->first_name_resolution_table also for second executions.
This was wrong as this could give wrong results if some joins had been optimized away between calls.
sql/sql_derived.cc:
Mark derived tables as internal temporary tables (INTERNAL_TMP_TABLE), not as NON_TRANSACTIONAL_TMP_TABLE.
This is more correct as the tables are not visible by the end user.
sql/sql_insert.cc:
Reset pos_in_table_list before calling fix_fields.
One of the consequences of the change of not caching all generated tables in Item_ident is that
pos_in_table_list needs to be correct in calls to fix_fields.
sql/sql_lex.cc:
More DBUG_PRINT
sql/sql_parse.cc:
Don't cache derivied tables as they may be temporary tables that are deleted before the next query
sql/sql_select.cc:
Reset table_vector. This was required as some code checked the vector to see if temporary tables had already been created.
sql/table.cc:
Mark tables with field translations as cacheable (as these will not disapper between stmt executions.
THD::thd->activate_stmt_arena_if_needed() should be used to temporary activating statement arena instead of direct usage of THD::set_n_backup_active_arena() because possible such scenario:
1) func1 saves current arena and activates copy1 of statement arena
2) func2 saves copy1 of statement arena setup by func1 and activates copy2
3) some changes made for copy 2
4) func2 stores changed copy2 back to statenet arena and activates copy1
5) func1 store unchanged copy1 back to statemnt arena (rewrite changed copy 2 so changes become lost) and activates arena which was before.
Materialization forced in case if rand() used in view or derived table to avoud several calls of rand for gting value of a field.
Fixed set variable uncachable flag from - it shouldbe a side effect not a random value.
The bug caused a memory overwrite in the function update_ref_and_keys()
It happened due to a wrong value of SELECT_LEX::cond_count. This value
historically was calculated by the fix_fields method. Now the logic of
calling this method became too complicated and, as a result, this value
is calculated not always correctly.
The patch changes the way how and when the values of SELECT_LEX::cond_count
and of SELECT_LEX::between_count are calculated. The new code does it just at
the beginning of update_ref_and_keys().
Other fix of maybe_null problem and revert of revno: 3608 "MDEV-3873 & MDEV-3876 & MDEV-3912 : Wrong result (extra rows) with ALL subquery from a MERGE view."
mysql_derived_merge_for_insert() should not be called for views or derived tables which are not put (directly or via other views) in main SELECT_LEX "join list".
-Added test and extra code to ensure we don't leave keyread on for a handler table.
-Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
mysql-test/include/default_mysqld.cnf:
Run test suite with smaller aria keybuffer size
mysql-test/suite/maria/maria3.result:
Run test suite with smaller aria keybuffer size
mysql-test/suite/sys_vars/r/aria_pagecache_buffer_size_basic.result:
Run test suite with smaller aria keybuffer size
sql/handler.cc:
Disable key read (extra safety if something went wrong)
sql/multi_range_read.cc:
Ensure we have don't leave keyread on for secondary_file
sql/opt_range.cc:
Simplify code with mark_columns_used_by_index_no_reset()
Ensure that read_keys_and_merge() disableds keyread if it enables it
sql/opt_subselect.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_derived.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_select.cc:
Use 'enable_keyread()' instead of calling HA_EXTRA_RESET. (Makes debugging easier)
Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
Remove not anymore used argument for create_internal_tmp_table()
More DBUG
sql/sql_select.h:
Remove not anymore used argument for create_internal_tmp_table()
Do not call, directly or indirectly, SQL_SELECT::test_quick_select()
for derived materialized tables / views when optimizing joins referring
to these tables / views to get cost estimates of materialization.
The current code does not create B-tree indexes for materialized
derived tables / views. So now it's not possible to get any estimates
for ranges conditions over the results of the materialization.
The function mysql_derived_create() must take into account the fact
that array of the KEY structures specifying the keys over a derived
table / view may be moved after the optimization phase if the
derived table / view is materialized.
If the expression for a derived table of a query contained a LIMIT
clause the estimate of the number of rows in this derived table
returned by the EXPLAIN command could be badly off since the
optimizer ignored the limit number from the LIMIT clause when
getting the estimate.
The call of the method SELECT_LEX_UNIT->set_limit added in the code
of mysql_derived_optimize() will be needed also in maria-5.5 where
parameters in the LIMIT clause are supported.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
Also:
1. simplified the code of the function mysql_derived_merge_for_insert.
2. moved merge of views/dt for multi-update/delete to the prepare stage.
3. the list of the references to the candidates for semi-join now is
allocated in the statement memory.
If a view/derived table is non-mergeable then the definition of the tmp table
to store the rows for it is created at the prepare stage. In this case if the
view definition uses outer joins and a view column belongs to an inner table
of one of them then the column should be considered as nullable independently
on nullability of the underlying column. If the underlying column happens to be
defined as non-nullable then the function create_tmp_field_from_item rather
than the function create_tmp_field_from_field should be employed to create
the definition of the interesting column in the tmp table.
Changed the code that processing of multi-updates and multi-deletes
with multitable views at the prepare stage.
A proper solution would be: never to perform any transformations of views
before and at the prepare stage. Yet it would require re-engineering
of the code that checks privileges and updatability of views.
Ultimately this re-engineering has to be done to provide a clean solution
for INSERT/UPDATE/DELETE statements that use views.
Fixed a valgrind problem in the function TABLE::use_index.
INSERT/UPDATE/DELETE statement that used a temptable view v1 could lead to
a crash if v1 was defined as a select from a mergeable view v2 that selected
rows from a temptable view v3.
When INSERT/UPDATE/DELETE uses a view that is not updatable then field
translation for the view should be created before the prepare phase.
When looking for the execution plan of a derived table to be materialized
JOIN::optimize finds out that all joined tables of the derived table
contain not more than one row then the derived table should be maretialized
at the optimization stage.
Added a test case for the bug.
Adjusted results in other test cases.
Resolved all conflicts, bad merges and fixed a few minor bugs in the code.
Commented out the queries from multi_update, view, subselect_sj, func_str,
derived_view, view_grant that failed either with crashes in ps-protocol or
with wrong results.
The failures are clear indications of some bugs in the code and these bugs
are to be fixed.
temptable views
The TABLE::key_read field indicates if the optimizer has found that row
retrieval only should access the index tree. The triggered assert
inside close_thread_table() checks that this field has been reset when
the table is about to be closed.
During normal execution, these fields are reset right before tables are
closed at the end of mysql_execute_command(). But in the case of errors,
tables are closed earlier. The patch for Bug#52044 refactored the open
tables code so that close_thread_tables() is called immediately if
opening of tables fails. At this point in the execution, it could
happend that all TABLE::key_read fields had not been properly reset,
therefore triggering the assert.
The problematic statement in this case was EXPLAIN where the query
accessed two derived tables and where the first derived table was
processed successfully while the second derived table was not.
Since it was an EXPLAIN, TABLE::key_read fields were not reset after
successful derived table processing since the state needs to be
accessible afterwards. When processing of the second derived table
failed, it's corresponding SELECT_LEX_UNIT was cleaned, which caused
it's TABLE::key_read fields to be reset. Since processing failed,
the error path of open_and_lock_tables() was entered and
close_thread_tables() was called. The assert was then triggered due
to the TABLE::key_read fields set during processing of the first
derived table.
This patch fixes the problem by adding a new derived table processor,
mysql_derived_cleanup() that is called after mysql_derived_filling().
It causes cleanup of all SELECT_LEX_UNITs to be called, resetting
all relevant TABLE::key_read fields.
Test case added to derived.test.
--Bug#52157 various crashes and assertions with multi-table update, stored function
--Bug#54475 improper error handling causes cascading crashing failures in innodb/ndb
--Bug#57703 create view cause Assertion failed: 0, file .\item_subselect.cc, line 846
--Bug#57352 valgrind warnings when creating view
--Recently discovered problem when a nested materialized derived table is used
before being populated and it leads to incorrect result
We have several modes when we should disable subquery evaluation.
The reasons for disabling are different. It could be
uselessness of the evaluation as in case of 'CREATE VIEW'
or 'PREPARE stmt', or we should disable subquery evaluation
if tables are not locked yet as it happens in bug#54475, or
too early evaluation of subqueries can lead to wrong result
as it happened in Bug#19077.
Main problem is that if subquery items are treated as const
they are evaluated in ::fix_fields(), ::fix_length_and_dec()
of the parental items as a lot of these methods have
Item::val_...() calls inside.
We have to make subqueries non-const to prevent unnecessary
subquery evaluation. At the moment we have different methods
for this. Here is a list of these modes:
1. PREPARE stmt;
We use UNCACHEABLE_PREPARE flag.
It is set during parsing in sql_parse.cc, mysql_new_select() for
each SELECT_LEX object and cleared at the end of PREPARE in
sql_prepare.cc, init_stmt_after_parse(). If this flag is set
subquery becomes non-const and evaluation does not happen.
2. CREATE|ALTER VIEW, SHOW CREATE VIEW, I_S tables which
process FRM files
We use LEX::view_prepare_mode field. We set it before
view preparation and check this flag in
::fix_fields(), ::fix_length_and_dec().
Some bugs are fixed using this approach,
some are not(Bug#57352, Bug#57703). The problem here is
that we have a lot of ::fix_fields(), ::fix_length_and_dec()
where we use Item::val_...() calls for const items.
3. Derived tables with subquery = wrong result(Bug19077)
The reason of this bug is too early subquery evaluation.
It was fixed by adding Item::with_subselect field
The check of this field in appropriate places prevents
const item evaluation if the item have subquery.
The fix for Bug19077 fixes only the problem with
convert_constant_item() function and does not cover
other places(::fix_fields(), ::fix_length_and_dec() again)
where subqueries could be evaluated.
Example:
CREATE TABLE t1 (i INT, j BIGINT);
INSERT INTO t1 VALUES (1, 2), (2, 2), (3, 2);
SELECT * FROM (SELECT MIN(i) FROM t1
WHERE j = SUBSTRING('12', (SELECT * FROM (SELECT MIN(j) FROM t1) t2))) t3;
DROP TABLE t1;
4. Derived tables with subquery where subquery
is evaluated before table locking(Bug#54475, Bug#52157)
Suggested solution is following:
-Introduce new field LEX::context_analysis_only with the following
possible flags:
#define CONTEXT_ANALYSIS_ONLY_PREPARE 1
#define CONTEXT_ANALYSIS_ONLY_VIEW 2
#define CONTEXT_ANALYSIS_ONLY_DERIVED 4
-Set/clean these flags when we perform
context analysis operation
-Item_subselect::const_item() returns
result depending on LEX::context_analysis_only.
If context_analysis_only is set then we return
FALSE that means that subquery is non-const.
As all subquery types are wrapped by Item_subselect
it allow as to make subquery non-const when
it's necessary.
mysql-test/r/derived.result:
test case
mysql-test/r/multi_update.result:
test case
mysql-test/r/view.result:
test case
mysql-test/suite/innodb/r/innodb_multi_update.result:
test case
mysql-test/suite/innodb/t/innodb_multi_update.test:
test case
mysql-test/suite/innodb_plugin/r/innodb_multi_update.result:
test case
mysql-test/suite/innodb_plugin/t/innodb_multi_update.test:
test case
mysql-test/t/derived.test:
test case
mysql-test/t/multi_update.test:
test case
mysql-test/t/view.test:
test case
sql/item.cc:
--removed unnecessary code
sql/item_cmpfunc.cc:
--removed unnecessary checks
--THD::is_context_analysis_only() is replaced with LEX::is_ps_or_view_context_analysis()
sql/item_func.cc:
--refactored context analysis checks
sql/item_row.cc:
--removed unnecessary checks
sql/item_subselect.cc:
--removed unnecessary code
--added DBUG_ASSERT into Item_subselect::exec()
which asserts that subquery execution can not happen
if LEX::context_analysis_only is set, i.e. at context
analysis stage.
--Item_subselect::const_item()
Return FALSE if LEX::context_analysis_only is set.
It prevents subquery evaluation in ::fix_fields &
::fix_length_and_dec at context analysis stage.
sql/item_subselect.h:
--removed unnecessary code
sql/mysql_priv.h:
--Added new set of flags.
sql/sql_class.h:
--removed unnecessary code
sql/sql_derived.cc:
--added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_lex.cc:
--init LEX::context_analysis_only field
sql/sql_lex.h:
--New LEX::context_analysis_only field
sql/sql_parse.cc:
--removed unnecessary code
sql/sql_prepare.cc:
--removed unnecessary code
--added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_select.cc:
--refactored context analysis checks
sql/sql_show.cc:
--added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_view.cc:
--added LEX::context_analysis_only analysis intialization/cleanup
