This bug manifested itself when executing queries with multiple reference
to a CTE specified by a query expression with union and having its
column names explicitly declared. In this case the server returned a bogus
error message about unknown column name. It happened because while for the
first reference to the CTE the names of the columns returned by the CTE
specification were properly changed to match the CTE definition for the
other references it was not done. This was a consequence of not quite
complete code of the function With_element::clone_parsed_spec() that forgot
to set the reference to the CTE definition for unit structures representing
non-first CTE references.
Approved by dmitry.shulga@mariadb.com
Due to this bug the server reported bogus messages about lack of SELECT
privileges for base tables used in the specifications of CTE tables.
It happened only if such a CTE were referred to at least twice.
For any non-recursive reference to CTE that is not primary the
specification of the CTE is cloned. The function check_table_access() is
called for such reference. The function checks privileges of the tables
referenced in the specification. As no name resolution was performed for
CTE references whose definitions occurred outside the specification before
the call of check_table_access() that was supposed to check the access
rights of the underlying tables these references were considered
as references to base tables rather than references to CTEs. Yet for CTEs
as well as for derived tables no privileges are needed and thus cannot
be granted.
The patch ensures proper name resolution of all references to CTEs before
any acl checks.
Approved by Oleksandr Byelkin <sanja@mariadb.com>
This bug could cause a crash when executing queries that used mutually
recursive CTEs with system variable big_tables set to 1. It happened due
to several bugs in the code that handled recursive table references
referred mutually recursive CTEs. For each recursive table reference a
temporary table is created that contains all rows generated for the
corresponding recursive CTE table on the previous step of recursion.
This temporary table should be created in the same way as the temporary
table created for a regular materialized derived table using the
method select_union::create_result_table(). In this case when the
temporary table is created it uses the select_union::TMP_TABLE_PARAM
structure as the parameter for the table construction. However the
code created the temporary table using just the function create_tmp_table()
and passed pointers to certain fields of the TMP_TABLE_PARAM structure
used for accumulation of rows of the recursive CTE table as parameters
for update. This was a mistake because now different temporary tables
cannot share some TMP_TABLE_PARAM fields in a general case. Besides,
depending on how mutually recursive CTE tables were defined and which
of them were referred in the executed query the select_union object
allocated for a recursive table reference could be allocated again after
the the temporary table had been created. In this case the TMP_TABLE_PARAM
object associated with the temporary table created for the recursive
table reference contained unassigned fields needed for execution when
Aria engine is employed as the engine for temporary tables.
This patch ensures that
- select_union object is created only once for any recursive table
reference
- any temporary table created for recursive CTEs uses its own
TMP_TABLE_PARAM structure
The patch also fixes a problem caused by incomplete cleanup of join tables
associated with recursive table references.
Approved by Oleksandr Byelkin <sanja@mariadb.com>
For table references to CTEs the field TABLE_LIST::db must be set to
an empty string as it's done for table references to derived tables in
order CTEs to be processed similar to how derived tables are processed.
Approved by Oleksandr Byelkin <sanja@mariadb.com>
This bug is the same as the bug MDEV-17024. The crashes caused by these
bugs were due to premature cleanups of the unit specifying recursive CTEs
that happened in some cases when there were several outer references the
same recursive CTE.
The problem of premature cleanups for recursive CTEs could be already
resolved by the correction in TABLE_LIST::set_as_with_table() introduced
in this patch. ALL other changes introduced by the patches for MDEV-17024
and MDEV-22748 guarantee that this clean-ups are performed as soon as
possible: when the select containing the last outer reference to a
recursive CTE is being cleaned up the specification of the recursive CTE
should be cleaned up as well.
With INFORMATION_SCHEMA set as the default database the check that a table
referred in the processed query is defined in INORMATION_SCHEMA must
be postponed until all CTE names can be identified.
Includes:
MDEV-17302 Add support for ALTER USER command in prepared statement
and
MDEV-17673 main.cte_recursive fails in bb-10.4-ps branch in --ps
Set correct SELECT_LEX linkage for recursive CTEs.
Do not delegate this job to TABLE_LIST::set_as_with_table,
because it is only run on prepare, while With_element::move_anchors_ahead
is run both on prepare and execute (fix by Igor)
This bug in the code of the function With_element::check_unrestricted_recursive()
could force a recursive CTE to be executed in a non-standard compliant mode
in which recursive UNION ALL could lead to an infinite execution. This
problem could occur only in the case when this CTE was used by another
recursive CTE at least twice.
This patch fills a serious flaw in the implementation of common table
expressions. Before this patch an attempt to prepare a statement from
a query with a parameter marker in a CTE that was used more than once
in the query ended up with a bogus error message. Similarly if a statement
in a stored procedure contained a CTE whose specification used a
local variables and this CTE was referred to more than once in the
statement then the server failed to execute the stored procedure returning
a bogus error message on a non-existing field.
The problems appeared due to incorrect handling of parameter markers /
local variables in CTEs that were referred more than once.
This patch fixes the problems by differentiating between the original
occurrences of a parameter marker / local variable used in the
specification of a CTE and the corresponding occurrences used
in copies of this specification. These copies are substituted
instead of non-first references to the CTE.
The idea of the fix and even some code were taken from the MySQL
implementation of the common table expressions.
The current code does not support recursive CTEs whose specifications
contain a mix of ALL UNION and DISTINCT UNION operations.
This patch catches such specifications and reports errors for them.
Usage of aggregate/window functions in non-recursive parts of recursive CTEs
is allowed. Error messages complaining about this were reported by mistake.
This bug manifested itself when the optimizer chose an execution plan with
an access of the recursive CTE in a recursive query by key and ARIA/MYISAM
temporary tables were used to store recursive tables.
The problem appeared due to passing an incorrect parameter to the call of
instantiate_tmp_table() in the function With_element::instantiate_tmp_tables().
the non-recursive CTE via prepared statement
The problem appears as the column names of the CTE were allocated on the
wrong MEMROOT and after the preparation of the statement they disappear.
To fix it in the procedure With_element::rename_columns_of_derived_unit
the CTE column names are now allocated in the permanent MEMROOT for the
prepared statements and stored procedures.
does not return error
Corrected the code of st_select_lex::find_table_def_in_with_clauses() for
a proper identification of CTE references used in embedded CTEs.
When identifying a table name the following should be taken into account:
a CTE name cannot be qualified with a database name, otherwise the table
name is considered as the name of a non-CTE table.
If the specification of a CTE contains a reference to a temporary table
then THD::open_temporary_table() must be called for this reference for
any occurrence of the CTE in the query. By mistake this was done only
for the first occurrences of CTEs.
The patch fixes this problem in With_element::clone_parsed_spec().
It also moves there the call of check_dependencies_in_with_clauses()
to its proper place before the call of check_table_access().
Additionally the patch optimizes the number of calls of the
function check_dependencies_in_with_clauses().
The support of embedded CTEs was not correct in the cases when
embedded CTEs were used multiple times. The problems occurred with
both non-recursive (bug mdev-13780) and recursive (bug mdev-14184)
embedded CTEs.
A reference to a CTE may occur not in the master of the CTE
specification. In this case if the reference to the CTE is
the first one the specification should be detached from its
master and attached to the referencing select.
Also fixed the TYPE column in the lines of the EXPLAIN output
created for CTE tables.
The bug happened when the specification of a recursive CTE had
no recursive references at the top level of the specification.
In this case the regular processing of derived table references
of the select containing a non-recursive reference to this
recursive CTE misses handling the specification unit.
At the preparation stage any non-recursive reference to a
recursive CTE must be handled after the preparation of the
specification unit for this CTE. So we have to force this
preparation when regular handling of derived tables does not
do it.
In the current code temporary tables we identified and opened before
other tables. CTE tables are identified in the same procedure as
regular tables. When a temporary table and a CTE table have the same
name T any reference to T that is in the scope of the CTE declaration
must be associated with this CTE. Yet it was not done properly.
When a reference to T was found in the scope of the declaration
of CTE T a pointer to this CTE was set in the reference. No check
that the reference had been already associated with a temporary table
was done. As a result, if the temporary table T had been created then
the reference to T was considered simultaneously as reference to the CTE
named T and as a reference to the temporary table named T. This
confused the code that were executed later and caused a crash of
the server.
Now when a table reference is associated with a CTE any previous
association with a temporary table is dropped.
This problem could be easily avoided if the temporary tables were
not identified prematurely.
as reference to CTE named T and
This patch fixed some problems that occurred with subqueries that
contained directly or indirectly recursive references to recursive CTEs.
1. A [NOT] IN predicate with a constant left operand and a non-correlated
subquery as the right operand used in the specification of a recursive CTE
was considered as a constant predicate and was evaluated only once.
Now such a predicate is re-evaluated after every iteration of the process
that produces the records of the recursive CTE.
2. The Exists-To-IN transformation could be applied to [NOT] IN predicates
with recursive references. This opened a possibility of materialization
for the subqueries used as right operands. Yet, materialization
is prohibited for the subqueries if they contain a recursive reference.
Now the Exists-To-IN transformation cannot be applied for subquery
predicates with recursive references.
The function st_select_lex::check_subqueries_with_recursive_references()
is called now only for the first execution of the SELECT.
When a CTE referring to another CTE from the same with clause
was used twice then the server could not find the second CTE and
reported a bogus error message.
This happened because for any unit that was created as a clone of
a CTE specification the pointer to the WITH clause that owned this CTE
was not set.
The method With_element::check_unrestricted_recursive() icorrectly performed
the check that no recursive reference is not encountered in inner parts of
outer joins. As a result the server reported errors for valid specifications
with outer joins.
The bug was caused by a wrong order of statements in With_clause::print().
As a result any view definition containing WITH clause with several
CTE specifications was put the frm file in a syntactically incorrect
form.
When a query containing a WITH clause is printed by EXPLAIN
EXTENDED command there should not be any data expansion in
the query specifications of the WITH elements of this WITH
clause.
The code for st_select_lex::find_table_def_in_with_clauses()
did not take into account the fact that the specs for mergeable
CTEs were cloned and were not processed by the function
With_element::check_dependencies_in_spec().
The bug was in the code of the recursive method
With_element::check_unrestricted_recursive. For recursive
calls of this method sel->get_with_element()->owner != owner.
Added comments.
Added reaction for exceeding maximum number of elements in with clause.
Added a test case to check this reaction.
Added a test case where the specification of a recursive table
uses two non-recursive with tables.
Temporary tables created for recursive CTE
were instantiated at the prepare phase. As
a result these temporary tables missed
indexes for look-ups and optimizer could not
use them.
Added test cases to check the fix.
Fixed the problem of wrong types of recursive tables when the type of anchor part does not coincide with the
type of recursive part.
Prevented usage of marerialization and subquery cache for subqueries with recursive references.
Introduced system variables 'max_recursion_level'.
Added a test case to test usage of this variable.
Added the check whether there are set functions in the specifications of recursive CTE.
Added the check whether there are recursive references in subqueries.
Introduced boolean system variable 'standards_compliant_cte'. By default it's set to 'on'.
When it's set to 'off' non-standard compliant CTE can be executed.
of mdev-8789.
Fixed a bug in TABLE_LIST::print.
Fixed another bug for the case when the definition of a
WITH table contained column list while the join in the main
query used two instances of this table.
