Commit graph

1677 commits

Author SHA1 Message Date
Oleksandr Byelkin
348f4c9f3b MDEV-30889: 1 - Allocation in Item_subselect::mark_as_dependent
Fix leack in Item_subselect::mark_as_dependent (allocation
of temporary list in statement memory inctroduced in
f4d5521043 )
2023-04-26 16:15:29 +02:00
Igor Babaev
c912fd3b29 Fixes of MDEV-30538 and MDEV-30586 for 10.4 adjusted for 11.0.
The commits for MDEV-30538 and MDEV-30586 could not be cherry-picked into
11.0 separately.
2023-03-15 17:35:22 -07:00
Monty
ae05097714 Fixed crashing bug in recursive SQL if write to tmp table would fail
This error was discovered while working on
MDEV-30540 Wrong result with IN list length reaching
           IN_PREDICATE_CONVERSION_THRESHOLD

If there is read error from handler::ha_rnd_next() during a recursive
query, st_select_lex_unit::exec_recursive() will crash as it will try to
get the error code from a structure that was deleted by the callee.
The code was using the construct:
   sl->join->exec();
   saved_error=sl->join->error;
This does not work as sl->join was freed by the exec() and sl->join would
be set to 0.
Fixed by having JOIN::exec() return the error code.
The included test case simulates the error in ha_rnd_next(), which causes
a crash without the patch.
scovered whle working on
MDEV-30540 Wrong result with IN list length reaching
           IN_PREDICATE_CONVERSION_THRESHOLD

If there is read error from handler::ha_rnd_next() during a recursive
query, st_select_lex_unit::exec_recursive() will crash as it will try to
get the error code from a structure that was deleted by the callee.
The code was using the construct:
   sl->join->exec();
   saved_error=sl->join->error;
This does not work as sl->join was freed by the exec() and sl->join was
set to 0.
Fixed by having JOIN::exec() return the error code.
The included test case simulates the error in ha_rnd_next(), which causes
a crash without the patch.
2023-03-02 13:11:54 +02:00
Marko Mäkelä
2e431ff7e6 Merge 10.11 into 11.0 2023-02-16 13:34:45 +02:00
Marko Mäkelä
0d55914d96 Merge 10.8 into 10.9 2023-02-16 10:25:34 +02:00
Marko Mäkelä
dbab3e8d90 Merge 10.6 into 10.8 2023-02-10 13:43:53 +02:00
Marko Mäkelä
6aec87544c Merge 10.5 into 10.6 2023-02-10 13:03:01 +02:00
Marko Mäkelä
c41c79650a Merge 10.4 into 10.5 2023-02-10 12:02:11 +02:00
Igor Babaev
bef20b5f36 MDEV-30538 Plans for SELECT and multi-table UPDATE/DELETE unexpectedly differ
This patch allowed transformation of EXISTS subqueries into equivalent
IN predicands at the top level of WHERE conditions for multi-table UPDATE
and DELETE statements. There was no reason to prohibit the transformation
for such statements. The transformation provides more opportunities of
using semi-join optimizations.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2023-02-03 11:17:03 -08:00
Rex
07f21cfb14 MDEV-21092,MDEV-21095,MDEV-29997: Optimizer Trace for index condition pushdown, partition pruning, exists-to-in
Add Optimizer Tracing for:
        - Index Condition Pushdown
        - Partition Pruning
        - Exists-to-IN optimization
2023-02-03 14:28:08 +03:00
Monty
5e5a8eda16 Derived tables and union can now create distinct keys
The idea is that instead of marking all select_lex's with DISTINCT, we
only mark those that really need distinct result.

Benefits of this change:
- Temporary tables used with derived tables, UNION, IN are now smaller
  as duplicates are removed already on the insert phase.
- The optimizer can now produce better plans with EQ_REF. This can be
  seen from the tests where several queries does not anymore materialize
  derived tables twice.
- Queries affected by 'in_predicate_conversion_threshold' where large IN
  lists are converted to sub query produces better plans.

Other things:
- Removed on duplicate call to sel->init_select() in
  LEX::add_primary_to_query_expression_body()
- I moved the testing of
  tab->table->pos_in_table_list->is_materialized_derived()
  in join_read_const_table() to the caller as it caused problems for
  derived tables that could be proven to be const tables.
  This also is likely to fix some bugs as if join_read_const_table()
  was aborted, the table was left marked as JT_CONST, which cannot
  be good.  I added an ASSERT there for now that can be removed when
  the code has been properly tested.
2023-02-02 22:32:57 +03:00
Oleksandr Byelkin
ebf2121529 Merge branch '10.8' into 10.9 2022-11-01 10:33:44 +01:00
Oleksandr Byelkin
1ebfa2af62 Merge branch '10.6' into 10.7 2022-10-29 19:22:04 +02:00
Marko Mäkelä
aeccbbd926 Merge 10.5 into 10.6
To prevent ASAN heap-use-after-poison in the MDEV-16549 part of
./mtr --repeat=6 main.derived
the initialization of Name_resolution_context was cleaned up.
2022-10-25 14:25:42 +03:00
Oleksandr Byelkin
55e07d9ade Merge branch '10.8' into 10.9 2022-10-04 13:23:13 +02:00
Oleksandr Byelkin
b6ebadaa66 Merge branch '10.6' into 10.7 2022-10-04 07:41:35 +02:00
Sergei Golubchik
900d7bf360 Merge branch '10.5' into 10.6 2022-10-02 22:14:21 +02:00
Sergei Golubchik
3a2116241b Merge branch '10.4' into 10.5 2022-10-02 14:38:13 +02:00
Sergei Golubchik
d4f6d2f08f Merge branch '10.3' into 10.4 2022-10-01 23:07:26 +02:00
Oleksandr Byelkin
47e9678982 MDEV-29022 add_slave destroy child list and has dead code
Nowdays subquery in a UNION's ORDER BY placed correctly in fake select,
the only problem was incorrect Name_resolution_contect is fixed by this
patch in parsing, so we do not need scanning/reseting of ORDER BY of
a union.
2022-09-27 09:56:21 +02:00
Marko Mäkelä
50d6966c50 Merge 10.8 into 10.9 2022-08-30 13:22:57 +03:00
Marko Mäkelä
b86be02ecf Merge 10.6 into 10.7 2022-08-30 13:02:42 +03:00
Marko Mäkelä
f410974f0f Merge 10.5 into 10.6 2022-08-30 13:01:16 +03:00
Marko Mäkelä
29fa9bcee0 Merge 10.4 into 10.5 2022-08-30 12:29:04 +03:00
Marko Mäkelä
7e574eb52c Merge 10.3 into 10.4 2022-08-30 12:17:33 +03:00
Igor Babaev
94e3f02db7 MDEV-29350 Crash when IN predicand is used in eliminated GROUP BY clause
This bug affected some queries with an IN/ALL/ANY predicand or an EXISTS
predicate whose subquery contained a GROUP BY clause that could be
eliminated. If this clause used a IN/ALL/ANY predicand whose left operand
was a single-value subquery then execution of the query caused a crash of
the server after invokation of remove_redundant_subquery_clauses().
The crash was caused by an attempt to exclude the unit for the single-value
subquery from the query tree for the second time by the function
Item_subselect::eliminate_subselect_processor().

This bug had been masked by the bug MDEV-28617 until a fix for the latter
that properly excluded units was pushed into 10.3.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2022-08-25 09:53:23 -07:00
Sergei Golubchik
bf2bdd1a1a Merge branch '10.8' into 10.9 2022-05-19 14:07:55 +02:00
Sergei Golubchik
fd132be117 Merge branch '10.6' into 10.7 2022-05-11 11:25:33 +02:00
Sergei Golubchik
3bc98a4ec4 Merge branch '10.5' into 10.6 2022-05-10 14:01:23 +02:00
Sergei Golubchik
ef781162ff Merge branch '10.4' into 10.5 2022-05-09 22:04:06 +02:00
Sergei Golubchik
a70a1cf3f4 Merge branch '10.3' into 10.4 2022-05-08 23:03:08 +02:00
Oleksandr Byelkin
9614fde1aa Merge branch '10.2' into 10.3 2022-05-03 10:59:54 +02:00
Oleg Smirnov
02c3babdec MDEV-28124 Server crashes in Explain_aggr_filesort::print_json_members
SHOW EXPLAIN/ANALYZE FORMAT=JSON tries to access items that have already been
freed by a call to free_items() during THD::cleanup_after_query().
The solution is to disallow APC calls including SHOW EXPLAIN/ANALYZE
just before the call to free_items().
2022-04-29 10:48:25 +03:00
Marko Mäkelä
638afc4acf Merge 10.6 into 10.7 2022-04-26 18:59:40 +03:00
Sergei Petrunia
5100b20b15 MDEV-26047: MariaDB server crash at Item_subselect::init_expr_cache_tracker
The cause of crash:
remove_redundant_subquery_clauses() removes redundant item expressions.
The primary goal of this is to remove the subquery items.
The removal process unlinks the subquery from SELECT_LEX tree, but does
not remove it from SELECT_LEX:::ref_pointer_array or from JOIN::all_fields.
Then, setup_subquery_caches() tries to wrap the subquery item in an
expression cache, which fails, the first reason for failure being that
the item doesn't have a query plan.

Solution: do not wrap eliminated items with expression cache.
(also added an assert to check that we do not attempt to execute them).

This may look like an incomplete fix: why don't we remove any mention
of eliminated item everywhere? The difficulties here are:
* items can be "un-removed" (see set_fake_select_as_master_processor)
* it's difficult to remove an element from ref_pointer_array: Item_ref
objects refer to elements of that array, so one can't shift elements in
it. Replacing eliminated subselect with a dummy Item doesn't look like a
good idea, either.
2022-04-26 15:21:21 +03:00
Sergei Petrunia
3c209bfc04 MDEV-25994: Crash with union of my_decimal type in ORDER BY clause
When single-row subquery fails with "Subquery reutrns more than 1 row"
error, it will raise an error and return NULL.

On the other hand, Item_singlerow_subselect sets item->maybe_null=0
for table-less subqueries like "(SELECT not_null_value)"  (*)

This discrepancy (item with maybe_null=0 returning NULL) causes the
code in Type_handler_decimal_result::make_sort_key_part() to crash.

Fixed this by allowing inference (*) only when the subquery is NOT a
UNION.
2022-04-22 13:57:16 +03:00
Marko Mäkelä
fae0ccad6e Merge 10.5 into 10.6 2022-04-21 17:46:40 +03:00
Marko Mäkelä
620c55e708 Merge 10.4 into 10.5 2022-04-21 15:33:50 +03:00
Marko Mäkelä
394784095e Merge 10.3 into 10.4 2022-04-21 11:33:59 +03:00
Alexander Barkov
9d734cdd61 Merge remote-tracking branch 'origin/10.2' into 10.3 2022-04-14 11:50:34 +04:00
Sergei Petrunia
5a8766a980 Better comments in Item_in_subselect::inject_in_to_exists_cond() 2022-04-07 15:40:34 +03:00
Marko Mäkelä
3d88f9f34c Merge 10.6 into 10.7 2022-02-25 16:09:16 +02:00
Marko Mäkelä
6daf8f8a0d Merge 10.5 into 10.6 2022-02-25 13:48:47 +02:00
Marko Mäkelä
b791b942e1 Merge 10.4 into 10.5 2022-02-25 13:27:41 +02:00
Marko Mäkelä
f5ff7d09c7 Merge 10.3 into 10.4 2022-02-25 13:00:48 +02:00
Marko Mäkelä
00b70bbb51 Merge 10.2 into 10.3 2022-02-25 10:43:38 +02:00
Sergei Petrunia
d140d27624 MDEV-22377: Subquery in an UPDATE query uses full scan instead of range
[Patch idea by Igor Babaev]

Symptom: for IN (SELECT ...) subqueries using IN-to-EXISTS transformation,
the optimizer was unable to make inferences using multiple equalities.

The cause is code Item_in_subselect::inject_in_to_exists_cond() which may
break invariants that Multiple-Equality code relies on. In particular, it
may produce a WHERE condition with an empty Item_cond::m_cond_equal.

Fixed this by making Item_cond::m_cond_equal.
2022-02-21 18:44:11 +03:00
Oleksandr Byelkin
9ed8deb656 Merge branch '10.6' into 10.7 2022-02-04 14:11:46 +01:00
Oleksandr Byelkin
f5c5f8e41e Merge branch '10.5' into 10.6 2022-02-03 17:01:31 +01:00
Oleksandr Byelkin
cf63eecef4 Merge branch '10.4' into 10.5 2022-02-01 20:33:04 +01:00
Oleksandr Byelkin
a576a1cea5 Merge branch '10.3' into 10.4 2022-01-30 09:46:52 +01:00
Oleksandr Byelkin
41a163ac5c Merge branch '10.2' into 10.3 2022-01-29 15:41:05 +01:00
Igor Babaev
d6ee351bbb Revert "MDEV-24454 Crash at change_item_tree"
This patch reverts the fixes of the bugs MDEV-24454 and MDEV-25631 from
the commit 3690c549c6.
It leaves the changes in plugin/feedback/feedback.cc and corresponding
test files introduced in this commit intact.

Proper fixes for the bug MDEV-24454 and MDEV-25631 will follow immediately.
2022-01-10 09:12:17 -08:00
Igor Babaev
8265d6d9f6 MDEV-22846 Server crashes in handler_index_cond_check on SELECT
If the optimizer decides to rewrites a NOT IN predicand of the form
  outer_expr IN (SELECT inner_col FROM ... WHERE subquery_where)
into the EXISTS subquery
  EXISTS (SELECT 1 FROM ... WHERE subquery_where AND
        (outer_expr=inner_col OR inner_col IS NULL))
then the pushed equality predicate outer_expr=inner_col can be used for
ref[or_null] access if inner_col is a reference to an indexed column.
In this case if there is a selective range condition over this column then
a Rowid filter may be employed coupled the with ref[or_null] access. The
filter is 'pushed' into the engine and in InnoDB currently it cannot be
used with index look-ups by primary key. The ref[or_null] access can be
used only when outer_expr is not NULL. Otherwise the original predicand
is evaluated to TRUE only if the result set returned by the query
 SELECT 1 FROM ... WHERE subquery_where
is empty. When performing this evaluation the executor switches to the
table scan by primary key. Before this patch the pushed filter still
remained marked as active and the engine tried to apply the filter. This
was incorrect and in InnoDB this attempt to use the filter led to an
assertion failure.

This patch fixes the problem by disabling usage of the filter when
outer_expr is evaluated to NULL.
2022-01-07 11:52:25 -08:00
Marko Mäkelä
b36d6f92a8 Merge 10.6 into 10.7 2021-09-30 11:01:07 +03:00
Marko Mäkelä
a49e394399 Merge 10.5 into 10.6 2021-09-30 10:38:44 +03:00
Marko Mäkelä
064cb58efe Merge 10.4 into 10.5
FIXME: Part of the MDEV-20699 test is disabled due to
nonderterministic result.
2021-09-30 09:04:43 +03:00
Marko Mäkelä
a10b63bf58 Merge 10.3 into 10.4 2021-09-29 16:03:02 +03:00
Marko Mäkelä
742b37a345 Merge 10.2 into 10.3 2021-09-29 15:04:20 +03:00
Oleksandr Byelkin
3690c549c6 MDEV-24454 Crash at change_item_tree
Use in_sum_func (and so nest_level) only in LEX to which SELECT lex belong to

Reduce usage of current_select (because it does not always point on the correct
 SELECT_LEX, for example with prepare.

Change context for all classes inherited from Item_ident (was only for Item_field) in case of pushing down it to HAVING.

Now name resolution context have to have SELECT_LEX reference if the context is present.

Fixed feedback plugin stack usage.
2021-09-27 11:00:51 +02:00
Sergei Golubchik
0299ec29d4 cleanup: MY_BITMAP mutex
in about a hundred of users of MY_BITMAP, only two were using its
built-in mutex, and only one of those two was actually needing it.

Remove the mutex from MY_BITMAP, remove all associated conditions
and checks in bitmap functions. Use an external LOCK_temp_pool
mutex and temp_pool_set_next/temp_pool_clear_bit acccessors.

Remove bitmap_init/bitmap_free, always use my_* versions.
2021-08-26 23:39:52 +02:00
Marko Mäkelä
cee37b5d26 Merge 10.6 into 10.7 2021-07-22 11:22:09 +03:00
Monty
fcbb2a1df1 Make marking/testing of top level item uniform
There where several different implementations of is_top_level_item(),
with different variable names and tests. In some cases the code used
'is_top_level_item()' as a test, in other cases it accessed the variable
directrly. This patch makes all usage of 'top_level_item' uniform.

The new implementation stores the 'is_tol_level_item()' flag as part
of base_flags. This saves 7 bytes in all items that previously stored
the flag in it's own bool.

I had to keep 'top_level_item()' virtual to ensure that Item_bool_const
item's will not be updated.  'is_top_level_item()' is not virtual
anymore.
2021-07-19 17:05:51 +03:00
Dmitry Shulga
fa45400d77 MDEV-26149: The test main.fetch_first fails in case it is run in PS mode
The root cause of test failure is that on optimization
of the statement clause 'order by (select 1)' in outer select
is done incorrect in case the statement run in PS mode.
2021-07-19 17:16:54 +07:00
Dmitry Shulga
9370c6e83c MDEV-16708: Unsupported commands for prepared statements
Withing this task the following changes were made:
- Added sending of metadata info in prepare phase for the admin related
  command (check table, checksum table, repair, optimize, analyze).

- Refactored implmentation of HELP command to support its execution in
  PS mode

- Added support for execution of LOAD INTO and XA- related statements
  in PS mode

- Modified mysqltest.cc to run statements in PS mode unconditionally
  in case the option --ps-protocol is set. Formerly, only those statements
  were executed using PS protocol that matched the hard-coded regular expression

- Fixed the following issues:
    The statement
      explain select (select 2)
    executed in regular and PS mode produces different results:

    MariaDB [test]> prepare stmt from "explain select (select 2)";
    Query OK, 0 rows affected (0,000 sec)
    Statement prepared
    MariaDB [test]> execute stmt;
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra          |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    |    1 | PRIMARY     | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | No tables used |
    |    2 | SUBQUERY    | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | No tables used |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    2 rows in set (0,000 sec)
    MariaDB [test]> explain select (select 2);
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra          |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    |    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | No tables used |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    1 row in set, 1 warning (0,000 sec)

    In case the statement
      CREATE TABLE t1 SELECT * FROM (SELECT 1 AS a, (SELECT a+0)) a
    is run in PS mode it fails with the error
      ERROR 1054 (42S22): Unknown column 'a' in 'field list'.

- Uniform handling of read-only variables both in case the SET var=val
  statement is executed as regular or prepared statememt.

- Fixed assertion firing on handling LOAD DATA statement for temporary tables

- Relaxed assert condition in the function lex_end_stage1() by adding
  the commands SQLCOM_ALTER_EVENT, SQLCOM_CREATE_PACKAGE,
  SQLCOM_CREATE_PACKAGE_BODY to a list of supported command

- Removed raising of the error ER_UNSUPPORTED_PS in the function
  check_prepared_statement() for the ALTER VIEW command

- Added initialization of the data memember st_select_lex_unit::last_procedure
  (assign NULL value) in the constructor

  Without this change the test case main.ctype_utf8 fails with the following
  report in case it is run with the optoin --ps-protocol.
    mysqltest: At line 2278: query 'VALUES (_latin1 0xDF) UNION VALUES(_utf8'a' COLLATE utf8_bin)' failed: 2013: Lost connection

- The following bug reports were fixed:
      MDEV-24460: Multiple rows result set returned from stored
                  routine over prepared statement binary protocol is
                  handled incorrectly
      CONC-519: mariadb client library doesn't handle server_status and
                warnign_count fields received in the packet
                COM_STMT_EXECUTE_RESPONSE.

  Reasons for these bug reports have the same nature and caused by
  missing loop iteration on results sent by server in response to
  COM_STMT_EXECUTE packet.

  Enclosing of statements for processing of COM_STMT_EXECUTE response
  in the construct like
    do
    {
      ...
    } while (!mysql_stmt_next_result());
  fixes the above mentioned bug reports.
2021-06-17 19:30:24 +02:00
Marko Mäkelä
65f1a42788 Merge 10.5 into 10.6 2021-06-09 16:50:58 +03:00
Marko Mäkelä
f4425d3a3d Merge 10.4 into 10.5 2021-06-08 16:03:53 +03:00
Marko Mäkelä
72b2489621 Merge 10.3 into 10.4 2021-06-08 15:02:40 +03:00
Igor Babaev
8149e4d0a1 MDEV-25682 Explain shows an execution plan different from actually executed
If a select query contained an ORDER BY clause that followed a LIMIT clause
or an ORDER BY clause or ORDER BY with LIMIT the EXPLAIN output for the
query showed an execution plan different from that was actually executed.

Approved by Roman Nozdrin <roman.nozdrin@mariadb.com>
2021-06-07 15:08:18 -07:00
Marko Mäkelä
860e754349 Merge 10.5 into 10.6 2021-05-26 11:22:40 +03:00
Marko Mäkelä
365cd08345 Merge 10.4 into 10.5 2021-05-26 09:47:28 +03:00
Igor Babaev
675716e1cb MDEV-23886 Reusing CTE inside a function fails with table doesn't exist
In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.

This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.

# Conflicts:
#	sql/sql_cte.cc
#	sql/sql_cte.h
#	sql/sql_lex.cc
#	sql/sql_lex.h
#	sql/sql_view.cc
#	sql/sql_yacc.yy
#	sql/sql_yacc_ora.yy
2021-05-25 21:48:54 -07:00
Marko Mäkelä
1dea7f7977 Merge 10.3 into 10.4 2021-05-25 15:38:57 +03:00
Igor Babaev
04de651725 MDEV-23886 Reusing CTE inside a function fails with table doesn't exist
In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.

This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.
2021-05-25 00:43:03 -07:00
Marko Mäkelä
1864a8ea93 Merge 10.2 into 10.3 2021-05-24 09:38:49 +03:00
Igor Babaev
43c9fcefc0 MDEV-23886 Reusing CTE inside a function fails with table doesn't exist
In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.

This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.
2021-05-21 16:00:35 -07:00
Sergei Petrunia
2087d47aae MDEV-22462: Item_in_subselect::create_single_in_to_exists_cond(JOIN *, Item **, Item **): Assertion `false' failed.
Item_in_subselect::create_single_in_to_exists_cond() should handle the
case where the subquery is a table-less select but it is not a result
of a UNION.

(Table-less subqueries like "(SELECT 1)" are "substituted" with their select
list, but table-less subqueries with WHERE or HAVING clause, like
"(SELECT 1 WHERE ...)" are not substituted. They are handled with regular
execution path)
2021-05-21 17:46:48 +03:00
Monty
08bc062e3c Remove some usage of Check_level_instant_set and Sql_mode_save
The reason for the removal are:
- Generates more code
  - Storing and retreving THD
  - Causes extra code and daata to be generated to handle possible throw
    exceptions (which never happens in MariaDB code)
- Uses more stack space

Other things:
- Changed convert_const_to_int() to use item->save_in_field_no_warnings(),
  which made the code shorter and simpler.
- Removed not needed code in Sp_handler::sp_create_routine()
- Added thd as argument to store_key.copy() to make function simpler
- Added thd as argument to some subselect* constructor that inherites
  from Item_subselect.
2021-05-19 22:54:12 +02:00
Sergei Golubchik
6de84e6f4e cleanup: Item::can_eval_in_optimize()
a helper method to check whether an item can be evaluated
in the query optimization phase (in and below JOIN::optimize()).
2021-05-19 22:27:53 +02:00
Monty
53b43f3078 Added full_name_cstring()
This returns a LEX_CSTRING and allows one to avoid strlen() calls.
2021-05-19 22:27:53 +02:00
Monty
b6ff139aa3 Reduce usage of strlen()
Changes:
- To detect automatic strlen() I removed the methods in String that
  uses 'const char *' without a length:
  - String::append(const char*)
  - Binary_string(const char *str)
  - String(const char *str, CHARSET_INFO *cs)
  - append_for_single_quote(const char *)
  All usage of append(const char*) is changed to either use
  String::append(char), String::append(const char*, size_t length) or
  String::append(LEX_CSTRING)
- Added STRING_WITH_LEN() around constant string arguments to
  String::append()
- Added overflow argument to escape_string_for_mysql() and
  escape_quotes_for_mysql() instead of returning (size_t) -1 on overflow.
  This was needed as most usage of the above functions never tested the
  result for -1 and would have given wrong results or crashes in case
  of overflows.
- Added Item_func_or_sum::func_name_cstring(), which returns LEX_CSTRING.
  Changed all Item_func::func_name()'s to func_name_cstring()'s.
  The old Item_func_or_sum::func_name() is now an inline function that
  returns func_name_cstring().str.
- Changed Item::mode_name() and Item::func_name_ext() to return
  LEX_CSTRING.
- Changed for some functions the name argument from const char * to
  to const LEX_CSTRING &:
  - Item::Item_func_fix_attributes()
  - Item::check_type_...()
  - Type_std_attributes::agg_item_collations()
  - Type_std_attributes::agg_item_set_converter()
  - Type_std_attributes::agg_arg_charsets...()
  - Type_handler_hybrid_field_type::aggregate_for_result()
  - Type_handler_geometry::check_type_geom_or_binary()
  - Type_handler::Item_func_or_sum_illegal_param()
  - Predicant_to_list_comparator::add_value_skip_null()
  - Predicant_to_list_comparator::add_value()
  - cmp_item_row::prepare_comparators()
  - cmp_item_row::aggregate_row_elements_for_comparison()
  - Cursor_ref::print_func()
- Removes String_space() as it was only used in one cases and that
  could be simplified to not use String_space(), thanks to the fixed
  my_vsnprintf().
- Added some const LEX_CSTRING's for common strings:
  - NULL_clex_str, DATA_clex_str, INDEX_clex_str.
- Changed primary_key_name to a LEX_CSTRING
- Renamed String::set_quick() to String::set_buffer_if_not_allocated() to
  clarify what the function really does.
- Rename of protocol function:
  bool store(const char *from, CHARSET_INFO *cs) to
  bool store_string_or_null(const char *from, CHARSET_INFO *cs).
  This was done to both clarify the difference between this 'store' function
  and also to make it easier to find unoptimal usage of store() calls.
- Added Protocol::store(const LEX_CSTRING*, CHARSET_INFO*)
- Changed some 'const char*' arrays to instead be of type LEX_CSTRING.
- class Item_func_units now used LEX_CSTRING for name.

Other things:
- Fixed a bug in mysql.cc:construct_prompt() where a wrong escape character
  in the prompt would cause some part of the prompt to be duplicated.
- Fixed a lot of instances where the length of the argument to
  append is known or easily obtain but was not used.
- Removed some not needed 'virtual' definition for functions that was
  inherited from the parent. I added override to these.
- Fixed Ordered_key::print() to preallocate needed buffer. Old code could
  case memory overruns.
- Simplified some loops when adding char * to a String with delimiters.
2021-05-19 22:27:48 +02:00
Monty
6079b46d8d Split item->flags into base_flags and with_flags
This was done to simplify copying of with_* flags

Other things:
- Changed Flags to C++ enums, which enables gdb to print
  out bit values for the flags. This also enables compiler
  errors if one tries to manipulate a non existing bit in
  a variable.
- Added set_maybe_null() as a shortcut as setting the
  MAYBE_NULL flags was used in a LOT of places.
- Renamed PARAM flag to SP_VAR to ensure it's not confused with persistent
  statement parameters.
2021-05-19 22:27:28 +02:00
Monty
7ca4e381f7 Removed Item::is_fixed() and Item::has_subquery()
One should instead use Item::fixed() and Item::with_subquery()

Removed Item::is_fixed() and has_subquery() and did the following replace:
replace is_fixed() fixed() -- *.*
replace 'has_subquery()' 'with_subquery()' -- *.*
2021-05-19 22:27:28 +02:00
Michael Widenius
9448548481 Remove calls to current_thd() in Item functions
- Added THD argument to functions that calls current_thd() or
  new without a mem_root argument:
  make_same(), set_comparator_func(), set_cmp_func(), set_cmp_func*(),
  set_aggregator() and prepare_sum_aggregators()
- Changed "new Class" to "new (thd->mem_root) Class"

Almost all changes mechanical, no logic changes.
2021-05-19 22:27:28 +02:00
Michael Widenius
3105c9e7a5 Change bitfields in Item to an uint16
The reason for the change is that neither clang or gcc can do efficient
code when several bit fields are change at the same time or when copying
one or more bits between identical bit fields.
Updated bits explicitely with & and | is MUCH more efficient than what
current compilers can do.
2021-05-19 22:27:28 +02:00
Michael Widenius
189d03dac5 Revert MDEV-14517 Cleanup for Item::with_subselect
Added back variable 'with_subquery' to Item class as a bit field.

This made the code shorter, faster (removed some virtual methods,
less code to create an initialized item etc) and made many Item's 7 bytes
smaller.

This is the last set of my patches the decreases the size of Item.

Some examples from gdb:
sizeof(Item):        144 -> 120
sizeof(Item_func)    208 -> 184
sizeof(Item_sum_max) 368 -> 344
2021-05-19 22:27:28 +02:00
Michael Widenius
ae39f4f6d6 Revert MDEV-16592 "Change Item::with_sum_func to a virtual method"
Added back variable 'with_sum_func' to Item class as a bit field.

This made the code shorter, faster (removed some virtual methods,
less code to create an initialized item etc) and made many Item's 7 bytes
smaller.

The code is also easier to understand as 'with_sum_func' is threated as any
other Item variable when creating or copying items.
2021-05-19 22:27:28 +02:00
Marko Mäkelä
b728c3dbd9 fixup dd6ad38068: remove dead code 2021-04-22 08:42:55 +03:00
Vicențiu Ciorbaru
13cf8f5e9a cleanup: Refactor select_limit in select lex
Replace
  * select_lex::offset_limit
  * select_lex::select_limit
  * select_lex::explicit_limit
with select_lex::Lex_select_limit

The Lex_select_limit already existed with the same elements and was used in
by the yacc parser.

This commit is in preparation for FETCH FIRST implementation, as it
simplifies a lot of the code.

Additionally, the parser is simplified by making use of the stack to
return Lex_select_limit objects.

Cleanup of init_query() too. Removes explicit_limit= 0 as it's done a bit later
in init_select() with limit_params.empty()
2021-04-21 14:08:58 +03:00
Sergei Petrunia
dd6ad38068 Code cleanup: merge walk_items_for_table_list with walk_table_functions_for_list 2021-04-21 12:32:58 +03:00
Marko Mäkelä
4930f9c94b Merge 10.5 into 10.6 2021-04-21 11:45:00 +03:00
Sergei Petrunia
4a10dd0253 MDEV-25380: JSON_TABLE: Assertion `join->best_read < double(1.797...) fails
The query used a subquery of this form:

SELECT ...
WHERE
   EXISTS( SELECT ...
           FROM JSON_TABLE(outer_ref, ..) as JT
           WHERE trivial_correlation_cond)

EXISTS-to-IN conversion code was unable to see that the subquery will
still be correlated after the trivial_correlation is removed, which
eventually caused a crash due to inability to construct a query plan.

Fixed by making Item_subselect::walk() also walk arguments of Table
Functions.
2021-04-21 10:21:46 +04:00
Marko Mäkelä
80ed136e6d Merge 10.4 into 10.5 2021-04-21 09:01:01 +03:00
Marko Mäkelä
a0588d54a2 Merge 10.3 into 10.4 2021-04-21 07:58:42 +03:00
Marko Mäkelä
75c01f39b1 Merge 10.2 into 10.3 2021-04-21 07:25:48 +03:00
Sergei Petrunia
ab5dc62545 MDEV-25407: EXISTS subquery with correlation in ON expression crashes
Make Item_subselect::walk() walk the ON expressions, too.
2021-04-16 17:15:22 +03:00
Monty
cccc96d66c Fixed wrong initializations of Dynamic_array
Other things:
- Added size() function to Dynamic_array()
2021-03-20 21:17:32 +02:00
Marko Mäkelä
be881ec457 Merge 10.4 into 10.5 2021-03-19 13:09:21 +02:00
Marko Mäkelä
44d70c01f0 Merge 10.3 into 10.4 2021-03-19 11:42:44 +02:00
Marko Mäkelä
19052b6deb Merge 10.2 into 10.3 2021-03-18 12:34:48 +02:00