The bug was that thd->lex->limit_rows_examined_cnt was not reset after
queries. It is reset in lex_start() at the start of the next query
execution.
This causes general_log_write(), which is called before lex_start(), to
exceed the limit. The effect is a crash or the next query would not be
executed.
Fixed by resetting limit_rows_examined_cnt at end of query.
Fix rpl suite tests added by MDEV-25039.
rpl_foreign_key_lock_table_insert.test is removed altogether because it
is unclear what the purpose of the test is. The changes of the patch
were done on the slave, yet all operations in the test were done on the
master. Nothing different could happen on the slave because it is
configured to be serial, so all transactions would run sequentially
anyway, and no validations were performed.
rpl_foreign_key_ddl_insert.test was renamed to
rpl_row_foreign_key_mdl.test and the test itself was re-written to be
a minimal test case to ensure that MDL locking behavior is different
pre- and post- patch. A few problems with the original test:
* No foreign-key locking was done on the slave because the table
engine was not InnoDB.
* rpl_fk_ddl.inc had inconsistent validation checking. I.e., the child
query validation checks were done on the master (which is incorrect)
and because the slave was configured to be serial, the two
transactions could not run concurrently on the slave anyway.
Issue:
On galera write node INSERT statements does not acquire MDL locks on it's all child
tables and thereby wsrep certification keys are also added for limited tables, but
on applier nodes it does acquire MDL locks for all child tables. This can result
into MDL BF-BF conflict on applier node when transactions referring to parent and
child tables are executed concurrently. For example:
Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4
Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4
On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.
On applier nodes Write_rows event is implicitly a REPLACE, deleting all conflicting
rows which can cause cascading FK actions and locks on foreign key children tables.
Solution:
For Galera applier nodes the Write_rows event is considered pure INSERT
which will never cause cascading FK actions and locks on foreign key children tables.
The Port field in the system table mysql.servers has type INT,
which translates to Field_long.
During parsing it is parsed as ulong_num, and in this patch we add
bound checks there.
In find_field_in_view(), we call field_it.create_item() which
creates item on a statement mem_root.
Then we set its name. Make sure the name is allocated on a statement
mem_root, too.
Run-time has semantics duplication in unireg_check, default_value and
flags, so all three must be in sync before FRM creation. Special
unireg_check values for temporal field types was introduced by
32b28f9298 WL#1266 "Separate auto-set logic from TIMESTAMP type."
Each ORDER and WHERE slot may generate split, see code like this:
if ((item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM) ||
item->with_window_func())
item->split_sum_func(thd, ref_ptrs, all_fields, SPLIT_SUM_SELECT);
Such kind of code is done in JOIN::prepare(), setup_order(),
setup_fields(), setup_group() and split_sum_func2() itself.
Since we are at the phase of ref_ptrs allocation, items are not fixed
yet and we cannot calculate precisely how much ref_ptrs is needed. We
can estimate at most how much is needed. In the worst case each window
function generates split on each ORDER BY field, GROUP BY field and
WHERE field, so the counts of these should be multiplied by window
funcs count.
As the split can be done in both setup_without_group() and
JOIN::prepare() simultaneously, the factor of window funcs should be
multiplied by 2.
The similar case may be with inner sumfunc items as of the condition
item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM
but factor of these is harder to predict at the stage of unfixed
items.
ft_handler isn't getting initialized for subqueries inside explain
delete/update queries. However, ft_handler is accessed inside ha_ft_read(),
and is the reason for NULL pointer exception.
This is not the case with non-explain delete/update queries, as
well as explain/non-explain select queries.
Follow the approach the SELECT statements are using in
JOIN::optimize_constant_subqueries(): remove SELECT_DESCRIBE
flag when invoking optimization of constant subqueries.
Single-table UPDATE/DELETEs have SELECT_LEX but don't have JOIN.
So, we make optimize_constant_subqueries() not to be a member
of JOIN class, and instead move it to SELECT_LEX, and then
invoke it from single-table UPDATE/DELETE as well as for SELECT queries.
Reason:
======
During InnoDB DDL, statistics updation fails due to lock wait
timeout and calls push_warning_printf() to generate warnings
but then returns success, causing the SQL layer
to attempt calling set_ok_status() when the diagnostics area
is already set.
Solution:
=========
By temporarily setting abort_on_warning to false around operations
that prevents warning to error escalation and restore the original
setting after calling HA_EXTRA_END_ALTER_COPY for alter operation.
of multi-table-styled DELETE from a view
Analysis:
The item_list of builtin_select stores the fields that are there in the
RETURNING clause.
During the "EXECUTE" command, a "dummy item" is added into the item_list
of the select_lex(builtin_select) representing DELETE during
Sql_cmd_delete::precheck(). This snippet that adds a dummy item is added
because columnstore needs for temporary table. Results are put into a
temporary table and to create a temporary table we need to know what
columns are there which we get from the select_lex->item_list.
As a result, the item_list now has an item even when there is not really
RETURNING clause, resulting in execution of the setup_returning_fields()
when it should have exited already.
Fix:
Instead of checking whether builint_select's item_list is empty to
determine whether there is RETURNING clause, use a flag.
Index merge and rowid filter should not be used together, however,
even if index merge is not chosen earlier in best_access_path, it may
be chosen again in make_join_select, inside ref_to_range. Therefore
this patch ensures that rowid filter is not used when index merge is
chosen there.
some I_S tables require "any non-SELECT privilege on the table".
If only SELECT was granted on the global level and something non-SELECT
on the schema level, then we need to check schema level privileges
explicitly, because check_grant() doesn't do that and get_all_tables()
doesn't look deeper if SELECT is present on the global level.
relax the assert, allowing '\n' at the end if the string is exactly
MYSQL_ERRMSG_SIZE-1 bytes long. It likely doesn't end with '\n' but
was truncated at the middle.
also, use MYSQL_ERRMSG_SIZE in my_error.c not a separate define
that must be "kept in sync"
* fail acl_load() if it was killed, this will cause all privileges to
be reset to their original pre-load values.
* only increment grant_version if privileges were, in fact, updated
in SIMULTANEOUS_ASSIGNMENT there is no need to switch value items
to new nullable copies of table Field's - they must refer to old
values in the row, which can never be null anyway.
skipping this redundant step simplifies moving field to record[1]
and back in fill_record()
if ha_partition::position() is asked for a position of a closed partition,
don't ask the underlying engine, just set the partition number.
in fact, the partition is open and can be perfectly used, the assert
is over-zealous. but in the future it might be actually closed.
Comparison between vector and scalar is invalid (ER_OPERAND_COLUMNS)
and handled by the parser. The problem is outer_context is missing
because relink_hack() cannot recover it due to
!builtin_select.first_inner_unit() condition. This condition was set
by previous relink hack called for previous expression some(select 1).
Since there can be arbitrary number of such expressions there seems to
be no point in such a limitation. MTR test do not fail without that
condition, so the fix proposes to remove it.
Pure aliases are not handled properly by Item_func_nextval::val_int().
add_table_to_list() does not create MDL request for pure aliases,
i.e. when there is no table_list->db set or TL_OPTION_ALIAS was
set. When the expression is not inside CTE the case with empty db is
handled by:
else if (!lex->with_cte_resolution && lex->copy_db_to(&db))
DBUG_RETURN(0);
So, table_list gets current database name and the query is failed with
ER_NO_SUCH_TABLE error.
The fix adds the case of is_pure_alias() for
Item_func_nextval::val_int() and fails it with ER_NOT_SEQUENCE2 error.
Note: semantics for TL_OPTION_ALIAS cannot be based on empty db, only
parser can set TL_OPTION_ALIAS as resolve_references_to_cte() relies
on TL_OPTION_ALIAS after copy_db_to().
1. Fix empty part_elem->id in prep_alter_part_table().
On auto-create newly added partition has id 0. It came from
set_up_default_partitions() for new part_info
(thd->work_part_info). vers_update_el_ids() can work only with
unassigned ids (UINT_MAX32), so we assign it explicitly on pushing
into tab_part_info.
2. If range value is out of TIMESTAMP_MAX_VALUE set it to
TIMESTAMP_MAX_VALUE, but only if the history partition is the last
one, otherwise push ER_DATA_OUT_OF_RANGE. Error is to create
multiple out-of-range partitions (e.g. with PARTITIONS clause in
CREATE TABLE).
default_used was missing as view is parsed on its own
lex. extend_table_list() decides maybe_need_prelocking based on
default_used and prelocking_strategy->handle_table() was skipped for
view, so internal_tables was not updated (they could be stale from
previous statement).
SELECT 1 union select 2 UNION SELECT 1 from a JOIN a b ON
(SELECT 1 FROM dual WHERE AAA)
Crashes during fix_outer_field while resolving field item AAA
In our resolver, once we have determined that a field item isn't
local to our select, we call Item::fix_outer_field(), which
iterates outwards towards the top level select, looking for where
our Item_field might be resolvable.
In our example here, the item isn't resolvable and we expose
fragility in the loop, which i will detail here.
After we initialize the variable 'outer_context' (to a context
containing /* select#3 */ select 1 AS `1` from (a join a b on
((subquery#4))) ) we enter a loop
│ 5927 for (;
│ 5928 outer_context;
│ 5929 outer_context= outer_context->outer_context)
│ 5930 {
│ 5931 select= outer_context->select_lex;
│ 5932 Item_subselect *prev_subselect_item=
│ 5933 last_checked_context->select_lex->master_unit()->item;
│ 5934 last_checked_context= outer_context;
here 'last_checked_context' is the context inner to the current
'outer_context', and we initialize prev_subselect_item to the
Item enclosing the unit containing this inner select.
So for the first iteration of the loop,
select: select #3
last_checked_context: from select #4 to select #3.
prev_subselect_item: item enclosing select #4 (where
field item AAA is defined)
The rest of the loop calls find_field_in_tables() /
resolve_ref_in_select_and_group() in an attempt to
resolve this item with this 'outer_context'.
After the item fails resolution, we move to an outer context
select: select #4294967295 (fake_select_lex)
last_checked_context: from select #3 to the fake select lex
containing the union (i.e. outermost)
prev_subselect_item: null, there is no Item that contains this,
it is the outermost select.
We still need to execute the rest of the loop to determine whether
AAA is resolvable here, but executing
│ 5937 place= prev_subselect_item->parsing_place;
We are now following a null pointer. We introduce a test for this
null pointer, indicating that we are now evaluating the outermost
select and we are not to try accessing the enclosing subselect item.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
When the optimizer_max_sel_arg_weight is set to 1, a nested query
crashed while tracing.
SEL_ARG object has a field named 'field', that is not set when the
type is other than KEY_RANGE. But, the field was accessed to store
its name, and weight to the trace. This resulted in a crash due to NULL
pointer.
Added a check to access field if the type is KEY_RANGE, and if not, just
trace the type.
Rename `Item::clone_item()` to `clone_constant()`, and do
the same for any overloads in descendant items.
The function returns non-NULL only for items that represent
constant literals.
Rename cloning methods of class Item and its descendants
in the following way:
(from) (to)
do_build_clone -> deep_copy
build_clone -> deep_copy_with_checks
do_get_copy -> shallow_copy
get_copy -> shallow_copy_with_checks
to better reflect their functionality.
Also make Item::deep_copy() and shallow_copy() protected.
Outside users should call deep_copy_with_checks()
and shallow_copy_with_checks().
SHOW EXPLAIN FOR, and EXPLAIN/DESC FOR CONNECTION should behave
identically. However, for a query with an addition expression containing
INTERVAL and NOT IN sub-select SHOW EXPLAIN FOR was correctly throwing
parse error, where as EXPLAIN/DESC FOR CONNECTION was crashing.
The reason for the crash is that select block was not initialized and
was being accessed inside the NOT IN sub-select in the EXPLAIN/DESC FOR
CONNECTION case.
Item_singerow_subselect::fix_length_and_dec() incorrectly calculated
its Item::max_length when the underlying expression was Item_int.
The reason of the problem:
Item_int has an optimized max_length to make CONCAT(1) create a VARCHAR(1)
column rather than a VARCHAR(2) column. Its max_length does not include one
extra character for the sign in case the value is positive but the value
is not marked as Item::unsigned==true.
So copying max_length from the underlying Item_int (with value==9)
in cases like this:
SELECT CONCAT((SELECT 9 FROM t0));
was not correct.
Implementing a new virtual method
Type_handler::Item_type_std_attributes_generic(const Item *item)
- The default implementation just copies attributes from "item" as is.
- In case of Type_handler_int_result it evaluates max_length
using item->decimal_precision() rather than item->max_length.
This works correctly for both "optimized" items like Item_int and
non-"optimized" Items whose max_length includes +1 for the sign
in case of signed expressions.
According to SQL standard, rows from `INFORMATION_SCHEMA.TRIGGERS` table
should be visible to users with non-SELECT privileges on the columns.
`ACTION_CONDITION`, `ACTION_STATEMENT` and `DEFINER` columns should be
visible only if the user is the owner of the schema.
MariaDB uses `TRIGGER` privilege instead of owner, which controls the
visibilty of all columns, including those which only need non-SELECT
privileges.
This fix
- Allows users with non-SELECT privileges- INSERT, DELETE or UPDATE,
to see rows in `INFORMATION_SCHEMA.TRIGGERS` table.
- Ensure `ACTION_CONDITION`, `ACTION_STATEMENT` and `DEFINER` columns
are `NULL` unless the user is the owner of the schema or has `TRIGGER`
privilege.
When GRANT EXECUTE ON PROCEDURE fails on the master, it will
erroneously be replicated and executed successfully on the slave.
This both breaks replication and is a security violation.
The underlying issue is that a failed GRANT EXECUTE ON PROCEDURE will
still be replicated when sql_mode does not have NO_AUTO_CREATE_USER.
This is because the function mysql_routine_grant() does not check if an
error occured while performing the GRANT before binlogging, it simply
always binlogs.
This patch fixes this problem by checking if an error happened
previously before binlogging, and if so, then skip binlogging.
Note there is still a broader issue in this area leading to replication
divergence. Reported in MDEV-29848, a partially-completed GRANT
statment (where some earlier GRANTS succeed and a later fails) will not
binlog. Note this affects all grant types, whereas the issue addressed
in this patch is limited to GRANT EXECUTE ON PROCEDURE. This patch
makes GRANT EXECUTE ON PROCEDURE binlogging behavior consistent with
the other grant types. A separate follow-up patch will address the
broader MDEV-29848 issue.
Also note that a test case in rpl_do_grant.test took advantage of
MDEV-38506 so a partially-failing REVOKE EXECUTE ON PROCEDURE would
still replicate. This test case is disabled with a TODO note to
re-enable it once MDEV-29848 is fixed
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
MDEV-29300 fix causes a wrong result by incorrectly removing a wrapper to
an item that needed to be wrapped for the correct result. Direct access
to the item causes an incorrect table reference to be used during
join evaluation. We revert that fix.
Our original problem query is this
SELECT (SELECT 0 GROUP BY c1 HAVING (SELECT c1)) FROM t1 group by c1;
JOIN::prepare on
/* select#2 */ select 0 group by t1.c1 having (subquery#3)
fixing t1.c1 in group by clause, calls fix_outer_field()
this item is resolved in an outer select (#1) and it is a grouping select,
so we wrap it in Item_outer_ref and set this item to unfixed for later
fixing in fix_inner_refs().
JOIN::prepare continues onto the having clause and fixes (subquery#3) which
calls initiates the prepare series of calls, leading to setup_fields on the
fields in this JOIN, one of which is an outer reference c1.
This is resolved to the item in the next most outer select in the group by
clause. This item has been wrapped with an unfixed Item_outer_ref.
It is found in resolve_ref_in_select_and_group() is it expected that
this item will have already been fixed, hence this call in
Item_field::fix_outer_field()
DBUG_ASSERT(*ref && (*ref)->fixed());
but as explained above, it isn't fixed and debug builds assert here.
Because this wrapper cannot be resolved here for reasons detailed in
fix_inner_refs, and we cannot remove this wrapper without potentially
returning an incorrect result, we have to relax this assertion.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
When MariaDB node has joined to MySQL Galera cluster if any writes
to existing tables are done cluster requests to use protocol
version 7 (max protocol version used by MySQL Galera cluster).
MariaDB had support to protocol version up to 4. While writeset
keys are prepared allow using protocol version 7, it has no
effect on writeset key format i.e. protocol version 4 and 7
are identical on that case.
Note that there is no mtr test case because currently there is no
way to test both MySQL and MariaDB in mtr.
Problem:
When building a pushable condition that can be pushed from HAVING into
WHERE, the server tries to clone aggregate functions. This is not
necessary because aggregate functions can not be pushed into WHERE
anyway.
Fix:
This fix introduces a check within `Item::build_pushable_cond` to skip
cloning aggregate functions.
Also fixes assert failure in MDEV-38492, by adding a missing copy method
for `Item_aggregate_ref`.
Problem was in wsrep_handle_mdl_conflict function was comparing
thd->lex->sql_command variable for granted MDL-lock.
There is two possible schedules:
(1) FLUSH TABLES ... FOR EXPORT that will take MDL-lock (granted_thd).
INSERT from other node is conflicting operation (request_thd)
and sees MDL-conflict. Because granted_thd has not executed anything
else thd->lex->sql_command == SQLCOM_FLUSH and this case was
correctly handled in wsrep_handle_mdl_conflict i.e. INSERT needs
to wait.
(2) FLUSH TABLES ... FOR EXPORT that will take MDL-lock (granted_thd).
SET SESSION wsrep_sync_wait=0; (granted_thd)
INSERT from other node is conflicting operation (request_thd)
However, thd->lex->sql_command is not stored to taken MDL-lock. Now
as granted_thd is executing SET thd->lex->sql_command != SQLCOM_FLUSH
and INSERT that is BF will abort it and that means also FTFE is
killed and MDL-lock relesed. This is incorrect as FTFE has written
file on filesystem and it can't be really killed.
In this fix wsrep_handle_mdl_conflict is refactored not to use
thd->lex->sql_command as a variable used for decisions. Instead
connection state can be determined also via THD members. E.g.:
* wsrep_thd_is_toi() || wsrep_thd_is_applying - ongoing TOI or applier
* wsrep_thd_is_BF - thread is brute force
* wsrep_thd_is_SR - thread is streaming replication thread
* thd->current_backup_stage != BACKUP_FINISHED - there's ongoing BACKUP
* thd->global_read_lock.is_acquired() - ongoing FTWRL
* thd->locked_tables_mode == LTM_LOCK_TABLES - ongoing FTFE or LOCK TABLES
Analysis:
When scanning json and reaching a path, the path type is
JSON_PATH_ARRAY_WILD. So it should not return any value. But the code
does not check that.
Fix:
Use path_setup_nwc() to check for range queries because json_value() should
not accept range queries.
The initial scanning of the GTID pos table in find_gtid_pos_tables_cb() used
the wrong call to commit the transaction, ha_commit_trans(thd, TRUE) instead
of trans_commit(thd). This could cause an assertion
Assertion '!thd->in_active_multi_stmt_transaction() || thd->in_multi_stmt_transaction_mode()' failed in mysql_execute_command
and possibly other issues due to incorrect state in the SQL thread THD.
Similar problem in rpl_slave_state::truncate_state_table() also fixed.
Note there is a sister patch cd88b0831f
which applied this same pattern which fixed a hang due to locks
not being released in the ha_commit_trans(thd, TRUE) call.
Test-case-by: Alice Sherepa <alice.sherepa@gmail.com>
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
number-to-time conversion was too eagerly capping the value.
A string "9000090" was invalid time, because of 90 seconds.
But number-to-time was capping first, validating later,
to 9000090->time worked. Let's fix it.
also, let's make invalid time values in a string field include
the field name in the warning message, just like invalid time
values in a numeric field do.
