The bug was that thd->lex->limit_rows_examined_cnt was not reset after
queries. It is reset in lex_start() at the start of the next query
execution.
This causes general_log_write(), which is called before lex_start(), to
exceed the limit. The effect is a crash or the next query would not be
executed.
Fixed by resetting limit_rows_examined_cnt at end of query.
In case if the view mysql.user was created (e.g. in 10.6) with a
pre- 18edb0959f server with an unexpected
character_set_client or collation_connection, e.g.
utf8mb3 and utf8mb3_general_ci, mysql_upgrade did not fix it to the
expected latin1 and latin1_swedish_ci.
Since 11.8 this could often lead to "Illegax mix of collations" errors
when querying mysql.user, because since 11.8 the default collation for
utf8mb3 is utf8mb3_uca1400_ai_ci, according to the default
@@character_set_collations. For consistency, it's better to fix the
problem starting from 11.4.
For example:
MariaDB [test]> select user,host,is_role from
-> mysql.user where is_role='N';
ERROR 1267 (HY000): Illegal mix of collations (utf8mb3_general_ci,COERCIBLE)
and (utf8mb3_uca1400_ai_ci,COERCIBLE) for operation '='
Fixing mariadb_system_tables_fix.sql to drop the view if it has non-standard
character set or collations, so it gets recreated again correctly by
the CREATE statement in mariadb_system_tables.sql
Fix rpl suite tests added by MDEV-25039.
rpl_foreign_key_lock_table_insert.test is removed altogether because it
is unclear what the purpose of the test is. The changes of the patch
were done on the slave, yet all operations in the test were done on the
master. Nothing different could happen on the slave because it is
configured to be serial, so all transactions would run sequentially
anyway, and no validations were performed.
rpl_foreign_key_ddl_insert.test was renamed to
rpl_row_foreign_key_mdl.test and the test itself was re-written to be
a minimal test case to ensure that MDL locking behavior is different
pre- and post- patch. A few problems with the original test:
* No foreign-key locking was done on the slave because the table
engine was not InnoDB.
* rpl_fk_ddl.inc had inconsistent validation checking. I.e., the child
query validation checks were done on the master (which is incorrect)
and because the slave was configured to be serial, the two
transactions could not run concurrently on the slave anyway.
Issue:
On galera write node INSERT statements does not acquire MDL locks on it's all child
tables and thereby wsrep certification keys are also added for limited tables, but
on applier nodes it does acquire MDL locks for all child tables. This can result
into MDL BF-BF conflict on applier node when transactions referring to parent and
child tables are executed concurrently. For example:
Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4
Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4
On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.
On applier nodes Write_rows event is implicitly a REPLACE, deleting all conflicting
rows which can cause cascading FK actions and locks on foreign key children tables.
Solution:
For Galera applier nodes the Write_rows event is considered pure INSERT
which will never cause cascading FK actions and locks on foreign key children tables.
The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.
Other things:
- Fixed some (not critical) memory leaks in mariabackup
The Port field in the system table mysql.servers has type INT,
which translates to Field_long.
During parsing it is parsed as ulong_num, and in this patch we add
bound checks there.
In find_field_in_view(), we call field_it.create_item() which
creates item on a statement mem_root.
Then we set its name. Make sure the name is allocated on a statement
mem_root, too.
Run-time has semantics duplication in unireg_check, default_value and
flags, so all three must be in sync before FRM creation. Special
unireg_check values for temporal field types was introduced by
32b28f9298 WL#1266 "Separate auto-set logic from TIMESTAMP type."
Each ORDER and WHERE slot may generate split, see code like this:
if ((item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM) ||
item->with_window_func())
item->split_sum_func(thd, ref_ptrs, all_fields, SPLIT_SUM_SELECT);
Such kind of code is done in JOIN::prepare(), setup_order(),
setup_fields(), setup_group() and split_sum_func2() itself.
Since we are at the phase of ref_ptrs allocation, items are not fixed
yet and we cannot calculate precisely how much ref_ptrs is needed. We
can estimate at most how much is needed. In the worst case each window
function generates split on each ORDER BY field, GROUP BY field and
WHERE field, so the counts of these should be multiplied by window
funcs count.
As the split can be done in both setup_without_group() and
JOIN::prepare() simultaneously, the factor of window funcs should be
multiplied by 2.
The similar case may be with inner sumfunc items as of the condition
item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM
but factor of these is harder to predict at the stage of unfixed
items.
ft_handler isn't getting initialized for subqueries inside explain
delete/update queries. However, ft_handler is accessed inside ha_ft_read(),
and is the reason for NULL pointer exception.
This is not the case with non-explain delete/update queries, as
well as explain/non-explain select queries.
Follow the approach the SELECT statements are using in
JOIN::optimize_constant_subqueries(): remove SELECT_DESCRIBE
flag when invoking optimization of constant subqueries.
Single-table UPDATE/DELETEs have SELECT_LEX but don't have JOIN.
So, we make optimize_constant_subqueries() not to be a member
of JOIN class, and instead move it to SELECT_LEX, and then
invoke it from single-table UPDATE/DELETE as well as for SELECT queries.
Reason:
======
During InnoDB DDL, statistics updation fails due to lock wait
timeout and calls push_warning_printf() to generate warnings
but then returns success, causing the SQL layer
to attempt calling set_ok_status() when the diagnostics area
is already set.
Solution:
=========
By temporarily setting abort_on_warning to false around operations
that prevents warning to error escalation and restore the original
setting after calling HA_EXTRA_END_ALTER_COPY for alter operation.
of multi-table-styled DELETE from a view
Analysis:
The item_list of builtin_select stores the fields that are there in the
RETURNING clause.
During the "EXECUTE" command, a "dummy item" is added into the item_list
of the select_lex(builtin_select) representing DELETE during
Sql_cmd_delete::precheck(). This snippet that adds a dummy item is added
because columnstore needs for temporary table. Results are put into a
temporary table and to create a temporary table we need to know what
columns are there which we get from the select_lex->item_list.
As a result, the item_list now has an item even when there is not really
RETURNING clause, resulting in execution of the setup_returning_fields()
when it should have exited already.
Fix:
Instead of checking whether builint_select's item_list is empty to
determine whether there is RETURNING clause, use a flag.
Index merge and rowid filter should not be used together, however,
even if index merge is not chosen earlier in best_access_path, it may
be chosen again in make_join_select, inside ref_to_range. Therefore
this patch ensures that rowid filter is not used when index merge is
chosen there.
some I_S tables require "any non-SELECT privilege on the table".
If only SELECT was granted on the global level and something non-SELECT
on the schema level, then we need to check schema level privileges
explicitly, because check_grant() doesn't do that and get_all_tables()
doesn't look deeper if SELECT is present on the global level.
in SIMULTANEOUS_ASSIGNMENT there is no need to switch value items
to new nullable copies of table Field's - they must refer to old
values in the row, which can never be null anyway.
skipping this redundant step simplifies moving field to record[1]
and back in fill_record()
if ha_partition::position() is asked for a position of a closed partition,
don't ask the underlying engine, just set the partition number.
in fact, the partition is open and can be perfectly used, the assert
is over-zealous. but in the future it might be actually closed.
Test was affected by incompletely closed preceding connections.
Wait for preceding connections to decrement Threads_connected
before testing ER_CON_COUNT_ERROR condition.
Test output was affected by incompletely closed preceding connections.
Wait for connections to leave I_S.PROCESSLIST before issuing
SHOW PROCESSLIST.
Also fixes similar failures in funcs_1.processlist_val_ps.
There was a missing NULL element terminator for --system's type
library definition.
This was causing a crash in find_type_eol when e.g. an incomplete
value was passed to --system where it keeps iterating until it
finds the NULL as a typelib element.
Fixed by appending a NullS to the definition.
Test case added.
Comparison between vector and scalar is invalid (ER_OPERAND_COLUMNS)
and handled by the parser. The problem is outer_context is missing
because relink_hack() cannot recover it due to
!builtin_select.first_inner_unit() condition. This condition was set
by previous relink hack called for previous expression some(select 1).
Since there can be arbitrary number of such expressions there seems to
be no point in such a limitation. MTR test do not fail without that
condition, so the fix proposes to remove it.
Pure aliases are not handled properly by Item_func_nextval::val_int().
add_table_to_list() does not create MDL request for pure aliases,
i.e. when there is no table_list->db set or TL_OPTION_ALIAS was
set. When the expression is not inside CTE the case with empty db is
handled by:
else if (!lex->with_cte_resolution && lex->copy_db_to(&db))
DBUG_RETURN(0);
So, table_list gets current database name and the query is failed with
ER_NO_SUCH_TABLE error.
The fix adds the case of is_pure_alias() for
Item_func_nextval::val_int() and fails it with ER_NOT_SEQUENCE2 error.
Note: semantics for TL_OPTION_ALIAS cannot be based on empty db, only
parser can set TL_OPTION_ALIAS as resolve_references_to_cte() relies
on TL_OPTION_ALIAS after copy_db_to().
1. Fix empty part_elem->id in prep_alter_part_table().
On auto-create newly added partition has id 0. It came from
set_up_default_partitions() for new part_info
(thd->work_part_info). vers_update_el_ids() can work only with
unassigned ids (UINT_MAX32), so we assign it explicitly on pushing
into tab_part_info.
2. If range value is out of TIMESTAMP_MAX_VALUE set it to
TIMESTAMP_MAX_VALUE, but only if the history partition is the last
one, otherwise push ER_DATA_OUT_OF_RANGE. Error is to create
multiple out-of-range partitions (e.g. with PARTITIONS clause in
CREATE TABLE).
default_used was missing as view is parsed on its own
lex. extend_table_list() decides maybe_need_prelocking based on
default_used and prelocking_strategy->handle_table() was skipped for
view, so internal_tables was not updated (they could be stale from
previous statement).
SELECT 1 union select 2 UNION SELECT 1 from a JOIN a b ON
(SELECT 1 FROM dual WHERE AAA)
Crashes during fix_outer_field while resolving field item AAA
In our resolver, once we have determined that a field item isn't
local to our select, we call Item::fix_outer_field(), which
iterates outwards towards the top level select, looking for where
our Item_field might be resolvable.
In our example here, the item isn't resolvable and we expose
fragility in the loop, which i will detail here.
After we initialize the variable 'outer_context' (to a context
containing /* select#3 */ select 1 AS `1` from (a join a b on
((subquery#4))) ) we enter a loop
│ 5927 for (;
│ 5928 outer_context;
│ 5929 outer_context= outer_context->outer_context)
│ 5930 {
│ 5931 select= outer_context->select_lex;
│ 5932 Item_subselect *prev_subselect_item=
│ 5933 last_checked_context->select_lex->master_unit()->item;
│ 5934 last_checked_context= outer_context;
here 'last_checked_context' is the context inner to the current
'outer_context', and we initialize prev_subselect_item to the
Item enclosing the unit containing this inner select.
So for the first iteration of the loop,
select: select #3
last_checked_context: from select #4 to select #3.
prev_subselect_item: item enclosing select #4 (where
field item AAA is defined)
The rest of the loop calls find_field_in_tables() /
resolve_ref_in_select_and_group() in an attempt to
resolve this item with this 'outer_context'.
After the item fails resolution, we move to an outer context
select: select #4294967295 (fake_select_lex)
last_checked_context: from select #3 to the fake select lex
containing the union (i.e. outermost)
prev_subselect_item: null, there is no Item that contains this,
it is the outermost select.
We still need to execute the rest of the loop to determine whether
AAA is resolvable here, but executing
│ 5937 place= prev_subselect_item->parsing_place;
We are now following a null pointer. We introduce a test for this
null pointer, indicating that we are now evaluating the outermost
select and we are not to try accessing the enclosing subselect item.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
When the optimizer_max_sel_arg_weight is set to 1, a nested query
crashed while tracing.
SEL_ARG object has a field named 'field', that is not set when the
type is other than KEY_RANGE. But, the field was accessed to store
its name, and weight to the trace. This resulted in a crash due to NULL
pointer.
Added a check to access field if the type is KEY_RANGE, and if not, just
trace the type.
SHOW EXPLAIN FOR, and EXPLAIN/DESC FOR CONNECTION should behave
identically. However, for a query with an addition expression containing
INTERVAL and NOT IN sub-select SHOW EXPLAIN FOR was correctly throwing
parse error, where as EXPLAIN/DESC FOR CONNECTION was crashing.
The reason for the crash is that select block was not initialized and
was being accessed inside the NOT IN sub-select in the EXPLAIN/DESC FOR
CONNECTION case.
Item_singerow_subselect::fix_length_and_dec() incorrectly calculated
its Item::max_length when the underlying expression was Item_int.
The reason of the problem:
Item_int has an optimized max_length to make CONCAT(1) create a VARCHAR(1)
column rather than a VARCHAR(2) column. Its max_length does not include one
extra character for the sign in case the value is positive but the value
is not marked as Item::unsigned==true.
So copying max_length from the underlying Item_int (with value==9)
in cases like this:
SELECT CONCAT((SELECT 9 FROM t0));
was not correct.
Implementing a new virtual method
Type_handler::Item_type_std_attributes_generic(const Item *item)
- The default implementation just copies attributes from "item" as is.
- In case of Type_handler_int_result it evaluates max_length
using item->decimal_precision() rather than item->max_length.
This works correctly for both "optimized" items like Item_int and
non-"optimized" Items whose max_length includes +1 for the sign
in case of signed expressions.
According to SQL standard, rows from `INFORMATION_SCHEMA.TRIGGERS` table
should be visible to users with non-SELECT privileges on the columns.
`ACTION_CONDITION`, `ACTION_STATEMENT` and `DEFINER` columns should be
visible only if the user is the owner of the schema.
MariaDB uses `TRIGGER` privilege instead of owner, which controls the
visibilty of all columns, including those which only need non-SELECT
privileges.
This fix
- Allows users with non-SELECT privileges- INSERT, DELETE or UPDATE,
to see rows in `INFORMATION_SCHEMA.TRIGGERS` table.
- Ensure `ACTION_CONDITION`, `ACTION_STATEMENT` and `DEFINER` columns
are `NULL` unless the user is the owner of the schema or has `TRIGGER`
privilege.
When GRANT EXECUTE ON PROCEDURE fails on the master, it will
erroneously be replicated and executed successfully on the slave.
This both breaks replication and is a security violation.
The underlying issue is that a failed GRANT EXECUTE ON PROCEDURE will
still be replicated when sql_mode does not have NO_AUTO_CREATE_USER.
This is because the function mysql_routine_grant() does not check if an
error occured while performing the GRANT before binlogging, it simply
always binlogs.
This patch fixes this problem by checking if an error happened
previously before binlogging, and if so, then skip binlogging.
Note there is still a broader issue in this area leading to replication
divergence. Reported in MDEV-29848, a partially-completed GRANT
statment (where some earlier GRANTS succeed and a later fails) will not
binlog. Note this affects all grant types, whereas the issue addressed
in this patch is limited to GRANT EXECUTE ON PROCEDURE. This patch
makes GRANT EXECUTE ON PROCEDURE binlogging behavior consistent with
the other grant types. A separate follow-up patch will address the
broader MDEV-29848 issue.
Also note that a test case in rpl_do_grant.test took advantage of
MDEV-38506 so a partially-failing REVOKE EXECUTE ON PROCEDURE would
still replicate. This test case is disabled with a TODO note to
re-enable it once MDEV-29848 is fixed
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
MDEV-29300 fix causes a wrong result by incorrectly removing a wrapper to
an item that needed to be wrapped for the correct result. Direct access
to the item causes an incorrect table reference to be used during
join evaluation. We revert that fix.
Our original problem query is this
SELECT (SELECT 0 GROUP BY c1 HAVING (SELECT c1)) FROM t1 group by c1;
JOIN::prepare on
/* select#2 */ select 0 group by t1.c1 having (subquery#3)
fixing t1.c1 in group by clause, calls fix_outer_field()
this item is resolved in an outer select (#1) and it is a grouping select,
so we wrap it in Item_outer_ref and set this item to unfixed for later
fixing in fix_inner_refs().
JOIN::prepare continues onto the having clause and fixes (subquery#3) which
calls initiates the prepare series of calls, leading to setup_fields on the
fields in this JOIN, one of which is an outer reference c1.
This is resolved to the item in the next most outer select in the group by
clause. This item has been wrapped with an unfixed Item_outer_ref.
It is found in resolve_ref_in_select_and_group() is it expected that
this item will have already been fixed, hence this call in
Item_field::fix_outer_field()
DBUG_ASSERT(*ref && (*ref)->fixed());
but as explained above, it isn't fixed and debug builds assert here.
Because this wrapper cannot be resolved here for reasons detailed in
fix_inner_refs, and we cannot remove this wrapper without potentially
returning an incorrect result, we have to relax this assertion.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
mysqltest had limited scripting capabilities, requiring complex
workarounds for mathematical calculations and string manipulations
in test cases. This commit solves these limitations by adding a new
`$(...)` syntax that enables direct evaluation of mathematical, logical,
and string expressions within test scripts.
Expression Evaluation (MDEV-36107):
- Recursive descent parser supporting arithmetic, logical, comparison,
and bitwise operators with proper precedence
- Support for integers (decimal, hex, binary), booleans, strings, and
NULL values
- Variable substitution within expressions
- Integration with existing mysqltest control flow
String Functions (MDEV-36108):
- Base conversion functions supporting bases 2-62
- String manipulation and processing functions
- Regular expression functions
- Conditional and numeric utility functions
The implementation enhances mysqltest's scripting capabilities while
maintaining full backward compatibility.
Test was affected by incompletely closed preceding connections.
Make test agnostic to concurrent connections by querying
InnoDB status only for connections that it uses.
This is an addition to 3b2169f0d1, which didn't handle a case when
preceding test has active transaction on disconnect.
