log_mmap(): If the MAP_SYNC|MAP_SHARED_VALIDATE operation (PMEM)
failed and the path is not in /dev/shm (which we treat as PMEM),
proceed to try regular MAP_SHARED read-only mapping. This allows
somewhat more efficient crash recovery, basically with an I/O
buffer that is not limited by innodb_log_buffer_size.
Reviewed by: Thirunarayanan Balathandayuthapani
Fix rpl suite tests added by MDEV-25039.
rpl_foreign_key_lock_table_insert.test is removed altogether because it
is unclear what the purpose of the test is. The changes of the patch
were done on the slave, yet all operations in the test were done on the
master. Nothing different could happen on the slave because it is
configured to be serial, so all transactions would run sequentially
anyway, and no validations were performed.
rpl_foreign_key_ddl_insert.test was renamed to
rpl_row_foreign_key_mdl.test and the test itself was re-written to be
a minimal test case to ensure that MDL locking behavior is different
pre- and post- patch. A few problems with the original test:
* No foreign-key locking was done on the slave because the table
engine was not InnoDB.
* rpl_fk_ddl.inc had inconsistent validation checking. I.e., the child
query validation checks were done on the master (which is incorrect)
and because the slave was configured to be serial, the two
transactions could not run concurrently on the slave anyway.
Issue:
On galera write node INSERT statements does not acquire MDL locks on it's all child
tables and thereby wsrep certification keys are also added for limited tables, but
on applier nodes it does acquire MDL locks for all child tables. This can result
into MDL BF-BF conflict on applier node when transactions referring to parent and
child tables are executed concurrently. For example:
Tables with foreign keys: t1<-t2<-t3<-t4
Conflicting transactions: INSERT t1 and DROP TABLE t4
Wsrep certification keys taken on write node:
- for INSERT t1: t1 and t2
- for DROP TABLE t4: t4
On applier node MDL BF-BF conflict happened between two transaction because
MDL locks on t1, t2, t3 and t4 were taken for INSERT t1, which conflicted
with MDL lock on t4 taken by DROP TABLE t4.
The Wsrep certification keys helps in resolving this MDL BF-BF conflict by
prioritizing and scheduling concurrent transactions. But to generate Wsrep
certification keys it needs to open and take MDL locks on all the child tables.
On applier nodes Write_rows event is implicitly a REPLACE, deleting all conflicting
rows which can cause cascading FK actions and locks on foreign key children tables.
Solution:
For Galera applier nodes the Write_rows event is considered pure INSERT
which will never cause cascading FK actions and locks on foreign key children tables.
Skip an all-zero pages in the index file.
They can happen normally if the ma_checkpoint_background
thread flushes some later page first (e.g. page 50 before page 48).
Also:
* don't do alloca() in a loop
* correct the check in ma_crypt_index_post_read_hook(),
the page can be completely full
* compilation failure in ma_open.c:1289:
comparison is always false due to limited range of data type
The backup of encrypted Aria tables was not supported.
Added support for this. One complication is that the page checksum is
for the not encrypted page. To be able to verify the checksum I have to
temporarly decrypt the page.
In the backup we store the encrypted pages.
Other things:
- Fixed some (not critical) memory leaks in mariabackup
The design of "binlog group commit" involves carrying some state across
transaction boundaries. This includes trx_t::commit_lsn, which keeps track
of how much write-ahead log needs to be written. Unfortunately, this
field was not reset in a commit where a log write was elided. That would
cause an unnecessary wait in a subsequent read-only transaction that
happened to reuse the same transaction object.
trx_deregister_from_2pc(): Reset trx->commit_lsn so that
an earlier write that was executed in the same client connection
will not result in an unnecessary wait during a subsequent read
operation.
trx_commit_complete_for_mysql(): Unless we are inside a binlog
group commit, reset trx->commit_lsn.
unlock_and_close_files(): Reset trx->commit_lsn after durably
writing the log, and remove a redundant log write call from some
callers.
trx_t::rollback_finish(): Clear commit_lsn, because a rolled-back
transaction will not need to be durably written.
trx_t::clear_and_free(): Wrapper function to suppress a debug check
in trx_t::free().
Also, remove some redundant ut_ad(!trx->will_lock) that will be checked
in trx_t::free().
Reviewed by: Vladislav Vaintroub
The Port field in the system table mysql.servers has type INT,
which translates to Field_long.
During parsing it is parsed as ulong_num, and in this patch we add
bound checks there.
In find_field_in_view(), we call field_it.create_item() which
creates item on a statement mem_root.
Then we set its name. Make sure the name is allocated on a statement
mem_root, too.
Run-time has semantics duplication in unireg_check, default_value and
flags, so all three must be in sync before FRM creation. Special
unireg_check values for temporal field types was introduced by
32b28f9298 WL#1266 "Separate auto-set logic from TIMESTAMP type."
Each ORDER and WHERE slot may generate split, see code like this:
if ((item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM) ||
item->with_window_func())
item->split_sum_func(thd, ref_ptrs, all_fields, SPLIT_SUM_SELECT);
Such kind of code is done in JOIN::prepare(), setup_order(),
setup_fields(), setup_group() and split_sum_func2() itself.
Since we are at the phase of ref_ptrs allocation, items are not fixed
yet and we cannot calculate precisely how much ref_ptrs is needed. We
can estimate at most how much is needed. In the worst case each window
function generates split on each ORDER BY field, GROUP BY field and
WHERE field, so the counts of these should be multiplied by window
funcs count.
As the split can be done in both setup_without_group() and
JOIN::prepare() simultaneously, the factor of window funcs should be
multiplied by 2.
The similar case may be with inner sumfunc items as of the condition
item->with_sum_func() && item->type() != Item::SUM_FUNC_ITEM
but factor of these is harder to predict at the stage of unfixed
items.
ft_handler isn't getting initialized for subqueries inside explain
delete/update queries. However, ft_handler is accessed inside ha_ft_read(),
and is the reason for NULL pointer exception.
This is not the case with non-explain delete/update queries, as
well as explain/non-explain select queries.
Follow the approach the SELECT statements are using in
JOIN::optimize_constant_subqueries(): remove SELECT_DESCRIBE
flag when invoking optimization of constant subqueries.
Single-table UPDATE/DELETEs have SELECT_LEX but don't have JOIN.
So, we make optimize_constant_subqueries() not to be a member
of JOIN class, and instead move it to SELECT_LEX, and then
invoke it from single-table UPDATE/DELETE as well as for SELECT queries.
Reason:
======
During InnoDB DDL, statistics updation fails due to lock wait
timeout and calls push_warning_printf() to generate warnings
but then returns success, causing the SQL layer
to attempt calling set_ok_status() when the diagnostics area
is already set.
Solution:
=========
By temporarily setting abort_on_warning to false around operations
that prevents warning to error escalation and restore the original
setting after calling HA_EXTRA_END_ALTER_COPY for alter operation.
some I_S tables require "any non-SELECT privilege on the table".
If only SELECT was granted on the global level and something non-SELECT
on the schema level, then we need to check schema level privileges
explicitly, because check_grant() doesn't do that and get_all_tables()
doesn't look deeper if SELECT is present on the global level.
relax the assert, allowing '\n' at the end if the string is exactly
MYSQL_ERRMSG_SIZE-1 bytes long. It likely doesn't end with '\n' but
was truncated at the middle.
also, use MYSQL_ERRMSG_SIZE in my_error.c not a separate define
that must be "kept in sync"
* fail acl_load() if it was killed, this will cause all privileges to
be reset to their original pre-load values.
* only increment grant_version if privileges were, in fact, updated
in SIMULTANEOUS_ASSIGNMENT there is no need to switch value items
to new nullable copies of table Field's - they must refer to old
values in the row, which can never be null anyway.
skipping this redundant step simplifies moving field to record[1]
and back in fill_record()
if ha_partition::position() is asked for a position of a closed partition,
don't ask the underlying engine, just set the partition number.
in fact, the partition is open and can be perfectly used, the assert
is over-zealous. but in the future it might be actually closed.
Clang upstream bug https://github.com/llvm/llvm-project/issues/173210
shows that a "int x=x" construct will in the intermediate representation
have code to read from x. With this generated sanitizer=undefined and
static analyzers will all see the uninitialized read and write.
Because clang has a stronger implementation of following paths to
uninitialized variables, "#define UNINIT_VAR(x) x" is the which
is what our release binaries use is the ideal path for this compiler.
Clang based compilers with error during compilation if any
uninitialized behaviour is detected at compile time because of
0c80ddb519.
Corrects MDEV-36542 - 6fd57f478f.
Test was affected by incompletely closed preceding connections.
Wait for preceding connections to decrement Threads_connected
before testing ER_CON_COUNT_ERROR condition.
Test output was affected by incompletely closed preceding connections.
Wait for connections to leave I_S.PROCESSLIST before issuing
SHOW PROCESSLIST.
Also fixes similar failures in funcs_1.processlist_val_ps.
There was a missing NULL element terminator for --system's type
library definition.
This was causing a crash in find_type_eol when e.g. an incomplete
value was passed to --system where it keeps iterating until it
finds the NULL as a typelib element.
Fixed by appending a NullS to the definition.
Test case added.
Comparison between vector and scalar is invalid (ER_OPERAND_COLUMNS)
and handled by the parser. The problem is outer_context is missing
because relink_hack() cannot recover it due to
!builtin_select.first_inner_unit() condition. This condition was set
by previous relink hack called for previous expression some(select 1).
Since there can be arbitrary number of such expressions there seems to
be no point in such a limitation. MTR test do not fail without that
condition, so the fix proposes to remove it.
Pure aliases are not handled properly by Item_func_nextval::val_int().
add_table_to_list() does not create MDL request for pure aliases,
i.e. when there is no table_list->db set or TL_OPTION_ALIAS was
set. When the expression is not inside CTE the case with empty db is
handled by:
else if (!lex->with_cte_resolution && lex->copy_db_to(&db))
DBUG_RETURN(0);
So, table_list gets current database name and the query is failed with
ER_NO_SUCH_TABLE error.
The fix adds the case of is_pure_alias() for
Item_func_nextval::val_int() and fails it with ER_NOT_SEQUENCE2 error.
Note: semantics for TL_OPTION_ALIAS cannot be based on empty db, only
parser can set TL_OPTION_ALIAS as resolve_references_to_cte() relies
on TL_OPTION_ALIAS after copy_db_to().
1. Fix empty part_elem->id in prep_alter_part_table().
On auto-create newly added partition has id 0. It came from
set_up_default_partitions() for new part_info
(thd->work_part_info). vers_update_el_ids() can work only with
unassigned ids (UINT_MAX32), so we assign it explicitly on pushing
into tab_part_info.
2. If range value is out of TIMESTAMP_MAX_VALUE set it to
TIMESTAMP_MAX_VALUE, but only if the history partition is the last
one, otherwise push ER_DATA_OUT_OF_RANGE. Error is to create
multiple out-of-range partitions (e.g. with PARTITIONS clause in
CREATE TABLE).
default_used was missing as view is parsed on its own
lex. extend_table_list() decides maybe_need_prelocking based on
default_used and prelocking_strategy->handle_table() was skipped for
view, so internal_tables was not updated (they could be stale from
previous statement).
SELECT 1 union select 2 UNION SELECT 1 from a JOIN a b ON
(SELECT 1 FROM dual WHERE AAA)
Crashes during fix_outer_field while resolving field item AAA
In our resolver, once we have determined that a field item isn't
local to our select, we call Item::fix_outer_field(), which
iterates outwards towards the top level select, looking for where
our Item_field might be resolvable.
In our example here, the item isn't resolvable and we expose
fragility in the loop, which i will detail here.
After we initialize the variable 'outer_context' (to a context
containing /* select#3 */ select 1 AS `1` from (a join a b on
((subquery#4))) ) we enter a loop
│ 5927 for (;
│ 5928 outer_context;
│ 5929 outer_context= outer_context->outer_context)
│ 5930 {
│ 5931 select= outer_context->select_lex;
│ 5932 Item_subselect *prev_subselect_item=
│ 5933 last_checked_context->select_lex->master_unit()->item;
│ 5934 last_checked_context= outer_context;
here 'last_checked_context' is the context inner to the current
'outer_context', and we initialize prev_subselect_item to the
Item enclosing the unit containing this inner select.
So for the first iteration of the loop,
select: select #3
last_checked_context: from select #4 to select #3.
prev_subselect_item: item enclosing select #4 (where
field item AAA is defined)
The rest of the loop calls find_field_in_tables() /
resolve_ref_in_select_and_group() in an attempt to
resolve this item with this 'outer_context'.
After the item fails resolution, we move to an outer context
select: select #4294967295 (fake_select_lex)
last_checked_context: from select #3 to the fake select lex
containing the union (i.e. outermost)
prev_subselect_item: null, there is no Item that contains this,
it is the outermost select.
We still need to execute the rest of the loop to determine whether
AAA is resolvable here, but executing
│ 5937 place= prev_subselect_item->parsing_place;
We are now following a null pointer. We introduce a test for this
null pointer, indicating that we are now evaluating the outermost
select and we are not to try accessing the enclosing subselect item.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
Rename `Item::clone_item()` to `clone_constant()`, and do
the same for any overloads in descendant items.
The function returns non-NULL only for items that represent
constant literals.
Rename cloning methods of class Item and its descendants
in the following way:
(from) (to)
do_build_clone -> deep_copy
build_clone -> deep_copy_with_checks
do_get_copy -> shallow_copy
get_copy -> shallow_copy_with_checks
to better reflect their functionality.
Also make Item::deep_copy() and shallow_copy() protected.
Outside users should call deep_copy_with_checks()
and shallow_copy_with_checks().
SHOW EXPLAIN FOR, and EXPLAIN/DESC FOR CONNECTION should behave
identically. However, for a query with an addition expression containing
INTERVAL and NOT IN sub-select SHOW EXPLAIN FOR was correctly throwing
parse error, where as EXPLAIN/DESC FOR CONNECTION was crashing.
The reason for the crash is that select block was not initialized and
was being accessed inside the NOT IN sub-select in the EXPLAIN/DESC FOR
CONNECTION case.
Item_singerow_subselect::fix_length_and_dec() incorrectly calculated
its Item::max_length when the underlying expression was Item_int.
The reason of the problem:
Item_int has an optimized max_length to make CONCAT(1) create a VARCHAR(1)
column rather than a VARCHAR(2) column. Its max_length does not include one
extra character for the sign in case the value is positive but the value
is not marked as Item::unsigned==true.
So copying max_length from the underlying Item_int (with value==9)
in cases like this:
SELECT CONCAT((SELECT 9 FROM t0));
was not correct.
Implementing a new virtual method
Type_handler::Item_type_std_attributes_generic(const Item *item)
- The default implementation just copies attributes from "item" as is.
- In case of Type_handler_int_result it evaluates max_length
using item->decimal_precision() rather than item->max_length.
This works correctly for both "optimized" items like Item_int and
non-"optimized" Items whose max_length includes +1 for the sign
in case of signed expressions.
According to SQL standard, rows from `INFORMATION_SCHEMA.TRIGGERS` table
should be visible to users with non-SELECT privileges on the columns.
`ACTION_CONDITION`, `ACTION_STATEMENT` and `DEFINER` columns should be
visible only if the user is the owner of the schema.
MariaDB uses `TRIGGER` privilege instead of owner, which controls the
visibilty of all columns, including those which only need non-SELECT
privileges.
This fix
- Allows users with non-SELECT privileges- INSERT, DELETE or UPDATE,
to see rows in `INFORMATION_SCHEMA.TRIGGERS` table.
- Ensure `ACTION_CONDITION`, `ACTION_STATEMENT` and `DEFINER` columns
are `NULL` unless the user is the owner of the schema or has `TRIGGER`
privilege.
When GRANT EXECUTE ON PROCEDURE fails on the master, it will
erroneously be replicated and executed successfully on the slave.
This both breaks replication and is a security violation.
The underlying issue is that a failed GRANT EXECUTE ON PROCEDURE will
still be replicated when sql_mode does not have NO_AUTO_CREATE_USER.
This is because the function mysql_routine_grant() does not check if an
error occured while performing the GRANT before binlogging, it simply
always binlogs.
This patch fixes this problem by checking if an error happened
previously before binlogging, and if so, then skip binlogging.
Note there is still a broader issue in this area leading to replication
divergence. Reported in MDEV-29848, a partially-completed GRANT
statment (where some earlier GRANTS succeed and a later fails) will not
binlog. Note this affects all grant types, whereas the issue addressed
in this patch is limited to GRANT EXECUTE ON PROCEDURE. This patch
makes GRANT EXECUTE ON PROCEDURE binlogging behavior consistent with
the other grant types. A separate follow-up patch will address the
broader MDEV-29848 issue.
Also note that a test case in rpl_do_grant.test took advantage of
MDEV-38506 so a partially-failing REVOKE EXECUTE ON PROCEDURE would
still replicate. This test case is disabled with a TODO note to
re-enable it once MDEV-29848 is fixed
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
MDEV-29300 fix causes a wrong result by incorrectly removing a wrapper to
an item that needed to be wrapped for the correct result. Direct access
to the item causes an incorrect table reference to be used during
join evaluation. We revert that fix.
Our original problem query is this
SELECT (SELECT 0 GROUP BY c1 HAVING (SELECT c1)) FROM t1 group by c1;
JOIN::prepare on
/* select#2 */ select 0 group by t1.c1 having (subquery#3)
fixing t1.c1 in group by clause, calls fix_outer_field()
this item is resolved in an outer select (#1) and it is a grouping select,
so we wrap it in Item_outer_ref and set this item to unfixed for later
fixing in fix_inner_refs().
JOIN::prepare continues onto the having clause and fixes (subquery#3) which
calls initiates the prepare series of calls, leading to setup_fields on the
fields in this JOIN, one of which is an outer reference c1.
This is resolved to the item in the next most outer select in the group by
clause. This item has been wrapped with an unfixed Item_outer_ref.
It is found in resolve_ref_in_select_and_group() is it expected that
this item will have already been fixed, hence this call in
Item_field::fix_outer_field()
DBUG_ASSERT(*ref && (*ref)->fixed());
but as explained above, it isn't fixed and debug builds assert here.
Because this wrapper cannot be resolved here for reasons detailed in
fix_inner_refs, and we cannot remove this wrapper without potentially
returning an incorrect result, we have to relax this assertion.
Approved by: Oleksandr "Sanja" Byelkin (sanja@mariadb.com)
mysqltest had limited scripting capabilities, requiring complex
workarounds for mathematical calculations and string manipulations
in test cases. This commit solves these limitations by adding a new
`$(...)` syntax that enables direct evaluation of mathematical, logical,
and string expressions within test scripts.
Expression Evaluation (MDEV-36107):
- Recursive descent parser supporting arithmetic, logical, comparison,
and bitwise operators with proper precedence
- Support for integers (decimal, hex, binary), booleans, strings, and
NULL values
- Variable substitution within expressions
- Integration with existing mysqltest control flow
String Functions (MDEV-36108):
- Base conversion functions supporting bases 2-62
- String manipulation and processing functions
- Regular expression functions
- Conditional and numeric utility functions
The implementation enhances mysqltest's scripting capabilities while
maintaining full backward compatibility.
