Commit graph

196337 commits

Author SHA1 Message Date
Denis Protivensky
b7718a1c1c MDEV-32738: Don't roll back high-prio txn waiting on a lock in InnoDB
DML transactions on FK-child tables also get table locks
on FK-parent tables. If there is a DML transaction holding
such a lock, and a TOI transaction starts, the latter
BF-aborts the former and puts itself into a waiting state.
If at this moment another DML transaction on FK-child table
starts, it doesn't check that the transaction waiting on
a parent table lock is TOI, and it erroneously BF-aborts
the waiting TOI transaction.

The fix: don't roll back high-priority transaction waiting
on a lock in InnoDB, instead roll back an incoming DML
transaction.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-07-08 23:36:21 +02:00
Brandon Nesterenko
744580d5a7 MDEV-32892: IO Thread Reports False Error When Stopped During Connecting to Primary
The IO thread can report error code 2013 into the error log when it
is stopped during the initial connection process to the primary, as
well as when trying to read an event. However, because the IO thread
is being stopped, its connection to the primary is force-killed by
the signaling thread (see THD::awake_no_mutex()), and thereby these
connection errors should be ignored.

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>
2024-07-08 10:39:17 -06:00
Alexander Barkov
d1e5fa8917 MDEV-34305 Redundant truncation errors/warnings with optimizer_trace enabled
my_like_range*() can create longer keys than Field::char_length().
This caused warnings during print_range().

Fix:

Suppressing warnings in print_range().
2024-07-08 18:01:01 +04:00
Anson Chung
df35072c8d Refactor GitLab cppcheck and update SAST ignorelists
Line numbers had to be removed from the ignorelists in order to be
diffed against since locations of the same findings can differ
across runs. Therefore preprocessing has to be done on the CI findings
so that it can be compared to the ignorelist and new findings can be
outputted. However, since line numbers have to be removed, a situation
occurs where it is difficult to reference the location of findings
in code given the output of the CI job.

To lessen this pain, change the cppcheck template to include
code snippets which make it easier to reference where in the code
the finding is referring to, even in the absence of line numbers.
Ignorelisting works as before since locations of the finding may
change but not the code it is referring to.

Furthermore, due to the innate difficulty in maintaining ignorelists
across branches and triaging new findings, allow failure as to not
have constantly failing pipelines as a result of a new findings that
have not been addressed yet.

Lastly, update SAST ignorelists to match the newly refactored cppcheck
job and the current state of the codebase.

All new code of the whole pull request, including one or several
files that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my
employer Amazon Web Services, Inc.
2024-07-08 10:51:48 +01:00
Anson Chung
215fab68db Perform simple fixes for cppcheck findings
Rectify cases of mismatched brackets and address
possible cases of division by zero by checking if
the denominator is zero before dividing.

No functional changes were made.

All new code of the whole pull request, including one or several
files that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my
employer Amazon Web Services, Inc.
2024-07-08 10:51:48 +01:00
Marko Mäkelä
72ceae7314 MDEV-34510: UBSAN: overflow on adding an unsigned offset
crc32_avx512(): Explicitly cast ssize_t(size) to make it clear that
we are indeed applying a negative offset to a pointer.
2024-07-08 18:17:29 +10:00
Monty
33964984ad MDEV-34522 Index for (specific) Aria table is created as corrupted
The issue was that when repairing an Aria table of row format PAGE and
the data file was bigger the 4G, the data file length was cut short
because of wrong parameters to MY_ALIGN().

The effect was that ALTER TABLE, OPTIMIZE TABLE or REPAIR TABLE would fail
on these tables, possibly corrupting them.
The MDEV also exposed a bug where error state was not propagated properly
to the upper level if the number of rows in the table changed.
2024-07-07 12:43:44 +03:00
Brandon Nesterenko
cbc1898e82 MDEV-25607: Auto-generated DELETE from HEAP table can break replication
The special logic used by the memory storage engine
to keep slaves in sync with the master on a restart can
break replication. In particular, after a restart, the
master writes DELETE statements in the binlog for
each MEMORY-based table so the slave can empty its
data. If the DELETE is not executable, e.g. due to
invalid triggers, the slave will error and fail, whereas
the master will never see the problem.

Instead of DELETE statements, use TRUNCATE to
keep slaves in-sync with the master, thereby bypassing
triggers.

Reviewed By:
===========
Kristian Nielsen <knielsen@knielsen-hq.org>
Andrei Elkin <andrei.elkin@mariadb.com>
2024-07-05 12:00:09 -06:00
Thirunarayanan Balathandayuthapani
834c013b64 MDEV-34519 innodb_log_checkpoint_now crashes when innodb_read_only is enabled
During read only mode, InnoDB doesn't allow checkpoint to happen.
So InnoDB should throw the warning when InnoDB tries to
force the checkpoint when innodb_read_only = 1 or
innodb_force_recovery = 6.
2024-07-05 15:26:05 +05:30
Hugo Wen
9e8546e2bd Fix a stack overflow in pinbox allocator
MariaDB supports a "wait-free concurrent allocator based on pinning addresses".
In `lf_pinbox_real_free()` it tries to sort the pinned addresses for better
performance to use binary search during "real free". `alloca()` was used to
allocate stack memory and copy addresses.

To prevent a stack overflow when allocating the stack memory the function checks
if there's enough stack space. However, the available stack size was calculated
inaccurately which eventually caused database crash due to stack overflow.

The crash was seen on MariaDB 10.6.11 but the same code defect exists on all
MariaDB versions.

A similar issue happened previously and the fix in fc2c1e43 was to add a
`ALLOCA_SAFETY_MARGIN` which is 8192 bytes. However, that safety margin is not
enough during high connection workloads.

MySQL also had a similar issue and the fix
https://github.com/mysql/mysql-server/commit/b086fda was to remove the use of
`alloca` and replace qsort approach by a linear scan through all pointers (pins)
owned by each thread.

This commit is mostly the same as it is the only way to solve this issue as:
1. Frame sizes in different architecture can be different.
2. Number of active (non-null) pinned addresses varies, so the frame
   size for the recursive sorting function `msort_with_tmp` is also hard
   to predict.
3. Allocating big memory blocks in stack doesn't seem to be a very good
   practice.

For further details see the mentioned commit in MySQL and the inline comments.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2024-07-05 13:26:37 +10:00
Galina Shalygina
6cb896a639 MDEV-29363: Constant subquery causing a crash in pushdown optimization
The crash is caused by the attempt to refix the constant subquery during
pushdown from HAVING into WHERE optimization.

Every condition that is going to be pushed into WHERE clause is first
cleaned up, then refixed. Constant subqueries are not cleaned or refixed
because they will remain the same after refixing, so this complicated
procedure can be omitted for them (introduced in MDEV-21184).
Constant subqueries are marked with flag IMMUTABLE_FL, that helps to miss
the cleanup stage for them. Also they are marked as fixed, so refixing is
also not done for them.
Because of the multiple equality propagation several references to the same
constant subquery can exist in the condition that is going to be pushed
into WHERE. Before this patch, the problem appeared in the following way.
After the first reference to the constant subquery is processed, the flag
IMMUTABLE_FL for the constant subquery is disabled.
So, when the second reference to this constant subquery is processed, the
flag is already disabled and the subquery goes through the procedure of
cleaning and refixing. That causes a crash.

To solve this problem, IMMUTABLE_FL should be disabled only after all
references to the constant subquery are processed, so after the whole
condition that is going to be pushed is cleaned up and refixed.

Approved by Igor Babaev <igor@maridb.com>
2024-07-04 13:46:19 +02:00
Oleksandr Byelkin
a4ef05d0d5 Fix compiler errors 2024-07-03 12:45:30 +02:00
Daniel Black
25c6e3e4b7 MDEV-34502 InnoDB debug mode build - asserts with Valgrind
Valgrind looks as the assertions as examining uninitalized values.

As the assertions are tested in other Debug builds we know
it isn't all invalid.

Account for Valgrind by removing the assertion under
the WITH_VALGRIND=1 compulation.
2024-07-03 20:07:04 +10:00
Monty
2739b5f5f8 MDEV-34494 Add server_uid global variable and add it to error log at startup
The feedback plugin server_uid variable and the calculate_server_uid()
function is moved from feedback/utils.cc to sql/mysqld.cc

server_uid is added as a global variable (shown in 'show variables') and
is written to the error log on server startup together with server version
and server commit id.
2024-07-02 11:26:13 +03:00
Monty
d8c9c5ead6 MDEV-34491 Setting log_slow_admin="" at startup should be converted to log_slow_admin=ALL
We have an issue if a user have the following in a configuration file:
log_slow_filter=""                  # Log everything to slow query log
log_queries_not_using_indexes=ON

This set log_slow_filter to 'not_using_index' which disables
slow_query_logging of most queries.
In effect, on should never use log_slow_filter="" in config files but
instead use log_slow_filter=ALL.

Fixed by changing log_slow_filter="" that comes either from a
configuration file or from the command line, when starting to the server,
to log_slow_filter=ALL.
A warning will be printed when this happens.

Other things:
- One can now use =ALL for any 'set' variable to set all options at once.
  (backported from 10.6)
2024-07-02 11:26:13 +03:00
Lena Startseva
090cecd5e8 MDEV-31933: Make working view-protocol + ps-protocol (running two protocols together)
Fix for v. 10.5
2024-07-02 10:11:33 +07:00
Alexander Barkov
d046b13e7b MDEV-20548 Unexpected error on CREATE..SELECT HEX(num)
Item_func_hex::fix_length_and_dec() evaluated a too short data type
for signed numeric arguments, which resulted in a 'Data too long for column'
error on CREATE..SELECT.

Fixing the code to take into account that a short negative
numer can produce a long HEX value: -1  -> 'FFFFFFFFFFFFFFFF'

Also fixing Item_func_hex::val_str_ascii_from_val_real().
Without this change, MTR test with HEX with negative float point arguments
failed on some platforms (aarch64, ppc64le, s390-x).
2024-07-01 18:50:32 +04:00
Daniel Black
e7b76f87c4 MDEV-34437 restrict port and extra-port to tcp valid values
extra_port and port are 16 bit numbers and not 32 bit as they are
tcp ports.

Restrict their value.
2024-07-01 17:43:12 +10:00
Daniel Black
54b7a879e9 MDEV-34313 WITHOUT_SERVER/WSREP postfix
Mismatched IF/ENDIF statements in cmake
caused a warning.
2024-06-29 10:24:09 +10:00
Lena Startseva
9e74a7f4f3 Removing MDEV-27871 from tastcases because it is not a bug 2024-06-28 16:45:50 +07:00
Meng-Hsiu Chiang
55db59f16d [MDEV-28162] Replace PFS_atomic with std::atomic<T>
PFS_atomic class contains wrappers around my_atomic_* operations, which
are macros to GNU atomic operations (__atomic_*). Due to different
implementations of compilers, clang may encounter errors when compiling
on x86_32 architecture.

The following functions are replaced with C++ std::atomic type in
performance schema code base:
  - PFS_atomic::store_*()
      -> my_atomic_store*
        -> __atomic_store_n()
    => std::atomic<T>::store()

  - PFS_atomic::load_*()
      -> my_atomic_load*
        -> __atomic_load_n()
    => std::atomic<T>::load()

  - PFS_atomic::add_*()
      -> my_atomic_add*
        -> __atomic_fetch_add()
    => std::atomic<T>::fetch_add()

  - PFS_atomic::cas_*()
    -> my_atomic_cas*
      -> __atomic_compare_exchange_n()
    => std::atomic<T>::compare_exchange_strong()

and PFS_atomic class could be dropped completely.

Note that in the wrapper memory order passed to original GNU atomic
extensions are hard-coded as `__ATOMIC_SEQ_CST`, which is equivalent to
`std::memory_order_seq_cst` in C++, and is the default parameter for
std::atomic_* functions.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services.
2024-06-27 09:27:12 +03:00
Daniel Black
d5bad49011 MDEV-34313: compile WITHOUT_SERVER and WSREP=ON
CMake WSREP=ON has some implications for client
executables so still present this as an option
when compiling WITHOUT_SERVER. In this case
default to ON for maximium compatibility of
the build client executables and libraries.
2024-06-26 12:42:25 +10:00
Daniel Black
53a4867837 MDEV-34313: compiler mariadb-binlog WITHOUT_SERVER
The log_event_old.cc is included by mysqlbinlog.cc.

With -DWITHOUT_SERVER the include path for the wsrep
include headers isn't there.

As these aren't needed by the mariadb-binlog, move
these to under a ifndef MYSQL_CLIENT preprocessor.

Caused by MDEV-18590
2024-06-26 10:21:58 +10:00
Yuchen Pei
ad0ee8cdb9
MDEV-33746 [fixup] Add suggested overrides in oqgraph 2024-06-25 14:22:25 +08:00
Yuchen Pei
01289dac41
MDEV-34361 Split my.cnf in the spider suite.
Just like the spider/bugfix suite.

One caveat is that my_2_3.cnf needs something under mysqld.2.3 group,
otherwise mtr will fail with something like:

There is no group named 'mysqld.2.3' that can be used to resolve
'port' for ...

This will allow new tests under the spider suite to use what is
needed. It also somehow fixes issues of running a test followed by
spider.slave_trx_isolation.
2024-06-25 13:45:04 +08:00
Yuchen Pei
aebd2397cc
MDEV-34404 Use safe_str in spider udfs to avoid passing NULL str 2024-06-25 13:45:04 +08:00
Dmitry Shulga
8b169949d6 MDEV-24411: Trigger doesn't work correctly with bulk insert
Executing an INSERT statement in PS mode having positional parameter
bound with an array could result in incorrect number of inserted rows
in case there is a BEFORE INSERT trigger that executes yet another
INSERT statement to put a copy of row being inserted into some table.

The reason for incorrect number of inserted rows is that a data structure
used for binding positional argument with its actual values is stored
in THD (this is thd->bulk_param) and reused on processing every INSERT
statement. It leads to consuming actual values bound with top-level
INSERT statement by other INSERT statements used by triggers' body.

To fix the issue, reset the thd->bulk_param temporary to the value nullptr
before invoking triggers and restore its value on finishing its execution.
2024-06-25 09:52:52 +07:00
Rex
d513a4ce74 MDEV-19520 Extend condition normalization to include 'NOT a'
Having Item_func_not items in item trees breaks assumptions during the
optimization phase about transformation possibilities in fix_fields().
Remove Item_func_not by extending normalization during parsing.

Reviewed by Oleksandr Byelkin (sanja@mariadb.com)
2024-06-25 04:51:29 +11:00
Marko Mäkelä
d9dd673fee MDEV-12008 fixup: Do not add a new error code
New error codes can only be added in the latest major version.
Adding ER_KILL_DENIED_HIGH_PRIORITY would shift by one all
error codes that were added in MariaDB Server 10.6 or later.

This amends commit 1001dae186

Suggested by: Sergei Golubchik
2024-06-24 12:08:13 +03:00
Marko Mäkelä
acc077ffa1 MDEV-34443 ha_innobase::info_low() does not distinguish HA_STATUS_VARIABLE_EXTRA
ha_innobase::info_low(): For HA_STATUS_VARIABLE without
HA_STATUS_VARIABLE_EXTRA, let us avoid unnecessary and costly updates
of the data_free statistics, which are only needed for SHOW TABLE STATUS.
This optimization had been enabled in
commit 247ecb7597 but not utilized until now.
2024-06-24 10:39:13 +03:00
Christian Hesse
3d2e54ff8c MDEV-20053: update systemd unit
Now that we have proper paths in `galera_recovery` changing the
directory is no longer required.
2024-06-22 17:19:49 +10:00
Christian Hesse
105473233d MDEV-20053: set @sbindir@ for scripts
The variable `sbindir` is never set for cmake. This adds borked paths to
`galera_recovery`, though it dit not break as the systemd unit changes
the dir to make the relative path work anyway.

Let's fix this nevertheless...
2024-06-22 17:19:49 +10:00
Rex
9e800eda86 MDEV-32583 UUID() should be treated as stochastic for the purposes of forcing query materialization
RAND() and UUID() are treated differently with respect to subquery
materialization both should be marked as uncacheable, forcing materialization.
Altered Create_func_uuid(_short)::create_builder().
Added comment in header about UNCACHEABLE_RAND meaning also unmergeable.
2024-06-22 13:26:49 +11:00
Thirunarayanan Balathandayuthapani
5979dcf95b MDEV-34435 Increase code coverage for debug_dbug test case during startup
- Few of test case should make sure that InnoDB does hit
the debug sync point during startup of the server.
InnoDB can remove the double quotes of debug point
in restart parameters.
2024-06-21 17:24:29 +05:30
Dave Gosselin
db0c28eff8 MDEV-33746 Supply missing override markings
Find and fix missing virtual override markings.  Updates cmake
maintainer flags to include -Wsuggest-override and
-Winconsistent-missing-override.
2024-06-20 11:32:13 -04:00
Thirunarayanan Balathandayuthapani
ab448d4b34 MDEV-34389 Avoid log overwrite in early recovery
- InnoDB tries to write FILE_CHECKPOINT marker during
early recovery when log file size is insufficient.
While updating the log checkpoint at the end of the recovery,
InnoDB must already have written out all pending changes
to the persistent files. To complete the checkpoint, InnoDB
has to write some log records for the checkpoint and to
update the checkpoint header. If the server gets killed
before updating the checkpoint header then it would lead
the logfile to be unrecoverable.

- This patch avoids FILE_CHECKPOINT marker during early
recovery and narrows down the window of opportunity to
make the log file unrecoverable.
2024-06-20 17:54:57 +05:30
Alexander Barkov
6cecf61a59 MDEV-34417 Wrong result set with utf8mb4_danish_ci and BNLH join
There were erroneous calls for charpos() in key_hashnr() and key_buf_cmp().
These functions are never called with prefix segments.

The charpos() calls were wrong. Before the change BNHL joins
- could return wrong result sets, as reported in MDEV-34417
- were extremely slow for multi-byte character sets, because
  the hash was calculated on string prefixes, which increased
  the amount of collisions drastically.

This patch fixes the wrong result set as reported in MDEV-34417,
as well as (partially) the performance problem reported in MDEV-34352.
2024-06-20 11:30:02 +04:00
Julius Goryavsky
2f0e7f665c galera: syncing SST scripts code with the following versions 2024-06-19 14:07:34 +02:00
Jan Lindström
1001dae186 MDEV-12008 : Change error code for Galera unkillable threads
Changed error code for Galera unkillable threads to
be ER_KILL_DENIED_HIGH_PRIORITY giving message

This is a high priority thread/query and cannot be killed
without the compromising consistency of the cluster

also a warning is produced
  Thread %lld is [wsrep applier|high priority] and cannot be killed

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-19 14:07:34 +02:00
Alexander Barkov
cfa6143453 MDEV-27966 Assertion fixed()' failed and Assertion fixed == 1' failed, both in Item_func_concat::val_str on SELECT after INSERT with collation utf32_bin on utf8_bin table
This problem was earlier fixed by this commit:

> commit 08c7ab404f
> Author: Aleksey Midenkov <midenok@gmail.com>
> Date:   Mon Apr 18 12:44:27 2022 +0300
>
>    MDEV-24176 Server crashes after insert in the table with virtual
>    column generated using date_format() and if()

Adding an mtr test only.
2024-06-19 10:01:30 +04:00
Brandon Nesterenko
6cab2f75fe MDEV-23857: replication master password length
After MDEV-4013, the maximum length of replication passwords was extended to
96 ASCII characters. After a restart, however, slaves only read the first 41
characters of MASTER_PASSWORD from the master.info file. This lead to slaves
unable to reconnect to the master after a restart.

After a slave restart, if a master.info file is detected, use the full
allowable length of the password rather than 41 characters.

Reviewed By:
============
Sergei Golubchik <serg@mariadb.com>
2024-06-18 07:21:18 -06:00
Brandon Nesterenko
0e25cc51a9 MDEV-34397: "delete si" rather than "my_free(si)" in THD::register_slave()
In the error case of THD::register_slave(), there is undefined
behavior of Slave_info si because it is allocated via malloc()
(my_malloc), and cleaned up via delete().

This patch makes these consistent by switching si's cleanup
to use my_free.
2024-06-18 07:20:41 -06:00
Souradeep Saha
10fbd1ce51 MDEV-34168: Extend perror utility to print link to KB page
As all MariaDB Server errors now have a dedicated web page, the
perror utility is extended to include a link to the KB page of
the corresponding error code.

All new code of the whole pull request, including one or several
files that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my
employer Amazon Web Services, Inc.
2024-06-18 13:25:39 +10:00
Sergei Petrunia
2eda310b15 Restore test coverage for MDEV-18956
(It was accidentally removed by fix for MDEV-28846)
2024-06-17 14:08:32 +03:00
Sergei Petrunia
0903276eae MDEV-30651: Assertion `sel->quick' in make_range_rowid_filters, followup
Review followup: RANGE_OPT_PARAM statement_should_be_aborted()
checks for thd->is_fatal_error and thd->is_error(). The first is
redundant when the second is present.
2024-06-17 14:08:32 +03:00
Sergei Petrunia
a2066b2400 MDEV-30651: Assertion `sel->quick' in make_range_rowid_filters
The optimizer deals with Rowid Filters this way:

1. First, range optimizer is invoked. It saves information
   about all potential range accesses.
2. A query plan is chosen. Suppose, it uses a Rowid Filter on
   index $IDX.
3. JOIN::make_range_rowid_filters() calls the range optimizer
again to create a quick select on index $IDX which will be used
to populate the rowid filter.

The problem: KILL command catches the query in step #3. Quick
Select is not created which causes a crash.

Fixed by checking if query was killed. Note: the problem also
affects 10.6, even if error handling for
SQL_SELECT::test_quick_select is different there.
2024-06-17 14:08:32 +03:00
Sergei Petrunia
b47bd3f8bf MDEV-33875: ORDER BY DESC causes ROWID Filter slowdown
Rowid Filter cannot be used with reverse-ordered scans, for the
same reason as IndexConditionPushdown cannot be.

test_if_skip_sort_order() already has logic to disable ICP when
setting up a reverse-ordered scan. Added logic to also disable
Rowid Filter in this case, factored out the code into
prepare_for_reverse_ordered_access(), and added a comment describing
the cause of this limitation.
2024-06-17 09:50:32 +03:00
Monty
956bcf8f49 Change mysqldump to use DO instead of 'SELECT' for storing sequences.
This avoids a lot of SETVAL() results when applying a mysqldump with
sequences.
2024-06-16 10:51:33 +03:00
Monty
fef32fd9ad MDEV-34406 Enhance mariadb_upgrade to print failing query in case of error
To make this possible, it was also necessary to enhance the mariadb
client with the option --print-query-on-error.
This option can also be very useful when running a batch of queries
through the mariadb client and one wants to find out where things goes
wrong.

TODO: It would be good to enhance mariadb_upgrade to not call the mariadb
client for executing queries but instead do this internally.  This
would have made this patch much easier!

Reviewed by: Sergei Golubchik <serg@mariadb.com>
2024-06-16 10:51:33 +03:00
Marko Mäkelä
4b4c371fe7 MDEV-34297 fixup: -Wconversion on 32-bit 2024-06-14 13:21:19 +03:00