Commit graph

78657 commits

Author SHA1 Message Date
Lena Startseva
9e74a7f4f3 Removing MDEV-27871 from tastcases because it is not a bug 2024-06-28 16:45:50 +07:00
Marko Mäkelä
4ca355d863 MDEV-33894: Resurrect innodb_log_write_ahead_size
As part of commit 685d958e38 (MDEV-14425)
the parameter innodb_log_write_ahead_size was removed, because it was
thought that determining the physical block size would be a sufficient
replacement.

However, we can only determine the physical block size on Linux or
Microsoft Windows. On some file systems, the physical block size
is not relevant. For example, XFS uses a block size of 4096 bytes
even if the underlying block size may be smaller.

On Linux, we failed to determine the physical block size if
innodb_log_file_buffered=OFF was not requested or possible.
This will be fixed.

log_sys.write_size: The value of the reintroduced parameter
innodb_log_write_ahead_size. To keep it simple, this is read-only
and a power of two between 512 and 4096 bytes, so that the previous
alignment guarantees are fulfilled. This will replace the previous
log_sys.get_block_size().

log_sys.block_size, log_t::get_block_size(): Remove.

log_t::set_block_size(): Ensure that write_size will not be less
than the physical block size. There is no point to invoke this
function with 512 or less, because that is the minimum value of
write_size.

innodb_params_adjust(): Add some disabled code for adjusting
the minimum value and default value of innodb_log_write_ahead_size
to reflect the log_sys.write_size.

log_t::set_recovered(): Mark the recovery completed. This is the
place to adjust some things if we want to allow write_size>4096.

log_t::resize_write_buf(): Refer to write_size.

log_t::resize_start(): Refer to write_size instead of get_block_size().

log_write_buf(): Simplify some arithmetics and remove a goto.

log_t::write_buf(): Refer to write_size. If we are writing less than
that, do not switch buffers, but keep writing to the same buffer.
Move some code to improve the locality of reference.

recv_scan_log(): Refer to write_size instead of get_block_size().

os_file_create_func(): For type==OS_LOG_FILE on Linux, always invoke
os_file_log_maybe_unbuffered(), so that log_sys.set_block_size() will
be invoked even if we are not attempting to use O_DIRECT.

recv_sys_t::find_checkpoint(): Read the entire log header
in a single 12 KiB request into log_sys.buf.

Tested with:
./mtr --loose-innodb-log-write-ahead-size=4096
./mtr --loose-innodb-log-write-ahead-size=2048
2024-06-27 16:38:08 +03:00
Marko Mäkelä
27a3366663 Merge 10.6 into 10.11 2024-06-27 10:26:09 +03:00
Yuchen Pei
d7042ec4da
Merge branch '10.5' into 10.6 2024-06-26 09:16:54 +08:00
Rex
d513a4ce74 MDEV-19520 Extend condition normalization to include 'NOT a'
Having Item_func_not items in item trees breaks assumptions during the
optimization phase about transformation possibilities in fix_fields().
Remove Item_func_not by extending normalization during parsing.

Reviewed by Oleksandr Byelkin (sanja@mariadb.com)
2024-06-25 04:51:29 +11:00
Marko Mäkelä
0076eb3d4e Merge 10.5 into 10.6 2024-06-24 13:09:47 +03:00
Marko Mäkelä
d9dd673fee MDEV-12008 fixup: Do not add a new error code
New error codes can only be added in the latest major version.
Adding ER_KILL_DENIED_HIGH_PRIORITY would shift by one all
error codes that were added in MariaDB Server 10.6 or later.

This amends commit 1001dae186

Suggested by: Sergei Golubchik
2024-06-24 12:08:13 +03:00
Rex
9e800eda86 MDEV-32583 UUID() should be treated as stochastic for the purposes of forcing query materialization
RAND() and UUID() are treated differently with respect to subquery
materialization both should be marked as uncacheable, forcing materialization.
Altered Create_func_uuid(_short)::create_builder().
Added comment in header about UNCACHEABLE_RAND meaning also unmergeable.
2024-06-22 13:26:49 +11:00
Thirunarayanan Balathandayuthapani
5979dcf95b MDEV-34435 Increase code coverage for debug_dbug test case during startup
- Few of test case should make sure that InnoDB does hit
the debug sync point during startup of the server.
InnoDB can remove the double quotes of debug point
in restart parameters.
2024-06-21 17:24:29 +05:30
Sergei Petrunia
49b4a6e26d Fix the testcase for MDEV-31558: log_slow_verbosity_innodb
log_slow_innodb.test sets "log_slow_verbosity_innodb_expected_matches"

in include/log_slow_grep.inc:
- Fix a typo: take value from log_slow_*VERBOSITY*_innodb_expected_matches
- Add a missing "--source include/log_grep.inc"
- two-line search patterns do not work, search for each line individually.
2024-06-21 10:53:40 +03:00
Vlad Lesin
0a199cb810 MDEV-34108 Inappropriate semi-consistent read in RC if innodb_snapshot_isolation=ON
The fixes in b8a6719889 have not disabled
semi-consistent read for innodb_snapshot_isolation=ON mode, they just allowed
read uncommitted version of a record, that's why the test for MDEV-26643 worked
well.

The semi-consistent read should be disabled on upper level in
row_search_mvcc() for READ COMMITTED isolation level.

Reviewed by Marko Mäkelä.
2024-06-20 16:11:54 +03:00
Thirunarayanan Balathandayuthapani
ab448d4b34 MDEV-34389 Avoid log overwrite in early recovery
- InnoDB tries to write FILE_CHECKPOINT marker during
early recovery when log file size is insufficient.
While updating the log checkpoint at the end of the recovery,
InnoDB must already have written out all pending changes
to the persistent files. To complete the checkpoint, InnoDB
has to write some log records for the checkpoint and to
update the checkpoint header. If the server gets killed
before updating the checkpoint header then it would lead
the logfile to be unrecoverable.

- This patch avoids FILE_CHECKPOINT marker during early
recovery and narrows down the window of opportunity to
make the log file unrecoverable.
2024-06-20 17:54:57 +05:30
Alexander Barkov
6cecf61a59 MDEV-34417 Wrong result set with utf8mb4_danish_ci and BNLH join
There were erroneous calls for charpos() in key_hashnr() and key_buf_cmp().
These functions are never called with prefix segments.

The charpos() calls were wrong. Before the change BNHL joins
- could return wrong result sets, as reported in MDEV-34417
- were extremely slow for multi-byte character sets, because
  the hash was calculated on string prefixes, which increased
  the amount of collisions drastically.

This patch fixes the wrong result set as reported in MDEV-34417,
as well as (partially) the performance problem reported in MDEV-34352.
2024-06-20 11:30:02 +04:00
Monty
3541bd63f0 MDEV-33582 Add more warnings to be able to better diagnose network issues
Changed the logged messages from errors to warnings
Also changed 'remain' to 'read_length' in the warning to make it more readable.
2024-06-20 09:53:01 +03:00
Vicențiu Ciorbaru
6382339144 MDEV-34311: Alter USER should reset all account limit counters
This commit introduces a reset of password errors counter on any alter user
command for the altered user. This is done so as to not require a
complete privilege system reload.
2024-06-19 23:08:35 +03:00
Iaroslav Babanin
5d49a2add7 MDEV-33935 fix deadlock counter
- The deadlock counter was moved from
Deadlock::find_cycle into Deadlock::report, because
the find_cycle method is called multiple times during deadlock
detection flow, which means it shouldn't have such side effects.
But report() can, which called only once for
a victim transaction.
- Also the deadlock_detect.test and *.result test case
has been extended to handle the fix.
2024-06-19 20:43:33 +03:00
Jan Lindström
ee974ca5e0 MDEV-31658 : Deadlock found when trying to get lock during applying
Problem was that there was two non-conflicting local idle
transactions in node_1 that both inserted a key to primary key.
Then two transactions from other nodes inserted also
a key to primary key so that insert from node_2 conflicted
one of the local transactions in node_1 so that there would
be duplicate key if both are committed. For this insert
from other node tries to acquire S-lock for this record
and because this insert is high priority brute force (BF)
transaction it will kill idle local transaction.

Concurrently, second insert from node_3 conflicts the second
idle insert transaction in node_1. Again, it tries to acquire
S-lock for this record and kills idle local transaction.

At this point we have two non-conflicting high priority
transactions holding S-lock on different records in node_1.
For example like this: rec s-lock-node2-rec s-lock-node3-rec rec.

Because these high priority BF-transactions do not wait
each other insert from node3 that has later seqno compared
to insert from node2 can continue. It will try to acquire
insert intention for record it tries to insert (to avoid
duplicate key to be inserted by local transaction). Hower,
it will note that there is conflicting S-lock in same gap
between records. This will lead deadlock error as we have
defined that BF-transactions may not wait for record lock
but we can't kill conflicting BF-transaction because
it has lower seqno and it should commit first.

BF-transactions are executed concurrently because their
values to primary key are different i.e. they do not
conflict.

Galera certification will make sure that inserts from
other nodes i.e these high priority BF-transactions
can't insert duplicate keys. Local transactions naturally
can but they will be killed when BF-transaction
acquires required record locks.

Therefore, we can allow situation where there is conflicting
S-lock and insert intention lock regardless of their seqno
order and let both continue with no wait. This will lead
to situation where we need to allow BF-transaction
to wait when lock_rec_has_to_wait_in_queue is called
because this function is also called from
lock_rec_queue_validate and because lock is waiting
there would be assertion in ut_a(lock->is_gap()
|| lock_rec_has_to_wait_in_queue(cell, lock));

lock_wait_wsrep_kill
  Add debug sync points for BF-transactions killing
  local transaction.

wsrep_assert_no_bf_bf_wait
  Print also requested lock information

lock_rec_has_to_wait
  Add function to handle wsrep transaction lock wait
  cases.

lock_rec_has_to_wait_wsrep
  New function to handle wsrep transaction lock wait
  exceptions.

lock_rec_has_to_wait_in_queue
  Remove wsrep exception, in this function all
  conflicting locks need to wait in queue.
  Conflicts between BF and local transactions
  are handled in lock_wait.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-19 14:09:11 +02:00
Jan Lindström
1001dae186 MDEV-12008 : Change error code for Galera unkillable threads
Changed error code for Galera unkillable threads to
be ER_KILL_DENIED_HIGH_PRIORITY giving message

This is a high priority thread/query and cannot be killed
without the compromising consistency of the cluster

also a warning is produced
  Thread %lld is [wsrep applier|high priority] and cannot be killed

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-19 14:07:34 +02:00
Marko Mäkelä
34813c1aa0 Merge 10.6 into 10.11 2024-06-19 15:04:07 +03:00
Andrei
387bdb2a2e MDEV-29934 rpl.rpl_start_alter_chain_basic, rpl.rpl_start_alter_restart_slave sometimes fail in BB with result content mismatch
rpl.rpl_start_alter_chain_basic was used to fail sporadically due
to a missed GTID master-slave synchronization which was necessary
because of the following SELECT from GTID-state table.

Fixed with arranging two synchronization pieces for two
chain slaves requiring that.

Note rpl.rpl_start_alter_restart_slave must have been fixed by
MDEV-30460 and 87e13722a9 (manual) merge commit.
2024-06-19 14:09:00 +03:00
Alexander Barkov
cfa6143453 MDEV-27966 Assertion fixed()' failed and Assertion fixed == 1' failed, both in Item_func_concat::val_str on SELECT after INSERT with collation utf32_bin on utf8_bin table
This problem was earlier fixed by this commit:

> commit 08c7ab404f
> Author: Aleksey Midenkov <midenok@gmail.com>
> Date:   Mon Apr 18 12:44:27 2022 +0300
>
>    MDEV-24176 Server crashes after insert in the table with virtual
>    column generated using date_format() and if()

Adding an mtr test only.
2024-06-19 10:01:30 +04:00
Brandon Nesterenko
6cab2f75fe MDEV-23857: replication master password length
After MDEV-4013, the maximum length of replication passwords was extended to
96 ASCII characters. After a restart, however, slaves only read the first 41
characters of MASTER_PASSWORD from the master.info file. This lead to slaves
unable to reconnect to the master after a restart.

After a slave restart, if a master.info file is detected, use the full
allowable length of the password rather than 41 characters.

Reviewed By:
============
Sergei Golubchik <serg@mariadb.com>
2024-06-18 07:21:18 -06:00
Souradeep Saha
10fbd1ce51 MDEV-34168: Extend perror utility to print link to KB page
As all MariaDB Server errors now have a dedicated web page, the
perror utility is extended to include a link to the KB page of
the corresponding error code.

All new code of the whole pull request, including one or several
files that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my
employer Amazon Web Services, Inc.
2024-06-18 13:25:39 +10:00
Andrei
c37b2a9f04 MDEV-30460 rpl.rpl_start_alter_restart_slave sometimes fails in BB with result length mismatch
The test was used to fail because of lacking a synchronization point
designating an expected for processing "CA_1" event has been indeed
taken into this phase.
The event apparently may not have even arrived at slave.

Fixed with deploying the missed synchronization.
2024-06-17 19:06:34 +03:00
Alexander Barkov
83d3ed4908 MDEV-34014 mysql_upgrade failed
Adding a new statement into scripts/sys_schema/before_setup.sql:

  ALTER DATABASE sys CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci;

to fix db.opt in case:
- the database `sys` was altered to unexpected CHARACTER SET or COLLATE values
- or db.opt was erroneously removed

to make sure that sys objects are always recreated using utf8mb3_general_ci.
2024-06-17 16:38:48 +04:00
Sergei Petrunia
2eda310b15 Restore test coverage for MDEV-18956
(It was accidentally removed by fix for MDEV-28846)
2024-06-17 14:08:32 +03:00
Sergei Petrunia
a2066b2400 MDEV-30651: Assertion `sel->quick' in make_range_rowid_filters
The optimizer deals with Rowid Filters this way:

1. First, range optimizer is invoked. It saves information
   about all potential range accesses.
2. A query plan is chosen. Suppose, it uses a Rowid Filter on
   index $IDX.
3. JOIN::make_range_rowid_filters() calls the range optimizer
again to create a quick select on index $IDX which will be used
to populate the rowid filter.

The problem: KILL command catches the query in step . Quick
Select is not created which causes a crash.

Fixed by checking if query was killed. Note: the problem also
affects 10.6, even if error handling for
SQL_SELECT::test_quick_select is different there.
2024-06-17 14:08:32 +03:00
Sergei Petrunia
ef9e3e73ed MDEV-30651: Assertion `sel->quick' in make_range_rowid_filters
(Variant for 10.6: return error code from SQL_SELECT::test_quick_select)
The optimizer deals with Rowid Filters this way:

1. First, range optimizer is invoked. It saves information
   about all potential range accesses.
2. A query plan is chosen. Suppose, it uses a Rowid Filter on
   index $IDX.
3. JOIN::make_range_rowid_filters() calls the range optimizer
again to create a quick select on index $IDX which will be used
to populate the rowid filter.

The problem: KILL command catches the query in step . Quick
Select is not created which causes a crash.

Fixed by checking if query was killed.
2024-06-17 12:50:43 +03:00
Sergei Petrunia
b47bd3f8bf MDEV-33875: ORDER BY DESC causes ROWID Filter slowdown
Rowid Filter cannot be used with reverse-ordered scans, for the
same reason as IndexConditionPushdown cannot be.

test_if_skip_sort_order() already has logic to disable ICP when
setting up a reverse-ordered scan. Added logic to also disable
Rowid Filter in this case, factored out the code into
prepare_for_reverse_ordered_access(), and added a comment describing
the cause of this limitation.
2024-06-17 09:50:32 +03:00
Marko Mäkelä
346a0c1402 Merge 10.6 into 10.11 2024-06-17 09:08:07 +03:00
Marko Mäkelä
e60acae655 Merge 10.5 into 10.6 2024-06-17 08:40:07 +03:00
Monty
956bcf8f49 Change mysqldump to use DO instead of 'SELECT' for storing sequences.
This avoids a lot of SETVAL() results when applying a mysqldump with
sequences.
2024-06-16 10:51:33 +03:00
Monty
fef32fd9ad MDEV-34406 Enhance mariadb_upgrade to print failing query in case of error
To make this possible, it was also necessary to enhance the mariadb
client with the option --print-query-on-error.
This option can also be very useful when running a batch of queries
through the mariadb client and one wants to find out where things goes
wrong.

TODO: It would be good to enhance mariadb_upgrade to not call the mariadb
client for executing queries but instead do this internally.  This
would have made this patch much easier!

Reviewed by: Sergei Golubchik <serg@mariadb.com>
2024-06-16 10:51:33 +03:00
Marko Mäkelä
5b89cab44f Merge 10.6 into 10.11 2024-06-13 08:16:49 +03:00
Marko Mäkelä
fc9005adc4 Merge 10.5 into 10.6 2024-06-12 07:51:28 +03:00
Marko Mäkelä
b81d717387 Merge 10.6 into 10.11 2024-06-11 12:50:10 +03:00
Sergei Golubchik
40dd5b8676 fix the test for --view 2024-06-10 21:03:53 +02:00
Brandon Nesterenko
fcd21d3e40 MDEV-34355: rpl.rpl_semi_sync_no_missed_ack_after_add_slave ‘server_3 should have sent…’
The problem is that the test could query the status variable
Rpl_semi_sync_slave_send_ack before the slave actually updated it.
This would result in an immediate --die assertion killing the rest
of the test. The bottom of this commit message has a small patch
that can be applied to reproduce the test failure.

This patch fixes the test failure by waiting for the variable to be
updated before querying its value.

diff --git a/sql/semisync_slave.cc b/sql/semisync_slave.cc
index 9ddd4c5c8d7..60538079fce 100644
--- a/sql/semisync_slave.cc
+++ b/sql/semisync_slave.cc
@@ -303,7 +303,10 @@ int Repl_semi_sync_slave::slave_reply(Master_info *mi)
     reply_res= DBUG_EVALUATE_IF("semislave_failed_net_flush", 1,
                                 net_flush(net));
     if (!reply_res)
+    {
+      sleep(1);
       rpl_semi_sync_slave_send_ack++;
+    }
   }
   DBUG_RETURN(reply_res);
 }
2024-06-10 12:27:20 -06:00
Alexander Barkov
3b80d23d02 mtr --skip-not-found did not skip suites
--skip-not-found switch tells mtr to skip not found tests instead of aborting.
But it failed to skip the test if the suite name was not found.

This problem also made the *last-N-failed builbot builders fail
to run `mtr --skip-not-found` if the last commit removed a file in
the mysql-test/include/ directory.

This commit fixes it, now the not found test is properly skipped,
no matter what component of the test name was not found:

$ ./mtr main.foo --skip-not-found foo.main
...
==============================================================================
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
foo.main                                 [ skipped ]  not found
main.foo                                 [ skipped ]  not found
--------------------------------------------------------------------------
2024-06-10 19:17:00 +02:00
Marko Mäkelä
27834ebc91 Merge 10.5 into 10.6 2024-06-10 15:22:15 +03:00
Alexander Barkov
21f56583bf MDEV-32376 SHOW CREATE DATABASE statement crashes the server when db name contains some unicode characters, ASAN stack-buffer-overflow
Adding the test for the length of lex->name into show_create_db().

Without this test writes beyond the end of db_name_buff were possible
upon a too long database name.
2024-06-10 09:31:14 +04:00
Jan Lindström
0172887980 MDEV-34269 : 10.11.8 cluster becomes inconsistent when using composite primary key and partitioning
This is regression from commit 3228c08fa8. Problem is that
when table storage engine is determined there should be
check is table partitioned and if it is then determine
partition implementing storage engine.

Reported bug is reproducible only with --log-bin so make
sure tests changed by 3228c08fa8 and new test are run
with --log-bin and binlog disabled.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-07 18:26:08 +02:00
Thirunarayanan Balathandayuthapani
4b4dbb23ea MDEV-34169 Don't allow innodb_open_files to be lesser than
number of non-user tablespace.

fil_space_t::try_to_close(): Don't try to close
the tablespace which is acquired by the caller of
the function

Added the suppression message in open_files_limit test case
2024-06-07 20:50:39 +05:30
Oleksandr Byelkin
77c4c0f256 MDEV-34203 Sandbox mode \- is not compatible with --binary-mode
"Process" sandbox short command put by masqldump to avoid an error.
2024-06-07 14:07:54 +02:00
Thirunarayanan Balathandayuthapani
b7a75fbb8a MDEV-34169 Don't allow innodb_open_files to be lesser than
number of non-user tablespace.

- InnoDB only closes the user tablespace when the number of open
files exceeds innodb_open_files limit. In that case, InnoDB should
make sure that innodb_open_files value should be greater
than number of undo tablespace, system and temporary tablespace files.
2024-06-07 15:37:11 +05:30
Marko Mäkelä
a687cf8661 Merge 10.5 into 10.6 2024-06-07 10:03:51 +03:00
Julius Goryavsky
c2d9762011 mtr: сhange the default setting for the port group size parameter
Some galera tests starts 6 galera nodes. Each galera node requires
three ports: 6*3 = 18. Plus 6 ports are needed for 6 mariadbd servers.
Since the number of ports is rounded up to 10 everywhere in mtr, we
will take 30 as the default value for the port group size parameter.
2024-06-06 19:31:28 +02:00
Daniele Sciascia
c1dc03974b MDEV-33523 Spurious deadlock error when wsrep_on=OFF
Avoid starting transactions in wsrep-lib side when wsrep is
disabled. It is unnecessary, and causes spurious deadlock errors on
transaction clean up.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-06 19:19:34 +02:00
Jan Lindström
d328705a12 MDEV-34170 : table gtid_slave_pos entries never been deleted with wsrep_gtid_mode = 0
Problem was that updates to mysql.gtid_slave_pos table were
replicated even when they were newer used and because that
newer deleted. Avoid replication of mysql.gtid_slave_pos
table if wsrep_gtid_mode=OFF.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-06 19:19:34 +02:00
Thirunarayanan Balathandayuthapani
a02773f7c0 MDEV-34057 Inconsistent FTS state in concurrent scenarios
Problem:
=======
- This commit is a merge of mysql commit 129ee47ef994652081a11ee9040c0488e5275b14.
InnoDB FTS can be in inconsistent state when sync operation
terminates the server before committing the operation. This
could lead to incorrect synced doc id and incorrect query results.

Solution:
========
- During sync commit operation, InnoDB should pass
the sync transaction to update the max doc id
in the config table.

fts_read_synced_doc_id() : This function is used
to read only synced doc id from the config table.
2024-06-06 19:09:13 +05:30