Commit graph

197459 commits

Author SHA1 Message Date
Vicențiu Ciorbaru
b482e87f26 Silence gcc-11 warnings 2023-02-03 13:09:48 +02:00
Oleksandr Byelkin
260f1fe7c3 Merge branch '10.8' into 10.9 2023-02-01 17:21:42 +01:00
Oleksandr Byelkin
d4310eb96a Merge branch '10.7' into 10.8 2023-02-01 17:19:48 +01:00
Oleksandr Byelkin
bc656c4fa5 Merge branch '10.6' into 10.7 2023-02-01 16:29:16 +01:00
Julius Goryavsky
4c79e15cc3 MDEV-30536: no expected deadlock in galera_insert_bulk test
Unstable test (galera_insert_bulk) temporarily disabled.
2023-02-01 15:57:22 +01:00
Oleksandr Byelkin
901870440b Fix results in real ps-protocol:
Starting wth 10.6 it goes via ps protocol so the subselect removed
only once on prepare.
2023-02-01 14:05:00 +01:00
Marko Mäkelä
1c926b6263 MDEV-30527 Assertion !m_freed_pages in mtr_t::start() on DROP TEMPORARY TABLE
mtr_t::commit(): Add special handling of
innodb_immediate_scrub_data_uncompressed for TEMPORARY TABLE.

This fixes a regression that was caused by
commit de4030e4d4 (MDEV-30400).
2023-02-01 10:55:49 +02:00
Marko Mäkelä
ccafd2d49d MDEV-30524 btr_cur_t::open_leaf() opens non-leaf page in BTR_MODIFY_LEAF mode
btr_cur_t::open_leaf(): When we have to reopen the root page in
a different mode, ensure that we will actually acquire a latch upfront,
instead of using RW_NO_LATCH. This prevents a race condition where
the index tree would be split between the time we released the
root page S latch and finally acquired a latch in
mtr->upgrade_buffer_fix(), actually on a non-leaf root page.

This race condition was introduced in
commit 89ec4b53ac (MDEV-29603).
2023-01-31 16:38:11 +02:00
Oleksandr Byelkin
de2d089942 Merge branch '10.8' into 10.9 2023-01-31 10:37:31 +01:00
Oleksandr Byelkin
638625278e Merge branch '10.7' into 10.8 2023-01-31 09:57:52 +01:00
Alexander Barkov
4a04c18af9 MDEV-25765 Mariabackup reduced verbosity option for log output 2023-01-31 12:49:14 +04:00
Oleksandr Byelkin
b923b80cfd Merge branch '10.6' into 10.7 2023-01-31 09:33:58 +01:00
Oleksandr Byelkin
c3a5cf2b5b Merge branch '10.5' into 10.6 2023-01-31 09:31:42 +01:00
Alexander Barkov
50b69641ef MDEV-29244 mariabackup --help output still referst to innobackupex
Changing the tool name in the "mariadb-backup --help" output
from "innobackupex" to "mariadb-backup".
2023-01-31 10:38:02 +04:00
Andrei
f8a85af8ca MDEV-30940: Revert "binlog.innodb_rc_insert_before_delete is disabled with MDEV-30490"
This reverts commit b2ea57e899,
as well as edits binlog.innodb_rc_insert_before_delete.test
to be safely runnable with any preceding test.

Note: manual 10.5 -> 10.6 merge is required to the test.
2023-01-30 21:28:21 +01:00
Andrei
b2ea57e899 binlog.innodb_rc_insert_before_delete is disabled with MDEV-30490 2023-01-30 13:25:26 +01:00
Oleksandr Byelkin
db8019ef00 Merge branch '10.4' into 10.5 2023-01-30 13:25:02 +01:00
Anel Husakovic
0e737f7898 MDEV-30186 Use of uninitialized value in substitution
Reviewer: <>
2023-01-30 12:35:36 +01:00
Oleksandr Byelkin
c8f2e9a5c0 Fix number of rows passing in case of EQ_REF 2023-01-30 11:49:42 +01:00
Jan Lindström
b05218e08f MDEV-30473 : Do not allow GET_LOCK() / RELEASE_LOCK() in cluster
Following tests do not test anymore what they intended to test

deleted:    suite/galera/t/MDEV-24143.test
deleted:    suite/galera/t/galera_bf_abort_get_lock.test
2023-01-30 08:55:35 +02:00
Sergei Petrunia
9c6fcdb85e MDEV-30218: Incorrect optimization for rowid_filtering, correction
Enable use of Rowid Filter optimization with eq_ref access.
Use the following assumptions:
- Assume index-only access cost is 50% of non-index-only access cost.
- Take into account that "Eq_ref access cache" reduces the number of
  lookups eq_ref access will make.
  = This means the number of Rowid Filter checks is reduced also
  = Eq_ref access cost is computed using that assumption (see
    prev_record_reads() call), so we should use it in all cost '
    computations.
2023-01-28 18:24:22 +01:00
Oleksandr Byelkin
a977054ee0 Merge branch '10.3' into 10.4 2023-01-28 18:22:55 +01:00
Andrei
6173a4a15b binlog.innodb_rc_insert_before_delete is disabled with MDEV-30490 2023-01-28 17:10:42 +02:00
Andrei
c73985f2ce MDEV-30010 post-push: fixing test results. 2023-01-28 15:21:23 +02:00
Oleksandr Byelkin
7fa02f5c0b Merge branch '10.4' into 10.5 2023-01-27 13:54:14 +01:00
Jan Lindström
49ee18eb42 MDEV-30473 : Do not allow GET_LOCK() / RELEASE_LOCK() in cluster
In 10.5 If WSREP_ON=ON do not allow RELEASE_ALL_LOCKS function.
Instead print clear error message.
2023-01-27 10:40:07 +02:00
Jan Lindström
696562ce55 MDEV-30473 : Do not allow GET_LOCK() / RELEASE_LOCK() in cluster
If WSREP_ON=ON do not allow GET_LOCK and RELEASE_LOCK functions.
Instead print clear error message.
2023-01-27 10:34:06 +02:00
Jan Lindström
844ddb1109 MDEV-30473 : Do not allow GET_LOCK() / RELEASE_LOCK() in cluster
If WSREP_ON=ON do not allow GET_LOCK and RELEASE_LOCK functions.
Instead print clear error message.
2023-01-27 08:39:32 +02:00
Jan Lindström
015fb54d45 MDEV-25037 : SIGSEGV in MDL_lock::hog_lock_types_bitmap
We should not call mdl_context.release_explicit_locks() in
Wsrep_client_service::bf_rollback() if client is quiting because
it will be done again in THD::cleanup().

Note that problem with GET_LOCK() / RELEASE_LOCK() will be fixed
on MDEV-30473.
2023-01-27 08:38:27 +02:00
Khem Raj
75bbf645a6 Add missing include <cstdio>
This is needed with GCC 13 and newer [1]

[1] https://www.gnu.org/software/gcc/gcc-13/porting_to.html

Signed-off-by: Khem Raj <raj.khem@gmail.com>
2023-01-27 12:43:38 +11:00
Sergei Petrunia
c8f9bb2718 MDEV-30218: Incorrect optimization for rowid_filtering, correction
Enable use of Rowid Filter optimization with eq_ref access.
Use the following assumptions:
- Assume index-only access cost is 50% of non-index-only access cost.
- Take into account that "Eq_ref access cache" reduces the number of
  lookups eq_ref access will make.
  = This means the number of Rowid Filter checks is reduced also
  = Eq_ref access cost is computed using that assumption (see
    prev_record_reads() call), so we should use it in all cost '
    computations.
2023-01-26 16:16:43 +03:00
Marko Mäkelä
672cdcbb93 MDEV-30404: Inconsistent updates of PAGE_MAX_TRX_ID on ROW_FORMAT=COMPRESSED pages
page_copy_rec_list_start(): Do not update the PAGE_MAX_TRX_ID
on the compressed copy of the page. The modification is supposed
to be logged as part of page_zip_compress() or page_zip_reorganize().
If the page cannot be compressed (due to running out of space),
then page_zip_decompress() must be able to roll back the changes.

This fixes a regression that was introduced in
commit 56f6dab1d0 (MDEV-21174).
2023-01-26 14:47:52 +02:00
Thirunarayanan Balathandayuthapani
81196469bb MDEV-30429 InnoDB: Failing assertion: stat_value != UINT64_UNDEFINED in storage/innobase/dict/dict0stats.cc line 3647
In dict_stats_analyze_index(), InnoDB sets the maximum value for
index_stats_t to indicate the bulk under bulk insert operation.
But InnoDB fails to empty the statistics of the table in that case.
2023-01-26 18:09:34 +05:30
Oleksandr Byelkin
dd24fa3063 Merge branch '10.3' into 10.4 2023-01-26 10:34:26 +01:00
Marko Mäkelä
e02ed04d17 MDEV-23855 fixup: Remove SRV_MASTER_CHECKPOINT_INTERVAL 2023-01-25 13:53:10 +02:00
Marko Mäkelä
82b18a8361 MDEV-29374 fixup: Suppress an error in a test 2023-01-25 10:56:07 +02:00
Jan Lindström
8bccba1d4a MDEV-30465 : Galera test galera_sr.MDEV-27615 takes 5mins
After d7d3ad69 we should use KILL_CONNECTION_HARD to interrupt
debug_sync waits. Test case uses debug_sync and then disconnects
connection from cluster.
2023-01-25 10:01:00 +02:00
Jan Lindström
509c7f66bd MDEV-27977 : galera.galera_UK_conflict fails with wrong result
Add wait_condition so that all rows expected are really
replicated before we check it.
2023-01-25 09:08:15 +02:00
Heiko Becker
15226a2822 Add missing include for std::runtime_error
Fixes the following error when building with gcc 13:

"tpool/aio_liburing.cc:64:18: error: 'runtime_error' is not a member of 'std'
   64 |       throw std::runtime_error("aio_uring()");"
2023-01-25 17:30:18 +11:00
Daniel Black
765291d63e CREDITS: re-instate Tencent Cloud
This was an accidential deletion looking at an outdated list.
2023-01-25 16:44:26 +11:00
Yuchen Pei
284810b3e8
MDEV-30370 Fixing spider hang when server aborts
This is Kentoku's patch for MDEV-22979 (e6e41f04f4 + 22a0097727),
which fixes 30370.

It changes the wait to a timed wait for the first sts thread, which
waits on server start to execute the init queries for spider. It also
flips the flag init_command to false when the sts thread is being
freed. With these changes the sts thread can check the flag regularly
and abort the init_queries when it finds out the init_command is
false. This avoids the deadlock that causes the problem in MDEV-30370.

It also fixes MDEV-22979 for 10.4, but not 10.5. I have not tested
higher versions for MDEV-22979.

A test has also been done on MDEV-29904 to avoid regression, given
MDEV-27233 is a similar problem and its patch caused the
regression. The test passes for 10.4-11.0.

However, this adhoc test only works consistently when placed in the
main testsuite. We should not place spider tests in the main suite, so
we do not include it in this commit. A patch for MDEV-27912 should fix
this problem and allow a proper test for MDEV-29904. See comments in
the jira ticket MDEV-30370/29904 for the adhoc testcase used for this
commit.
2023-01-25 12:12:17 +11:00
Vladislav Vaintroub
2279dddad6 MDEV-30457 Windows, signtool error "No file digest algorithm specified."
Add /fd parameter. It is now mandatory for the recent versions of
signtool
2023-01-24 20:51:53 +01:00
Andrei
7fe932444d MDEV-30323 Some DDLs like ANALYZE can complete on parallel slave out of order
ANALYZE was observed to race over a preceding in binlog order DML
in updating the binlog and slave gtid states.

Tagging ANALYZE and other admin class commands in binlog by the fixes
of MDEV-17515 left a flaw allowing such race leading to
the gtid mode out-of-order error.
This is fixed now to observe by ADMIN commands the ordered access to
the slave gtid status variables and binlog.
2023-01-24 20:18:03 +02:00
Andrei
3aa04c0deb MDEV-30010 Slave (additional info): Commit failed due to failure of an earlier commit on which this one depends Error_code: 1964
This commit merely adds is a Read-Committed version MDEV-30225 test
solely to prove the RC isolation yields ROW binlog format as it is
supposed to per docs.
2023-01-24 19:39:44 +02:00
Igor Babaev
f513d71538 MDEV-30081 Crash with splitting from constant mergeable derived table
This bug manifested itself in very rare situations when splitting
optimization was applied to a materialized derived table with group clause
by key over a constant meargeable derived table that was in inner part of
an outer join. In this case the used tables for the key to access the
split table incorrectly was evaluated to a not empty table map.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2023-01-24 08:46:41 -08:00
Brandon Nesterenko
d69e835787 MDEV-29639: Seconds_Behind_Master is incorrect for Delayed, Parallel Replicas
Problem
========
On a parallel, delayed replica, Seconds_Behind_Master will not be
calculated until after MASTER_DELAY seconds have passed and the
event has finished executing, resulting in potentially very large
values of Seconds_Behind_Master (which could be much larger than the
MASTER_DELAY parameter) for the entire duration the event is
delayed. This contradicts the documented MASTER_DELAY behavior,
which specifies how many seconds to withhold replicated events from
execution.

Solution
========
After a parallel replica idles, the first event after idling should
immediately update last_master_timestamp with the time that it began
execution on the primary.

Reviewed By
===========
Andrei Elkin <andrei.elkin@mariadb.com>
2023-01-24 08:11:35 -07:00
Marko Mäkelä
4d9fe4032b Merge 10.8 into 10.9 2023-01-24 14:59:42 +02:00
Marko Mäkelä
fa543a0f62 Merge 10.7 into 10.8 2023-01-24 14:52:25 +02:00
Marko Mäkelä
cea50896d2 Merge 10.6 into 10.7 2023-01-24 14:35:36 +02:00
Marko Mäkelä
de4030e4d4 MDEV-30400 Assertion height == btr_page_get_level(...) on INSERT
This also fixes part of MDEV-29835 Partial server freeze
which is caused by violations of the latching order that was
defined in https://dev.mysql.com/worklog/task/?id=6326
(WL#6326: InnoDB: fix index->lock contention). Unless the
current thread is holding an exclusive dict_index_t::lock,
it must acquire page latches in a strict parent-to-child,
left-to-right order. Not all cases of MDEV-29835 are fixed yet.
Failure to follow the correct latching order will cause deadlocks
of threads due to lock order inversion.

As part of these changes, the BTR_MODIFY_TREE mode is modified
so that an Update latch (U a.k.a. SX) will be acquired on the
root page, and eXclusive latches (X) will be acquired on all pages
leading to the leaf page, as well as any left and right siblings
of the pages along the path. The DEBUG_SYNC test innodb.innodb_wl6326
will be removed, because at the time the DEBUG_SYNC point is hit,
the thread is actually holding several page latches that will be
blocking a concurrent SELECT statement.

We also remove double bookkeeping that was caused due to excessive
information hiding in mtr_t::m_memo. We simply let mtr_t::m_memo
store information of latched pages, and ensure that
mtr_memo_slot_t::object is never a null pointer.
The tree_blocks[] and tree_savepoints[] were redundant.

buf_page_get_low(): If innodb_change_buffering_debug=1, to avoid
a hang, do not try to evict blocks if we are holding a latch on
a modified page. The test innodb.innodb-change-buffer-recovery
will be removed, because change buffering may no longer be forced
by debug injection when the change buffer comprises multiple pages.
Remove a debug assertion that could fail when
innodb_change_buffering_debug=1 fails to evict a page.
For other cases, the assertion is redundant, because we already
checked that right after the got_block: label. The test
innodb.innodb-change-buffering-recovery will be removed, because
due to this change, we will be unable to evict the desired page.

mtr_t::lock_register(): Register a change of a page latch
on an unmodified buffer-fixed block.

mtr_t::x_latch_at_savepoint(), mtr_t::sx_latch_at_savepoint():
Replaced by the use of mtr_t::upgrade_buffer_fix(), which now
also handles RW_S_LATCH.

mtr_t::set_modified(): For temporary tables, invoke
buf_page_t::set_modified() here and not in mtr_t::commit().
We will never set the MTR_MEMO_MODIFY flag on other than
persistent data pages, nor set mtr_t::m_modifications when
temporary data pages are modified.

mtr_t::commit(): Only invoke the buf_flush_note_modification() loop
if persistent data pages were modified.

mtr_t::get_already_latched(): Look up a latched page in mtr_t::m_memo.
This avoids many redundant entries in mtr_t::m_memo, as well as
redundant calls to buf_page_get_gen() for blocks that had already
been looked up in a mini-transaction.

btr_get_latched_root(): Return a pointer to an already latched root page.
This replaces btr_root_block_get() in cases where the mini-transaction
has already latched the root page.

btr_page_get_parent(): Fetch a parent page that was already latched
in BTR_MODIFY_TREE, by invoking mtr_t::get_already_latched().
If needed, upgrade the root page U latch to X.
This avoids bloating mtr_t::m_memo as well as performing redundant
buf_pool.page_hash lookups. For non-QUICK CHECK TABLE as well as for
B-tree defragmentation, we will invoke btr_cur_search_to_nth_level().

btr_cur_search_to_nth_level(): This will only be used for non-leaf
(level>0) B-tree searches that were formerly named BTR_CONT_SEARCH_TREE
or BTR_CONT_MODIFY_TREE. In MDEV-29835, this function could be
removed altogether, or retained for the case of
CHECK TABLE without QUICK.

btr_cur_t::left_block: Remove. btr_pcur_move_backward_from_page()
can retrieve the left sibling from the end of mtr_t::m_memo.

btr_cur_t::open_leaf(): Some clean-up.

btr_cur_t::search_leaf(): Replaces btr_cur_search_to_nth_level()
for searches to level=0 (the leaf level). We will never release
parent page latches before acquiring leaf page latches. If we need to
temporarily release the level=1 page latch in the BTR_SEARCH_PREV or
BTR_MODIFY_PREV latch_mode, we will reposition the cursor on the
child node pointer so that we will land on the correct leaf page.

btr_cur_t::pessimistic_search_leaf(): Implement new BTR_MODIFY_TREE
latching logic in the case that page splits or merges will be needed.
The parent pages (and their siblings) should already be latched on
the first dive to the leaf and be present in mtr_t::m_memo; there
should be no need for BTR_CONT_MODIFY_TREE. This pre-latching almost
suffices; it must be revised in MDEV-29835 and work-arounds removed
for cases where mtr_t::get_already_latched() fails to find a block.

rtr_search_to_nth_level(): A SPATIAL INDEX version of
btr_search_to_nth_level() that can search to any level
(including the leaf level).

rtr_search_leaf(), rtr_insert_leaf(): Wrappers for
rtr_search_to_nth_level().

rtr_search(): Replaces rtr_pcur_open().

rtr_latch_leaves(): Replaces btr_cur_latch_leaves(). Note that unlike
in the B-tree code, there is no error handling in case the sibling
pages are corrupted.

rtr_cur_restore_position(): Remove an unused constant parameter.

btr_pcur_open_on_user_rec(): Remove the constant parameter
mode=PAGE_CUR_GE.

row_ins_clust_index_entry_low(): Use a new
mode=BTR_MODIFY_ROOT_AND_LEAF to gain access to the root page
when mode!=BTR_MODIFY_TREE, to write the PAGE_ROOT_AUTO_INC.

BTR_SEARCH_TREE, BTR_CONT_SEARCH_TREE: Remove.

BTR_CONT_MODIFY_TREE: Note that this is only used by
rtr_search_to_nth_level().

btr_pcur_optimistic_latch_leaves(): Replaces
btr_cur_optimistic_latch_leaves().

ibuf_delete_rec(): Acquire exclusive ibuf.index->lock in order
to avoid a deadlock with ibuf_insert_low(BTR_MODIFY_PREV).

btr_blob_log_check_t(): Acquire a U latch on the root page,
so that btr_page_alloc() in btr_store_big_rec_extern_fields()
will avoid a deadlock.

btr_store_big_rec_extern_fields(): Assert that the root page latch
is being held.

Tested by: Matthias Leich
Reviewed by: Vladislav Lesin
2023-01-24 14:09:21 +02:00