Commit graph

195795 commits

Author SHA1 Message Date
Marko Mäkelä
4c2e971841 MDEV-33052 MSAN use-of-uninitialized-value in buf_read_ahead_linear()
buf_read_ahead_linear(): Suppress a warning of comparing potentially
uninitialized FIL_PAGE_PREV and FIL_PAGE_NEXT fields.
2023-12-18 10:37:11 +02:00
Marko Mäkelä
4ae105a37d Merge 10.4 into 10.5 2023-12-18 08:59:07 +02:00
Marko Mäkelä
f98d2ef5b4 MDEV-33009 Server hangs for a long time with innodb_undo_log_truncate=ON
trx_purge_truncate_history(): Release buf_pool.flush_list_mutex
before starting a rescan of buf_pool.flush_list, to ensure that
the buf_flush_page_cleaner thread (which may be holding buf_pool.mutex)
will be able to proceed.

This fixes up commit a0f02f7438 (MDEV-32757).

Tested by: Axel Schwenke
Reviewed by: Vladislav Lesin
2023-12-15 15:38:31 +02:00
Sergei Golubchik
87a5d16911 add another missing result file
see also 4eca64e331
2023-12-15 12:11:26 +01:00
Thirunarayanan Balathandayuthapani
59a984b4d8 MDEV-32725 innodb.import_update_stats accesses uninitialized ib_table->stat_n_rows
- InnoDB should write all zeros into a table and its indexes
statistics members when table is unreadable.
2023-12-15 15:43:19 +05:30
Sergei Golubchik
a2c6d61db8 don't use dynstr_append() in mysqltest.cc
followup for ad796aaa94
2023-12-15 10:26:48 +01:00
Sisi Huang
c5d7036e1a MDEV-32942 Fix Memory Leak in my_print_defaults with Non-Existing Config Files 2023-12-15 10:22:40 +11:00
Sergei Golubchik
4eca64e331 add missing result file 2023-12-14 20:25:58 +01:00
Marko Mäkelä
852e1383e3 MDEV-21245 InnoDB: Using a partial-field key prefix in search
ha_innobase::compare_key_parts(): If a full column index is
being replaced with a column prefix index, return Compare_keys::NotEqual.
2023-12-14 09:57:38 +11:00
Daniel Black
2c60d43d7d MDEV-33006 Missing required privilege CONNECTION ADMIN
opt_kill_long_query_type being an enum could be 0 corresponding
to ALL. When ALL is specified, the CONNECTION ADMIN is still
required.

Also check REPLICA MONITOR privilege and make the tests
find the results by recording stderr.

Noticed thanks to bug report by Tim van Dijen.

Fixes: 79b58f1ca8
2023-12-14 09:08:43 +11:00
Rex
b4712242dd MDEV-31279 Crash when lateral derived is guaranteed to return no rows
Consider this query
SELECT t1.* FROM t1, (SELECT t2.b FROM t2 WHERE NOT EXISTS
(SELECT 1 FROM t3) GROUP BY b) sq where sq.b = t1.a;

If SELECT 1 FROM t3 is expensive, for example t3 has >
thd->variables.expensive_subquery_limit, first evaluation is deferred to
mysql_derived_fill().  There it is noted that, in the above case
 NOT EXISTS (SELECT 1 FROM t3) is constant and false.

This causes the join variable zero_result_cause to be set to
"Impossible WHERE noticed after reading const tables" and the handler
for this join is never "opened" via handler::ha_open.

When mysql_derived_fill() is called for the next group of results, this
unopened handler is not taken into account.

reviewed by Igor Babaev (igor@mariadb.com)
2023-12-14 06:14:24 +12:00
Marko Mäkelä
c17aca2f11 MDEV-18322 Assertion "wrong page type" on instant ALTER TABLE
row_ins_clust_index_entry_low(): Invoke btr_set_instant() in the same
mini-transaction that has successfully inserted the metadata record.
In this way, if inserting the metadata record fails before any
undo log record was written for it, the index root page will remain
consistent.

innobase_instant_try(): Remove the btr_set_instant() call.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Matthias Leich
2023-12-13 15:01:50 +02:00
Daniel Black
fbe604d883 MDEV-32795: ALTER SEQUENCE IF NOT EXISTS non_existing_seq Errors rather than note
Like all IF NOT EXISTS syntax, a Note should be generated.

The original commit of Seqeuences cleared the IF NOT EXISTS part
in the sql/sql_yacc.yy with lex->create_info.init(). Without this
bit set there was no way it could do anything other than error.

To remedy this removal, the sql_yacc.yy components have been
minimised as they where all set at the beginning of the ALTER.
This way the opt_if_not_exists correctly set the IF_EXISTS flag.

In MDEV-13005 (bb4dd70e7c) the error code changed, requiring
ER_UNKNOWN_SEQUENCES to be handled in the function
No_such_table_error_handler::handle_condition.
2023-12-13 17:48:03 +11:00
Daniel Black
7504985daf MDEV-21587: disk.disk{_notembedded} test result
Allow for a CI system to be almost out of space, or having so
little use, that the Total space is the same as available or used.

Thanks Otto Kekäläinen for the bug report and testing.
2023-12-13 08:54:39 +11:00
Sergei Glushchenko
6193d0cabb MDEV-20286 mariabackup fails when innodb_max_dirty_pages_pct contains a fraction (is not an integer)
This is a port of the Percona Server commit 5265f42e290573e9591f8ca28ab66afc051f89a3

which is the same as their bug PXB-1807: xtrabackup does not accept fractional values for
innodb_max_dirty_pages_pct

Problem:

Variable specified as double in MySQL server, but read as long in the
xtrabackup. This causes xtrabackup to fail at startup when the value
contains decimal point.

Fix:

Make xtrabackup to interpret the value as double to be compatible with
server.
2023-12-12 16:49:12 +00:00
Sergei Golubchik
e472b682e0 MDEV-32839 LONG UNIQUE gives error when used with REPLACE
calculate auto-inc value even if long duplicate check fails -
this is what the engine does for normal uniques.

auto-inc value is needed if it's a REPLACE
2023-12-12 15:21:43 +01:00
Sergei Golubchik
81609d8625 cleanup: remove innodb-specific code around update_auto_increment()
removed dead code
2023-12-12 15:21:43 +01:00
Sergei Golubchik
c5904702cd cleanup: remove partition-specific code around update_auto_increment() 2023-12-12 15:21:43 +01:00
Marko Mäkelä
68e7909be9 MDEV-31000 Assertion failed on ALTER TABLE...page_compressed=1
ha_innobase::check_if_supported_inplace_alter(): On ALTER_OPTIONS,
if innodb_file_per_table=1 and the table resides in the system tablespace,
require that the table be rebuilt (and moved to an .ibd file).

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Matthias Leich
2023-12-12 14:40:45 +02:00
Marko Mäkelä
9f5078a1d7 MDEV-20139 innodb.innodb_buffer_pool_dump_pct failed in buildbot with timeout
This test was using a sleep of 1 second in an attempt to ensure that the
timestamp that is part of an InnoDB status string would increase.
This not only prolongs the test execution time by 1+1 seconds, but it
also is inaccurate. It is possible that the actual sleep duration is
less than a second.

Let us wait for the creation of the file ib_buffer_pool and then wait
for the buffer pool dump completion. In that way, the test can complete
in a dozen or two milliseconds (1% of the previous duration) and work
more reliably.
2023-12-12 11:43:23 +02:00
Daniele Sciascia
61daac54d6 MDEV-27806 GTIDs diverge in Galera cluster after CTAS
Add OPTION_GTID_BEGIN to applying side thread. This is needed to avoid
intermediate commits when CREATE TABLE AS SELECT is applied, causing
one more GTID to be consumed with respect to executing node.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-12-12 05:55:34 +01:00
Daniele Sciascia
9ab7dbc3be MDEV-28971 SEQUENCEs do not work with streaming replication
Return an error if user attempts to use SEQUENCEs in combination with
streaming replication in a Galera cluster. This is currently not
supported.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-12-12 05:55:34 +01:00
Marko Mäkelä
bd01029255 MDEV-29972 Crash emitting "Unsupported meta-data version number" error message
row_import_read_meta_data(): Use ER_NOT_SUPPORTED_YET instead of
ER_IO_READ_ERROR to have a matching error message pattern.
2023-12-12 15:09:31 +11:00
Yuchen Pei
b94ae3870b
[fixup] galera: Fix an accidental logical inversion in a recent merge
The merge was 13dd787530
2023-12-12 11:27:20 +11:00
Brandon Nesterenko
8dad51481b MDEV-10653: SHOW SLAVE STATUS Can Deadlock an Errored Slave
AKA rpl.rpl_parallel, binlog_encryption.rpl_parallel fails in
buildbot with timeout in include

A replication parallel worker thread can deadlock with another
connection running SHOW SLAVE STATUS. That is, if the replication
worker thread is in do_gco_wait() and is killed, it will already
hold the LOCK_parallel_entry, and during error reporting, try to
grab the err_lock. SHOW SLAVE STATUS, however, grabs these locks in
reverse order. It will initially grab the err_lock, and then try to
grab LOCK_parallel_entry. This leads to a deadlock when both threads
have grabbed their first lock without the second.

This patch implements the MDEV-31894 proposed fix to optimize the
workers_idle() check to compare the last in-use relay log’s
queued_count==dequeued_count for idleness. This removes the need for
workers_idle() to grab LOCK_parallel_entry, as these values are
atomically updated.

Huge thanks to Kristian Nielsen for diagnosing the problem!

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>
Andrei Elkin <andrei.elkin@mariadb.com>
2023-12-11 07:45:23 -07:00
Kristian Nielsen
5ca63b2b8b MDEV-26632: multi source replication filters breaking GTID semantic
Add a test case that demonstrates a working setup as described in MDEV-26632.
This requires --gtid-ignore-duplicates=1 and --gtid-strict-mode=0.

In A->B->C, B filters some (but not all) events from A. C is promoted to
create A->C->B, and the current GTID position in B contains a GTID from A that
is not present in C (due to filtering). Demonstrate that B can still connect
with GTID to C, starting at the "hole" in the binlog stream on C originating
from A.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2023-12-11 12:04:49 +01:00
Kristian Nielsen
50ce001afd MDEV-13792: innodb.purge_thread_shutdown failed in buildbot with wrong result (sporadic)
Omit `state` when selecting processlist to verify which threads are running.
The state changes as threads are running (enter_state()), and this causes
sporadic test failures.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2023-12-11 12:04:39 +01:00
Kristian Nielsen
da9ffca908 MDEV-29816 rpl.rpl_parallel_29322 occasionally fails in BB
Make sure the old binlog dump thread is not still running when manipulating
binlog files; otherwise there is a small chance it will see an invalid
partial file and report an I/O error.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2023-12-11 12:02:58 +01:00
Marko Mäkelä
1e80601b82 MDEV-16264 fixup: Remove a useless test
Let us remove a test that frequently fails with a result difference.
This test had been added in fc279d7ea2
to cover a bug in thd_destructor_proxy(), which was replaced with
simpler logic in 5e62b6a5e0 (MDEV-16264).
2023-12-11 10:37:12 +02:00
Marko Mäkelä
7e34bb5ce1 MDEV-11905: Simplify encryption.innodb_encrypt_discard_import
The test was populating unnecessarily large tables and
restarting the server several times for no real reason.

Let us hope that a smaller version of the test will produce more
stable results. Occasionally, some unencrypted contents in the table t2
was revealed in the old test.
2023-12-11 10:31:49 +02:00
Alexander Barkov
03ee23bcbb MDEV-17226 Column Data in Truncated on UNION to the length of the first value if using REPLACE
This problem was earlier fixed by:
  commit 55b2788800

Adding MTR tests only.
2023-12-11 10:42:37 +04:00
Dmitry Shulga
9bf50a0eec MDEV-32965: Assertion `thd->active_stmt_arena_to_use()-> is_stmt_prepare_or_first_sp_execute() || thd->active_stmt_arena_to_use()-> is_conventional() || thd->active_stmt_arena_to_use()->state == Query_arena::STMT_SP_QUERY_ARGUMENTS' failed
This patch fixes too strong condition in assert at the method
  Item_func_group_concat::fix_fields
that is true in case of a stored routine and obviously broken
for a prepared statement.
2023-12-11 12:27:11 +07:00
Marko Mäkelä
5775df0127 MDEV-20142 encryption.innodb_encrypt_temporary_tables fails
The data type of the column INFORMATION_SCHEMA.GLOBAL_STATUS.VARIABLE_VALUE
is a character string. Therefore, if we want to compare some values as
integers, we must explicitly cast them to integer type, to avoid an
awkward comparison where '10'<'9' because the first digit is smaller.
2023-12-10 13:19:21 +02:00
Alexander Barkov
2d775fd01a Cleanup: Removing the unused method Type_handler::get_handler_by_cmp_type
It's not used in 10.5+
2023-12-10 08:21:38 +04:00
Daniel Black
02d67cecb6 MDEV-32043 mariadb-upgrade should remove bundled plugins from mysql.plugin
Also in the startup, lets not "Error" on attempting to install a
mysql.plugin that is already there. We use the 'if_not_exists'
parameter to true to downgrade this to a "Note".

Also corrects: MDEV-32041 "plugin already loaded" should be a Warning, not an Error
2023-12-09 11:59:02 +11:00
Marko Mäkelä
a356a940d2 MDEV-32971 Assertion !recv_sys.is_corrupt_fs() failed on recovery
recv_recovery_from_checkpoint_start(): Relax a too strict debug assertion
that occasionally fails in the test encryption.innodb-redo-nokeys
when fil_ibd_load() returns FIL_LOAD_INVALID due to missing crypt_info.

This assertion had been removed in MariaDB Server 10.8 as part of
commit 685d958e38 (MDEV-14425).
2023-12-08 10:55:13 +02:00
Marko Mäkelä
c8346c0bac MDEV-31939 Adaptive flush recommendation ignores dirty ratio and checkpoint age
buf_flush_page_cleaner(): Pass pct_lwm=srv_max_dirty_pages_pct_lwm
(innodb_max_dirty_pages_pct_lwm) to
page_cleaner_flush_pages_recommendation() unless the dirty page ratio
of the buffer pool is below that. Starting with
commit d4265fbde5 we used to always
pass pct_lwm=0.0, which was not intended.

Reviewed by: Vladislav Vaintroub
2023-12-08 09:31:34 +02:00
Yuchen Pei
d7c6d306fb
Merge branch '10.4' into 10.5 2023-12-08 18:02:23 +11:00
Brandon Nesterenko
9be7e03f70 MDEV-32953: main.rpl_mysqldump_slave Fails with "Master binlog wasn’t deleted" Assertion
Because --delete-master-logs immediately purges logs after flushing,
it is possible the binlog dump thread would still be using the old
log when the purge executes, disallowing the file from being
deleted.

This patch institutes a work-around in the test as follows:
 1) temporarily stop the slave so there is no chance the old binlog
is still being referenced.
 2) set master_use_gtid=Slave_pos so the slave can still appear
up-to-date on the master after the master flushes/purges its logs
(while the slave is offline). Otherwise (i.e. if using binlog
file/pos), the slave would point to a purged log file, and receive
an error immediately upon connecting to the master.

Reviewed By
============
Andrei Elkin <andrei.elkin@mariadb.com>
2023-12-07 09:27:15 -07:00
Thirunarayanan Balathandayuthapani
d5a6ea36f3 MDEV-32242 innodb.doublewrite test case always gets skipped
- Split the doublewrite test into two test (doublewrite,
doublewrite_debug) to reduce the execution time of the test
- Removed big_test tag for the newly added test case
- Made doublewrite test as non-debug test
- Added search pattern to make sure that InnoDB uses doublewrite buffer
- Replaced all kill_mysqld.inc with shutdown_mysqld.inc and
zero shutdown timeout
- Removed the case where fsp_flags got corrupted. Because from commit
3da5d047b8 (MDEV-31851) onwards,
doublewrite buffer removes the conversion the fsp flags from buggy
10.1 format

Thanks to Marko Mäkelä for providing the non-debug test
2023-12-07 18:44:28 +05:30
Yuchen Pei
c33ca17c17
Merge branch '10.4' into 10.5 2023-12-07 17:52:44 +11:00
Yuchen Pei
13dd787530
Merge branch '10.4' into 10.5 2023-12-07 16:38:00 +11:00
Yuchen Pei
ba94778d2c
MDEV-32753 Make spider init queries compatible with oracle sql mode
Remove ORACLE from the (session) sql_mode in connections made with sql
service to run init queries

The connection is new and the global variable value takes effect
rather than the session value from the caller of spider_db_init.
2023-12-07 16:01:42 +11:00
Sergei Golubchik
69389c03b1
MDEV-32683 Spider engine does not load with non-default alter-algorithm
specify algorithm/lock explicitly, don't depend on server settings
2023-12-07 16:01:41 +11:00
Yuchen Pei
9259b4b849
MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func 2023-12-07 16:01:41 +11:00
Yuchen Pei
30af987259
MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
This should fix certain CI builds where the spider suite test files
and the main suite test files do not follow the same relative paths
relations as the mariadb source.
2023-12-07 16:01:41 +11:00
Yuchen Pei
70283aca34
MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
$MYSQLD_CMD uses .1 as the defaults-group-suffix, which could cause
the use of the default port (3306) or socket, which will fail in
environment where these defaults are already in use by another server.

Adding an extra --defaults-group-suffix=.1.1 does not help, because
the first flag wins.

So we use $MYSQLD_LAST_CMD instead, which uses the correct suffix.

The extra innodb buffer pool warning is irrelevant to the goal of the
test (running --wsrep-recover with --plug-load-add=ha_spider should
not cause hang)
2023-12-07 16:01:41 +11:00
Yuchen Pei
d8f5d2bef0
MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
Fix spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218) while
preventing regression on old ones (MDEV-30370, MDEV-29904)

Two things are changed:

First, Spider initialisation is made fully synchronous, i.e. it no
longer happens in a background thread. Adapted from the original fix
by nayuta for MDEV-27233. This change itself would cause failure when
spider is initialised early, by plugin-load-add, due to dependency on
Aria and udf function creation, which are fixed in the second and
third parts below. Requires SQL Service, thus porting earlier versions
requires MDEV-27595

Second, if spider is initialised before udf_init(), create udf by
inserting into `mysql.func`, otherwise do it by `CREATE FUNCTION` as
usual. This change may be generalised in MDEV-31401.

Also factor out some clean-up queries from deinit_spider.inc for use
of spider init tests.

A minor caveat is that early spider initialisation will fail if the
server is bootstrapped for the first time, due to missing `mysql`
database which needs to be created by the bootstrap script.
2023-12-07 16:01:41 +11:00
Yuchen Pei
afe63ec614
MDEV-27095 clean up spd_init_query.h
Removing procedures that were created and dropped during init.

This also fixes a race condition where mtr test with
plugin-load-add=ha_spider.so causes post test check to fail as it
expects the procedures to still be there.
2023-12-07 16:01:40 +11:00
Yuchen Pei
f0af56be01
MDEV-27095 installing one spider plugin should not trigger others
There are several plugins in ha_spider: spider, spider_alloc_mem,
spider_wrapper_protocols, spider_rewrite etc.

INSTALL PLUGIN foo SONAME ha_spider causes all the other ones to be
installed by the init queries where foo is any of the plugins.

This introduces unnecessary complexiy. For example it reads
mysql.plugins to find all other plugins, causing the hack of moving
spider plugin init to a separate thread.

To install all spider related plugins, install soname ha_spider should
be used instead.

This also fixes spurious rows in mysql.plugin when installing say only
the spider plugin with `plugin-load-add=SPIDER=ha_spider.so`:

select * from mysql.plugin;
name	dl
spider_alloc_mem	ha_spider.so # should not be here
spider_wrapper_protocols	ha_spider.so # should not be here

Adapted from part of the reverted commit
c160a115b8.
2023-12-07 16:01:40 +11:00