Commit graph

25599 commits

Author SHA1 Message Date
Yuchen Pei
9e9e0b99ad
MDEV-30170 ha_spider::delete_table() should report table not exist
All Spider tables are recorded in the system table
mysql.spider_tables. Deleting a spider table removes the corresponding
rows from the system table, among other things. This patch makes it so
that if spider could not find any record in the system table to delete
for a given table, it should correctly report that no such Spider
table exists.
2024-01-11 12:43:47 +11:00
Yuchen Pei
7801c6d22d
MDEV-29002 Spider: remove SPIDER_CONN::loop_check_meraged_last
The field is assigned but unused, and it causes heap-use-after-free.
2024-01-11 11:41:52 +11:00
Sergei Golubchik
761d5c8987 MDEV-33092 Undefined reference to concurrency on Solaris
remove thr_setconcurrency()
followup for 8bbcaab160

Fix by Rainer Orth
2024-01-10 10:16:20 +01:00
Yuchen Pei
c9902a20b3
Merge branch '10.4' into 10.5 2024-01-10 18:01:46 +11:00
Yuchen Pei
bc3d416a17
MDEV-29718 Fix spider detection of same data node server
When the host is not specified, it defaults to localhost.
2024-01-10 16:37:36 +11:00
Yuchen Pei
eabc74aaef
MDEV-33008 Fix spider table discovery
A new column was introduced to the show index output in 10.6 in
f691d9865b

Thus we update the check of the number of columns to be at least 13,
rather than exactly 13.

Also backport an err number and format from 10.5 for better error
messages when the column number is wrong.
2024-01-10 16:36:39 +11:00
Rainer Orth
2310f659f5 MDEV-8941 Compile on Solaris (SPARC) fails with errors in filamvct.cpp
There are a large number of uses of `strerror` in the codebase,
the local declaration in `storage/connect/tabvct.cpp` is the only one.

Given that none is needed elsewhere, I conclude that this instance can
simply be removed.
2024-01-10 13:48:23 +11:00
Marko Mäkelä
0b612619ef MDEV-33098: Fix some instrumentation for innodb.doublewrite_debug
buf_flush_page_cleaner(): A continue or break inside DBUG_EXECUTE_IF
actually is a no-op. Use an explicit call to _db_keyword_() to
actually avoid advancing the checkpoint.

buf_flush_list_now_set(): Invoke os_aio_wait_until_no_pending_writes()
to ensure that the page write to the system tablespace is completed.
2024-01-08 14:36:54 +02:00
Marko Mäkelä
613d019497 MDEV-33160 show_status_array() calls various functions via incompatible pointer
In commit b4ff64568c the
signature of mysql_show_var_func was changed, but not all functions
of that type were adjusted.

When the server is configured with `cmake -DWITH_ASAN=ON` and
compiled with clang, runtime errors would be flagged for invoking
functions through an incompatible function pointer.

Reviewed by: Michael 'Monty' Widenius
2024-01-04 12:50:05 +02:00
Marko Mäkelä
cc5c0eda4c MDEV-33156 Crash on innodb_buf_flush_list_now=ON and innodb_force_recovery=6
srv_start(): Move a read only mode startup tweak from
innodb_init_params() to the correct location. Also if
innodb_force_recovery=6 we will disable the doublewrite buffer,
because InnoDB must run in read-only mode to prevent further corruption.

This change only affects debug checks. Whenever srv_read_only_mode holds,
the buf_pool.flush_list will be empty, that is, there will be no writes
of persistent InnoDB data pages.

Reviewed by: Thirunarayanan Balathandayuthapani
2024-01-03 12:08:21 +02:00
Vladislav Vaintroub
569381df83 MDEV-33046 fixup. Do not try to schedule timer without dict_stats_start()
dict_stats_schedule() is executed for --innodb-force-recovery=3+,
but timer is not created.
2023-12-21 20:20:47 +01:00
Vladislav Vaintroub
7cc332b724 MDEV-33046 - delete unnecessary synchronization with dict_stats_mutex
Timer has internal synchronization, so that calling
set_time concurrently is not a problem.
2023-12-21 11:52:21 +01:00
Marko Mäkelä
a3dd7ea09f Merge 10.4 into 10.5 2023-12-21 11:30:32 +02:00
Yuchen Pei
c73417c68e
MDEV-32986 Make regexp operator work in spider group by handler
In spider_db_mbase_util::print_item_func(), if the sql item_func has
an UNKNOWN_FUNC type, by default the spider group by handler (gbh)
transform infix to prefix. But regexp should remain infix, so we add
an if condition to account for this.
2023-12-21 10:31:12 +11:00
Julius Goryavsky
96e49c76a8 Revert "MDEV-32964: Expect DB_INTERRUPTED from wsrep_row_upd_check_foreign_constraints"
This reverts commit ddd8a90812 (additional correction required)
2023-12-20 14:20:40 +01:00
Denis Protivensky
ddd8a90812 MDEV-32964: Expect DB_INTERRUPTED from wsrep_row_upd_check_foreign_constraints
Remove DB_LOCK_WAIT return code check as it should have been resolved to
one of the other errors by that point.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-12-20 12:31:17 +01:00
Marko Mäkelä
476ff0927a MDEV-33062 innodb_undo_log_truncate=ON prevents fast shutdown
trx_purge_truncate_history(): If a fast shutdown has been initiated,
disregard innodb_undo_log_truncate=ON and return.
2023-12-19 14:45:39 +02:00
Marko Mäkelä
4c2e971841 MDEV-33052 MSAN use-of-uninitialized-value in buf_read_ahead_linear()
buf_read_ahead_linear(): Suppress a warning of comparing potentially
uninitialized FIL_PAGE_PREV and FIL_PAGE_NEXT fields.
2023-12-18 10:37:11 +02:00
Marko Mäkelä
4ae105a37d Merge 10.4 into 10.5 2023-12-18 08:59:07 +02:00
Marko Mäkelä
f98d2ef5b4 MDEV-33009 Server hangs for a long time with innodb_undo_log_truncate=ON
trx_purge_truncate_history(): Release buf_pool.flush_list_mutex
before starting a rescan of buf_pool.flush_list, to ensure that
the buf_flush_page_cleaner thread (which may be holding buf_pool.mutex)
will be able to proceed.

This fixes up commit a0f02f7438 (MDEV-32757).

Tested by: Axel Schwenke
Reviewed by: Vladislav Lesin
2023-12-15 15:38:31 +02:00
Sergei Golubchik
87a5d16911 add another missing result file
see also 4eca64e331
2023-12-15 12:11:26 +01:00
Thirunarayanan Balathandayuthapani
59a984b4d8 MDEV-32725 innodb.import_update_stats accesses uninitialized ib_table->stat_n_rows
- InnoDB should write all zeros into a table and its indexes
statistics members when table is unreadable.
2023-12-15 15:43:19 +05:30
Sergei Golubchik
4eca64e331 add missing result file 2023-12-14 20:25:58 +01:00
Marko Mäkelä
852e1383e3 MDEV-21245 InnoDB: Using a partial-field key prefix in search
ha_innobase::compare_key_parts(): If a full column index is
being replaced with a column prefix index, return Compare_keys::NotEqual.
2023-12-14 09:57:38 +11:00
Marko Mäkelä
c17aca2f11 MDEV-18322 Assertion "wrong page type" on instant ALTER TABLE
row_ins_clust_index_entry_low(): Invoke btr_set_instant() in the same
mini-transaction that has successfully inserted the metadata record.
In this way, if inserting the metadata record fails before any
undo log record was written for it, the index root page will remain
consistent.

innobase_instant_try(): Remove the btr_set_instant() call.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Matthias Leich
2023-12-13 15:01:50 +02:00
Sergei Golubchik
81609d8625 cleanup: remove innodb-specific code around update_auto_increment()
removed dead code
2023-12-12 15:21:43 +01:00
Marko Mäkelä
68e7909be9 MDEV-31000 Assertion failed on ALTER TABLE...page_compressed=1
ha_innobase::check_if_supported_inplace_alter(): On ALTER_OPTIONS,
if innodb_file_per_table=1 and the table resides in the system tablespace,
require that the table be rebuilt (and moved to an .ibd file).

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Matthias Leich
2023-12-12 14:40:45 +02:00
Marko Mäkelä
bd01029255 MDEV-29972 Crash emitting "Unsupported meta-data version number" error message
row_import_read_meta_data(): Use ER_NOT_SUPPORTED_YET instead of
ER_IO_READ_ERROR to have a matching error message pattern.
2023-12-12 15:09:31 +11:00
Marko Mäkelä
a356a940d2 MDEV-32971 Assertion !recv_sys.is_corrupt_fs() failed on recovery
recv_recovery_from_checkpoint_start(): Relax a too strict debug assertion
that occasionally fails in the test encryption.innodb-redo-nokeys
when fil_ibd_load() returns FIL_LOAD_INVALID due to missing crypt_info.

This assertion had been removed in MariaDB Server 10.8 as part of
commit 685d958e38 (MDEV-14425).
2023-12-08 10:55:13 +02:00
Marko Mäkelä
c8346c0bac MDEV-31939 Adaptive flush recommendation ignores dirty ratio and checkpoint age
buf_flush_page_cleaner(): Pass pct_lwm=srv_max_dirty_pages_pct_lwm
(innodb_max_dirty_pages_pct_lwm) to
page_cleaner_flush_pages_recommendation() unless the dirty page ratio
of the buffer pool is below that. Starting with
commit d4265fbde5 we used to always
pass pct_lwm=0.0, which was not intended.

Reviewed by: Vladislav Vaintroub
2023-12-08 09:31:34 +02:00
Thirunarayanan Balathandayuthapani
d5a6ea36f3 MDEV-32242 innodb.doublewrite test case always gets skipped
- Split the doublewrite test into two test (doublewrite,
doublewrite_debug) to reduce the execution time of the test
- Removed big_test tag for the newly added test case
- Made doublewrite test as non-debug test
- Added search pattern to make sure that InnoDB uses doublewrite buffer
- Replaced all kill_mysqld.inc with shutdown_mysqld.inc and
zero shutdown timeout
- Removed the case where fsp_flags got corrupted. Because from commit
3da5d047b8 (MDEV-31851) onwards,
doublewrite buffer removes the conversion the fsp flags from buggy
10.1 format

Thanks to Marko Mäkelä for providing the non-debug test
2023-12-07 18:44:28 +05:30
Yuchen Pei
c33ca17c17
Merge branch '10.4' into 10.5 2023-12-07 17:52:44 +11:00
Yuchen Pei
13dd787530
Merge branch '10.4' into 10.5 2023-12-07 16:38:00 +11:00
Yuchen Pei
ba94778d2c
MDEV-32753 Make spider init queries compatible with oracle sql mode
Remove ORACLE from the (session) sql_mode in connections made with sql
service to run init queries

The connection is new and the global variable value takes effect
rather than the session value from the caller of spider_db_init.
2023-12-07 16:01:42 +11:00
Sergei Golubchik
69389c03b1
MDEV-32683 Spider engine does not load with non-default alter-algorithm
specify algorithm/lock explicitly, don't depend on server settings
2023-12-07 16:01:41 +11:00
Yuchen Pei
9259b4b849
MDEV-32485 Fix Spider upgrade failure caused by duplication in mysql.func 2023-12-07 16:01:41 +11:00
Yuchen Pei
30af987259
MDEV-32507 Spider: Use $MTR_SUITE_DIR for init-file files
This should fix certain CI builds where the spider suite test files
and the main suite test files do not follow the same relative paths
relations as the mariadb source.
2023-12-07 16:01:41 +11:00
Yuchen Pei
70283aca34
MDEV-32515 Use $MYSQLD_LAST_CMD in spider/bugfix.mdev_30370
$MYSQLD_CMD uses .1 as the defaults-group-suffix, which could cause
the use of the default port (3306) or socket, which will fail in
environment where these defaults are already in use by another server.

Adding an extra --defaults-group-suffix=.1.1 does not help, because
the first flag wins.

So we use $MYSQLD_LAST_CMD instead, which uses the correct suffix.

The extra innodb buffer pool warning is irrelevant to the goal of the
test (running --wsrep-recover with --plug-load-add=ha_spider should
not cause hang)
2023-12-07 16:01:41 +11:00
Yuchen Pei
d8f5d2bef0
MDEV-22979 MDEV-27233 MDEV-28218 Fixing spider init bugs
Fix spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218) while
preventing regression on old ones (MDEV-30370, MDEV-29904)

Two things are changed:

First, Spider initialisation is made fully synchronous, i.e. it no
longer happens in a background thread. Adapted from the original fix
by nayuta for MDEV-27233. This change itself would cause failure when
spider is initialised early, by plugin-load-add, due to dependency on
Aria and udf function creation, which are fixed in the second and
third parts below. Requires SQL Service, thus porting earlier versions
requires MDEV-27595

Second, if spider is initialised before udf_init(), create udf by
inserting into `mysql.func`, otherwise do it by `CREATE FUNCTION` as
usual. This change may be generalised in MDEV-31401.

Also factor out some clean-up queries from deinit_spider.inc for use
of spider init tests.

A minor caveat is that early spider initialisation will fail if the
server is bootstrapped for the first time, due to missing `mysql`
database which needs to be created by the bootstrap script.
2023-12-07 16:01:41 +11:00
Yuchen Pei
afe63ec614
MDEV-27095 clean up spd_init_query.h
Removing procedures that were created and dropped during init.

This also fixes a race condition where mtr test with
plugin-load-add=ha_spider.so causes post test check to fail as it
expects the procedures to still be there.
2023-12-07 16:01:40 +11:00
Yuchen Pei
f0af56be01
MDEV-27095 installing one spider plugin should not trigger others
There are several plugins in ha_spider: spider, spider_alloc_mem,
spider_wrapper_protocols, spider_rewrite etc.

INSTALL PLUGIN foo SONAME ha_spider causes all the other ones to be
installed by the init queries where foo is any of the plugins.

This introduces unnecessary complexiy. For example it reads
mysql.plugins to find all other plugins, causing the hack of moving
spider plugin init to a separate thread.

To install all spider related plugins, install soname ha_spider should
be used instead.

This also fixes spurious rows in mysql.plugin when installing say only
the spider plugin with `plugin-load-add=SPIDER=ha_spider.so`:

select * from mysql.plugin;
name	dl
spider_alloc_mem	ha_spider.so # should not be here
spider_wrapper_protocols	ha_spider.so # should not be here

Adapted from part of the reverted commit
c160a115b8.
2023-12-07 16:01:40 +11:00
Yuchen Pei
13896f73df
MDEV-28683 Spider: create conn on demand when direct delete
We do this in 10.4 at the same place where in 10.5+ dml_init() is
called which does the same thing, among other things.
2023-12-07 11:02:31 +11:00
Thirunarayanan Balathandayuthapani
d5fc34db4c MDEV-29092 FOREIGN_KEY_CHECKS does not prevent non-copy alter from creating invalid FK structure
Problem:
========
- InnoDB should have two keys on the same column for the self
referencing foreign key relation.

Solution:
=========
- Allow self referential foreign key relation to work with one
key.
2023-12-06 14:29:17 +05:30
Yuchen Pei
9d15c3e35a
MDEV-29020 Reduce default spider bg sts/crd thread counts 2023-12-05 20:59:58 +11:00
Yuchen Pei
ddd5449c57
[fixup] post-merge spider fixup
MDEV-32524: a couple missed magic numbers
MDEV-26247: a couple missed goto statements that could lead to memory leak
2023-12-05 14:33:16 +11:00
Sergei Golubchik
98a39b0c91 Merge branch '10.4' into 10.5 2023-12-02 01:02:50 +01:00
Sergei Golubchik
5f89045221 MDEV-22230 fix failing test
followup for 5bb31bc882
2023-11-30 13:56:26 +01:00
Marko Mäkelä
89a5a8d234 MDEV-32269 InnoDB after ALTER TABLE…IMPORT TABLESPACE may not be crash safe
mtr_t::commit(): If IMPORT TABLESPACE is first-time-dirtying blocks,
acquire both log_sys.mutex and log_sys.flush_order_mutex to assign
a valid m_commit_lsn so that the block will be inserted into the
correct position of buf_pool.flush_list.

This fixes occasional debug assertion failures when running the
regression test suite.

Reviewed by: Vladislav Lesin
2023-11-30 09:43:36 +02:00
Marko Mäkelä
cd79f10211 MDEV-31441 BLOB corruption on UPDATE of PRIMARY KEY with FOREIGN KEY
row_upd_clust_rec_by_insert(): If we are resuming from a lock wait,
reset the 'disowned' flag of the BLOB pointers in 'entry' that we
copied from 'rec' on which we had invoked btr_cur_disown_inherited_fields()
before the lock wait started. In this way, the inserted record with
the updated PRIMARY KEY value will have the BLOB ownership associated
with itself, like it is supposed to be.

Note: If the lock wait had been aborted, then rollback would have
invoked btr_cur_unmark_extern_fields() and no corruption would be possible.

Reviewed by: Vladislav Lesin
Tested by: Matthias Leich
2023-11-29 18:57:57 +02:00
Vlad Lesin
968061fd9c MDEV-28682 gcol.gcol_purge contaminates further execution of innodb.gap_locks
ha_innobase::extra() invokes check_trx_exists() unconditionally even for
not supported operations. check_trx_exists() creates and registers trx_t
object if THD does not contain pointer to it. If ha_innobase::extra() does
not support some operation, it just invokes check_trx_exists() and quites.
If check_trx_exists() creates and registers new trx_t object for such
operation, it will never be freed and deregistered.

For example, if ha_innobase::extra() is invoked from purge thread with
operation = HA_EXTRA_IS_ATTACHED_CHILDREN, like it goes in
gcol.gcol_purge test, trx_t object will be registered, but not
deregisreted, and this causes innodb.gap_lock failure, as "SHOW ENGINE
INNODB STATUS" shows information about unexpected transaction at the end
of trx_sys.trx_list.

The fix is not to invoke check_trx_exists() for unsupported operations
in ha_innobase::extra().

Reviewed by: Marko Mäkelä
2023-11-29 19:50:52 +03:00