Commit graph

197238 commits

Author SHA1 Message Date
Lena Startseva
7bef86ea74 MDEV-31460: main.order_by_pack_big fails with view-protocol
Fixed tests:
main.order_by_pack_big - disabled view-protocol for some queries
because the view is created with wrong column name if column
name > 64 symbols
2023-09-15 17:58:28 +07:00
Yuchen Pei
0f870914d4
Merge branch '10.5' into 10.6 2023-09-15 15:23:37 +10:00
Yuchen Pei
cf81626307
Merge branch '10.4' into 10.5 2023-09-15 15:21:48 +10:00
Yuchen Pei
18990f0073
MDEV-32157 MDEV-28856 Spider: Tests, documentation, small fixes and cleanups
Removed some redundant hint related string literals from
spd_db_conn.cc

Clean up SPIDER_PARAM_*_[CHAR]LEN[S]

Adding tests covering monitoring_kind=2. What it does is that it reads
from mysql.spider_link_mon_servers with matching db_name, table_name,
link_id, and does not do anything about that...

How monitoring_* can be useful: in the deprecated spider high
availability feature, when one remote fails, spider will try another
remote, which apparently makes use of these table parameters.

A test covering the query_cache_sync table param. Some further tests
on some spider table params.

Wrapper should be case insensitive.

Code documentation on spider priority binary tree.

Add an assertion that static_key_cardinality is always -1. All tests
pass still
2023-09-15 13:15:33 +10:00
Yuchen Pei
3b3200e24a
MDEV-32157 MDEV-28856 Spider: drop server in tests
This helps eliminate "server exists" failures

Also, spider/bugfix.mdev_29676, when enabled after MDEV-29525 is
pushed will fail because we have not --recorded the result. But the
failure will only emerge when working on MDEV-31138 where we manually
re-enable this test, so let's worry about that then.
2023-09-15 13:12:55 +10:00
Yuchen Pei
b70d8fbf18
Merge branch '10.5' into 10.6 2023-09-15 12:12:46 +10:00
Yuchen Pei
68a002071b
MDEV-29502 Fix some issues with spider direct aggregate
The direct aggregate mechanism sems to be only intended to work when
otherwise a full table scan query will be executed from the spider
node and the aggregation done at the spider node too. Typically this
happens in sub_select(). In the test spider.direct_aggregate_part
direct aggregate allows to send COUNT statements directly to the data
nodes and adds up the results at the spider node, instead of iterating
over the rows one by one at the spider node.

By contrast, the group by handler (GBH) typically sends aggregated
queries directly to data nodes, in which case DA does not improve the
situation here.

That is why we should fix it by disabling DA when GBH is used.

There are other reasons supporting this change. First, the creation of
GBH results in a call to change_to_use_tmp_fields() (as opposed to
setup_copy_fields()) which causes the spider DA function
spider_db_fetch_for_item_sum_funcs() to work on wrong items. Second,
the spider DA function only calls direct_add() on the items, and the
follow-up add() needs to be called by the sql layer code. In
do_select(), after executing the query with the GBH, it seems that the
required add() would not necessarily be called.

Disabling DA when GBH is used does fix the bug. There are a few
other things included in this commit to improve the situation with
spider DA:

1. Add a session variable that allows user to disable DA completely,
this will help as a temporary measure if/when further bugs with DA
emerge.

2. Move the increment of direct_aggregate_count to the spider DA
function. Currently this is done in rather bizarre and random
locations.

3. Fix the spider_db_mbase_row creation so that the last of its row
field (sentinel) is NULL. The code is already doing a null check, but
somehow the sentinel field is on an invalid address, causing the
segfaults. With a correct implementation of the row creation, we can
avoid such segfaults.
2023-09-15 12:08:25 +10:00
Yuchen Pei
e95e9a221f
Merge branch '10.4' into 10.5 2023-09-15 12:04:44 +10:00
Yuchen Pei
96760d3acc
MDEV-31787 MDEV-26151 Add a test exercising non-0 spider_casual_read
Also:
- clean up spider_check_and_get_casual_read_conn() and
  spider_check_and_set_autocommit()
- remove a couple of commented out code blocks
2023-09-15 11:15:41 +10:00
Yuchen Pei
d59334da94
MDEV-31673 [fixup] Fixing indentation from previous mdev-31673 patch 2023-09-15 11:01:16 +10:00
Anel Husakovic
15cd8542cf MDEV-32004: Cosmetic fixes
- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Anel Husakovic
8d6ae0f2f9 MDEV-32004: Remove extra server_<num>_1 connections during initialization
- Remove extra connections in the form of `server_number_1` for the same server
  during initialization of servers in the `rpl_init.inc` file.
- Remove disconnecting and reconnecting to the same connections,
  since they are not used by the test.
- Update comments about the above.

- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Anel Husakovic
2534e5bc0b MDEV-32004: Parse error in mtr tests when using rpl_check_server_ids parameter
- Fix the calling of the assertion condition when `rpl_check_server_ids` parameter is used.
- Fix comments regarding the default usage and configuration files
extension in this case.

- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Marko Mäkelä
6a470db552 Merge 10.5 into 10.6 2023-09-14 15:25:53 +03:00
Marko Mäkelä
81e60f1a0a MDEV-32163 Crash recovery fails after DROP TABLE in system tablespace
fseg_free_extent(): After fsp_free_extent() succeeded, properly
mark the affected pages as freed. We failed to write FREE_PAGE records.

This bug was revealed or caused by
commit e938d7c18f (MDEV-32028).
2023-09-14 15:17:27 +03:00
Anel Husakovic
b1ab4ec4e2 Remove duplicated default client include from replication my.cnf
- `default_client` is included already in rpl_1slave_base.cnf`, so
remove it from `my.cnf`
- Remove option group for `mysqld` server as and add comment how to
override specific settings for specific server

- Reviewer: <brandon.nesterenko@mariadb.com>
2023-09-14 12:56:41 +02:00
Yuchen Pei
d8e9f3d981
MDEV-31673 MDEV-29502 Remove spider_db_handler::need_lock_before_set_sql_for_exec
This function trivially returns false
2023-09-14 16:37:34 +10:00
Yuchen Pei
cb1965bd9d
Merge branch '10.4' into 10.5 2023-09-14 16:30:11 +10:00
Marko Mäkelä
0f9acce3f2 Merge 10.5 into 10.6 2023-09-14 09:01:15 +03:00
Marko Mäkelä
cce76df5cc Fix cmake -DWITH_INNODB_AHI=OFF
This fixes up commit 6cc88c3db1

Thanks to Markus Mäkelä for reporting the build failure.
2023-09-14 08:58:41 +03:00
Brandon Nesterenko
1407f99963 MDEV-31177: SHOW SLAVE STATUS Last_SQL_Errno Race Condition on Errored Slave Restart
The SQL thread and a user connection executing SHOW SLAVE STATUS
have a race condition on Last_SQL_Errno, such that a slave which
previously errored and stopped, on its next start, SHOW SLAVE STATUS
can show that the SQL Thread is running while the previous error is
also showing.

The fix is to move when the last error is cleared when the SQL
thread starts to occur before setting the status of
Slave_SQL_Running.

Thanks to Kristian Nielson for his work diagnosing the problem!

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>
Kristian Nielson <knielsen@knielsen-hq.org>
2023-09-13 12:01:47 -06:00
Brandon Nesterenko
7de0c7b569 MDEV-31038: rpl.rpl_xa_prepare_gtid_fail clean up
- Removed commented out and unused lines.
- Updated test to reference true failure of timeout
  rather than deadlock
- Switched save variables from MTR to user
- Forced relay-log purge to not potentially re-execute
  an already prepared transaction
2023-09-13 10:59:26 -06:00
Daniel Black
1831f8e4d7 MDEV-31369 Disable TLS v1.0 and 1.1 for MariaDB
Remove TLSv1.1 from the default tls_version system variable.

Output a warning if TLSv1.0 or TLSv1.1 are selected.

Thanks Tingyao Nian for the feature request.
2023-09-13 20:17:29 +10:00
Sergei Golubchik
9e9cefde2a post-merge fix 2023-09-13 12:10:43 +02:00
Oleg Smirnov
5fe8d0d559 MDEV-31315 Add client_ed25519.dll to the list of plugins shipped with HeidiSQL
There is a list of plugins in the WiX configuration file for HeidiSQL,
and the installer only installs DLLs from that list although the HeidiSQL
portable archive may include other plugins.

This commit adds client_ed25519.dll to this list and also rearranges
the list alphabetically, so it is easier to verify its contents
2023-09-13 11:17:27 +10:00
Marko Mäkelä
d20a4da23d MDEV-32150 InnoDB reports corruption on 32-bit platforms with ibd files sizes > 4GB
buf_read_page_low(): Use 64-bit arithmetics when computing the
file byte offset. In other calls to fil_space_t::io() the offset
was being computed correctly, for example by
buf_page_t::physical_offset().
2023-09-12 15:16:31 +03:00
Marko Mäkelä
736901b443 MDEV-30100 fixup: Remove a failing debug assertion
trx_purge_truncate_history(): Remove a debug assertion that
had originally been added in
commit 0de3be8cfd (MDEV-30671).
In trx_t::commit_empty() we do not have any efficient way to rewind
rseg.needs_purge to an accurate value that would satisfy this
debug assertion.

Note: No correctness property should be violated here. At the point
where the debug assertion was located, we had already established
that purge_sys.sees(rseg.needs_purge) holds, that is, it is safe
to remove everything from rseg.
2023-09-12 12:25:51 +03:00
Marko Mäkelä
3c840ae746 MDEV-26782 fixup: Remove dead code
trx_undo_reuse_cached(): Assert that this is being invoked on the
persistent rollback segment of the transaction, and remove dead code
that was handling cached temporary undo log. This was missed in
commit 51e62cb3b3 (MDEV-26782).
2023-09-12 12:03:35 +03:00
sjaakola
a3cbc44b24 MDEV-31833 replication breaks when using optimistic replication and replica is a galera node
MariaDB async replication SQL thread was stopped for any failure
in applying of replication events and error message logged for the failure
was: "Node has dropped from cluster". The assumption was that event applying
failure is always due to node dropping out.
With optimistic parallel replication, event applying can fail for natural
reasons and applying should be retried to handle the failure. This retry
logic was never exercised because the slave SQL thread was stopped with first
applying failure.

To support optimistic parallel replication retrying logic this commit will
now skip replication slave abort, if node remains in cluster (wsrep_ready==ON)
and replication is configured for optimistic or aggressive retry logic.

During the development of this fix, galera.galera_as_slave_nonprim test showed
some problems. The test was analyzed, and it appears to need some attention.
One excessive sleep command was removed in this commit, but it will need more
fixes still to be fully deterministic. After this commit galera_as_slave_nonprim
is successful, though.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-12 02:37:30 +02:00
Julius Goryavsky
1adfdfbd90 galera: wsrep-lib sumbodule update 2023-09-12 00:38:48 +02:00
Daniele Sciascia
ef4b59fa5c MDEV-32051 Failed to insert streaming client
- Deterministic test to reproduce the warning
- Update wsrep-lib to fix the issue

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-12 00:37:12 +02:00
Jan Lindström
fee138a123 MDEV-31988 : galera_partition test: assertion due to unallowed state transition
Test case is starting too many servers that are not really
needed for original problem testing. This fix reduces
number of servers to make test case smaller and more
robust.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-12 00:31:54 +02:00
Jan Lindström
632a503ce7 MDEV-29861 : Galera "notify" test cases hang
Problem was that if wsrep_notify_cmd was set it was called
with a new status "joined" it tries to connect to the server
to update some table, but the server isn't initialized yet,
it's not listening for connections. So the server waits for the
script to finish, script waits for mariadb client to connect,
and the client cannot connect, because the server isn't listening.

Fix is to call script only when Galera has already formed a
view or when it is synched or donor.

This fix also enables following test cases:
* galera.MW-284
* galera.galera_binlog_checksum
* galera_var_notify_ssl_ipv6

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-11 23:06:15 +02:00
Thirunarayanan Balathandayuthapani
a03b8cd0a2 MDEV-32145 Disable read-ahead for temporary tablespace
- Lifetime of temporary tables is expected to be short, it would
seem to make sense to assume that all temporary tablespace pages
will remain in the buffer pool. It doesn't make sense to have
read-ahead for pages of temporary tablespace
2023-09-11 18:02:53 +05:30
Marko Mäkelä
cdd2fa7fc5 MDEV-32134 InnoDB hang in buf_flush_wait_LRU_batch_end()
buf_flush_page_cleaner(): Before finishing a batch, wake up any threads
that are waiting for buf_pool.done_flush_LRU.

This should fix a hung shutdown that we observed
after SET GLOBAL innodb_buffer_pool_size started was executed
to shrink the InnoDB buffer pool.
2023-09-11 14:54:50 +03:00
Marko Mäkelä
466d9f5ff3 MDEV-32103 InnoDB ALTER TABLE is not crash-safe
Starting with commit 4ff5311dec
log_write_up_to(trx->commit_lsn, true) in DDL operations could end up
being a no-op, because trx->commit_lsn would be 0.

trx_flush_log_if_needed(): Revert an incorrect attempt to ensure
that DDL operations are crash-safe.

trx_t::commit(std::vector<pfs_os_file_t> &), ha_innobase::rename_table():
Set trx_t::flush_log_later so that trx_t::commit_in_memory() will
retain trx_t::commit_lsn for the final durability call.

Tested by: Matthias Leich
2023-09-11 14:54:17 +03:00
Marko Mäkelä
4a8291fc5f MDEV-30531 Corrupt index(es) on busy table when using FOREIGN KEY
lock_wait(): Never return the transient error code DB_LOCK_WAIT.
In commit 78a04a4c22 (MDEV-29869)
some assignments assign trx->error_state = DB_SUCCESS were removed,
and it was possible that the field was left at its initial value
DB_LOCK_WAIT.

The test case for this is nondeterministic; without this fix, it
would only occasionally fail.

Reviewed by: Vladislav Lesin
2023-09-11 14:52:05 +03:00
Marko Mäkelä
e039720bf3 MDEV-32096 Parallel replication lags because innobase_kill_query() may fail to interrupt a lock wait
lock_sys_t::cancel(trx_t*): Remove, and merge to its only caller
innobase_kill_query().

innobase_kill_query(): Before reading trx->lock.wait_lock,
do acquire lock_sys.wait_mutex, like we did before
commit e71e613353 (MDEV-24671).
In this way, we should not miss a recently started lock wait
by the killee transaction.

lock_rec_lock(): Add a DEBUG_SYNC "lock_rec" for the test case.

lock_wait(): Invoke trx_is_interrupted() before entering the wait,
in case innobase_kill_query() was invoked some time earlier and
some longer-running operation did not check for interrupts.
As suggested by Vladislav Lesin, do not overwrite
trx->error_state==DB_INTERRUPTED with DB_SUCCESS.
This would avoid a call to trx_is_interrupted() when the test is
modified to use the DEBUG_SYNC point lock_wait_start instead of lock_rec.
Avoid some redundant loads of trx->lock.wait_lock; cache the value
in the local variable wait_lock.

Deadlock::check_and_resolve(): Take wait_lock as a parameter and
return wait_lock (or -1 or nullptr). We only need to reload
trx->lock.wait_lock if lock_sys.wait_mutex had been released
and reacquired.

trx_t::error_state: Correctly document the data member.

trx_lock_t::was_chosen_as_deadlock_victim: Clarify that other threads
may set the field (or flags in it) while holding lock_sys.wait_mutex.

Thanks to Johannes Baumgarten for reporting the problem and testing
the fix, as well as to Kristian Nielsen for suggesting the fix.

Reviewed by: Vladislav Lesin
Tested by: Matthias Leich
2023-09-11 14:51:02 +03:00
Marko Mäkelä
0dd25f28f7 Merge 10.5 into 10.6 2023-09-11 14:46:39 +03:00
Marko Mäkelä
ef569c324d MDEV-21679 fixup for s390x
Some s390x environments include
https://github.com/madler/zlib/pull/410
and a more pessimistic compressBound: (sourceLen * 16 + 2308) / 8 + 6.
Let us adjust the recently enabled tests accordingly.
2023-09-11 12:32:44 +03:00
Sergei Golubchik
d890aca6b5 "un-skip" more skipped tests
* version_compile_os can be "linux-systemd", not equal to "Linux"
* main.no-threads forces no-threads scheduler, a check whether it
  has one_thread_per_connection is guaranteed to fail.
2023-09-11 11:23:26 +02:00
Marko Mäkelä
384eb570a6 MDEV-32144 Debug assertion failure w == MAYBE_NOP in mtr_t::memcpy()
trx_undo_write_trx_xid(): Silence the debug assertion by passing
a template parameter that causes us to not care that the contents of
the page did not actually change and no log record would be written.
This debug assertion could fail if XA PREPARE was executed multiple
times with the same XID.
2023-09-11 11:48:15 +03:00
Marko Mäkelä
f8f7d9de2c Merge 10.4 into 10.5 2023-09-11 11:29:31 +03:00
Marko Mäkelä
65c99207e0 MDEV-23841: Memory leak in innodb_monitor_validate()
innodb_monitor_validate(): Let item_val_str() allocate the memory
in THD, so that it will be available to innodb_monitor_update().
In this way, there is no need to allocate another buffer, and
no problem if the call to innodb_monitor_update() is skipped due
to an invalid value that is passed to another configuration parameter.

There are some other callers to st_mysql_sys_var::val_str()
that validate configuration parameters that are related to FULLTEXT INDEX,
but they will allocate memory by invoking thd_strmake().
2023-09-11 10:27:21 +03:00
Marko Mäkelä
5299f0c45e MDEV-21664 Add opt files for have_innodb_Xk.inc
Currently include/have_innodb_4k.inc etc. files only check that the
server is running with the corresponding page size. I think it would
be more convenient if they actually enforced the setting.
2023-09-11 09:09:02 +03:00
Marko Mäkelä
d4fd4ae4cf MDEV-21679 innodb_zip.index_large_prefix_4k fails with ER_TOO_BIG_ROWSIZE
The test innodb_zip.index_large_prefix_4k would not run unless it is
invoked as
./mtr --mysqld=--innodb-page-size=4k innodb_zip.index_large_prefix_4k

This test was originally developed to cover an option that was removed
in commit 0c92794db3. Starting with
MariaDB Server 10.2, which introduced innodb_default_row_format=dynamic,
the option innodb_large_prefix had become useless.

Let us remove some of the stale tests and adjust the outcome to the
expected behaviour.
2023-09-11 09:06:01 +03:00
Marko Mäkelä
86f6129ca2 MDEV-21678 innodb_gis.gis_split_nan fails with ER_CANT_CREATE_GEOMETRY_OBJECT
Let us avoid inserting the rows fid=714 and fid=715, because we would
evaluate g=NULL for them, and NULL values are not allowed in InnoDB
SPATIAL INDEX.

Also, let the test run on any page size, and on non-debug builds.
2023-09-11 08:12:58 +03:00
Sergei Golubchik
7d7ea79916 fix "Undefined subroutine &Manager::mtr_lastlinesfromfile"
and "Undefined subroutine &Manager::move"
2023-09-09 08:20:05 +02:00
Sergei Golubchik
fba4abf3b9 MDEV-32128 wrong table name in innodb's "row too big" errors 2023-09-08 19:15:33 +02:00
Sergei Golubchik
a6c0184534 MDEV-31970 ASAN errors in grn_obj_unlink / ha_mroonga::clear_indexes upon index operations
from upstream
2023-09-08 19:12:35 +02:00