Commit graph

195714 commits

Author SHA1 Message Date
Anel Husakovic
15cd8542cf MDEV-32004: Cosmetic fixes
- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Anel Husakovic
8d6ae0f2f9 MDEV-32004: Remove extra server_<num>_1 connections during initialization
- Remove extra connections in the form of `server_number_1` for the same server
  during initialization of servers in the `rpl_init.inc` file.
- Remove disconnecting and reconnecting to the same connections,
  since they are not used by the test.
- Update comments about the above.

- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Anel Husakovic
2534e5bc0b MDEV-32004: Parse error in mtr tests when using rpl_check_server_ids parameter
- Fix the calling of the assertion condition when `rpl_check_server_ids` parameter is used.
- Fix comments regarding the default usage and configuration files
extension in this case.

- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Marko Mäkelä
81e60f1a0a MDEV-32163 Crash recovery fails after DROP TABLE in system tablespace
fseg_free_extent(): After fsp_free_extent() succeeded, properly
mark the affected pages as freed. We failed to write FREE_PAGE records.

This bug was revealed or caused by
commit e938d7c18f (MDEV-32028).
2023-09-14 15:17:27 +03:00
Anel Husakovic
b1ab4ec4e2 Remove duplicated default client include from replication my.cnf
- `default_client` is included already in rpl_1slave_base.cnf`, so
remove it from `my.cnf`
- Remove option group for `mysqld` server as and add comment how to
override specific settings for specific server

- Reviewer: <brandon.nesterenko@mariadb.com>
2023-09-14 12:56:41 +02:00
Yuchen Pei
d8e9f3d981
MDEV-31673 MDEV-29502 Remove spider_db_handler::need_lock_before_set_sql_for_exec
This function trivially returns false
2023-09-14 16:37:34 +10:00
Yuchen Pei
cb1965bd9d
Merge branch '10.4' into 10.5 2023-09-14 16:30:11 +10:00
Brandon Nesterenko
1407f99963 MDEV-31177: SHOW SLAVE STATUS Last_SQL_Errno Race Condition on Errored Slave Restart
The SQL thread and a user connection executing SHOW SLAVE STATUS
have a race condition on Last_SQL_Errno, such that a slave which
previously errored and stopped, on its next start, SHOW SLAVE STATUS
can show that the SQL Thread is running while the previous error is
also showing.

The fix is to move when the last error is cleared when the SQL
thread starts to occur before setting the status of
Slave_SQL_Running.

Thanks to Kristian Nielson for his work diagnosing the problem!

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>
Kristian Nielson <knielsen@knielsen-hq.org>
2023-09-13 12:01:47 -06:00
Brandon Nesterenko
7de0c7b569 MDEV-31038: rpl.rpl_xa_prepare_gtid_fail clean up
- Removed commented out and unused lines.
- Updated test to reference true failure of timeout
  rather than deadlock
- Switched save variables from MTR to user
- Forced relay-log purge to not potentially re-execute
  an already prepared transaction
2023-09-13 10:59:26 -06:00
Daniel Black
1831f8e4d7 MDEV-31369 Disable TLS v1.0 and 1.1 for MariaDB
Remove TLSv1.1 from the default tls_version system variable.

Output a warning if TLSv1.0 or TLSv1.1 are selected.

Thanks Tingyao Nian for the feature request.
2023-09-13 20:17:29 +10:00
Sergei Golubchik
9e9cefde2a post-merge fix 2023-09-13 12:10:43 +02:00
Oleg Smirnov
5fe8d0d559 MDEV-31315 Add client_ed25519.dll to the list of plugins shipped with HeidiSQL
There is a list of plugins in the WiX configuration file for HeidiSQL,
and the installer only installs DLLs from that list although the HeidiSQL
portable archive may include other plugins.

This commit adds client_ed25519.dll to this list and also rearranges
the list alphabetically, so it is easier to verify its contents
2023-09-13 11:17:27 +10:00
Marko Mäkelä
d20a4da23d MDEV-32150 InnoDB reports corruption on 32-bit platforms with ibd files sizes > 4GB
buf_read_page_low(): Use 64-bit arithmetics when computing the
file byte offset. In other calls to fil_space_t::io() the offset
was being computed correctly, for example by
buf_page_t::physical_offset().
2023-09-12 15:16:31 +03:00
sjaakola
a3cbc44b24 MDEV-31833 replication breaks when using optimistic replication and replica is a galera node
MariaDB async replication SQL thread was stopped for any failure
in applying of replication events and error message logged for the failure
was: "Node has dropped from cluster". The assumption was that event applying
failure is always due to node dropping out.
With optimistic parallel replication, event applying can fail for natural
reasons and applying should be retried to handle the failure. This retry
logic was never exercised because the slave SQL thread was stopped with first
applying failure.

To support optimistic parallel replication retrying logic this commit will
now skip replication slave abort, if node remains in cluster (wsrep_ready==ON)
and replication is configured for optimistic or aggressive retry logic.

During the development of this fix, galera.galera_as_slave_nonprim test showed
some problems. The test was analyzed, and it appears to need some attention.
One excessive sleep command was removed in this commit, but it will need more
fixes still to be fully deterministic. After this commit galera_as_slave_nonprim
is successful, though.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-12 02:37:30 +02:00
Julius Goryavsky
1adfdfbd90 galera: wsrep-lib sumbodule update 2023-09-12 00:38:48 +02:00
Daniele Sciascia
ef4b59fa5c MDEV-32051 Failed to insert streaming client
- Deterministic test to reproduce the warning
- Update wsrep-lib to fix the issue

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-12 00:37:12 +02:00
Jan Lindström
fee138a123 MDEV-31988 : galera_partition test: assertion due to unallowed state transition
Test case is starting too many servers that are not really
needed for original problem testing. This fix reduces
number of servers to make test case smaller and more
robust.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-12 00:31:54 +02:00
Jan Lindström
632a503ce7 MDEV-29861 : Galera "notify" test cases hang
Problem was that if wsrep_notify_cmd was set it was called
with a new status "joined" it tries to connect to the server
to update some table, but the server isn't initialized yet,
it's not listening for connections. So the server waits for the
script to finish, script waits for mariadb client to connect,
and the client cannot connect, because the server isn't listening.

Fix is to call script only when Galera has already formed a
view or when it is synched or donor.

This fix also enables following test cases:
* galera.MW-284
* galera.galera_binlog_checksum
* galera_var_notify_ssl_ipv6

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-11 23:06:15 +02:00
Marko Mäkelä
ef569c324d MDEV-21679 fixup for s390x
Some s390x environments include
https://github.com/madler/zlib/pull/410
and a more pessimistic compressBound: (sourceLen * 16 + 2308) / 8 + 6.
Let us adjust the recently enabled tests accordingly.
2023-09-11 12:32:44 +03:00
Sergei Golubchik
d890aca6b5 "un-skip" more skipped tests
* version_compile_os can be "linux-systemd", not equal to "Linux"
* main.no-threads forces no-threads scheduler, a check whether it
  has one_thread_per_connection is guaranteed to fail.
2023-09-11 11:23:26 +02:00
Marko Mäkelä
384eb570a6 MDEV-32144 Debug assertion failure w == MAYBE_NOP in mtr_t::memcpy()
trx_undo_write_trx_xid(): Silence the debug assertion by passing
a template parameter that causes us to not care that the contents of
the page did not actually change and no log record would be written.
This debug assertion could fail if XA PREPARE was executed multiple
times with the same XID.
2023-09-11 11:48:15 +03:00
Marko Mäkelä
f8f7d9de2c Merge 10.4 into 10.5 2023-09-11 11:29:31 +03:00
Marko Mäkelä
65c99207e0 MDEV-23841: Memory leak in innodb_monitor_validate()
innodb_monitor_validate(): Let item_val_str() allocate the memory
in THD, so that it will be available to innodb_monitor_update().
In this way, there is no need to allocate another buffer, and
no problem if the call to innodb_monitor_update() is skipped due
to an invalid value that is passed to another configuration parameter.

There are some other callers to st_mysql_sys_var::val_str()
that validate configuration parameters that are related to FULLTEXT INDEX,
but they will allocate memory by invoking thd_strmake().
2023-09-11 10:27:21 +03:00
Marko Mäkelä
5299f0c45e MDEV-21664 Add opt files for have_innodb_Xk.inc
Currently include/have_innodb_4k.inc etc. files only check that the
server is running with the corresponding page size. I think it would
be more convenient if they actually enforced the setting.
2023-09-11 09:09:02 +03:00
Marko Mäkelä
d4fd4ae4cf MDEV-21679 innodb_zip.index_large_prefix_4k fails with ER_TOO_BIG_ROWSIZE
The test innodb_zip.index_large_prefix_4k would not run unless it is
invoked as
./mtr --mysqld=--innodb-page-size=4k innodb_zip.index_large_prefix_4k

This test was originally developed to cover an option that was removed
in commit 0c92794db3. Starting with
MariaDB Server 10.2, which introduced innodb_default_row_format=dynamic,
the option innodb_large_prefix had become useless.

Let us remove some of the stale tests and adjust the outcome to the
expected behaviour.
2023-09-11 09:06:01 +03:00
Marko Mäkelä
86f6129ca2 MDEV-21678 innodb_gis.gis_split_nan fails with ER_CANT_CREATE_GEOMETRY_OBJECT
Let us avoid inserting the rows fid=714 and fid=715, because we would
evaluate g=NULL for them, and NULL values are not allowed in InnoDB
SPATIAL INDEX.

Also, let the test run on any page size, and on non-debug builds.
2023-09-11 08:12:58 +03:00
Sergei Golubchik
7d7ea79916 fix "Undefined subroutine &Manager::mtr_lastlinesfromfile"
and "Undefined subroutine &Manager::move"
2023-09-09 08:20:05 +02:00
Sergei Golubchik
fba4abf3b9 MDEV-32128 wrong table name in innodb's "row too big" errors 2023-09-08 19:15:33 +02:00
Sergei Golubchik
a6c0184534 MDEV-31970 ASAN errors in grn_obj_unlink / ha_mroonga::clear_indexes upon index operations
from upstream
2023-09-08 19:12:35 +02:00
Sergei Golubchik
3267606bcd MDEV-31966 Server crash upon inserting into Mroonga table with compressed column 2023-09-08 19:12:35 +02:00
Sergei Golubchik
53fd63254f remove groonga examples
to follow the similar upstream change
2023-09-08 19:12:35 +02:00
Daniel Black
1815719a5b oqgraph: remove clang warnings
-Wdeprecated-copy-with-user-provided-copy was causing a few errors on
things that where defined in a way that was implicit. By removing code
it now compiles without warnings.

tested with fc38 / clang-16
2023-09-08 10:50:15 +01:00
Marko Mäkelä
34c283ba1b MDEV-32132 DROP INDEX followed by CREATE INDEX may corrupt data
ibuf_set_bitmap_for_bulk_load(): Port a bug fix that was made as part of
commit 165564d3c3 (MDEV-30009)
in MariaDB Server 10.5.19.
2023-09-08 11:28:21 +03:00
Alexander Barkov
5544ea2eda MDEV-32130 Port MySQL test on protocol bug #106352 to MariaDB
Port the test case from MySQL to MariaDB:

MySQL fix Bug#33813951, Change-Id: I2448e3f2f36925fe70d882ae5681a6234f0d5a98.

Function test_simple_temporal() from MySQL ported from C++ to pure C.

This includes one change:
-  DIE_UNLESS(field->type == MYSQL_TYPE_DATETIME);
+  DIE_UNLESS(field->type == MYSQL_TYPE_TIMESTAMP);

The bound param of SELECT ? is TIMESTAMP in this code.
MySQL returns it back as DATETIME. MariaDB preserves TIMESTAMP.

Code packaged for commit by Daniel Black.
2023-09-08 18:20:57 +10:00
Kristian Nielsen
e937a64d46 MDEV-10356: rpl.rpl_parallel_temptable failure due to incorrect commit optimization of temptables
The problem was that parallel replication of temporary tables using
statement-based binlogging could overlap the COMMIT in one thread with a DML
or DROP TEMPORARY TABLE in another thread using the same temporary table.
Temporary tables are not safe for concurrent access, so this caused
reference to freed memory and possibly other nastiness.

The fix is to disable the optimisation with overlapping commits of one
transaction with the start of a later transaction, when temporary tables are
in use. Then the following event groups will be blocked from starting until
the one using temporary tables is completed.

This also fixes occasional test failures of rpl.rpl_parallel_temptable seen
in Buildbot.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2023-09-07 14:40:05 +02:00
Sergei Golubchik
d762e9d943 MDEV-32093 long uniques break old->new replication
recalculate long unique hash in Write_rows_log_event
and Update_rows_log_event.

normally generated columns (stored and indexed virtual)
are deterministic and their values don't need to be recalculated
on the slave as they're already present in the row image.
but the long unique hash function was changed in MDEV-27653,
so a row event from the old master will have the old hash,
but a table created on the new slave will need a new hash.
2023-09-06 22:38:41 +02:00
Sachin
c53cb718b7 MDEV-22722 Assertion "inited==NONE" failed in handler::ha_index_init on the slave during UPDATE
test case only
2023-09-06 22:38:41 +02:00
Sergei Golubchik
65b3c89430 MDEV-32015 insert into an empty table fails with hash unique
don't enable bulk insert when table->s->long_unique_table
2023-09-06 22:38:41 +02:00
Sergei Golubchik
382c543f53 MDEV-32012 hash unique corrupts index on virtual blobs
as always when copying record[0] aside one needs to detach
Field_blob::value's from it, and restore them when record[0]
is restored from a backup.
2023-09-06 22:38:41 +02:00
Sergei Golubchik
4d96dba767 MDEV-25369 mysqlbinlog (mariadb-binlog) -T/--table option
clarify the help text
2023-09-06 22:38:41 +02:00
Sergei Golubchik
fe86d04ea7 MDEV-30904 "rpm --setugids" breaks PAM authentication
move user/group creation from %post to %pre as Fedora packaging
guidelines say. This allows to use %attr() to set the correct
ownership of files
2023-09-06 22:38:41 +02:00
Sergei Golubchik
e78ce63291 MDEV-17711 Assertion `arena_for_set_stmt== 0' failed in LEX::set_arena_for_set_stmt upon SET STATEMENT
restore SET STATEMENT variables between statements in a multi-statement
2023-09-06 22:38:41 +02:00
Sergei Golubchik
28f7725731 wolfssl: enable chacha cyphers and secure negotiation
compaitibility with:
* chacha - mobile devices
* secure negotiation - openssl 3
2023-09-06 22:38:41 +02:00
Monty
b08474435f Fix compression tests for s390x
The problem is that s390x is not using the default bzip library we use
on other platforms, which causes compressed string lengths to be differnt
than what mtr tests expects.

Fixed by:
- Added have_normal_bzip.inc, which checks if compress() returns the
  expected length.
- Adjust the results to match the expected one
  - main.func_compress.test & archive.archive
- Don't print lengths that depends on compression library
  - mysqlbinlog compress tests & connect.zip
- Don't print DATA_LENGTH for SET column_compression_zlib_level=1
  - main.column_compression
2023-09-05 12:34:39 +03:00
Aleksey Midenkov
a49b9314c1 MDEV-30836 MTR hangs after tests have completed
The problem is in manager/worker communication when worker sends
WARNINGS and then TESTRESULT. If manager yet didn't read WARNINGS
response both responses get into the same buffer, can_read() will
indicate we have data only once and we must read all the data from the
socket at once. Otherwise TESTRESULT response is lost and manager
waits it forever.

The fix now instead of single line reads the socket in a loop. But if
there is only one response in the buffer the second read will be
blocked waiting until new data arrives. That can be overcame by
blocking(0) which sets the handle into non-blocking mode. If there is
no data second read just returns undef.

The problem is non-blocking mode is not supported by all perl flavors
on Windows. Strawberry and ActiveState do not support it. Cygwin and
MSYS2 do support. There is some ioctl() hack that was known to "work"
but it doesn't do what is expected (it does not return data when there
is data). So for Windows if it is not Cygwin we disable the fix.
2023-09-05 17:43:15 +10:00
Aleksey Midenkov
848b3af816 MDEV-30836 MTR MSYS2 fix attempt
MSYS2 is basically Cygwin, except it has more easy installation (but
with tools which are not used) and it has some more control of path
conversion via MSYS2_ARG_CONV_EXCL and MSYS2_ENV_CONV_EXCL. So it
should be more Windows-friendly than Cygwin.

Installation

Similar to Cygwin, except installing patch requires additional command
run from shell:

    pacman -S patch

MSYS2 still doesn't work as it returns wierd "Bad address" when
exec-ing forked process from create_process(). Same exec from
standalone perl -e runs just fine... :(
2023-09-05 17:43:15 +10:00
Aleksey Midenkov
640cd404af MDEV-30836 MTR Cygwin fix
Cygwin is more Unix-oriented. It does not treat \n as \r\n in regexps
(fixed by \R), it supplies Unix-style paths (fixed by
mixed_path()). It does some cleanup on paths when running exe, so it
will be different in exe output (like with $exe_mysqld, comparing
basename() is enough).

Cygwin installation

1. Just install latest perl version (only base package) and
   patchutils from cygwin-setup;
2. Don't forget to add c:\cygwin64\bin into system path
   before any other perl flavors;
3. There is path-style conflict (see below), you must replace
   c:\cygwin64\bin\sh.exe with the wrapper. Run MTR with
   --cygwin-subshell-fix=do for that. Make sure you are running Cygwin
   perl for the option to work.
4. Restart buildbot via net stop buildbot; net start buildbot

Path-style conflict of Cygwin-ish Perl

Some exe paths are passed to mysqltest which are executed by a native
call. This requires native-style paths (\-style). These exe paths also
executed by Perl itself. Either by MTR itself which is not so
critical, but also by tests' --perl blocks which is impossible to
change. And if Perl detects shell-expansion or uses pipe command it
passess this exe path to /bin/sh which is Cygwin-compiled bash that
cannot work with \-style (or at least in -c processing). Thus we require
\-style on some parts of MTR execution and /-style on another parts.

The examples of tests which cover these different parts are:

    main.mysqlbinlog_row_compressed \
    main.sp_trans_log

That could be great to force Perl to use something different from
/bin/sh, but unfortunately /bin/sh is compiled-in into binary. So the
only solution left is to overwrite /bin/sh with some wrapper script
which passes the command to cmd.exe instead of bash.
2023-09-05 17:43:15 +10:00
Aleksey Midenkov
4ed583031a MDEV-30836 MTR Cygwin subshell wrapper fix
See "Path-style conflict" in "MDEV-30836 MTR Cygwin fix" for explanation.

To install subshell fix use --cygwin-subshell-fix=do
To uninstall use --cygwin-subshell-fix=remove

This works only from Cygwin environment. As long as perl on PATH is
from Cygwin you are on Cygwin environment. Check it with

     perl --version

     This is perl 5, version 36, subversion 1 (v5.36.1) built for
     x86_64-cygwin-threads-multi
2023-09-05 17:43:15 +10:00
Aleksey Midenkov
0815a3b6b5 MDEV-30836 run_test_server() refactored
run_test_server() is actually manager main loop. We move out this
function into Manager package and split into run() and
parse_protocol(). The latter is needed for the fix. Moving into
separate package helps to make some common variables which was local
to run_test_server().

Functions from the main package is now prefixed with main:: (should be
reorganized somehow later or auto-imported).
2023-09-05 17:17:44 +10:00
Aleksey Midenkov
92fb31f0b1 MDEV-30836 MTR misc improvements
1. Better logging and error reporting;
2. Worker process title;
3. Some comments

Worker process title example:

 446209 pts/2    R+     0:00 mysql-test-run.pl worker[01] :42146 -> :35027 versioning.view
 446210 pts/2    S+     0:00 mysql-test-run.pl worker[02] :42150 -> :35027 versioning.view
 446211 pts/2    S+     0:00 mysql-test-run.pl worker[03] :42154 -> :35027 versioning.foreign
 446212 pts/2    S+     0:00 mysql-test-run.pl worker[04] :42160 -> :35027 versioning.autoinc

Manager-worker localhost socket connection is represented by a pair :source -> :destination ports.

-vv Now adds --verbose to mysqltest as well, see var/mysqltest.log for the output.
2023-09-05 17:17:44 +10:00