Commit graph

1591 commits

Author SHA1 Message Date
Alexander Barkov
36eba98817 MDEV-19123 Change default charset from latin1 to utf8mb4
Changing the default server character set from latin1 to utf8mb4.
2024-07-11 10:21:07 +04:00
Alexander Barkov
c4bf4ce948 Merge remote-tracking branch 'origin/11.2' into 11.4 2024-06-17 15:46:39 +04:00
Marko Mäkelä
d34289a3e2 Merge 10.11 into 11.1 2024-06-17 09:21:50 +03:00
Marko Mäkelä
fc9005adc4 Merge 10.5 into 10.6 2024-06-12 07:51:28 +03:00
Alexander Barkov
3b80d23d02 mtr --skip-not-found did not skip suites
--skip-not-found switch tells mtr to skip not found tests instead of aborting.
But it failed to skip the test if the suite name was not found.

This problem also made the *last-N-failed builbot builders fail
to run `mtr --skip-not-found` if the last commit removed a file in
the mysql-test/include/ directory.

This commit fixes it, now the not found test is properly skipped,
no matter what component of the test name was not found:

$ ./mtr main.foo --skip-not-found foo.main
...
==============================================================================
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
foo.main                                 [ skipped ]  not found
main.foo                                 [ skipped ]  not found
--------------------------------------------------------------------------
2024-06-10 19:17:00 +02:00
Yuchen Pei
2d3e2c58b6
Merge branch '10.11' into 11.1 2024-05-31 10:54:31 +10:00
Marko Mäkelä
5ba542e9ee Merge 10.5 into 10.6 2024-05-30 14:27:07 +03:00
Oleksandr Byelkin
99b370e023 Merge branch '11.2' into 11.4 2024-05-21 19:38:51 +02:00
Dave Gosselin
4911ec1a5b mtr on FreeBSD detects core count for --parallel=auto 2024-05-16 10:07:03 +10:00
Sergei Golubchik
f0a5412037 Merge branch '11.0' into 11.1 2024-05-13 09:52:30 +02:00
Sergei Golubchik
f9807aadef Merge branch '10.11' into 11.0 2024-05-12 12:18:28 +02:00
Sergei Golubchik
7b53672c63 Merge branch '10.5' into 10.6 2024-05-08 20:06:00 +02:00
Dave Gosselin
89084c2ea4 MDEV-33078 SysInfo.pm reports incorrect CPU count on macOS
When running on macOS, MTR will ask the operating system for the core count when --parallel=auto
2024-05-07 09:08:57 +10:00
Kristian Nielsen
16aa4b5f59 Merge from 10.4 to 10.5
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-04-15 17:46:49 +02:00
Sergei Golubchik
41296a07c8 Merge branch '10.5' into 10.6 2024-04-11 13:58:22 +02:00
Sergei Golubchik
12d448fde9 mtr: increase more timeouts under debuggers
in particular, debug_sync timeout and wait_for_pos timeout
2024-04-06 00:32:29 +02:00
Marko Mäkelä
683fbced6b Merge 11.0 into 11.1 2024-03-28 12:15:36 +02:00
Vladislav Vaintroub
5abf0fea51 mtr - synchronize output between different threads on Windows.
An attempt to fix lost output sometimes seen on buildbot.
2024-03-18 22:03:21 +01:00
Marko Mäkelä
d73baa402a Merge 10.11 into 11.0 2024-02-20 12:02:01 +02:00
Oleksandr Byelkin
fa69b085b1 Merge branch '11.3' into 11.4 2024-02-15 13:53:21 +01:00
Vladislav Vaintroub
5c5242a674 mtr - synchronize output between different threads on Windows.
An attempt to fix lost output sometimes seen on buildbot.
2024-02-06 20:03:41 +01:00
Sergei Golubchik
68f0af2bf1 test SSL MitM attack
verify that --ssl-verify-server-cert detects cert mismatch,
but with --disable-ssl-verify-server-cert the connection succeeds
2024-02-04 22:19:00 +01:00
Sergei Golubchik
1ef1bab99e MDEV-31855 validate ssl certificates using client password
if the client enabled --ssl-verify-server-cert, then
the server certificate is verified as follows:

* if --ssl-ca or --ssl-capath were specified, the cert must have
  a proper signature by the specified CA (or CA in the path)
  and the cert's hostname must match the server's hostname.
  If the cert isn't signed or a hostname is wrong - the
  connection is aborted.

* if MARIADB_OPT_TLS_PEER_FP was used and the fingerprint matches,
  the connection is allowed, if it doesn't match - aborted.

* If the connection uses unix socket or named pipes - it's allowed.
  (consistent with server's --require-secure-transport behavior)

otherwise the cert is still in doubt, we don't know if we can trust
it or there's an active MitM in progress.

* If the user has provided no password or the server requested an
  authentication plugin that sends the password in cleartext -
  the connection is aborted.

* Perform the authentication. If the server accepts the password,
  it'll send SHA2(scramble || password hash || cert fingerprint)
  with the OK packet.

* Verify the SHA2 digest, if it matches - the connection is allowed,
  otherwise it's aborted.
2024-02-04 22:18:55 +01:00
Oleksandr Byelkin
d21cb43db1 Merge branch '11.2' into 11.3 2024-02-04 16:42:31 +01:00
Sergei Golubchik
b6680e0101 Merge branch '11.0' into 11.1 2024-02-02 11:30:47 +01:00
Marko Mäkelä
9374772ecd Merge 10.11 into 11.0 2024-01-19 09:07:48 +02:00
Yuchen Pei
d06b6de305
Merge branch '10.5' into 10.6 2024-01-11 12:59:22 +11:00
Yuchen Pei
c9902a20b3
Merge branch '10.4' into 10.5 2024-01-10 18:01:46 +11:00
Sergei Golubchik
ac0ce44519 ./mtr --skip-not-found should skip combinations too
With the result like

encryption.innochecksum 'debug'   [ skipped ] combination not found

instead of

*** ERROR: Could not run encryption.innochecksum with 'debug' combination(s)
2024-01-08 13:57:51 +01:00
Oleksandr Byelkin
34272bd6a5 Merge branch '11.2' into 11.3 2023-11-14 18:33:03 +01:00
Oleksandr Byelkin
0f5613a25f Merge branch '11.0' into 11.1 2023-11-08 18:03:08 +01:00
Oleksandr Byelkin
48af85db21 Merge branch '10.11' into 11.0 2023-11-08 17:09:44 +01:00
Oleksandr Byelkin
b83c379420 Merge branch '10.5' into 10.6 2023-11-08 15:57:05 +01:00
Oleksandr Byelkin
6cfd2ba397 Merge branch '10.4' into 10.5 2023-11-08 12:59:00 +01:00
Marko Mäkelä
7b842f1536 Merge 11.2 into 11.3 2023-10-27 10:48:29 +03:00
Sergei Golubchik
78cd45b29a ./mtr --skip-not-found
New mtr option --skip-not-found makes it to show not found tests
as skipped

main.a                                   [ skipped ]  not found

(but only if the test was specified with the suite name)
and not error out early with

mysql-test-run: *** ERROR: Could not find 'a' in 'main' suite

This is useful in buildbot, on builders that generate the list
of tests dynamically.
2023-10-23 17:40:03 +02:00
Marko Mäkelä
9b2a65e41a Merge 11.0 into 11.1 2023-10-19 08:26:16 +03:00
Marko Mäkelä
be24e75229 Merge 10.11 into 11.0 2023-10-19 08:12:16 +03:00
Vladislav Vaintroub
f197f9a527 MDEV-32387 Windows - mtr output on is messed up with large MTR_PARALLEL.
Windows C runtime does not implement line buffering mode for stdio.

This sometimes makes output from different tests interleaved in MTR
MTR relies on this buffering (lines won't  output until "\n") to correctly
work in parallel scenarios.

Implement do-it-yourself line buffering on Windows, to workaround.
2023-10-10 20:53:00 +02:00
Sergei Golubchik
82174dae06 MDEV-32104 remove deprecated features
In particular:

* @@debug
  deprecated since 5.5.37
* sr_YU locale
  deprecated since 10.0.11
* "engine_condition_pushdown" in the @@optimizer_switch
  deprecated since 10.1.1
* @@date_format, @@datetime_format, @@time_format, @@max_tmp_tables
  deprecated since  10.1.2
* @@wsrep_causal_reads
  deprecated since 10.1.3
* "parser" in mroonga table comment
  deprecated since 10.2.11
2023-09-30 14:43:12 +02:00
Sergei Golubchik
04b7b3a0ca mtr bug: even if the test is skipped, process combinations
otherwise, e.g.

  ./mtr main.mysql_install_db_win_admin,innodb

results in sporadic

mysql-test-run: *** ERROR: Could not run main.mysql_install_db_win_admin with 'innodb' combination(s)

depending on whether it'll process the skip (not windows admin)
or the innodb.combinations first (if skip is processed first,
innodb combination wasn't, making the further code think that the test
doesn't have innodb combination)
2023-09-29 10:44:09 +02:00
Marko Mäkelä
0dd25f28f7 Merge 10.5 into 10.6 2023-09-11 14:46:39 +03:00
Marko Mäkelä
f8f7d9de2c Merge 10.4 into 10.5 2023-09-11 11:29:31 +03:00
Aleksey Midenkov
848b3af816 MDEV-30836 MTR MSYS2 fix attempt
MSYS2 is basically Cygwin, except it has more easy installation (but
with tools which are not used) and it has some more control of path
conversion via MSYS2_ARG_CONV_EXCL and MSYS2_ENV_CONV_EXCL. So it
should be more Windows-friendly than Cygwin.

Installation

Similar to Cygwin, except installing patch requires additional command
run from shell:

    pacman -S patch

MSYS2 still doesn't work as it returns wierd "Bad address" when
exec-ing forked process from create_process(). Same exec from
standalone perl -e runs just fine... :(
2023-09-05 17:43:15 +10:00
Aleksey Midenkov
640cd404af MDEV-30836 MTR Cygwin fix
Cygwin is more Unix-oriented. It does not treat \n as \r\n in regexps
(fixed by \R), it supplies Unix-style paths (fixed by
mixed_path()). It does some cleanup on paths when running exe, so it
will be different in exe output (like with $exe_mysqld, comparing
basename() is enough).

Cygwin installation

1. Just install latest perl version (only base package) and
   patchutils from cygwin-setup;
2. Don't forget to add c:\cygwin64\bin into system path
   before any other perl flavors;
3. There is path-style conflict (see below), you must replace
   c:\cygwin64\bin\sh.exe with the wrapper. Run MTR with
   --cygwin-subshell-fix=do for that. Make sure you are running Cygwin
   perl for the option to work.
4. Restart buildbot via net stop buildbot; net start buildbot

Path-style conflict of Cygwin-ish Perl

Some exe paths are passed to mysqltest which are executed by a native
call. This requires native-style paths (\-style). These exe paths also
executed by Perl itself. Either by MTR itself which is not so
critical, but also by tests' --perl blocks which is impossible to
change. And if Perl detects shell-expansion or uses pipe command it
passess this exe path to /bin/sh which is Cygwin-compiled bash that
cannot work with \-style (or at least in -c processing). Thus we require
\-style on some parts of MTR execution and /-style on another parts.

The examples of tests which cover these different parts are:

    main.mysqlbinlog_row_compressed \
    main.sp_trans_log

That could be great to force Perl to use something different from
/bin/sh, but unfortunately /bin/sh is compiled-in into binary. So the
only solution left is to overwrite /bin/sh with some wrapper script
which passes the command to cmd.exe instead of bash.
2023-09-05 17:43:15 +10:00
Aleksey Midenkov
4ed583031a MDEV-30836 MTR Cygwin subshell wrapper fix
See "Path-style conflict" in "MDEV-30836 MTR Cygwin fix" for explanation.

To install subshell fix use --cygwin-subshell-fix=do
To uninstall use --cygwin-subshell-fix=remove

This works only from Cygwin environment. As long as perl on PATH is
from Cygwin you are on Cygwin environment. Check it with

     perl --version

     This is perl 5, version 36, subversion 1 (v5.36.1) built for
     x86_64-cygwin-threads-multi
2023-09-05 17:43:15 +10:00
Aleksey Midenkov
92fb31f0b1 MDEV-30836 MTR misc improvements
1. Better logging and error reporting;
2. Worker process title;
3. Some comments

Worker process title example:

 446209 pts/2    R+     0:00 mysql-test-run.pl worker[01] :42146 -> :35027 versioning.view
 446210 pts/2    S+     0:00 mysql-test-run.pl worker[02] :42150 -> :35027 versioning.view
 446211 pts/2    S+     0:00 mysql-test-run.pl worker[03] :42154 -> :35027 versioning.foreign
 446212 pts/2    S+     0:00 mysql-test-run.pl worker[04] :42160 -> :35027 versioning.autoinc

Manager-worker localhost socket connection is represented by a pair :source -> :destination ports.

-vv Now adds --verbose to mysqltest as well, see var/mysqltest.log for the output.
2023-09-05 17:17:44 +10:00
Marko Mäkelä
07494006dd Merge 10.5 into 10.6 2023-08-22 09:36:35 +03:00
Marko Mäkelä
f9cc29824b Merge 10.4 into 10.5 2023-08-22 09:01:34 +03:00
Oleksandr Byelkin
c062b351f0 Make vgdb call more universal. 2023-08-21 13:00:34 +02:00