Commit graph

3991 commits

Author SHA1 Message Date
Sergei Krivonos
7d6617e966 MDEV-19129: Xcode compatibility update: mysql-test-run.pl: rename $opt_vs_config to $multiconfig to use with other cmake multiconfig generators 2021-10-21 16:48:00 +03:00
Sergei Krivonos
2e844a08f7 MDEV-19129: Xcode compatibility update: mysql-test-run.pl 2021-10-21 12:24:54 +03:00
Sergei Golubchik
1223cfe1d3 MDEV-25802 mtr: race condition if the test quickly restarts twice
expect file is always removed before starting a server.
So if it exists here, it means the server started successfully,
mysqltest continued executing the test, created a new expect file,
and shut down the server. All while we were waiting for the server
to start.

In other words, if the expect file exists, the server did actually start.
Even if it isn't running now.

This fixes occasional failures of innodb.log_corruption (in 10.6)
2021-07-06 16:12:13 +02:00
Sergei Golubchik
6a466db00a MDEV-25857 MTR should report at least last test that was executed in case of shutdown and not-completed
* return a success/failure value from mysqld_start()
  and don't error out / exit in mysqld_start(), the caller will do
* pass the correct $mysqld object into check_expected_crash_and_restart()
  instead of searching for it inside. Search in the caller
* so that when a failed restart changes $mysqld->{proc}, mtr would
  still detect it as a failed mysqld (by updating $proc to match)

also: log the server command line into the server error log
2021-07-06 16:12:13 +02:00
Dmitriy Karpovskiy
f776fa96b4 MDEV-24135: Print warnings in XML, save test retries in XML, save the combinations in XML, replace the special symbols in the XML comment 2021-04-12 21:34:37 +02:00
Sergei Golubchik
dd9e5827a6 mtr --gdb: fix for --rr and for a warning
use _RR_TRACE_DIR=dir instead of -o dir, as the former can store
multiple traces in dir (if, e.g., the test restarts mysqld)

suppress uninitialized warning when $exe is undefined (--manual-XXX)
2021-03-01 13:22:34 +01:00
Sergei Golubchik
feacc0aaf2 unify mtr handling of debuggers
"debugger" is anything that wraps execution of a target
binary (mysqld or mysqltest). Currently the list includes:
gdb, ddd, dbx, lldb, valgrind, strace, ktrace, rr,
devenv, windbg, vsjitdebugger.

for every debugger xxx, mtr will recognize four options:
--xxx, --boot-xxx, --manual-xxx, --client-xxx.
They all support an optional "=string" argument. String
being a semicolon-separated list of commands (e.g. for gdb)
or one (not semicolon-separated) command line of options
(e.g. for valgrind). Or both (e.g. --gdb='-quiet -nh;info files'

In embedded both --xxx and --client-xxx work.

Functionality changed/removed:
* --rr-args is gone
* --rr-dir is gone
* --manual-debug is gone
* --debugger={devenv|vc|windbg|vc_express|vsjitdebugger} is gone
* --strace-option is gone
* --stracer={strace|ktrace} is gone
* --valgrind only enables it for the server, not for everything
* --valgrind-all is gone
* --valgrind-mysqltest is gone
* --valgrind-mysqld is gone
* --valgrind-options is gone
* --valgrind-option is gone
* --valgrind-path is gone
* --callgrind is gone
* one cannot combine --valgrind --gdb anymore
* valgrind report doesn't add a fake test line to the output
* vc and vcexpress on windows are no longer supported
2021-02-22 19:43:08 +01:00
Sergei Golubchik
3b0b4e614c cleanup: remove dead code in mtr 2021-02-22 19:27:12 +01:00
Sergei Golubchik
1a99958545 mtr: --client-gdb='<gdb commands>' 2021-01-24 11:35:55 +01:00
Anel Husakovic
a82209ca31 MDEV-24177 && MDEV-24178
- MDEV-24177: main.sp2 test fails: Result length mismatch
- MDEV-24178: main.upgrade_MDEV-19650 test fails: Result length mismatch

Reviewed by: serg@mariadb.com
2020-12-07 09:56:06 +01:00
Anel Husakovic
cd927dd345 MDEV-23769: MTR can abort before it prints the test result summary
- Patch is solving generating report on warning
        To repeat the error run single worker:
        ```
        ./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st --force --parallel 1
        ```
        or `N` workers with `N+1` tests with failures and `force`
        ```
        ./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st grant5 --force --parallel 2
        ```
- Patch is doing cosmetic fix of `current_test` log file which holds the old log value of test `CURRENT TEST:..` in `mark_log()` in case of `unknown option` and as such
  the logic which is using it's content doesn't output valid log content and doesn't generate valid `$test->{'comment'}` message.asdf

- Closing the socket/handler after the removing the handler from IO for
consistency

Reviewed by: serg@mariadb.com
2020-11-10 21:20:44 +01:00
Marko Mäkelä
620ea816ad Merge 10.1 into 10.2 2020-10-21 14:02:04 +03:00
Anel Husakovic
d1af93a5e8 Update mtr help
- Based on patch: d6a983351c5a454bd0cb113852f
- Update combination example for 10.2 (commit 2a3fe45dd2 added change
for 10.3+)
```
==============================================================================

TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
rpl.rpl_invoked_features 'innodb,mix'    [ pass ]   1677
rpl.rpl_invoked_features 'innodb,row'    [ pass ]   3516
rpl.rpl_invoked_features 'innodb,stmt'   [ pass ]   1609
--------------------------------------------------------------------------
```
- `gdb` option will be added during the merge
2020-10-20 17:35:33 +02:00
Anel Husakovic
0627c4ae21 Updated mtr help
- Updated combination example to use `innodb_plugin`
```
$ ./mysql-test/mtr rpl.rpl_invoked_features # no rpl.rpl_invoked_features,mix,xtradb_plugin

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
rpl.rpl_invoked_features 'innodb_plugin,mix' [ pass ]    904
rpl.rpl_invoked_features 'mix,xtradb'    [ pass ]   1707
rpl.rpl_invoked_features 'innodb_plugin,row' [ pass ]    927
rpl.rpl_invoked_features 'row,xtradb'    [ pass ]    828
rpl.rpl_invoked_features 'innodb_plugin,stmt' [ pass ]    855
rpl.rpl_invoked_features 'stmt,xtradb'   [ pass ]    952
--------------------------------------------------------------------------
```
- `gdb` option can take optional gdb arguments (good documentation in
KB)
2020-10-20 17:33:42 +02:00
Aleksey Midenkov
2b832151ad MDEV-23787 mtr --rr fixes
1. rr record -h randomizes number of processors. Disable THREAD_POOL_SIZE check.

2. check for kernel.perf_event_paranoid for user-friendly error message.
2020-10-06 15:07:06 +03:00
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Daniel Black
beec8404fa MDEV-17076: mtr int options aren't negative 2020-07-28 09:08:36 +10:00
Daniel Black
99af3cbc85 MDEV-17076: mtr max-{core,datadir} 0 means 0
There was no ability to set the mtr arguments of:
* --max-save-core; and
* --max-save-datadir

to 0. This is desireable in an automatied scenario where space
is limited hence targeting 10.1 branch.

We take away the 0 means unlimited aspect for these,
however, perl can handle some big numbers so they may as well be
close enough to unlimited for all meaningful purposes.
2020-07-28 09:08:36 +10:00
Teodor Mircea Ionita
1ce97358bb MDEV-17076: increment only if saving occurs 2020-07-28 09:08:36 +10:00
Marko Mäkelä
ea2bc974dc Merge 10.1 into 10.2 2020-07-01 12:03:55 +03:00
Aleksey Midenkov
e3104c4a8c MDEV-22179 rr support for mtr
* --rr-dir to change store dir
* --rr-arg doesn't enable --rr (good for scripts)
* Bootstrap is saved to rr.boot
2020-06-24 10:15:21 +03:00
Daniel Black
37c88445e3 mtr: use env for perl
On FreeBSD, perl isn't in /usr/bin, its in /usr/local/bin or
elsewhere in the path.

Like storage/{maria/unittest/,}ma_test_* , we use /usr/bin/env to
find perl and run it.
2020-06-23 03:24:46 +02:00
Aleksey Midenkov
009ef36d9a MDEV-22179 rr support for mtr review
* --rr-arg instead of --rr_option
* Bootstrap saved to rr.bootstrap
* Replication slaves are saved to rr.N dirs
* Perl coding fixes
2020-06-22 12:28:11 +05:30
Sachin
804ed12e0e MDEV-22179 rr(record and replay) support for mtr
This feature adds the support for rr in mtr. These 2 options are added
--rr         run   the mysqld in rr record mode
--rr_option= run the rr with custom record option, for multiple
             options use --rr_option= for each option.
             For example
              ./mtr main.view --rr_option=-h --rr_option=-u --rr_option=-c=23
--boot-rr    run the mysqld performing bootstrap in rr record mode

Recording are stored in mysql-test/var/rr folder.
To run recording please run
rr replay var/rr/mysql-X

Limitations
Restart will create a new recording.
Repeat will work on same recording , So might be harder to debug.
If test create the multiple instance of mariadb all will be stored in var/rr
2020-06-22 12:28:09 +05:30
Sergei Golubchik
29ed04cb6d add stress suite to the list of default suites to run 2020-06-04 23:52:11 +02:00
Sergei Golubchik
9fd8f1b264 mtr: update titlebar when the test ends, not when it starts
otherwise it reaches "0 tests left" state and then waits for
a few minutes  for all workers to complete their tests.

show failures. account for retries.
2020-05-27 15:56:40 +02:00
Marko Mäkelä
e380f44742 Merge 10.1 into 10.2 2020-05-20 11:13:40 +03:00
Rasmus Johansson
2c4a2f2007 MDEV-22636 XML output for mtr doesn't work with valgrind option 2020-05-19 15:24:39 +00:00
Rasmus Johansson
395ed66b3b MDEV-22636 XML output for mtr doesn't work with valgrind option 2020-05-19 15:23:29 +00:00
Rasmus Johansson
e9a33a5392 MDEV-22631 some test causes MTR interruption without generating summary and XML 2020-05-19 10:51:28 +00:00
Marko Mäkelä
6bc4444d7c Merge 10.1 into 10.2 2020-05-13 11:12:31 +03:00
Vladislav Vaintroub
8c4b526121 Windows, mtr : Fix "Subroutine HAVE_WIN32_CONSOLE redefined at (eval 25) line 1." 2020-05-07 00:40:48 +02:00
Vladislav Vaintroub
1af74d523a postfix after e3f5789ac0 - var/log/stdout.log contains escape sequences. 2020-05-05 12:49:29 +02:00
Rasmus Johansson
95fa7bc89d MDEV-22273 jUnit patch: xml test result differs from MTR output in case if retry 2020-05-04 15:53:04 +00:00
Oleksandr Byelkin
ca091e6372 Merge branch '10.1' into 10.2 2020-05-02 08:44:17 +02:00
Oleksandr Byelkin
23c6fb3e62 Merge branch '5.5' into 10.1 2020-04-30 17:36:41 +02:00
Daniel Black
de8c9b538f
mysql-test-run.pl - fix strict subs in HAVE_WIN32_CONSOLE (#1521)
Fix mtr error:

Bareword "HAVE_WIN32_CONSOLE" not allowed while "strict subs" in use at mysql-test-run.pl line 387.
Execution of mysql-test-run.pl aborted due to compilation errors.

Added in e3f5789ac0
2020-04-30 04:03:24 +02:00
Vladislav Vaintroub
e3f5789ac0 mysql-test-run.pl - show remaining test count and estimated time on Windows
Port this functionality from to Windows.
It requires Win32::Console module, which is already included into
Strawberry perl
2020-04-29 22:39:44 +02:00
Marko Mäkelä
c06845d6f0 Merge 10.1 into 10.2 2020-04-27 13:28:13 +03:00
Anel Husakovic
bc1be39972 Fix failure for ipv6 not enabled
In case of ipv6 not enabled tests like `main.ipv6, rpl.rpl_ipv6` failed on
aarch buildbot.
Fix it by following commits 70dcb46e98 and 0bae1957dd for
`10.2`.
2020-04-24 14:05:42 +02:00
Elena Stepanova
5d856760fb MDEV-22349 MTR re-bootstrap modifies environment variable MYSQLD_BOOTSTRAP_CMD 2020-04-23 18:54:36 +03:00
Marko Mäkelä
9e43ca8e8c Merge 10.1 into 10.2 2020-04-15 15:05:45 +03:00
Daniel Black
ae688808fa mtr: Only old windows patch-2.5.9 needs --binary
Windows GNU patch 2.7.6 is ok without it.

So account for the old buildbot version for now.

Linux works without it.

--binary fails on FreeBSD-12.0:

$ patch --version
patch 2.0-12u11 FreeBSD
$ patch --binary
patch: unrecognized option `--binary'
2020-04-14 16:58:53 +03:00
Marko Mäkelä
ccaec18b39 Merge 10.1 into 10.2 2020-04-14 16:13:35 +03:00
Vicențiu Ciorbaru
0b7a79c6b0 Revert "mtr: remove --binary from patch args"
This reverts commit 1749a68968.

The reason why we need --binary for patch is because of a bug in
patch.exe 2.5.9. We need to supply binary otherwise the patch program
crashes.
2020-04-13 16:25:32 +03:00
Daniel Black
1749a68968 mtr: remove --binary from patch args
This causes problems on FreeBSD which doesn't have a patch
that supports this.

Linux and Windows don't require it either.

Was added in c39877071a without
explaination.
2020-04-13 13:49:39 +03:00
Sergei Golubchik
e40ed0e881 fix incorrect merge 2020-04-12 18:07:40 +02:00
Rasmus Johansson
5720db2b43 MDEV-22176 Add JUnit support to MTR to generate XML test result
A new parameter has been added called xml-report, with which the
filename of the XML file is given to which the XML result is
written. There is also xml-package for adding a package value in
the XML output. Example usage:
./mysql-test-run.pl main.events_bugs innodb.count_distinct
main.explain_json innodb.file_format_defaults json.json_no_table
--suite=main,innodb,json --force --xml-report=build123456789.xml
--xml-package=simpletestrun
2020-04-07 09:27:51 +00:00
Marko Mäkelä
c9ec1cc751 Merge 10.1 into 10.2 2020-03-20 15:51:30 +02:00
Sergei Golubchik
5c1ed707a3 mtr: update heuristics for --parallel=auto
to work better for CPUs with more than 2000 bogomips.
old behavior is preserved if less than 2500 bogomips.
2020-03-14 12:38:26 +01:00