Commit graph

4110 commits

Author SHA1 Message Date
Sergei Golubchik
9ecdf860ce missing ' 2022-08-10 09:14:17 +02:00
Aleksey Midenkov
1848804840 MDEV-29023 MTR hangs after multiple failures
Passing $opt_parallel as $childs is wrong: child can be killed before
it connects and you will never decrement $childs for this.

Another problem is (and that is the cause of this bug): child can be
killed and never close server socket. This can happen f.ex. after
unmaskable KILL signal. In such case the socket is closed by reaping
the child but that never happens inside reading the socket loop in
run_test_server().

The proper design is the waitless reap of children inside the socket
loop and if there is no more children we finish the socket loop. Since
there is Windows variation where we don't control the children via
waitpid(), all the clients must normally close the socket and only
this can finish the socket loop. For Unix variation we reckon that
case as all children closed the socket but not all yet died and for
that we do final waiting waitpid() (was done before the patch as
well).

To be more complete, we now handle 3 end-of-game scenarios in Unix:

   1. all children closed socket, all children died: everything is
      handled by the socket loop;

   2. all children closed socket, not all yet died: we wait for alive
      children to die after exiting the socket loop;

   3. not all children closed socket, all children died: everything is
      handled by the socket loop.

For Windows end-of-game scenario is only one:

   All children close the socket.
2022-07-18 23:16:18 +03:00
Aleksey Midenkov
7ca5c7d8f9 MDEV-29023 waitpid() cleanup
The case for "Unknown process $ret_pid exited" is never known to be
valid. Such state is not documented for waitpid().
2022-07-18 23:16:18 +03:00
Aleksey Midenkov
1d307ed48a MDEV-29025 MTR doesn't print stack trace for bootstrap crashes 2022-07-18 23:16:18 +03:00
Aleksey Midenkov
1bdcffb028 MDEV-29025 Refactoring: moved out core_wanted() out of mysql-test-run.pl 2022-07-18 23:16:17 +03:00
Aleksey Midenkov
e9be5428a2 MDEV-28931 MTR prints detailed stack trace unconditionally
66832e3a introduced change that prints core dumps in very detailed
format. That's completely out of user-friendliness but serves as a
measure for debugging hard-reproducible bugs.

The proper way to implement this:

  1. it must be controlled by command-line and environment variable;
  2. detailed traces must be default for buildbots only, for user
     invocations normal stack traces should be printed.

Options for control are: MTR_PRINT_CORE and --print-core that accept
the following values:

  no	         Don't print core
  short	       	 Print stack trace of failed thread
  medium	 Print stack traces of all threads
  detailed       Print all stack traces with debug context
  custom:<code>  Use debugger commands <code> to print stack trace

Default setting is: short (see env_or_default() call in pre_setup())

For environment variable wrong values are silently ignored (falls back
to default setting, see env_or_default()).

Command-line option --print-core (or -C) overrides environment
variable. Its default value is 'short' if not specified explicitly
(same env_or_default() call in pre_setup()). Explicit values are
checked for validity.

--print-method option can specify by which debugger we print
cores. For Windows there is only one choice: cdb. For Unix the values
are: gdb, dbx, lldb, auto. Default value is: auto

In 'auto' we try to use all possible debuggers until success.
2022-07-18 23:16:17 +03:00
Aleksey Midenkov
ce7820eb83 MDEV-28931 --verbose option is too verbose
GetOpt::Long bundling option for convenient one-char verbosity levels:

  -v    General verbosity (file and execute operations)
  -vv   High verbosity (algorithmic considerations)
  -vvv  Debug verbosity (anything else)
2022-07-18 23:16:17 +03:00
Sergei Golubchik
f31e935c3e mtr: fix a race condition
if a test can run in cond1,cond2 and cond1,cond3 then they can happen
to run in parallel and both wanting to create test,cond1.result~
2022-06-15 14:55:35 +02:00
Alexander Barkov
9d734cdd61 Merge remote-tracking branch 'origin/10.2' into 10.3 2022-04-14 11:50:34 +04:00
Shunpoco
767d8d8335 MDEV-27448 MTR returns success (zero) upon invalid option
I change from `exit;` to `exit(1);` on a function `usage()`.
When we try to run mtr with a wrong option, a function `usage()` is called with the wrong option as its argument. In this case, because the function call `exit` in a first if statement, we get exit status 0.
2022-04-12 08:17:32 +10:00
Sergei Golubchik
a36fc80aeb Merge branch '10.2' into 10.3 2022-02-10 20:23:56 +01:00
Vladislav Vaintroub
ae33a006f7 MDEV-27738 Windows : mysql-test-run --extern does not work
Use portable quoting in mtr_add_arg.
2022-02-08 10:14:50 +01:00
Oleksandr Byelkin
41a163ac5c Merge branch '10.2' into 10.3 2022-01-29 15:41:05 +01:00
Daniel Black
68b3fa8865 MDEV-27289: mtr test for WITH_SERVER_EMBEDDED=ON reenable
mtr is checking the wrong path for the embedded executable
on out of tree builds.

The is_embedded.inc tests are also checking the version rather
than the MTR MYSQL_EMBEDDED environment variable.

As a result, a few tests are out of date in the result recordings.
2022-01-27 10:36:39 +11:00
Sergei Golubchik
153b75b576 Merge branch '10.2' into 10.3 2021-12-06 22:23:07 +01:00
Sergei Golubchik
214cad8c3b fix ./mtr --manual warning after f5441ef4da 2021-12-06 18:50:19 +01:00
Marko Mäkelä
289721de9a Merge 10.2 into 10.3 2021-11-29 10:33:06 +02:00
Sergei Golubchik
f5441ef4da MDEV-26972 MTR worker aborts after server restart failure
restore the old behavior where without a debugger mtr does not
wait for mysqld to start. It was broken in feacc0aaf2
2021-11-26 17:15:41 +01:00
Marko Mäkelä
f9b856b052 Merge 10.2 into 10.3 2021-10-21 17:39:34 +03:00
Sergei Krivonos
7d6617e966 MDEV-19129: Xcode compatibility update: mysql-test-run.pl: rename $opt_vs_config to $multiconfig to use with other cmake multiconfig generators 2021-10-21 16:48:00 +03:00
Marko Mäkelä
e4a7c15dd6 Merge 10.2 into 10.3 2021-10-21 13:41:04 +03:00
Sergei Krivonos
2e844a08f7 MDEV-19129: Xcode compatibility update: mysql-test-run.pl 2021-10-21 12:24:54 +03:00
Sergei Golubchik
6190a02f35 Merge branch '10.2' into 10.3 2021-07-21 20:11:07 +02:00
Sergei Golubchik
1223cfe1d3 MDEV-25802 mtr: race condition if the test quickly restarts twice
expect file is always removed before starting a server.
So if it exists here, it means the server started successfully,
mysqltest continued executing the test, created a new expect file,
and shut down the server. All while we were waiting for the server
to start.

In other words, if the expect file exists, the server did actually start.
Even if it isn't running now.

This fixes occasional failures of innodb.log_corruption (in 10.6)
2021-07-06 16:12:13 +02:00
Sergei Golubchik
6a466db00a MDEV-25857 MTR should report at least last test that was executed in case of shutdown and not-completed
* return a success/failure value from mysqld_start()
  and don't error out / exit in mysqld_start(), the caller will do
* pass the correct $mysqld object into check_expected_crash_and_restart()
  instead of searching for it inside. Search in the caller
* so that when a failed restart changes $mysqld->{proc}, mtr would
  still detect it as a failed mysqld (by updating $proc to match)

also: log the server command line into the server error log
2021-07-06 16:12:13 +02:00
Marko Mäkelä
6e6318b29b Merge 10.2 into 10.3 2021-04-13 10:26:01 +03:00
Dmitriy Karpovskiy
f776fa96b4 MDEV-24135: Print warnings in XML, save test retries in XML, save the combinations in XML, replace the special symbols in the XML comment 2021-04-12 21:34:37 +02:00
Marko Mäkelä
ddbc612692 Merge 10.2 into 10.3 2021-03-03 09:41:50 +02:00
Sergei Golubchik
dd9e5827a6 mtr --gdb: fix for --rr and for a warning
use _RR_TRACE_DIR=dir instead of -o dir, as the former can store
multiple traces in dir (if, e.g., the test restarts mysqld)

suppress uninitialized warning when $exe is undefined (--manual-XXX)
2021-03-01 13:22:34 +01:00
Sergei Golubchik
0ab1e3914c Merge branch '10.2' into 10.3 2021-02-22 22:42:27 +01:00
Sergei Golubchik
feacc0aaf2 unify mtr handling of debuggers
"debugger" is anything that wraps execution of a target
binary (mysqld or mysqltest). Currently the list includes:
gdb, ddd, dbx, lldb, valgrind, strace, ktrace, rr,
devenv, windbg, vsjitdebugger.

for every debugger xxx, mtr will recognize four options:
--xxx, --boot-xxx, --manual-xxx, --client-xxx.
They all support an optional "=string" argument. String
being a semicolon-separated list of commands (e.g. for gdb)
or one (not semicolon-separated) command line of options
(e.g. for valgrind). Or both (e.g. --gdb='-quiet -nh;info files'

In embedded both --xxx and --client-xxx work.

Functionality changed/removed:
* --rr-args is gone
* --rr-dir is gone
* --manual-debug is gone
* --debugger={devenv|vc|windbg|vc_express|vsjitdebugger} is gone
* --strace-option is gone
* --stracer={strace|ktrace} is gone
* --valgrind only enables it for the server, not for everything
* --valgrind-all is gone
* --valgrind-mysqltest is gone
* --valgrind-mysqld is gone
* --valgrind-options is gone
* --valgrind-option is gone
* --valgrind-path is gone
* --callgrind is gone
* one cannot combine --valgrind --gdb anymore
* valgrind report doesn't add a fake test line to the output
* vc and vcexpress on windows are no longer supported
2021-02-22 19:43:08 +01:00
Sergei Golubchik
3b0b4e614c cleanup: remove dead code in mtr 2021-02-22 19:27:12 +01:00
Sergei Golubchik
60ea09eae6 Merge branch '10.2' into 10.3 2021-02-01 13:49:33 +01:00
Sergei Golubchik
1a99958545 mtr: --client-gdb='<gdb commands>' 2021-01-24 11:35:55 +01:00
Oleksandr Byelkin
25561435e0 Merge branch '10.2' into 10.3 2020-12-23 19:28:02 +01:00
Anel Husakovic
a82209ca31 MDEV-24177 && MDEV-24178
- MDEV-24177: main.sp2 test fails: Result length mismatch
- MDEV-24178: main.upgrade_MDEV-19650 test fails: Result length mismatch

Reviewed by: serg@mariadb.com
2020-12-07 09:56:06 +01:00
Marko Mäkelä
150f447af1 Merge 10.2 into 10.3 2020-11-12 10:37:21 +02:00
Anel Husakovic
cd927dd345 MDEV-23769: MTR can abort before it prints the test result summary
- Patch is solving generating report on warning
        To repeat the error run single worker:
        ```
        ./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st --force --parallel 1
        ```
        or `N` workers with `N+1` tests with failures and `force`
        ```
        ./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st grant5 --force --parallel 2
        ```
- Patch is doing cosmetic fix of `current_test` log file which holds the old log value of test `CURRENT TEST:..` in `mark_log()` in case of `unknown option` and as such
  the logic which is using it's content doesn't output valid log content and doesn't generate valid `$test->{'comment'}` message.asdf

- Closing the socket/handler after the removing the handler from IO for
consistency

Reviewed by: serg@mariadb.com
2020-11-10 21:20:44 +01:00
Marko Mäkelä
e3d692aa09 Merge 10.2 into 10.3 2020-10-22 08:26:28 +03:00
Marko Mäkelä
620ea816ad Merge 10.1 into 10.2 2020-10-21 14:02:04 +03:00
Anel Husakovic
d1af93a5e8 Update mtr help
- Based on patch: d6a983351c5a454bd0cb113852f
- Update combination example for 10.2 (commit 2a3fe45dd2 added change
for 10.3+)
```
==============================================================================

TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
rpl.rpl_invoked_features 'innodb,mix'    [ pass ]   1677
rpl.rpl_invoked_features 'innodb,row'    [ pass ]   3516
rpl.rpl_invoked_features 'innodb,stmt'   [ pass ]   1609
--------------------------------------------------------------------------
```
- `gdb` option will be added during the merge
2020-10-20 17:35:33 +02:00
Anel Husakovic
0627c4ae21 Updated mtr help
- Updated combination example to use `innodb_plugin`
```
$ ./mysql-test/mtr rpl.rpl_invoked_features # no rpl.rpl_invoked_features,mix,xtradb_plugin

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
rpl.rpl_invoked_features 'innodb_plugin,mix' [ pass ]    904
rpl.rpl_invoked_features 'mix,xtradb'    [ pass ]   1707
rpl.rpl_invoked_features 'innodb_plugin,row' [ pass ]    927
rpl.rpl_invoked_features 'row,xtradb'    [ pass ]    828
rpl.rpl_invoked_features 'innodb_plugin,stmt' [ pass ]    855
rpl.rpl_invoked_features 'stmt,xtradb'   [ pass ]    952
--------------------------------------------------------------------------
```
- `gdb` option can take optional gdb arguments (good documentation in
KB)
2020-10-20 17:33:42 +02:00
Aleksey Midenkov
2b832151ad MDEV-23787 mtr --rr fixes
1. rr record -h randomizes number of processors. Disable THREAD_POOL_SIZE check.

2. check for kernel.perf_event_paranoid for user-friendly error message.
2020-10-06 15:07:06 +03:00
Oleksandr Byelkin
c32f71af7e Merge branch '10.2' into 10.3 2020-08-03 13:41:29 +02:00
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Alexander Barkov
d63631c3fa MDEV-19632 Replication aborts with ER_SLAVE_CONVERSION_FAILED upon CREATE ... SELECT in ORACLE mode
- Adding optional qualifiers to data types:
    CREATE TABLE t1 (a schema.DATE);
  Qualifiers now work only for three pre-defined schemas:

    mariadb_schema
    oracle_schema
    maxdb_schema

  These schemas are virtual (hard-coded) for now, but may turn into real
  databases on disk in the future.

- mariadb_schema.TYPE now always resolves to a true MariaDB data
  type TYPE without sql_mode specific translations.

- oracle_schema.DATE translates to MariaDB DATETIME.

- maxdb_schema.TIMESTAMP translates to MariaDB DATETIME.

- Fixing SHOW CREATE TABLE to use a qualifier for a data type TYPE
  if the current sql_mode translates TYPE to something else.

The above changes fix the reported problem, so this script:

    SET sql_mode=ORACLE;
    CREATE TABLE t2 AS SELECT mariadb_date_column FROM t1;

is now replicated as:

    SET sql_mode=ORACLE;
    CREATE TABLE t2 (mariadb_date_column mariadb_schema.DATE);

and the slave can unambiguously treat DATE as the true MariaDB DATE
without ORACLE specific translation to DATETIME.

Similar,

    SET sql_mode=MAXDB;
    CREATE TABLE t2 AS SELECT mariadb_timestamp_column FROM t1;

is now replicated as:

    SET sql_mode=MAXDB;
    CREATE TABLE t2 (mariadb_timestamp_column mariadb_schema.TIMESTAMP);

so the slave treats TIMESTAMP as the true MariaDB TIMESTAMP
without MAXDB specific translation to DATETIME.
2020-08-01 07:43:50 +04:00
Daniel Black
beec8404fa MDEV-17076: mtr int options aren't negative 2020-07-28 09:08:36 +10:00
Daniel Black
99af3cbc85 MDEV-17076: mtr max-{core,datadir} 0 means 0
There was no ability to set the mtr arguments of:
* --max-save-core; and
* --max-save-datadir

to 0. This is desireable in an automatied scenario where space
is limited hence targeting 10.1 branch.

We take away the 0 means unlimited aspect for these,
however, perl can handle some big numbers so they may as well be
close enough to unlimited for all meaningful purposes.
2020-07-28 09:08:36 +10:00
Teodor Mircea Ionita
1ce97358bb MDEV-17076: increment only if saving occurs 2020-07-28 09:08:36 +10:00
Marko Mäkelä
1df1a63924 Merge 10.2 into 10.3 2020-07-02 06:17:51 +03:00