Commit graph

56 commits

Author SHA1 Message Date
unknown
fe1bb69862 BUG #15895810 - REQUIRE ADDITIONAL INFORMATION WITH THE --RESULT-FILE OPTION OF MTR 2012-11-20 15:33:45 +05:30
Bjorn Munch
cb15b0712d WL #5680 MTR results written to file with well defined format
Added --result-file option, which will produce var/mtr-results.txt
Output has a simple format:

<tag> : <value>  for general info on test run
{
  <tag> : <value>
  ....
}                for each test

Output from failed tests are included but may be truncated.
See WL for more details.
2011-05-09 16:07:43 +02:00
Bjorn Munch
5cfa1ed166 Attempt at solving missing end of mtr output on Windows 2011-02-28 16:11:53 +01:00
Bjorn Munch
4a54e5adb1 Attempt at solving missing end of mtr output on Windows 2011-02-28 16:10:35 +01:00
Bjorn Munch
2c4f6e5c63 Bug #11766640 (59789) Hook the invocation of unit tests in MTR.
Added code to call 'ctest' if the needed cmake file is present
Will do so unless tests/suited named on mtr command line
Also add option to turn on/off
Will be made to look like a test 'unit-test' which counts towards total
Extracts summary report and any test failures from ctest output
Addendum: added override to turn off in PB, add back in selected invocations
2011-02-21 13:56:43 +01:00
Bjorn Munch
d651529112 Fixed copyright headers in mtr src files 2011-01-18 11:21:37 +01:00
Bjorn Munch
5a85609d6b Fixed copyright headers in mtr src files 2011-01-18 11:03:44 +01:00
Bjorn Munch
7b64f3a8ec Bug #59182 output of mysql-test-run.pl - mismatch between col names and actual col contents
New patch, avoid global $opt_parallel
I still prefer not to print workerid when not doing parallel
2011-01-12 10:27:46 +01:00
Bjorn Munch
8937cbd5ae upmerge 58896,58900,59002 2011-01-11 15:15:25 +01:00
Bjorn Munch
f3bc9f699c Bug #58896 MTR should recognise combinations as experimental without needing wildcards
Added a pattern match to cover combinations
Added to readme file
2011-01-11 10:53:22 +01:00
Bjorn Munch
bf89d47516 Bug #56787 MTR completion report should be more informative
Added counts of skipped test, inclusing how many by test itself
Also fixed misspelling in the (hitherto unused) variable name
2010-09-20 10:12:39 +02:00
Bjorn Munch
c9d57b0c0f Bug #54364 Allow multiple --experimental command line options for MTR
Convert --experimental into a multi option
Size of diff is caused by intenting code put into a for loop
2010-06-10 10:34:16 +02:00
Bjorn Munch
afb8b414da Bug #47663 mtr --parallel has weird output
Some output is written, some is not
Finally concluded it's a Perl bug: after running with parallel threads
  for a while, print suddenly ignores all but the first argument.
Workaround: concatenate all the arguments into one, except in output that
  only comes before we start running tests
2009-11-04 13:42:22 +01:00
Bjorn Munch
c22826bfad Bug #48240 "Test suite timeout" and "Too many tests failed" is masked from status page
Prepend "Completed", "Timeout" or "Too many failed" to summary line
2009-10-26 14:28:51 +01:00
Bjorn Munch
dcf6aae407 Bug #48130 Expected failures should not count towards max-test-fail
Test batches may be terminated too early
Avoid counting exp-fail tests
2009-10-17 18:34:56 +02:00
Bjorn Munch
4d676f314c Bug #47075 Wildcards in experimental test names destroyed when tested first time
Extract substr into local variable
2009-09-03 08:19:54 +02:00
Bjorn Munch
ad365a5c07 Bug #45190 mtr should report some statistics even if aborting after too many tests fail
Added calls to mtr_report_stats() also after timeout or too many failures
2009-06-02 10:00:45 +02:00
Bjorn Munch
4404b6a915 Bug #43659 MTR2 should report worker data for tests when using --parallel > 1
Adding "w# " before the test result
Also enable --parallel=2 so this can be tested.
2009-05-28 13:02:03 +02:00
Vladislav Vaintroub
2943d2b7e9 Bug #42804 --parallel option does not work for MTR under ActiveState
perl 

The problem here was the method how MTR gets its unique thread ids.
Prior to this patch, the method to do it was to maintain a global 
table of pid,mtr_unique_id) pairs. The table was backed by a text 
file. The table was cleaned up one in a while and dead processes leaking
unique_ids were determined with with kill(0) or with scripting tasklist
on Windows.

This method is flawed specifically on native Windows Perl. fork() is 
implemented with starting a new thread, give it a syntetic negative PID
(threadID*(-1)), until this thread creates a new process with exec()
However,  neither tasklist nor any other native Windows tool can cope with
negative perl PIDs. This lead to incorrect determination of dead process 
and reusing already used mtr_unique_id.

The patch introduces alternative portable  method of solving unique-id 
problem. When a process needs a unique id in range [min...max], it just 
starts  to open files named min, min+1,...max in a loop . After file is 
opened, we do non-blocking flock(). When flock() succeeds, process has 
allocated the ID. When process dies, file is unlocked . Checks for zombies 
are not necessary.

Since the change would create a co-existence problems with older version
of MTR, because of different way to calculate IDs, the default ID range
is changed from 250-299 to 300-349.

Another fix that was necessary enable --parallel option was to serialize 
spawn() calls on Windows. specifically, IO redirects needed to be protected.

This patch also fixes hanging CRTL-C (as described in Bug #38629) for the
"new"  MTR. The fix was already in 6.0 and is now downported.
2009-04-23 13:35:02 +02:00
Bjorn Munch
ba70e8eac7 Bug #43917 MTR2 does not report accurate test statistics when using the 'repeat=n'
option
In practice, only the last run of the test was counted
Add a separate counter rep_failures for failures before last run
2009-04-01 13:58:30 +02:00
Bjorn Munch
3cf777e1a2 Bug #43148 MTR2 + PB2 should display correct number of failures
Retried tests are reported the same as ordinary ones
Prepend "retry-" to fail or pass for retried tests
2009-03-09 12:59:34 +01:00
Daniel Fischer
fff57e9dfc address review comments 2009-02-25 15:00:17 +01:00
Daniel Fischer
bde5f92b31 Bug#42888 for MySQL 5.1. Add collections of test runs to make it both configurable and transparent what kinds of tests we run during integration testing. Implement filter list in mysql-test-run.pl to filter out failures of experimental test cases, using a new status code "exp-fail", so we can tell regressions ("fail") from failures of test cases that are still in development ("exp-fail"). 2009-02-24 12:53:34 +01:00
Bjorn Munch
d3f4c75e55 Bug #42535 MTR2 output format for skipped tests should be identical to MTR 1
Changed [ skip ] to [ skipped ] and removed optional .
2009-02-02 11:26:19 +01:00
Luis Soares
0030ff1457 BUG#42485: Pushbuild2 and some build tools do not work together with MTR2
It seems that the length of the thick line printed by mtr when printing the 
suite name differs from mtr1 and mtr2, affecting the mtr filtering by PB2.
      
This patch addresses it by restoring the thick line length to 78 (original
length) instead of 60 (the one in mtr2).
2009-02-01 14:14:39 +01:00
Magnus Svensson
ad94278ba0 WL#4198 mtr.pl v2
- Improve error message for "found warnings in server log file"
2009-01-28 10:28:48 +01:00
Magnus Svensson
8a73a4beca Merge 2009-01-27 15:10:15 +01:00
Luis Soares
47e19cf2d5 merge: 5.1-rpl (with merge from main) -> 5.1-rpl 2009-01-26 17:06:39 +01:00
Magnus Svensson
bbdd0576ee WL#4189 mtr.pl v2
- Clear test variables "comment" and "logfile" to make sure thay aren't
  already set from previous run of same test
 - Print warning if test result already set and set it anyway
2009-01-24 13:02:27 +01:00
Luis Soares
df8543868d merge: 5.1 -> 5.1-rpl
conflicts:
  Text conflict in client/mysqltest.cc
  Text conflict in mysql-test/include/wait_until_connected_again.inc
  Text conflict in mysql-test/lib/mtr_report.pm
  Text conflict in mysql-test/mysql-test-run.pl
  Text conflict in mysql-test/r/events_bugs.result
  Text conflict in mysql-test/r/log_state.result
  Text conflict in mysql-test/r/myisam_data_pointer_size_func.result
  Text conflict in mysql-test/r/mysqlcheck.result
  Text conflict in mysql-test/r/query_cache.result
  Text conflict in mysql-test/r/status.result
  Text conflict in mysql-test/suite/binlog/r/binlog_index.result
  Text conflict in mysql-test/suite/binlog/r/binlog_innodb.result
  Text conflict in mysql-test/suite/rpl/r/rpl_packet.result
  Text conflict in mysql-test/suite/rpl/t/rpl_packet.test
  Text conflict in mysql-test/t/disabled.def
  Text conflict in mysql-test/t/events_bugs.test
  Text conflict in mysql-test/t/log_state.test
  Text conflict in mysql-test/t/myisam_data_pointer_size_func.test
  Text conflict in mysql-test/t/mysqlcheck.test
  Text conflict in mysql-test/t/query_cache.test
  Text conflict in mysql-test/t/rpl_init_slave_func.test
  Text conflict in mysql-test/t/status.test
2009-01-23 13:22:05 +01:00
Magnus Svensson
9c76ec9323 Bug#39972 mysql-test-run.pl: Confusing error message under special conditions
- Fix problem with for example ./mtr --timer, caused by new version of "Getopt::Long"
 - Evaluating the "$opt" variable as a string, returns the name of the parameter
   to be modified instead of "Getopt::Long::Callback" which is the class name
2009-01-21 11:17:16 +01:00
Luis Soares
b2cdc3b6cb merge: 5.1 -> 5.1-rpl
conflicts:
  Text conflict in mysql-test/lib/mtr_report.pm
  Text conflict in mysql-test/mysql-test-run.pl
2009-01-08 19:03:56 +00:00
Bjorn Munch
d2e0e3ce35 Bug #40924 please print timestamp when mtr fails
Added text "Test ended at <time>" after failure reports
2008-12-03 14:11:03 +01:00
Magnus Svensson
27e8584271 Bug #38181 Please print more debug info when tests fail 2008-08-04 21:54:44 +02:00
Magnus Svensson
43e396c8d5 Turn off timestamping 2008-06-19 11:20:45 +02:00
unknown
74cdcba997 (temporarily)Turn on timestamp by default 2008-05-20 11:48:52 +02:00
unknown
1e1918141b Make printout of $name nicer
Remove debug printouts


mysql-test/lib/mtr_report.pm:
  Make printout of $name nicer
mysql-test/mysql-test-run.pl:
  Remove debug printouts
2008-05-09 11:50:36 +02:00
unknown
e70c291046 Add printouts
mysql-test/lib/mtr_report.pm:
  Add 'mtr_print'
2008-05-05 22:55:47 +02:00
unknown
7ff37ca6e1 Tune reporting 2008-05-04 23:22:41 +02:00
unknown
80377ad95e Set "timediff" to off by default 2008-05-04 21:43:49 +02:00
unknown
69d26cb560 Move opt_timer to mtr_report.pm 2008-05-04 21:40:40 +02:00
unknown
f30f2b1523 Correctly calculate failed tests 2008-05-04 21:21:11 +02:00
unknown
1e9f11d582 Print tests that has been run but shouldn't and vice versa
Use opt_start_timeout also for ndb
Remove the .result files generated by 'check_testcase'


mysql-test/lib/mtr_report.pm:
  Use "failures" to keep track of if test has previously failed
2008-05-02 12:11:04 +02:00
unknown
645db270ca Move code that saves datadirs to test server
Remove mtr_report_test_failed


mysql-test/lib/mtr_report.pm:
  Remvoe mtr_report_test_failed
2008-04-27 23:35:38 +02:00
unknown
59f38e7268 Run all check-warning and check-testcase in parallel, this also give the
oportunity to check if any other process fails during this.
Add possiblitiy to store "user_data" in a Safeprocess
Add function to get pid from a SafeProcess


mysql-test/lib/My/SafeProcess.pm:
  Add possiblitiy to store "user_data" in a Safeprocess
  Add function to get pid from a SafeProcess
mysql-test/lib/mtr_report.pm:
  Print and detected test case sideffect also in the test server
mysql-test/mysql-test-run.pl:
  Run all check-warning and check-testcase in parallel, this also give the
  oportunity to check if any other process fails during this.
2008-04-27 21:31:32 +02:00
unknown
586d8cd454 Merge pilot.mysql.com:/data/msvensson/mysql/mysql-5.1-rpl
into  pilot.mysql.com:/data/msvensson/mysql/mysql-5.1-maint2


mysql-test/lib/mtr_cases.pm:
  Auto merged
mysql-test/mysql-test-run.pl:
  Auto merged
mysql-test/lib/mtr_report.pm:
  Manual merge
2008-04-25 08:23:57 +02:00
unknown
570c4c4a89 Print name(of thread) also in mtr_report, testserver does not set name
so it will still look the same, but makes debugging much easier
2008-04-24 20:24:24 +02:00
unknown
d4a9ed4b50 Replace all - with - in $opt name 2008-04-24 14:15:15 +02:00
unknown
adc257481c Add support for running in parallel
mysql-test/lib/My/Options.pm:
  Allow $VAR as option
mysql-test/lib/My/SafeProcess.pm:
  Remove printouts
mysql-test/lib/My/Test.pm:
  New BitKeeper file ``mysql-test/lib/My/Test.pm''
2008-04-24 13:02:53 +02:00
unknown
2c4ca51050 Check warnings in servers error log as part of test case
BitKeeper/deleted/.del-rpl_bug33931-slave.opt:
  Delete: mysql-test/suite/rpl/t/rpl_bug33931-slave.opt
mysql-test/include/default_mysqld.cnf:
  Set a default name for "log-bin"
mysql-test/mysql-test-run.pl:
  Check for warnings in mysqld error log files after each testcase,
  using SQL
mysql-test/lib/mtr_cases.pm:
  Make mtr_match into a perl module
mysql-test/lib/mtr_match.pm:
  Make mtr_match into a perl module
mysql-test/lib/mtr_report.pm:
  Make mtr_match into a perl module
  Print warnings if testcase failed from warnings
mysql-test/r/information_schema.result:
  Be more selective which databases and tables are select in the queries
mysql-test/r/mysql_upgrade.result:
  Update result, mysql_upgrade will check _all_ databases
mysql-test/r/mysqlcheck.result:
  Update result, mysql_upgrade should check _all_ databases
mysql-test/r/sp-destruct.result:
  Be more selective which databases and tables are select in the queries
mysql-test/r/sp-error.result:
  Backup and restore mysql.proc table
mysql-test/r/sp-security.result:
  Be more selective which databases and tables are select in the queries
mysql-test/r/sp.result:
  Be more selective which databases and tables are select in the queries
mysql-test/suite/rpl/r/rpl_bug33931.result:
  Move the setting of debug flag into the test file instead of in -slave.opt
  Add supression
mysql-test/suite/rpl/r/rpl_idempotency.result:
  Add supression
  Add master-slave-end.inc
mysql-test/suite/rpl/t/rpl_bug33931.test:
  Move the setting of debug flag into the test file instead of in -slave.opt
  Add supression
mysql-test/suite/rpl/t/rpl_idempotency.test:
  Add supression
  Add master-slave-end.inc
mysql-test/t/information_schema.test:
  Be more selective which databases and tables are select in the queries
mysql-test/t/sp-destruct.test:
  Be more selective which databases and tables are select in the queries
mysql-test/t/sp-error.test:
  Backup and restore mysql.proc table
mysql-test/t/sp-security.test:
  Be more selective which databases and tables are select in the queries
mysql-test/t/sp.test:
  Be more selective which databases and tables are select in the queries
mysql-test/include/check-warnings.test:
  New BitKeeper file ``mysql-test/include/check-warnings.test''
mysql-test/include/mtr_warnings.sql:
  New BitKeeper file ``mysql-test/include/mtr_warnings.sql''
2008-04-08 16:51:26 +02:00