Problem was the parsing of test suite files for various tags and options.
This was done inefficiently, and include files were re-parsed for every
place they were included. This caused a delay of 20 seconds or so before
the first test started to run.
By parsing more efficiently and re-using first parse for subsequent
inclusion of the same file, time spent parsing is reduced to less than
1 second, and start appears instantaneous.
(With this patch, full ./mtr runs in 3 minutes on my laptop (release
build.)
mysql-test/suite/innodb_plugin/t/innodb_bug52663.test:
Test is fairly slow, so try to avoid getting stuck with it at the end
while other workers are idle.
- Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria"
- Fixed wrong assert in Aria
Now need to merge with latest xtradb before pushing
sql/ha_partition.cc:
Ensure that m_ordered_rec_buffer is not freed before close.
sql/mysqld.cc:
Changed to use opt_stack_trace instead of opt_pstack.
Removed references to pstack
sql/partition_element.h:
Ensure that connect_string is initialized
storage/maria/ma_key_recover.c:
Fixed wrong assert
The patch also fixes a race in rpl_stop_slave.test.
On machines with lots of CPU and memory, something like `mtr --parallel=10`
can speed up the test suite enormously. However, we have a few test cases
that run for long (several minutes), and if we are unlucky and happen to
schedule those towards the end of the test suite, we end up with most
workers idle while waiting for the last slow test to end, significantly
delaying the finish of the entire suite.
Improve this by marking the offending tests as taking "long", and trying
to schedule those tests early. This reduces the time towards the end of
the test suite run where some workers are waiting with nothing to do for
the remaining workers each to finish their last test.
Also, the rpl_stop_slave test had a race which could cause it to take
a 300 seconds debug_sync timeout; this is fixed.
Testing on a 4-core 8GB machine, this patch speeds up the test suite with
around 30% for --parallel=10 (debug build), allowing to run the entire
suite in 5 minutes.
It does work in general, the problem here was that the test name
'alter_table' matches 'main.alter_table-big' which has already been found.
Fixed by matching more explicitly (with/without suite name)
There were actually more problems in this area:
Slaves (if any) were unconditionally restarted, this appears unnecessary.
Sort criteria were suboptimal, included the test name.
Added logic to "reserve" a sequence of tests with same config for one thread
Got rid of sort_criteria hash, put it into the test case itself
Adds little sanity check that expected worker picks up test
Fixed some tests that may fail if starting on running server
Some of these fail only if *same* test is repeated.
Finally, special sorting of tests that do --force-restart
* expanding unknown env variable does not abort mtr
* have_archive, blackhole, innodb - preload the corresponding engine
* all options from .opt files get equal treatment, all are searched for special options,
not only -{master,slave}.opt as before (which ignored suite.opt and --mysqld=...)
* --plugin-load gets special treatment too - all instances are merged into one
* federated test fixed to preload federated
* no automatic --loose-skip-innodb added by mtr based on the test name.
instead loose-skip-innodb is now in the default_mysqld.cnf
* have_innodb_plugin.inc is changed to give a verbose "skip" message
(instead of "require: true")
* My::Suite class. It's support in mtr, and everywhere
* support for suite.pm
* when sorting tests, take combinations into account
* support for SUITENAME_COMBINATIONS
* no special treatment for innodb_plugin in mtr_cases.pm
* two special pre-created config groups: ENV and OPT
* allow option names to start from #
* allow magic option to have an argument
* remove dead code
* fix @-substitution to works as expected
* new processes take the value of $opt_verbose automatically, no need to pass it to a constructor
* innodb_plugin suite uses suite.pm and combinations file to test as much as possible
(innodb plugin, xtradb plugin, xtradb static - whatever available)
* besides test-master.opt and test-slave.opt a test.opt file is also
loaded, both for master and slave
* .opt files for all included files are loaded too
* progress report in the xterm titlebar
If one compiles innodb_plugin, then the tests in suite/innodb_plugin will use the plugin. If not and xtradb is used, the tests will use xtradb.
mysql-test/include/have_innodb_plugin.inc:
Test both for innodb_plugin and xtradb
mysql-test/include/have_real_innodb_plugin.inc:
Test if we are using innodb_plugin (but not xtradb)
mysql-test/include/have_xtradb.inc:
Test if xtradb is used
mysql-test/lib/mtr_cases.pm:
Enable easy testing of innodb_plugin
mysql-test/mysql-test-run.pl:
Added supression for difference between xtradb & innodb_plugin
mysql-test/suite/innodb_plugin/r/innodb-index-ip.result:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/r/innodb-index-xb.result:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/r/innodb-index.result:
Move tests away that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/r/innodb-ip.result:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/r/innodb-xb.result:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/r/innodb.result:
Move tests away that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/r/innodb_bug21704-xb.result:
Test result differ for xtradb
mysql-test/suite/innodb_plugin/r/innodb_bug46000.result:
Remove (not needed) error message not given by MariaDB
mysql-test/suite/innodb_plugin/r/innodb_bug49164-xb.result:
Test result differs for xtradb
mysql-test/suite/innodb_plugin/r/innodb_bug49164.result:
Update results
mysql-test/suite/innodb_plugin/r/innodb_bug53591.result:
Remove (not needed) error message not given by MariaDB
mysql-test/suite/innodb_plugin/r/innodb_bug54679.result:
Updated result file
mysql-test/suite/innodb_plugin/r/innodb_mysql.result:
Updated result file
mysql-test/suite/innodb_plugin/t/disabled.def:
Disable some tests that depends on newer version of XtraDB
mysql-test/suite/innodb_plugin/t/innodb-index-ip.test:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/t/innodb-index-xb.test:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/t/innodb-index.test:
Move tests away that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/t/innodb-ip.test:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/t/innodb-xb.test:
Tests from innodb-index that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/t/innodb.test:
Move tests away that gave different results for innodb_plugin and xtradb
mysql-test/suite/innodb_plugin/t/innodb_bug21704-xb.test:
Test result differ for xtradb
mysql-test/suite/innodb_plugin/t/innodb_bug21704.test:
Test result differ for xtradb
mysql-test/suite/innodb_plugin/t/innodb_bug53591.test:
Test results only makes sence for innodb_plugin (things works ok for xtradb)
sql/sql_table.cc:
Don't set HA_CREATE_USED_ROW_FORMAT for create table (only for update_create_info) if ROW_FORMAT is not used.
storage/innodb_plugin/handler/ha_innodb.cc:
Fixed wrong error message from innodb.
This is needed as MariaDB properly handles errors from ha_index_init()
storage/xtradb/handler/ha_innodb.cc:
Update base information for XtraDB so that one can use informationschema.plugins to check if one is using XtraDB
* fully support --mysqld=--plugin-load=xxxx
* uniformly support all loadable plugins, no need to hard-code
every new plugin in mtr
* autodetect MTR_VS_CONFIG on windows
MTR will ignore fully qualified test name entries in disabled.def
lists. Therefore, it would still run the test case, even if it is
listed.
This patch fix this by extending the check when marking the test
case as disabled to take into consideration not only the cases that
contain the simple test name but also those that contain fully
qualified test names.
Fixed some bugs introduced in 5.1.47
Disabled some tests until we have merged with latest Xtradb
configure.in:
Added testing if valgrind/memcheck.h exists
storage/pbxt/src/ha_pbxt.cc:
LOCK_plugin is not anymore locked in init
they are ignored to a new test suite "innodb_plugin".
Remove a hack in mtr that was deployed to run the builtin InnoDB tests against
the InnoDB Plugin. Also detect if a test is an 'innodb plugin test' and if so
then transparently replace the builtin InnoDB with the InnoDB Plugin.
Disable pbxt for test cases not using pbxt (speeds up test suite)
extra/comp_err.c:
Added cast to get rid of compiler warning
extra/libevent/kqueue.c:
Added cast to get rid of compiler warning
mysql-test/lib/mtr_cases.pm:
Use --skip-pbxt for test cases that doesn't need pbxt
Collect default-storage-engine from suite.opt file (should actually be my.cnf file, but that wasn't easy to do)
mysql-test/suite/pbxt/t/suite.opt:
Added marker for mysql-test-run.pl that this suite require pbxt
mysys/mf_keycache.c:
Use LINT_INIT() to remove compiler warnings
mysys/my_gethostbyname.c:
Remove compiler warnings
sql/handler.cc:
Reset variable that may be used uninitialized
sql/item.h:
Remove compiler warning
sql/mysqld.cc:
Use LINT_INIT() to remove compiler warnings
sql/sql_class.h:
Remove compiler warning
sql/sql_table.cc:
Ensure variable is always set (to remove compiler warning)
sql/sql_view.cc:
Use LINT_INIT() to remove compiler warnings
storage/maria/ma_loghandler.c:
Use LINT_INIT() to remove compiler warnings
storage/myisammrg/ha_myisammrg.cc:
Fixed wrong type to printf
storage/myisammrg/myrg_open.c:
Use LINT_INIT() to remove compiler warnings
storage/xtradb/include/ut0lst.h:
Trivial change of macro to remove compiler warning
strings/ctype-ucs2.c:
Use LINT_INIT() to remove compiler warnings
strings/ctype-utf8.c:
Use LINT_INIT() to remove compiler warnings
support-files/compiler_warnings.supp:
Suppress some not relevant warnings
unittest/mysys/waiting_threads-t.c:
Don't use ftruncate() as this gives warning about ignored return value
Some logic would group by suite always
Disable this if using --noreorder
Also fix getting array from collect_one_suite() in this case
Amended according to previous comment
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
Fix bad merge causing error when specifying test case in non-default suite for mysql-test-run.
Implement the ability to add extra flags and configure options when running BUILD/xxx scripts.
Improve unclear help text in mysql-test-run
BUILD/FINISH.sh:
Implement the ability to add extra flags and configure options when running BUILD/xxx scripts.
BUILD/SETUP.sh:
Implement the ability to add extra flags and configure options when running BUILD/xxx scripts.
extra/yassl/taocrypt/include/block.hpp:
Fix some compiler warnings.
mysql-test/lib/mtr_cases.pm:
Fix bad merge causing error when specifying test case in non-default suite for mysql-test-run.
Also remove some non-essential differences to mysql version to simplify future merges.
mysql-test/mysql-test-run.pl:
Improve help texts.
plugin/fulltext/plugin_example.c:
Fix some compiler warnings.
vio/viosslfactories.c:
Fix some compiler warnings.
applied revisions: r6185, r6186, r6189, r6194
r6185 - only code changes incorporated, changesets which change
innodb tests in the main mysql suite are discarded
r61889 - Fixes BUG#48782
Detailed revision comments:
r6185 | marko | 2009-11-17 16:44:20 +0200 (Tue, 17 Nov 2009) | 16 lines
branches/zip: Report duplicate table names
to the client connection, not to the error log. This change will allow
innodb-index.test to be re-enabled. It was previously disabled, because
mysql-test-run does not like output in the error log.
row_create_table_for_mysql(): Do not output anything to the error log
when reporting DB_DUPLICATE_KEY. Let the caller report the error.
Add a TODO comment that the dict_table_t object is apparently not freed
when an error occurs.
create_table_def(): Convert InnoDB table names to the character set
of the client connection for reporting. Use my_error(ER_WRONG_COLUMN_NAME)
for reporting reserved column names. Report my_error(ER_TABLE_EXISTS_ERROR)
when row_create_table_for_mysql() returns DB_DUPLICATE_KEY.
rb://206
r6186 | vasil | 2009-11-17 16:48:14 +0200 (Tue, 17 Nov 2009) | 4 lines
branches/zip:
Add ChangeLog entry for r6185.
r6189 | marko | 2009-11-18 11:36:18 +0200 (Wed, 18 Nov 2009) | 5 lines
branches/zip: ha_innobase::add_index(): When creating the primary key
and the table is being locked by another transaction,
do not attempt to drop the table. (Bug #48782)
Approved by Sunny Bains over IM
r6194 | vasil | 2009-11-19 09:24:45 +0200 (Thu, 19 Nov 2009) | 5 lines
branches/zip:
Increment version number from 1.0.5 to 1.0.6 since 1.0.5 was just released
by MySQL and we will soon release 1.0.6.
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
- disabled main.innodb_bug47777.test with InnoDB plugin
until fix for plugin is applied.
- disabled main.innodb-autoinc.test (failing)
- re-enabled main.innodb_bug39438.test
- added error message suppression to innodb_bug39438, as
requested by InnoDB/Oracle
- reverted change to main.innodb_bug34300 as plugin specific.
Knowledge of no SSL support is not used
Skip tests the same way e.g. innodb tests are
Does not refer to have_ssl_communication.inc,
will add this when merging to 6.0-codebase
