Before this fix, the test output for perfschema.server_init would
vary between executions, because some of the objects tested were
not guaranteed to exist in all configurations / code paths.
This fix removes these weak tests.
Also, comments referring to abandonned code have been cleaned up.
tree for embedded server
Test case for bug #56251 "Deadlock with INSERT
DELAYED and MERGE tables" can't be run against
embedded server. Embedded server converts all
DELAYED INSERTs into ordinary INSERTs and this
test can't work properly if such conversion
happens.
Moved this test from merge.test to delayed.test
which is skipped if test suite is run with
--embedded-server option.
Subselect executes twice, at JOIN::optimize stage
and at JOIN::execute stage. At optimize stage
Innodb prebuilt struct which is used for the
retrieval of column values is initialized in.
ha_innobase::index_read(), prebuilt->sql_stat_start is true.
After QUICK_ROR_INTERSECT_SELECT finished his job it
restores read_set/write_set bitmaps with initial values
and deactivates one of the handlers used by
QUICK_ROR_INTERSECT_SELECT in JOIN::cleanup
(it's the case when we reuse original handler as one of
handlers required by QUICK_ROR_INTERSECT_SELECT object).
On second subselect execution inactive handler is activated
in QUICK_RANGE_SELECT::reset, file->ha_index_init().
In ha_index_init Innodb prebuilt struct is reinitialized
with inappropriate read_set/write_set bitmaps. Further
reinitialization in ha_innobase::index_read() does not
happen as prebuilt->sql_stat_start is false.
It leads to partial retrieval of required field values
and we get a mix of field values from different records
in the record buffer.
The fix is to reset
read_set/write_set bitmaps as these values
are required for proper intialization of
internal InnoDB struct which is used for
the retrieval of column values
(see build_template(), ha_innodb.cc)
mysql-test/include/index_merge_ror_cpk.inc:
test case
mysql-test/r/index_merge_innodb.result:
test case
mysql-test/r/index_merge_myisam.result:
test case
sql/opt_range.cc:
if ROR merge scan is used we need to reset
read_set/write_set bitmaps as these values
are required for proper intialization of
internal InnoDB struct which is used for
the retrieval of column values
(see build_template(), ha_innodb.cc)
adding new indexes
A fast alter table requires that the existing (old) table
and indices are unchanged (i.e only new indices can be
added). To verify this, the layout and flags of the old
table/indices are compared for equality with the new.
The PACK_KEYS option is a no-op in InnoDB, but the flag
exists, and is used in the table compare. We need to
check this (table) option flag before deciding whether an
index should be packed or not. If the table has
explicitly set PACK_KEYS to 0, the created indices should
not be marked as packed/packable.
The problem was that RENAME TABLE caused an assert if the system variable
lower_case_table_names was 2 (default on Mac OS X) and the old table name
was given in upper case. This caused lowercase_table2.test to fail.
The assert checks that an exclusive metadata lock is held by the connection
trying to do RENAME TABLE - specificially during updates of table triggers.
The assert was triggered since the check is case sensitive and the lock
was held on the normalized (lower case) version of the table name.
This patch fixes the problem by making sure a normalized version of the
table name is used for the metadata lock check, while using a non-normalized
version of the table name for the rename of trigger files. The same is done
for ALTER TABLE ... RENAME.
Regression testing for the bug itself is already covered by
lowercase_table2.test. Additional coverage added to lowercase_fs_off.test.
Before this fix, the server could crash inside a memcpy when reading data
from the EVENTS_WAITS_CURRENT / HISTORY / HISTORY_LONG tables.
The root cause is that the length used in a memcpy could be corrupted,
when another thread writes data in the wait record being read.
Reading unsafe data is ok, per design choice, and the code does sanitize
the data in general, but did not sanitize the length given to memcpy.
The fix is to also sanitize the schema name / object name / file name
length when extracting the data to produce a row.
tables".
Attempting to issue an INSERT DELAYED statement for a MERGE
table might have caused a deadlock if it happened as part of
a transaction or under LOCK TABLES, and there was a concurrent
DDL or LOCK TABLES ... WRITE statement which tried to lock one
of its underlying tables.
The problem occurred when a delayed insert handler thread tried
to open a MERGE table and discovered that to do this it had also
to open all underlying tables and hence acquire metadata
locks on them. Since metadata locks on the underlying tables were
not pre-acquired by the connection thread executing INSERT DELAYED,
attempts to do so might lead to waiting. In this case the
connection thread had to wait for the delayed insert thread.
If the thread which was preventing the lock on the underlying table
from being acquired had to wait for the connection thread (due to
this or other metadata locks), a deadlock occurred.
This deadlock was not detected by the MDL deadlock detector since
waiting for the handler thread by the connection thread is not
represented in the wait-for graph.
This patch solves the problem by ensuring that the delayed
insert handler thread never tries to open underlying tables
of a MERGE table. Instead open_tables() is aborted right after
the parent table is opened and a ER_DELAYED_NOT_SUPPORTED
error is emitted (which is passed to the connection thread and
ultimately to the user).
mysql-test/r/merge.result:
Added test for bug #56251 "Deadlock with INSERT DELAYED and
MERGE tables".
mysql-test/t/merge.test:
Added test for bug #56251 "Deadlock with INSERT DELAYED and
MERGE tables".
sql/sql_base.cc:
Changed open_n_lock_single_table() to take prelocking strategy
as an argument instead of always using DML_prelocking_strategy.
sql/sql_base.h:
Changed open_n_lock_single_table() to take prelocking strategy
as an argument instead of always using DML_prelocking_strategy.
Added a version of this function which is compatible with old
signature.
sql/sql_insert.cc:
When opening MERGE table in delayed insert thread stop and emit
ER_DELAYED_NOT_SUPPORTED right after opening main table and
before opening underlying tables. This ensures that we won't
try to acquire metadata lock on underlying tables which might
lead to a deadlock.
This is achieved by using special prelocking strategy which
abort open_tables() process as soon as we discover that we
have opened table with engine which doesn't support delayed
inserts.
Bug#56657: Test still uses "--exec rm -f ..." which is non-portable
Bug#56601: Test uses Unix path for temporary file, fails, and writes misleading message
Several tests that was written in a non portable way (failed on windows)
Fixed by
1) backporting the fix for replace_result to also apply to list_files
(mysqltest from mysql-trunk)
2) replacing all #p#/#sp#/#tmp# to #P#/#SP#/#TMP#/
(innodb always converts filenames to lower case in windows).
3) replacing '--exec rm -f' with '--remove_files_wildcard'
4) replacing a perl snippet with '--write_file'
client/mysqltest.cc:
backport from mysql-trunk to allow replace_result to apply
also on list_files
mysql-test/suite/parts/inc/partition_check_drop.inc:
Compensate for differences between innodb on windows vs unix.
Using mysqltest command, instead of unix command to remove files.
mysql-test/suite/parts/inc/partition_crash.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_fail.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_layout.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_layout_check1.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/inc/partition_layout_check2.inc:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/r/partition_recover_myisam.result:
updated result
mysql-test/suite/parts/r/partition_special_myisam.result:
updated result
mysql-test/suite/parts/t/part_supported_sql_func_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter1_1_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter1_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter2_1_1_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter2_1_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter2_2_2_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_alter4_innodb.test:
Test takes very long time, require --big flag
mysql-test/suite/parts/t/partition_debug_sync_innodb.test:
compensate for differences between innodb on windows vs unix
mysql-test/suite/parts/t/partition_recover_myisam.test:
more generic suppression (failed in windows)
mysql-test/suite/parts/t/partition_special_myisam.test:
Using portable mysqltest command 'write_file' instead of perl snippet.
Implemented post review comments.
Added --force to the mysql_upgrade command in the test scripts,
so that the test output does not depends on whether other tests involving an
upgrade have been executed or not in the same test suite execution.
ORDER BY computed col
GROUP BY implies ORDER BY in the MySQL dialect of SQL. Therefore, when an
index on the first table in the query is used, and that index satisfies
ordering according to the GROUP BY clause, the query optimizer estimates the
number of tuples that need to be read from this index. If there is a LIMIT
clause, table statistics on tables following this 'sort table' are employed.
There may be a separate ORDER BY clause however, which mandates reading the
whole 'sort table' anyway. But the previous estimate was left untouched.
Fixed by removing the estimate from EXPLAIN output if GROUP BY is used in
conjunction with an ORDER BY clause that mandates using a temporary table.
The problem was that issuing XA END when the XA transaction was
already ended, caused an assertion. This assertion tests that
the server does not try to send OK to the client if there has
already been an error reported. The bug was only noticeable on
debug versions of the server.
The reason for the problem was that the trans_xa_end() function
reported success if the transaction was at XA_IDLE state at the
end regardless of any errors occured during processing of
trans_xa_end(). So if the transaction state was XA_IDLE already,
reported errors would be ignored.
This patch fixes the problem by having trans_xa_end() take into
consideration any reported errors. The patch also fixes a similar
bug with XA PREPARE.
Test case added to xa.test.
Version "5.1.42 SUSE MySQL RPM"
When a query was using a DATE or DATETIME value formatted
using different formatting than "yyyy-mm-dd HH:MM:SS", a
query with a greater-or-equal '>=' condition matched only
greater values in an indexed TIMESTAMP column.
The problem was introduced by the fix for the bug 46362
and partially solved (for DATE and DATETIME columns only)
by the fix for the bug 47925.
The stored_field_cmp_to_item function has been modified
to take into account TIMESTAMP columns like we do for
DATE and DATETIME columns.
mysql-test/r/type_timestamp.result:
Test case for bug #55779.
mysql-test/t/type_timestamp.test:
Test case for bug #55779.
sql/item.cc:
Bug #55779: select does not work properly in mysql server
Version "5.1.42 SUSE MySQL RPM"
The stored_field_cmp_to_item function has been modified
to take into account TIMESTAMP columns like we do for
DATE and DATETIME.
Before this fix, the server could crash during shutdown,
due to race conditions, that occured when killing the server.
In particular, the performance schema instrumentation handle,
PSI_server, and the performance schema itself would be cleaned up
too soon, causing race conditions with a running kill server thread.
The specifics of the race condition found are that:
the main thread executing "PSI_server= NULL" can cause crashes in
other threads still running, which are executing
"if (PSI_server != NULL) PSI_server->xxx()"
as part of the performance schema instrumentation.
While the bug was reported for the kill server thread,
in theory the same crash could happen with the signal thread,
as found by code analysis.
The correct fix would be to only shutdown the performance schema
and set PSI_server to NULL after every other thread is guaranteed
to be completed, including the kill_server_thread.
However, due to the existing mysqld server design, this is not the case.
See in particular bug number 56666.
The work around used to fix this race condition is to simply not
perform the call to shutdown_performance_schema() when the server exits,
and to keep the PSI_server pointer unchanged.
This will cause memory leaks to be reported by tools like valgrind,
but no memory leak actually happen because the process is about to exit().
As a result, the file mysql-test/valgrind.supp has been updated
to filter out these false positive messages.
This code has been tested with running in a loop the following
tests in parallel, which have been known to fail with race conditions
in the past:
- rpl_change_master
- binlog_max_extension
- events_restart
- rpl_heartbeat_basic
and no crash of test failure has been seen with the changed code.
to 5.5 (removed one test case as it is no longer valid).
mysql-test/r/select.result:
Removed a part of the test case for bug#48291 since it is not
valid anymore. The comments for the removed part were actually
describing a side-effect from the problem addressed by the
addendum patch for bug #54190.
mysql-test/t/select.test:
Removed a part of the test case for bug#48291 since it is not
valid anymore. The comments for the removed part were actually
describing a side-effect from the problem addressed by the
addendum patch for bug #54190.
table causes assert failure".
Attempting to use FLUSH TABLE table_list WITH READ LOCK
statement for a MERGE table led to an assertion failure if
one of its children was not present in the list of tables
to be flushed. The problem was not visible in non-debug builds.
The assertion failure was caused by the fact that in such
situations FLUSH TABLES table_list WITH READ LOCK implementation
tried to use (e.g. lock) such child tables without acquiring
metadata lock on them. This happened because when opening tables
we assumed metadata locks on all tables were already acquired
earlier during statement execution and a such assumption was
false for MERGE children.
This patch fixes the problem by ensuring at open_tables() time
that we try to acquire metadata locks on all tables to be opened.
For normal tables such requests are satisfied instantly since
locks are already acquired for them. For MERGE children metadata
locks are acquired in normal fashion.
Note that FLUSH TABLES merge_table WITH READ LOCK will lock for
read both the MERGE table and its children but will flush only
the MERGE table. To flush children one has to mention them in table
list explicitly. This is expected behavior and it is consistent with
usage patterns for this statement (e.g. in mysqlhotcopy script).
mysql-test/r/flush.result:
Added test case for bug #55273 "FLUSH TABLE tm WITH READ LOCK
for Merge table causes assert failure".
mysql-test/t/flush.test:
Added test case for bug #55273 "FLUSH TABLE tm WITH READ LOCK
for Merge table causes assert failure".
sql/sql_base.cc:
Changed lock_table_names() to support newly introduced
MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag.
sql/sql_base.h:
Introduced MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag for
open_tables() and lock_table_names() which allows to skip
acquiring of global and schema-scope locks when SNW, SNRW or
X metadata locks are acquired.
sql/sql_reload.cc:
Changed "FLUSH TABLES table_list WITH READ LOCK" code not to
cause assert about missing metadata locks when MERGE table is
flushed without one of its underlying tables.
To achieve this we no longer call open_and_lock_tables() with
MYSQL_OPEN_HAS_MDL_LOCK flag so this function automatically
acquires metadata locks on MERGE children if such lock has
not been already acquired at earlier stage. Instead we call
this function with MYSQL_OPEN_SKIP_SCOPED_MDL_LOCK flag to
suppress acquiring of global IX lock in order to keep FLUSH
TABLES table_list WITH READ LOCK compatible with FLUSH TABLE
WITH READ LOCK.
Also changed implementation to use lock_table_names() function
for pre-acquiring of metadata locks instead of custom code.
To implement this change moved setting of open_type member for
table list elements to parser.
sql/sql_yacc.yy:
Now we set acceptable type of table for FLUSH TABLES table_list
WITH READ LOCK at parsing time instead of execution time.
result
Row subqueries producing no rows were not handled as UNKNOWN
values in row comparison expressions.
That was a result of the following two problems:
1. Item_singlerow_subselect did not mark the resulting row
value as NULL/UNKNOWN when no rows were produced.
2. Arg_comparator::compare_row() did not take into account that
a whole argument may be NULL rather than just individual scalar
values.
Before bug#34384 was fixed, the above problems were hidden
because an uninitialized (i.e. without any stored value) cached
object would appear as NULL for scalar values in a row subquery
returning an empty result. After the fix
Arg_comparator::compare_row() would try to evaluate
uninitialized cached objects.
Fixed by removing the aforementioned problems.
mysql-test/r/row.result:
Added a test case for bug #54190.
mysql-test/r/subselect.result:
Updated the result for a test relying on wrong behavior.
mysql-test/t/row.test:
Added a test case for bug #54190.
sql/item_cmpfunc.cc:
If either of the argument rows is NULL, return NULL as the
result of comparison.
sql/item_subselect.cc:
Adjust null_value for Item_singlerow_subselect depending on
whether a row has been produced by the row subquery.
Item_func_spatial_collection::fix_length_and_dec()
changed to use argument's print() method to print
the ER_ILLEGAL_VALUE_FOR_TYPE error.
mysql-test/r/gis.result:
Fix for bug#56679: gis.test: valgrind error
- test result adjusted.
sql/item_geofunc.h:
Fix for bug#56679: gis.test: valgrind error
- use argument's print() method instead of improper val_str()
call in the Item_func_spatial_collection::fix_length_and_dec(), as
it's applicable only for constant items.
With recent changes in the performance schema default sizing parameters,
the memory used by a mysqld binary increased accordingly.
This negatively affects the MTR test suite,
because running several tests in parallel now consumes more ressources.
The fix is to leave the default production values unchanged,
and to configure the MTR environment to limit memory
used when running tests in the test suite, which is ok
because only a few objects are typically used within a test script.
This fix:
- changed the default configuration in MTR to use less memory
- adjusted the performance schema tests accordingly
Note that 1,000 mutex instances was too short and caused test failures
in the past in team trees, so the default used is now 10,000 in MTR.
The amount of memory used by the performance schema itself
can be observed with the statement SHOW ENGINE PERFORMANCE_SCHEMA STATUS
ALTER TABLE on a MERGE table could cause a deadlock with two
other connections if we reached a situation where:
1) A connection doing ALTER TABLE can't upgrade to MDL_EXCLUSIVE on the
parent table, but holds TL_READ_NO_INSERT on the child tables.
2) A connection doing DELETE on a child table can't get TL_WRITE on it
since ALTER TABLE holds TL_READ_NO_INSERT.
3) A connection doing SELECT on the parent table can't get TL_READ on
the child tables since TL_WRITE is ahead in the lock queue, but holds
MDL_SHARED_READ on the parent table preventing ALTER TABLE from upgrading.
For regular tables, this deadlock is avoided by having ALTER TABLE
take a MDL_SHARED_NO_WRITE metadata lock on the table. This prevents
DELETE from acquiring MDL_SHARED_WRITE on the table before ALTER TABLE
tries to upgrade to MDL_EXCLUSIVE. In the example above, SELECT would
therefore not be blocked by the pending DELETE as DELETE would not be
able to enter TL_WRITE in the table lock queue.
This patch fixes the problem for merge tables by using the same metadata
lock type for child tables as for the parent table. The child tables will
in this case therefore be locked with MDL_SHARED_NO_WRITE, preventing
DELETE from acquiring a metadata lock and enter into the table lock queue.
Change in behavior: By taking the same metadata lock for child tables
as for the parent table, LOCK TABLE on the parent table will now also
implicitly lock the child tables. Since LOCK TABLE on the parent table
now takes more than one metadata lock, it is possible for LOCK TABLE
... WRITE on the parent table or child tables to give ER_LOCK_DEADLOCK
error.
Test case added to mdl_sync.test.
Merge.test/.result has been updated to reflect the change to LOCK TABLE.
