Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock
- Incompatible change: truncate no longer resorts to a row by
row delete if the storage engine does not support the truncate
method. Consequently, the count of affected rows does not, in
any case, reflect the actual number of rows.
- Incompatible change: it is no longer possible to truncate a
table that participates as a parent in a foreign key constraint,
unless it is a self-referencing constraint (both parent and child
are in the same table). To work around this incompatible change
and still be able to truncate such tables, disable foreign checks
with SET foreign_key_checks=0 before truncate. Alternatively, if
foreign key checks are necessary, please use a DELETE statement
without a WHERE condition.
Problem description:
The problem was that for storage engines that do not support
truncate table via a external drop and recreate, such as InnoDB
which implements truncate via a internal drop and recreate, the
delete_all_rows method could be invoked with a shared metadata
lock, causing problems if the engine needed exclusive access
to some internal metadata. This problem originated with the
fact that there is no truncate specific handler method, which
ended up leading to a abuse of the delete_all_rows method that
is primarily used for delete operations without a condition.
Solution:
The solution is to introduce a truncate handler method that is
invoked when the engine does not support truncation via a table
drop and recreate. This method is invoked under a exclusive
metadata lock, so that there is only a single instance of the
table when the method is invoked.
Also, the method is not invoked and a error is thrown if
the table is a parent in a non-self-referencing foreign key
relationship. This was necessary to avoid inconsistency as
some integrity checks are bypassed. This is inline with the
fact that truncate is primarily a DDL operation that was
designed to quickly remove all data from a table.
mysql-test/suite/innodb/t/innodb-truncate.test:
Add test cases for truncate and foreign key checks.
Also test that InnoDB resets auto-increment on truncate.
mysql-test/suite/innodb/t/innodb.test:
FK is not necessary, test is related to auto-increment.
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
mysql-test/suite/innodb/t/innodb_mysql.test:
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
Use delete instead of truncate, test is used to check
the interaction of FKs, triggers and delete.
mysql-test/suite/parts/inc/partition_check.inc:
Fix typo.
mysql-test/suite/sys_vars/t/foreign_key_checks_func.test:
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
mysql-test/t/mdl_sync.test:
Modify test case to reflect and ensure that truncate takes
a exclusive metadata lock.
mysql-test/t/trigger-trans.test:
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
sql/ha_partition.cc:
Reorganize the various truncate methods. delete_all_rows is now
passed directly to the underlying engines, so as truncate. The
code responsible for truncating individual partitions is moved
to ha_partition::truncate_partition, which is invoked when a
ALTER TABLE t1 TRUNCATE PARTITION p statement is executed.
Since the partition truncate no longer can be invoked via
delete, the bitmap operations are not necessary anymore. The
explicit reset of the auto-increment value is also removed
as the underlying engines are now responsible for reseting
the value.
sql/handler.cc:
Wire up the handler truncate method.
sql/handler.h:
Introduce and document the truncate handler method. It assumes
certain use cases of delete_all_rows.
Add method to retrieve the list of foreign keys referencing a
table. Method is used to avoid truncating tables that are
parent in a foreign key relationship.
sql/share/errmsg-utf8.txt:
Add error message for truncate and FK.
sql/sql_lex.h:
Introduce a flag so that the partition engine can detect when
a partition is being truncated. Used to give a special error.
sql/sql_parse.cc:
Function mysql_truncate_table no longer exists.
sql/sql_partition_admin.cc:
Implement the TRUNCATE PARTITION statement.
sql/sql_truncate.cc:
Change the truncate table implementation to use the new truncate
handler method and to not rely on row-by-row delete anymore.
The truncate handler method is always invoked with a exclusive
metadata lock. Also, it is no longer possible to truncate a
table that is parent in some non-self-referencing foreign key.
storage/archive/ha_archive.cc:
Rename method as the description indicates that in the future
this could be a truncate operation.
storage/blackhole/ha_blackhole.cc:
Implement truncate as no operation for the blackhole engine in
order to remain compatible with older releases.
storage/federated/ha_federated.cc:
Introduce truncate method that invokes delete_all_rows.
This is required to support partition truncate as this
form of truncate does not implement the drop and recreate
protocol.
storage/heap/ha_heap.cc:
Introduce truncate method that invokes delete_all_rows.
This is required to support partition truncate as this
form of truncate does not implement the drop and recreate
protocol.
storage/ibmdb2i/ha_ibmdb2i.cc:
Introduce truncate method that invokes delete_all_rows.
This is required to support partition truncate as this
form of truncate does not implement the drop and recreate
protocol.
storage/innobase/handler/ha_innodb.cc:
Rename delete_all_rows to truncate. InnoDB now does truncate
under a exclusive metadata lock.
Introduce and reorganize methods used to retrieve the list
of foreign keys referenced by a or referencing a table.
storage/myisammrg/ha_myisammrg.cc:
Introduce truncate method that invokes delete_all_rows.
This is required in order to remain compatible with earlier
releases where truncate would resort to a row-by-row delete.
thd->in_sub_stmt
In a precursor patch for Bug#52044
(revid:bzr/kostja@stripped), a
number of reorganizations of code was made. In addition some
assertions were added to ensure the correct transactional state.
The reorganization had a small glitch so statements that was
active in the query cache was not followed by a
statement commit/rollback (this code was removed). A section
in the trans_commit_stmt/trans_rollback_stmt code is to
clear the thd->transaction.stmt list of affected storage
engines. When a new statement is initiated, an assert
introduced by the 523044 patch checks if this list is cleared.
When the query cache is accessed, this list may be populated,
and since it's not committed it will not be cleared.
This fix adds explicit statement commit or rollback for
statements that is contained in the query cache.
for ALTER TABLE + MERGE tables
The patch for Bug#56292 changed how metadata locks are taken for MERGE
tables. After the patch, locking the MERGE table will also lock the
children tables with the same metadata lock type. This means that
LOCK TABLES on a MERGE table also will implicitly do LOCK TABLES on
the children tables.
A consequence of this change, is that it is possible to do LOCK TABLES
on a child table both explicitly and implicitly with the same statement
and that these two locks can be of different strength. For example,
LOCK TABLES child READ, merge WRITE.
In LOCK TABLES mode, we are not allowed to take new locks and each
statement must therefore try to find an existing TABLE instance with
a suitable lock. The code that searched for a suitable TABLE instance,
only considered table level locks. If a child table was locked twice,
it was therefore possible for this code to find a TABLE instance with
suitable table level locks but without suitable metadata lock.
This problem caused the assert in upgrade_shared_lock_to_exclusive()
to be triggered as it tried to upgrade a MDL_SHARED lock to
EXCLUSIVE. The problem was a regression caused by the patch for
Bug#56292.
This patch fixes the problem by partially reverting the changes
done by Bug#56292. Now, the children tables will only use the
same metadata lock as the MERGE table for MDL_SHARED_NO_WRITE
when not in locked tables mode. This means that LOCK TABLE
on a MERGE table will not implicitly lock the children tables.
This still fixes the original problem in Bug#56292 without
causing a regression.
Test case added to merge.test.
- fix ~20 warnings about redefinition of HAVE_DLOPEN
This was caused by new ADD_DEFINITIONS(-DHAVE_DLOPEN) inside client library CMakeLists.txt
- fix dlerror() message which ("static build?" ) which is only confusing on Windows,
there is no "static build"
The error was introduced by typo in variable name (errormsg instead of correct
errmsg)
- Also, precache HAVE_PEERCRED to OFF in cmake\os\WindowsCache.cmake
(to avoid useless system check).
- Also, add missing check for errors from FormatMessage().
- Also, remove annoying CMake debug printout of MALLOC_LIB if this is not set.
connectors plugins
Implemented changes needed to keep the client plugin API compatible with
the existing plugins :
1. Provided an options() client plugin API to let the application pass
options to the plugin after loading it
2. Added "License" (const char *) to specify the client plugin's license
3. Added "mysql_api" as a placeholder that the client library can use
to pass function pointers to the plugin so that the plugin can call the
C lib back.
4. Updated the existing client plugins to comply with the API change.
5. Added more detailed error message generation for Windows.
Quoting from the bug report:
The pstack library has been included in MySQL since version
4.0.0. It's useless and should be removed.
Details: According to its own documentation, pstack only works
on Linux on x86 in 32 bit mode and requires LinuxThreads and a
statically linked binary. It doesn't really support any Linux
from 2003 or later and doesn't work on any other OS.
detector". This patch addresses performance regression in OLTP_RO/MyISAM
test on Windows introduced by the fix for bug #56405. Thus it makes
original patch acceptable as a solution for bug #56585 "Slowdown of
readonly sysbench benchmarks (e.g point_select) on Windows 5.5".
With this patch, MySQL will use native Windows condition variables and
reader-writer locks if they are supported by the OS.
This speeds up MyISAM and the effect comes mostly from using native
rwlocks. Native conditions improve scalability with higher number of
concurrent users in other situations, e.g for prlocks.
Benchmark numbers for this patch as measured on Win2008R2 quad
core machine are attached to the bug report.
( direct link http://bugs.mysql.com/file.php?id=15883 )
Note that currently we require at least Windows7/WS2008R2 for
reader-writer locks, even though native rwlock is available also on Vista.
Reason is that "trylock" APIs are missing on Vista, and trylock is used in
the server (in a single place in query cache).
While this patch could have been written differently, to enable the native
rwlock optimization also on Vista/WS2008 (e.g using native locks everywhere
but portable implementation in query cache), this would come at the
expense of the code clarity, as it would introduce a new "try-able" rwlock
type, to handle Vista case.
Another way to improve performance for the special case
(OLTP_RO/MYISAM/Vista) would be to eliminate "trylock" usage from server,
but this is outside of the scope here.
Native conditions variables are used beginning with Vista though the effect
of using condition variables alone is not measurable in this benchmark.
But when used together with native rwlocks on Win7, native conditions improve
performance in high-concurrency OLTP_RO/MyISAM (128 and more sysbench
users).
This assert was triggered if DELETE was done on a view that
referenced another view which in turn (directly or indirectly)
referenced more than one table.
Delete from a view referencing more than one table (a join view)
is not supported and is supposed to give ER_VIEW_DELETE_MERGE_VIEW
error. Before this error was reported from the multi table
delete code, an assert verified that the view from the DELETE statement
had more than one underlying table. However, this assert did not take
into account that the view could refer to another view which in turn
referenced the actual tables.
This patch fixes the problem by adjusting the assert to take this
possibility into account. This problem was only noticeable on debug
builds of the server. On release builds, ER_VIEW_DELETE_MERGE_VIEW
was correctly reported.
Test case added to delete.test.
tcmalloc lib (libtcmalloc_minimal.so) is missing from mysql 5.5 binary
Ensure that a "-DMALLOC_LIB=/foo/bar/libtcmalloc_minimal.so"
will ensure that this library gets included in the tar.gz
binary package.
It depends on the build environment to set this variable
to the correct path name, if it is available.
