Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock
- Incompatible change: truncate no longer resorts to a row by
row delete if the storage engine does not support the truncate
method. Consequently, the count of affected rows does not, in
any case, reflect the actual number of rows.
- Incompatible change: it is no longer possible to truncate a
table that participates as a parent in a foreign key constraint,
unless it is a self-referencing constraint (both parent and child
are in the same table). To work around this incompatible change
and still be able to truncate such tables, disable foreign checks
with SET foreign_key_checks=0 before truncate. Alternatively, if
foreign key checks are necessary, please use a DELETE statement
without a WHERE condition.
Problem description:
The problem was that for storage engines that do not support
truncate table via a external drop and recreate, such as InnoDB
which implements truncate via a internal drop and recreate, the
delete_all_rows method could be invoked with a shared metadata
lock, causing problems if the engine needed exclusive access
to some internal metadata. This problem originated with the
fact that there is no truncate specific handler method, which
ended up leading to a abuse of the delete_all_rows method that
is primarily used for delete operations without a condition.
Solution:
The solution is to introduce a truncate handler method that is
invoked when the engine does not support truncation via a table
drop and recreate. This method is invoked under a exclusive
metadata lock, so that there is only a single instance of the
table when the method is invoked.
Also, the method is not invoked and a error is thrown if
the table is a parent in a non-self-referencing foreign key
relationship. This was necessary to avoid inconsistency as
some integrity checks are bypassed. This is inline with the
fact that truncate is primarily a DDL operation that was
designed to quickly remove all data from a table.
Introduce a MySQL maintainer/developer mode that enables
a set of warning options for the C/C++ compiler. This mode
is intended to help improve the overall quality of the code.
The warning options are:
C_WARNINGS="-Wall -Wextra -Wunused -Wwrite-strings -Werror"
CXX_WARNINGS="$C_WARNINGS -Wno-unused-parameter"
Since -Wall is essentially a moving target, autoconf checks
are not run with warning options enabled, in particualr -Werror.
This decision might be revisited in the future. The patch also
fixes a mistake in the makefiles, where automake CXXFLAGS would
be set to CFLAGS.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
The atomic operations implementation on 5.1 has a few problems,
which might cause tests to abort randomly. Since no code in 5.1
uses atomic operations, simply remove the code.
(make relies GNU extentions). The patch was partially
backport from 6.0.
Original comment:
bug#30708: make relies GNU extensions. Now that we no longer use
BitKeeper we can safely remove the SCCS handling with no loss of
functionality.
The fix inserts newline and comma characters as appropriate
into the constraint reporting code to match the formatting
required by SHOW CREATE TABLE. Additionally, a erroneously
duplicated copy of check_if_incompatible_data() was removed
from db2i_constraints.cc since the correct version is already
in ha_ibmdb2i.cc.
This fix changes the character set used within the
IBMDB2I handler to hash table names to information
about open tables. Previously, tables with names
that differed only in letter case would hash to the
same data structure. This caused incorrect behavior
or errors when two such tables were in use simultaneously.
Recently, the "#define" directives mapping the old names to the new ones
have been removed, so now all callers must use the new names.
This change was missing in the DB2 storage handler modules.
With ibmdb2i_create_index_option set to 1, creating an IBMDB2I table
with a primary key should produce an additional index that uses EBCDIC
hexadecimal sorting. However, this does not work. Adding indexes that
are not primary keys does work. The ibmdb2i_create_index_option should
be honoured when creating a table with a primary key.
This patch adds code to the create() function to check for the value
of the ibmdb2i_create_index_option variable and, when appropriate, to
generate a *HEX-based shadow index in DB2 for the primary key. Previously
this behavior was limited to secondary indexes.
Additionally, this patch restricts the creation of shadow indexes to
cases in which a non-*HEX sort sequence is used, as the documentation
for ibmdb2i_create_index_option describes. Previously, the shadow index
would in some cases be created even when the MySQL-specific index used
*HEX sorting, leading to redundant indexes.
Finally, the code used to generate the list of fields for indexes
and the code used to generate the SQL statement for the shadow
indexes has been refactored into individual functions.
Some collations were causing IBMDB2I to report
inaccurate key range estimations to the optimizer
for LIKE clauses that select substrings. This can
be seen by running EXPLAIN. This problem primarily
affects multi-byte and unicode character sets.
This patch involves substantial changes to several
modules. There are a number of problems with the
character set and collation handling. These problems
have been or are being fixed, and a comprehensive
test has been included which should provide much
better coverage than there was before. This test
is enabled only for IBM i 6.1, because that version
has support for the greatest number of collations.
Creating an IBMDB2I table with the macce character set
is successful, but any attempt to insert data into the
table was failing.
This was happening because the character set name "macce"
is not a valid iconv descriptor for IBM i PASE. This patch
adds an override to convertTextDesc to use the equivalent
valid iconv descriptor "IBM-1282" instead.
Some collations--including cp1250_czech_cs,latin2_czech_cs,
ucs2/utf8_czech_ci, ucs2/utf8_danish_ci--are not being
sorted correctly by the IBMDB2I storage engine. This
was being caused because the sort order used by DB2 is
incompatible with the order expected by MySQL.
This patch removes support for the cp1250_czech_cs and
latin2_czech_cs collations because it has been determined
that the sort order used by DB2 is incompatible with the
order expected by MySQL. Users needing a czech collation
with IBMDB2I are encouraged to use a Unicode-based collation
instead of these single-byte collations. This patch also
modifies the DB2 sort sequence used for ucs2/utf8_czech_ci
and ucs2/utf8_danish_ci collations to better match the
sorting expected by MySQL. This will only affect indexes
or tables that are newly created through the IBMDB2I storage
engine. Existing IBMDB2I tables will retain the old sort
sequence until recreated.
Running a SELECT query over an IBMDB2I table with a cp1250 character set
was producing an error 2027 (ibmdb2i error 2027: Error converting single-byte
sort sequence to UCS-2).
The QMY_DESCRIBE_RANGE API was returning error 2027 to the storage engine
because the CCSID used for a cp1250 column (870) does not match the CCSID
used by the DB2 sort sequences associated with cp1250_* collations (1153).
This was because the storage engine relies on a set of system APIs to
determine which CCSID value most closely matches a particular MySQL
character set. However, in the case of cp1250, the system is returning
CCSID 870, which does not have a codepoint for the euro symbol, making it
an incorrect match.
This patch overrides the selection of a compatible CCSID to always return
1153 for cp1250.
wmemset was being used to fill the row buffers.
wmemset was intended to fill the buffer with
16-bit UCS2 pad values. However, the 64-bit
version of wmemset uses 32-bit wide characters
and thus filled the buffer incorrectly. In some
cases, the null byte map would be overwritten,
causing ctype_utf8.test and ibmdb2i_rir.test to
fail, giving the error message CPF5035.
This patch eliminates the use of wmemset to fill
the row buffer. wmemset has been replaced with
memset16, which always fills memory with 16-bit
values.
Occasionally, if both the partition_pruning
and partition_range tests are run sequentially
against the IBMDB2I engine, the partition_range
test will fail.
Compiler padding on a 64-bit build allowed
garbage data in the hash key used for
caching open iconv descriptors. As a
result, cached descriptors were not found,
and multiple duplicate iconv descriptors
were opened for a single character set.
Eventually, the maximum number of open
iconv descriptors was reached, and further
iconv_open() calls would fail, leading the
storage engine to report incorrectly that
the character set was not supported.
This patch widens the 16-bit members of the
hash key to 32 bits to eliminate compiler
padding. The entire length of the hash key
is now initialized correctly on both 32-bit
and 64-bit builds.
In order to better support the usage of
IBMDB2I tables from within RPG programs,
the storage engine should ensure that the
RCDFMT name is consistent and predictable
for DB2 tables.
This patch appends a "RCDFMT <name>"
clause to the CREATE TABLE statement
that is passed to DB2. <name> is
generated from the original name of
the table itself. This ensures a
consistent and deterministic mapping
from the original table.
For the sake of simplicity only
the alpha-numeric characters are
preserved when generating the new
name, and these are upper-cased;
other characters are replaced with
an underscore (_). Following DB2
system identifier rules, the name
always begins with an alpha-character
and has a maximum of ten characters.
If no usable characters are found in
the table name, the name X is used.
When a user selected an unsupported character set for an
IBMDB2I table, error 2501 or 2511 may have been returned,
giving the appearance of an internal programming error.
This patch consolidates these errors into a single descriptive
error message for the common case of an unsupported character
set.
The new error number is 2504 and indicates a user error.
The errors 2501 and 2511 remain to indicate cases of internal
programming errors.
The storage engine was not correctly handling the case in
which rnd_pos is executed for a handler without a preceding
rnd_next or index read operation. As a result, an unitialized
file handle was sometimes being passed to the QMY_READ API.
The fix clears the rrnAssocHandle at the beginning of each
read operation and then checks to see whether it has been
set to a valid handle value before attempting to use it
in rnd_pos. If rrnAssocHandle has not been set by a previous
read operation, rnd_pos instead falls back to the use of the
currently active handle.
On IBM i 5.4, schemas with names that are longer
than 8 characters and contain digits or an underscore
cannot contain IBMDB2I tables, even though this should
theoritically be possible if all alpha characters
are uppercase.
THe current patch fixes the IBMDB2I engine to
allow digits and the underscore(_) to be used in
schema names longer than 8 characters on IBM i 5.4.
In some circumstances, when a table is created with
the IBMDB2I engine, the CREATE TABLE statement will
return successfully but the table will not exist.
The current patch addresses the above issue and causes
CREATE to fail and report and error to the user.
The utf8_swedish_ci and ucs2_swedish_ci
collations do not work with indexes on
IBMDB2I tables.
The current patch adds the mapping for
ucs2_swedish collation and removes the
ucs2_spanish2 mapping which is not
supported by any version of the operating
system.
Modify plugins.m4 configuration framework so that plugins which are
not built still get added to the source distribution during make dist.
This came up now because we can only build ibmdb2i on i5/OS, and we
can't bootstrap our source dist on that platform. The solution is to
specify DIST_SUBDIRS containing all plugins, separate from SUBDIRS
which contains the plugins which are actually built.
This ibmdb2i code is from the ibmdb2i-ga3-src.zip file, with a patch
to plug.in to disable the plugin if the PASE environment isn't available.
