Inserting a negative value in the autoincrement column of a
partitioned innodb table was causing the value of the auto
increment counter to wrap around into a very large positive
value. The consequences are the same as if a very large positive
value was inserted into a column, e.g. reduced autoincrement
range, failure to read autoincrement counter.
The current patch ensures that before calculating the next
auto increment value, the current value is within the positive
maximum allowed limit.
The partitioning clause is only a very long single line, which is very
hard to interpret for a human. This patch breaks the partitioning
syntax into one line for the partitioning type, and one line per
partition/subpartition.
breaks auto increment
The auto_increment value was not initialized if
the first statement after opening a table was
an 'UPDATE'.
solution was to check initialize if it was not,
before trying to increase it in update.
It is a very big test and as such it takes a lot of time.
Solution is to divide the test in two parts, one for testing increasing
column size and one for decreasing size.
The innodb branch does extended tests (that myisam is not) due to the
$do_pk_tests variabel, that is the reason why the innodb branch takes
longer.
No increase of memory usage in innodb was found when analyzing, (tested
with looping some millions time of create/drop and alter commands)
The memory exhaust discovered in the test is due to mysqltest which
stores the result in memory (result-file) and this was the biggest
result file in the test framework, so by dividing the test into two
parts also cuts the memory usage of mysqltest.
Problem was a mutex added in bug n 27405 for solving a problem
with auto_increment in partitioned innodb tables.
(in ha_partition::write_row over partitions file->ha_write_row)
Solution is to use the patch for bug#33479, which refines the
usage of mutexes for auto_increment.
Backport of bug-33479 from 6.0:
Bug-33479: auto_increment failures in partitioning
Several problems with auto_increment in partitioning
(with MyISAM, InnoDB. Locking issues, not handling
multi-row INSERTs properly etc.)
Changed the auto_increment handling for partitioning:
Added a ha_data variable in table_share for storage engine specific data
such as auto_increment value handling in partitioning, also see WL 4305
and using the ha_data->mutex to lock around read + update.
The idea is this:
Store the table's reserved auto_increment value in
the TABLE_SHARE and use a mutex to, lock it for reading and updating it
and unlocking it, in one block. Only accessing all partitions
when it is not initialized.
Also allow reservations of ranges, and if no one has done a reservation
afterwards, lower the reservation to what was actually used after
the statement is done (via release_auto_increment from WL 3146).
The lock is kept from the first reservation if it is statement based
replication and a multi-row INSERT statement where the number of
candidate rows to insert is not known in advance (like INSERT SELECT,
LOAD DATA, unlike INSERT VALUES (row1), (row2),,(rowN)).
This should also lead to better concurrancy (no need to have a mutex
protection around write_row in all cases)
and work with any local storage engine.
partition is corrupt
The main problem was that ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR
PARTITION took another code path (over mysql_alter_table instead of
mysql_admin_table) which differs in two ways:
1) alter table opens the tables in a different way than admin tables do
resulting in returning with error before it tried the command
2) alter table does not start to send any diagnostic rows to the client
which the lower admin functions continue to use -> resulting in
assertion crash
The fix:
Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION to use
the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE t.
Adding check in mysql_admin_table to setup the partition list for
which partitions that should be used.
Partitioned tables will still not work with
REPAIR TABLE/PARTITION USE_FRM, since that requires moving partitions
to tables, REPAIR TABLE t USE_FRM, and check that the data still
fulfills the partitioning function and then move the table back to
being a partition.
NOTE: I have removed the following functions from the handler
interface:
analyze_partitions, check_partitions, optimize_partitions,
repair_partitions
Since they are not longer needed.
THIS ALTERS THE STORAGE ENGINE API
manually resolved conflicts:
Text conflict in client/mysqltest.c
Contents conflict in mysql-test/include/have_bug25714.inc
Text conflict in mysql-test/include/have_ndbapi_examples.inc
Text conflict in mysql-test/mysql-test-run.pl
Text conflict in mysql-test/suite/parts/inc/partition_check_drop.inc
Text conflict in mysql-test/suite/parts/inc/partition_layout.inc
Text conflict in mysql-test/suite/parts/inc/partition_layout_check1.inc
Text conflict in mysql-test/suite/parts/inc/partition_layout_check2.inc
Text conflict in mysql-test/suite/parts/r/partition_alter1_1_2_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter1_1_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter1_2_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter2_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_alter3_innodb.result
Text conflict in mysql-test/suite/parts/r/partition_alter3_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_basic_innodb.result
Text conflict in mysql-test/suite/parts/r/partition_basic_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_basic_symlink_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_engine_myisam.result
Text conflict in mysql-test/suite/parts/r/partition_syntax_myisam.result
Text conflict in mysql-test/suite/rpl_ndb/t/disabled.def
Text conflict in mysql-test/t/disabled.def
Problem was that ha_partition had HA_FILE_BASED flag set
(since it uses a .par file), but after open it uses the first partitions
flags, which results in different case handling for create and for
open.
Solution was to change the underlying partition name so it was consistent.
(Only happens when lower_case_table_names = 2, i.e. Mac OS X and storage
engines without HA_FILE_BASED, like InnoDB and Memory.)
(Recommit after adding rename of check_lowercase_names to
get_canonical_filename, and moved it from handler.h to mysql_priv.h)
NOTE: if a mixed case name for a partitioned table was created when
lower_case_table_name = 2 it should be renamed or dropped before using
the updated version (See bug#37402 for more info)
The problem is that relying on the output of the 'ls' command is not
portable as its behavior is not the same between systems and it might
even not be available at all in (Windows).
So I added list_files that relies on the portable mysys library instead.
(and also list_files_write_file and list_files_append_file,
since the test was using '--exec ls' in that way.)
Split the test into 3 smaller parts, reducing single test run time.
NDB variant still disabled. The runtime for a single test is greater than the runtime
for the Innodb variant that was timing out. The test works, but disabled for Build systems.
Bug#36033 - Test funcs_1.processlist_val_ps fails on Windows
Bug#36034 - Test parts.part_supported_sql_func_<eng> fails on Windows
Bug#36036 - Test parts.partition_alter1_<eng> fails on Windows
Bug#36037 - Test parts.partition_alter2_<eng> fails on Windows
Bug#36038 - Test parts.partition_basic_<engine> failing on Windows
Bug#36039 - Test parts.partition_engine_<eng> fails on Windows
Bug#36040 - Test parts.partition_syntax_<eng> fails on Windows
Changeset to fix multiple 5.1.24-rc build bugs on Windows platforms.
Many of these bugs had the same root causes.
Fixed a missed case in the patch for Bug#31931.
Also makes Bug#33722 a duplicate of Bug#31931.
Added tests for better coverage.
Replaced some legacy function calls.
Since it used /tmp-dir, it continues to fail on the same server
until the /tmp-dir is cleaned. (Another problem was that it
uses DATA/INDEX DIR without checking for symlink, which is needed)
Solution:
Moved all DATA/INDEX DIR test to a new partition_basic_symlink.inc file
and use this for myisam and innodb, also requiring symlinks and
not_windows.
(i.e. removed DATA/INDEX DIR use from several tests)
Problem was that the mix of handlers was not consistent between
CREATE and ALTER
changed so that it works like:
- All partitions must use the same engine
AND it must be the same as the table.
- if one does NOT specify an engine on the table level
then one must either NOT specify any engine on any
partition/subpartition OR for ALL partitions/subpartitions
Note: that after a table have been created, the storage engine
is specified for all parts of the table (table/partition/subpartition)
and so when using alter, one does not need to specify it (unless one
wants to change the storage engine, then one have to specify it on the
table level)
Bug#31610 Remove outdated and redundant tests:
partition_02myisam partition_03ndb
Bug#32405 testsuite parts: partition_char_myisam wrong content
and cleanup of testsuite
- remove/correct wrong comments
- remove workarounds for fixed bugs
- replace error numbers with error names
- exclude subtests from execution which fail now because of
new limitations for partitioning functions
- remove code for the no more intended dual use
fast test in regression tests/slow test in testsuite
- analyze and fix problems with partition_char_innodb
- fix problems caused by last change of error numbers
- Introduce error name to error number mapping which makes
maintenance after next error renumbering easier
Bug#31481 test suite parts: Many tests fail because of changed server error codes
Bug#31243 Test "partition_basic_myisam" truncates path names
+ minor cleanup
Bug #30316: Some "parts" tests fail because the server uses "--secure-file-priv"
Bug #30341: Test suite "parts" needs to be adapted to the new rules disallowing many functio
Bug #30408: Suite "parts" needs bug numbers updated
Bug #30411: Suite "parts" needs bug numbers updated: ER_UNIQUE_KEY_NEED_ALL_FIELDS_IN_PF
Bug #30576: part_supported_sql_func_innodb.test tries to LOAD DATA outside of var dir
Bug #30581: partition_value tests use disallowed CAST() function
Included are some general fixes to allow the "parts" test suite to be run
successfully. This includes disabling a few tests or parts of tests,
cleaning up the test cases and their results, etc. Basically, these tests
have not been run for some time, and had suffered some bit rot.
The bugs were fixed as a single changeset, because in some ways they depend
on each other. I couldn't be sure I'd updated all the error codes (for
bugs 30408 and 30411) without also adapting to the new allowed functions
rules (bug 30341), and vice versa.
