A few of the failures was because of missing sync_slave_to_master in
the test suite.
However, the biggest reason for most faulures was that in case of
ALTER PARTITION the master writes the query to the binary log before
it has updated the .frm and .par files. This causes a problem for an
S3 slave as it will start execute the ALTER PARTITION but get old .frm and
.par files from S3 which causes "open table" to fail, either with an error
or in some case with a crash.
Fixed
Incorrect syntax for SYSTEM_TIME partition. work_part_info is detected
as HASH partition. We cannot add partition of different type neither
we cannot reorganize SYSTEM_TIME into/from different type
partitioning.
The sidefix for version until 10.5 corrects the message:
"For LIST partitions each partition must be defined"
System versioning assertion fix. Since DROP SYSTEM VERSIONING does not
change list of dropped keys we should handle a special case.
Caused by MDEV-19751. This fix deprecates MDEV-17091.
Default (empty) field list in partitioning by KEY() clause is assigned
from primary key. If primary key is changed the partitioning field
list is changed as well, so repartitioning required. Not applicable to
any non-primary keys as default field list may be taken only from
primary key.
MDEV-21398 Deadlock (server hang) or assertion failure in
Diagnostics_area::set_error_status upon ALTER under lock
This failure could only happen if one locked the same table
multiple times and then did an ALTER TABLE on the table.
Major change is to change all instances of
table->m_needs_reopen= true;
to
table->mark_table_for_reopen();
The main fix for the problem was to ensure that we mark all
instances of the table in the locked_table_list and when we
reopen the tables, we first close all tables before reopening
and locking them.
Other things:
- Don't call thd->locked_tables_list.reopen_tables if there
are no tables marked for reopen. (performance)
MDEV-22488 test failures: parts.partition_debug_innodb /
parts.partition_debug_myisam
The reason for the failure was a wrong printf() that accessed not existing
things on the stack.
The reason the falure was hard to find was that the partition_debug_...
tests disables core dumps, so there was no trace that the server had
crashed in the logs.
Fixed by fixing the faulty push_warning_printf() and splitting the tests
into two parts, one that test failures (with core dumps enabled) and one
part that test crash recovery.
The review and test splitting was done by Monty
- ALTER_ALGORITHM should be substituted when there is no mention of
algorithm in alter statement.
- Introduced algorithm(thd) in Alter_info. It returns the
user requested algorithm. If user doesn't specify algorithm explicitly then
it returns alter_algorithm variable.
- changed algorithm() to get_algorithm(thd) to return algorithm name for
displaying the error.
- set_requested_algorithm(algo_value) to avoid direct assignment on
requested_algorithm variable.
- Avoid direct access of requested_algorithm to encapsulate
requested_algorithm variable
This is continuation of MDEV-22153 bug when contiguity of history
partitions is broken. ha_partition::open_read_partitions() can not
handle non-contiguous list of default partitions.
Fix: when default partition is dropped convert list of partitions to
non-default.
MDEV-22088 S3 partitioning support
All ALTER PARTITION commands should now work on S3 tables except
REBUILD PARTITION
TRUNCATE PARTITION
REORGANIZE PARTITION
In addition, PARTIONED S3 TABLES can also be replicated.
This is achived by storing the partition tables .frm and .par file on S3
for partitioned shared (S3) tables.
The discovery methods are enchanced by allowing engines that supports
discovery to also support of the partitioned tables .frm and .par file
Things in more detail
- The .frm and .par files of partitioned tables are stored in S3 and kept
in sync.
- Added hton callback create_partitioning_metadata to inform handler
that metadata for a partitoned file has changed
- Added back handler::discover_check_version() to be able to check if
a table's or a part table's definition has changed.
- Added handler::check_if_updates_are_ignored(). Needed for partitioning.
- Renamed rebind() -> rebind_psi(), as it was before.
- Changed CHF_xxx hadnler flags to an enum
- Changed some checks from using table->file->ht to use
table->file->partition_ht() to get discovery to work with partitioning.
- If TABLE_SHARE::init_from_binary_frm_image() fails, ensure that we
don't leave any .frm or .par files around.
- Fixed that writefrm() doesn't leave unusable .frm files around
- Appended extension to path for writefrm() to be able to reuse to function
for creating .par files.
- Added DBUG_PUSH("") to a a few functions that caused a lot of not
critical tracing.
>= M_TOT_PARTS' FAILED.
This patch is taken from MySQL, originally written by Mattias Jonsson
Here follows the original commit message:
Problem in handle_alter_part_error(),
result in altered partition_info object was still used
if table was under LOCK TABLES.
Solution was to always close and destroy all table
and table_share instances if exclusive mdl lock was
possible.
If not succeeding in get an exlusive lock (only possible
during rollback of DDL), at least close and destroy this
table instance.
rb#7361.
Approved by Mikael and Aditya.
In main.index_merge_myisam we remove the test that was added in
commit a2d24def8c because
it duplicates the test case that was added in
commit 5af12e4635.
- Fixed mysql_prepare_create_table() constraint duplicate checking;
- Refactored period constraint handling in mysql_prepare_alter_table():
* No need to allocate new objects;
* Keep old constraint name but exclude it from dup checking by automatic_name;
- Some minor memory leaks fixed;
- Some conceptual TODOs.
close_all_tables_for_name() is always preceded by
wait_while_table_is_used(), which makes tdc_remove_table() redundant.
The only (now fixed) exception was close_cached_tables().
Part of MDEV-17882 - Cleanup refresh version
Assertion `old_part_id == m_last_part' failed in ha_partition::update_row or `part_id == m_last_part' in ha_partition::delete_row upon UPDATE/DELETE after dropping versioning
PRIMARY KEY change hadn't been treated as partition reorganization in case of partitioning by KEY() (without parameters).
* set `*partition_changed= true` in the described case.
* since add/drop system versioning does not affect alter_info->key_list, it required separate attention
When there are E empty partitions left, auto-create N new empty
partitions for SYSTEM_TIME partitioning rotated by INTERVAL/LIMIT and
marked by AUTO_INCREMENT keyword. Syntax change: AUTO_INCREMENT
keyword (or shorter AUTO may be used instead) after LIMIT/INTERVAL
clause.
CREATE OR REPLACE TABLE t (x INT) WITH SYSTEM VERSIONING
PARTITION BY SYSTEM_TIME LIMIT 100000 AUTO_INCREMENT;
CREATE OR REPLACE TABLE t (x INT) WITH SYSTEM VERSIONING
PARTITION BY SYSTEM_TIME INTERVAL 1 WEEK AUTO_INCREMENT;
The current revision implements hard-coded values of 1 for E and N. As
well as auto-creation threshold MinInterval = 1 hour, MinLimit = 1000.
The name for newly added partition will be first chosen as "pX", where
X is partition number and "p" is hard-coded name prefix. If this name
is already occupied, the X will be incremented until the resulting
name will be free to use.
ALTER TABLE ADD PARTITION is now always fast. If there some history
partition overflow occurs manual ALTER TABLE REBUILD PARTITION is
needed.
* Explicit STARTS syntax
* SHOW CREATE
* Default STARTS rounding depending on INTERVAL type
* Warn when STARTS timestamp is later than query time
* Fix uninitialized Lex->create_last_non_select_table under
mysql_unpack_partition()
Default STARTS rounding depending on INTERVAL type
If STARTS clause is omitted, default one is assigned with value
derived from query timestamp. The rounding is done on STARTS value
depending on INTERVAL type:
SECOND: no rounding is done;
MINUTE: timestamp seconds is set to 0;
HOUR: timestamp seconds and minutes are set to 0;
DAY, WEEK, MONTH and YEAR: timestamp seconds, minutes and hours are
set to 0 (the date of rotation is kept as current date).
The code in convert_charset_partition_constant() did not
take into account that the call for item->safe_charset_converter()
can return NULL when conversion is not safe.
Note, 10.2 was not affected. The test for NULL presents in 10.2,
but it disappeared in 10.3 in a mistake. Restoring the test.
The old code to print partition values was too complicated:
- it created new Items for character set conversion purposes.
- it mixed string conversion and partition error reporting
in the same code blocks.
Simplifying the code as follows:
- Adding helper methods String::can_be_safely_convert_to() and
String::append_introducer_and_hex().
- Adding DBUG_EXECUTE_IF("generate_partition_syntax_for_frm", push_warning...)
into generate_partition_syntax_for_frm(), to test the PARTITON
clause written to FRM. Adding test partition_utf8-debug.test for this.
- Removing functions get_cs_converted_part_value_from_string() and
get_cs_converted_string_value. Changing get_partition_column_description()
to use Type_handler::partition_field_append_value() instead.
This makes SHOW CREATE TABLE and SELECT FROM I_S.PARTITIONS
use the same code path.
- Changing Type_handler::partition_field_append_value() not to
call convert_charset_partition_constant(), to avoid creating a new Item
for string conversion pursposes.
Rewritting the code to use only String methods.
- Removing error reporting code (ER_PARTITION_FUNCTION_IS_NOT_ALLOWED)
from Type_handler::partition_field_append_value().
The error is correctly detected and reported on the caller level.
So error reporting was redundant here.
Also:
- Moving methods Type_handler::partition_field_*() from sql_partition.cc
to sql_type.cc. This fixes compilation problem with -DPLUGIN_PARTITION=NO,
earlier introduced by the patch for MDEV-20831.
This clause in CREATE TABLE:
PARTITION BY LIST COLUMNS (inet6column)
(PARTITION p1 VALUES IN ('::'))
was erroneously written to frm file as:
PARTITION BY LIST COLUMNS(inet6column)
(PARTITION p1 VALUES IN (_binary 0x3A3A))
I.e. the text value '::' was converted to HEX representation
and prefixed with _binary.
A simple fix could write `_latin1 0x3A3A` instead of `_binary 0x3A3A`,
but in case of INET6 we don't need neither character set introducers,
nor HEX encoding, because text representation of INET6 values consist
of pure ASCII characters.
So this patch changes the above clause to be printed as:
PARTITION BY LIST COLUMNS(inet6column)
(PARTITION p1 VALUES IN ('::'))
Details:
The old code in check_part_field() was not friendly to pluggable data types.
Replacing this function to two new virtual methods in Type_handler:
virtual bool partition_field_check(const LEX_CSTRING &field_name,
Item *item_expr) const;
virtual bool partition_field_append_value(String *str,
Item *item_expr,
CHARSET_INFO *field_cs,
partition_value_print_mode_t mode)
const;
so data type plugins can decide whether they need to use character set
introducer and/or hex encoding when printing partition values.
InnoDB intentionally (it's a documented behavior) ignores changing of
DATA DIRECTORY and INDEX DIRECTORY for partitions. Though we should
issue warning when this happens.
