Commit graph

358 commits

Author SHA1 Message Date
Aleksey Midenkov
9b46d8e5c4 MDEV-23968 CREATE TEMPORARY TABLE .. LIKE (system versioned table) returns error if unique index is defined in the table
- Remove row_start/row_end from keys in fix_create_like();

- Disable manual adding of implicit row_start/row_end to indexes on
  CREATE TABLE. INVISIBLE_SYSTEM fields are unoperable by user;

- Fix memory leak on allocation of Key_part_spec.
2020-10-20 10:49:54 +03:00
Aleksey Midenkov
ddea8f6a39 MDEV-23779 Error upon querying the view, that selecting from versioned table with partitions
PARTITION clause in SELECT means query is non-versioned (see
WITH_PARTITION_STORAGE_ENGINE in vers_setup_conds()).

vers_setup_conds() expands such query to SYSTEM_TIME_ALL which is then
added to VIEW specification. When VIEW is queried both clauses
PARTITION and FOR SYSTEM_TIME ALL lead to ER_VERS_QUERY_IN_PARTITION
(same place WITH_PARTITION_STORAGE_ENGINE).

Fix removes FOR SYSTEM_TIME ALL from VIEW by accessing original
SYSTEM_TIME clause: the one specified in parser. As a side-effect
EXPLAIN SELECT displays SYSTEM_TIME specified in SELECT which is
user-friendly.
2020-10-20 10:49:54 +03:00
Aleksey Midenkov
a3c379ea61 MDEV-23799 CREATE .. SELECT wrong result on join versioned table
For join to work correctly versioning condition must be added to table
on_expr. Without that JOIN_CACHE gets expression (1)

  trigcond(xtitle.row_end = TIMESTAMP'2038-01-19 06:14:07.999999') and
  trigcond(xtitle.elementId = x.`id` and xtitle.pkey = 'title')

instead of (2)

  trigcond(xtitle.elementId = x.`id` and xtitle.pkey = 'title')

for join_null_complements(). It is NULL-row of xtitle for
complementing the join and the above comparisons of course FALSE, but
trigcond (Item_func_trig_cond) makes them TRUE via its trig_var
property which is bound to some boolean properties of JOIN_TAB.

Expression (2) evaluated to TRUE because its trig_var is bound to
first_inner_tab->not_null_compl. The expression (1) does not evaluate
correctly because row_end comparison's trig_var is bound to
first_inner->found earlier. As a result JOIN_CACHE::check_match()
skipped the row for join_null_complements().

When we add versioning condition to table's on_expr the optimizer in
make_join_select() distributes conditions differently. tmp_cond
inherits on_expr value and in Good case it is full expression

xgender.elementId = x.`id` and xgender.pkey = 'gender' and
xgender.row_end = TIMESTAMP'2038-01-19 06:14:07.999999'

while in Bad case it is only

xgender.elementId = x.`id` and xgender.pkey = 'gender'.

Later in Good row_end condition is optimized out and we get one
trigcond in form of (2).
2020-10-20 10:49:54 +03:00
Nikita Malyavin
34f2be3b29 MDEV-16023 Unfortunate error message WARN_VERS_PART_FULL (partition <name> is full) when rotation time for the last interval passed
* remove one case of WARN_VERS_PART_FULL
2020-07-29 14:32:07 +10:00
Aleksey Midenkov
af83ed9f0e MDEV-20661 Virtual fields are not recalculated on system fields value assignment
Fix stale virtual field value in 4 cases: when virtual field depends
on row_start/row_end in timestamp/trx_id versioned table. row_start
dep is recalculated in vers_update_fields() (SQL and InnoDB
layer). row_end dep is recalculated on history row insert.
2020-07-20 18:28:08 +03:00
Aleksey Midenkov
af57c65809 MDEV-22061 InnoDB: Assertion of missing row in sec index row_start upon REPLACE on a system-versioned table
make_versioned_helper() appended new update field unconditionally
while it should check if this field already exists in update vector.

Misc renames to conform versioning prefix. vers_update_fields() name
conforms with sql layer TABLE::vers_update_fields().
2020-07-20 18:28:07 +03:00
Aleksey Midenkov
b633b6a9d8 MDEV-22906 Disallow system_versioning_asof in DML
system_versioning_asof does not influence on multi-delete,
multi-update, insert-select, replace-select.
2020-06-16 10:43:53 +03:00
Aleksey Midenkov
05693cf214 MDEV-22112 Assertion `tab_part_info->part_type == RANGE_PARTITION || tab_part_info->part_type == LIST_PARTITION' failed in prep_alter_part_table
Incorrect syntax for SYSTEM_TIME partition. work_part_info is detected
as HASH partition. We cannot add partition of different type neither
we cannot reorganize SYSTEM_TIME into/from different type
partitioning.

The sidefix for version until 10.5 corrects the message:
"For LIST partitions each partition must be defined"
2020-06-04 12:12:49 +03:00
Marko Mäkelä
2e1d10ecac Add end-of-test markers to ease merges 2020-05-30 10:48:27 +03:00
Aleksey Midenkov
19da9a51ae MDEV-16937 Strict SQL with system versioned tables causes issues
Respect system fields in NO_ZERO_DATE mode.

This is the subject for refactoring in MDEV-19597
2020-05-28 22:22:20 +03:00
Aleksey Midenkov
dd9773b723 MDEV-22413 Server hangs upon UPDATE on a view reading from versioned partitioned table
UPDATE gets access to history records because versioning conditions
are not set for VIEW. This leads to endless loop of inserting history
records when clustered index is rebuilt and ha_rnd_next() returns
newly inserted history record.

Return back original behavior of failing on write-locked table in
historical query.

35b679b9 assumed that SELECT_LEX::lock_type influences anything, but
actually at this point table is already locked. Original bug report
was 
2020-05-28 22:22:19 +03:00
Aleksey Midenkov
dac1280a65 MDEV-18794 Assertion `!m_innodb' failed in ha_partition::cmp_ref upon SELECT from partitioned table
System versioning assertion fix. Since DROP SYSTEM VERSIONING does not
change list of dropped keys we should handle a special case.

Caused by MDEV-19751. This fix deprecates MDEV-17091.
2020-05-28 20:55:59 +03:00
Aleksey Midenkov
0932c5804d MDEV-20515 multi-update tries to position updated table by null reference
Cause

Join tmp table inserts null row because of OUTER JOIN, that's
expected. Since `multi_update::prepare2()` converted
`Item_temptable_rowid` into `Item_field` (28dbdf3)
`multi_update::send_data()` accesses join tmp record directly and
treats it as a normal row ignoring null status of ref field. NULL ref
field is then treated as normal in `multi_update::do_updates()` which
tries to position updated table by reference 0.

Note that reference 0 may be valid reference and the first row of
table can be wrongly updated (see multi_update.test).

Fix

Do not add row into multi-update tmp table in case of null ref
field. Join tmp table does not have null_row status at this time (as
well as `STATUS_NULL_ROW`) and cannot be skipped by these properties
(see first comment in multi_update::send_data()). But it has all null
fields (including the ref field).
2020-04-02 20:48:38 +03:00
Aleksey Midenkov
ba34f409ad MDEV-21688 Assertion or ER_WARN_DATA_OUT_OF_RANGE upon ALTER on previously versioned table
Earlier skip of history row. Cleanup of dead code for VTMD.
2020-04-02 20:48:38 +03:00
Aleksey Midenkov
44c6c7a923 MDEV-21342 Assertion in set_ok_status() upon spatial field error on system-versioned table
SQL_SELECT::check_quick() returns error status only
test_quick_select() returns -1. Fix error handling when lower frames
throw error, but it is ignored by test_quick_select(). Fix return
status for out-of-memory errors which are obviously must be processed
as error in upper frames.
2020-04-02 20:48:38 +03:00
Nikita Malyavin
9149017bb8
MDEV-17091 - Assertion failed after dropping versioning
Assertion `old_part_id == m_last_part' failed in ha_partition::update_row or `part_id == m_last_part' in ha_partition::delete_row upon UPDATE/DELETE after dropping versioning

PRIMARY KEY change hadn't been treated as partition reorganization in case of partitioning by KEY() (without parameters).

* set `*partition_changed= true` in the described case.
* since add/drop system versioning does not affect alter_info->key_list, it required separate attention
2020-04-02 22:37:36 +10:00
Aleksey Midenkov
006f6f97b1 MDEV-17798 System variable system_versioning_asof accepts wrong values 2020-02-02 15:13:29 +03:00
Aleksey Midenkov
14e6f0251c MDEV-20955 versioning.update failed in buildbot with wrong result code
Race condition when innodb_lock_wait_timeout (default 50 seconds)
exceeds for 'send update', but information_schema.innodb_lock_waits
still sees this wait or it my exit by timeout. My occur on overloaded
host.
2020-02-02 15:13:29 +03:00
Nikita Malyavin
4923604ee2 MDEV-18865 Assertion `t->first->versioned_by_id()' failed in innodb_prepare_commit_versioned
Cause:
* row_start != 0 treated as it exists. Probably, possible row permutations had not been taken in mind.

Solution:
* Checking both row_start and row_end is correct, so versioned() function is used
2019-12-29 12:16:04 +02:00
Nikita Malyavin
720e9bd5be MDEV-18875 Assertion `thd->transaction.stmt.ha_list == __null || trans == &thd->transaction.stmt' failed or bogus ER_DUP_ENTRY upon ALTER TABLE with versioning
Cause:
* when autocommit=0 (or transaction is issued by user),
 `ha_commit_trans` is called twice on ALTER TABLE, causing a duplicated
 insert into `transaction_registry` (ER_DUP_ENTRY).

Solution:
* ALTER TABLE makes an implicit commit by a second call. We actually
 need to make an insert only when it is a real commit. So is_real
 variable is additionally checked.
2019-12-29 12:16:04 +02:00
Aleksey Midenkov
be92dce613 MDEV-21234 Server crashes in in setup_on_expr upon 3rd execution of SP
Versioned conditions in on_expr can not be rebuilt at optimization
stage on non-conventional arena.
2019-12-05 23:53:07 +03:00
Aleksey Midenkov
d759f764f6 MDEV-21233 Assertion `m_extra_cache' failed in ha_partition::late_extra_cache
Incorrect assertion of EXTRA_CACHE for
HA_EXTRA_PREPARE_FOR_UPDATE. The latter is related to read cache, but
must operate without it as a noop.

Related to Bug#55458 and MDEV-20441.
2019-12-05 15:25:30 +03:00
Aleksey Midenkov
477629d2cd MDEV-21147 Assertion `marked_for_read()' upon UPDATE on versioned table via view
Unit prepare prematurely fixed field which must be fixed via
setup_conds() to correctly update table->covering_keys.

Call vers_setup_conds() directly instead, because actually everything
else is not needed.
2019-12-04 07:10:41 +03:00
Aleksey Midenkov
cef2b34f25 MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED
Wrong assertion condition. SYSTEM_TIME_ALL indicates that
vers_setup_conds() is done. In case FOR SYSTEM_TIME ALL is specified
in command the assertion passes but not checks anything.
2019-12-03 15:28:41 +03:00
Aleksey Midenkov
db32d9457e MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED
Don't do skip_setup_conds() unless all errors are checked.

Fixes following errors:
      ER_PERIOD_NOT_FOUND
      ER_VERS_QUERY_IN_PARTITION
      ER_VERS_ENGINE_UNSUPPORTED
      ER_VERS_NOT_VERSIONED
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
a7cf0db3d8 MDEV-21011 Table corruption reported for versioned partitioned table after DELETE: "Found a misplaced row"
LIMIT history partitions cannot be checked by existing algorithm of
check_misplaced_rows() because working history partition is
incremented each time another one is filled. The existing algorithm
gets record and tries to decide partition id for it by
get_partition_id(). For LIMIT history it will just get first
non-filled partition.

To fix such partitions it is required to do REBUILD instead of REPAIR.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
6dd41e008e MDEV-21155 Assertion with versioned table upon DELETE from view of view after replacing first view
When view is merged by DT_MERGE_FOR_INSERT it is then skipped from
processing and doesn't update WHERE clause with
vers_setup_conds(). Note that view itself cannot work in
vers_setup_conds() because it doesn't have row_start, row_end
fields. Thus it is required to descend down to material TABLE_LIST
through calls of mysql_derived_prepare() and run vers_setup_conds()
from there. Luckily, all views (views of views, views of views of
views, etc.) are linked in one list through next_global pointer, so we
can skip all views of views and get straight to non-view TABLE_LIST by
checking its merge_underlying_list property for zero value (it is
assigned by DT_MERGE_FOR_INSERT for merged derived tables).

We have to do that only for UPDATE and DELETE. Other DML commands
don't use WHERE clause.

MDEV-21146 Assertion `m_lock_type == 2' in handler::ha_drop_table upon LOAD DATA

LOAD DATA does not use WHERE and the above call of vers_setup_conds()
is not needed. unit->prepare() led to wrongly locked temporary table.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
97aa07abf5 MDEV-21147 Assertion `marked_for_read()' upon UPDATE on versioned table via view
"write set" for replication finally got its correct place
(mark_columns_per_binlog_row_image()). When done generally in
mark_columns_needed_for_update() it affects optimization
algorithm. used_key_is_modified, query_plan.using_io_buffer are
wrongly set and that leads to wrong prepare_for_keyread() which limits
read_set.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
498a96a478 MDEV-20441 ER_CRASHED_ON_USAGE upon update on versioned Aria table
Turn read cache off for update and multi-update for versioned
table. no_cache is reinited on each TABLE open because it is
applicable for specific algorithms.

As a side fix vers_insert_history_row() honors vers_write setting.

Aria with row_format=fixed uses IO_CACHE of type READ_CACHE for
sequential read in update loop. When history row is inserted inside
this loop the cache misses it and fails with error.

TODO:

Currently maria_extra() does not support SEQ_READ_APPEND. Probably it
might be possible to use this type of cache.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
0076dce2c8 MDEV-18727 improve DML operation of System Versioning
MDEV-18957 UPDATE with LIMIT clause is wrong for versioned partitioned tables

UPDATE, DELETE: replace linear search of current/historical records
with vers_setup_conds().

Additional DML cases in view.test
2019-11-22 14:29:03 +03:00
Aleksey Midenkov
7ae0be25a6 MDEV-20812 Unexpected ER_ROW_IS_REFERENCED_2 upon DELETE from versioned table with FK
MDEV-16210 original case was wrongly allowed versioned DELETE from
referenced table where reference is by non-primary key. InnoDB UPDATE
has optimization for new rows not changing its clustered index
position. In this case InnoDB doesn't update all secondary indexes and
misses the one holding the referenced key. The fix was to disable this
optimization for versioned DELETE. In case of versioned DELETE we
forcely update all secondary indexes and therefore check them for
constraints.

But the above fix raised another problem with versioned DELETE on
foreign table side. In case when there was no corresponding record in
referenced table (illegal foreign reference can be done with "set
foreign_key_checks=off") there was spurious constraint check (because
versioned DELETE is actually UPDATE) and hence the operation failed
with constraint error.

MDEV-16210 tried to fix the above problem by checking foreign table
instead of referenced table and that at least was illegal.

Constraint check is done by row_ins_check_foreign_constraint() no
matter what kind of table is checked, referenced or foreign
(controlled by check_ref argument).

Referenced table is checked by row_upd_check_references_constraints().
Foreign table is checked by row_ins_check_foreign_constraints().

Current fix rolls back the wrong fix for the above problem and
disables referenced table check for DELETE on foreign side by
introducing `check_foreign` argument which when set to *false* skips
row_ins_check_foreign_constraints() call.
2019-10-15 13:29:18 +03:00
Aleksey Midenkov
a92f3146d2 MDEV-19406 Assertion on updating view of join with versioned table
TABLE::mark_columns_needed_for_update(): use_all_columns() assigns
pointer of all_set into read_set and write_set, but this is not good
since all_set is changed later by
TABLE::mark_columns_used_by_index_no_reset().

Do column_bitmaps_signal() whenever we change read_set/write_set.
2019-10-10 00:20:34 +03:00
Aleksey Midenkov
647a38818a MDEV-16130 wrong error message adding AS ROW START to versioned table 2019-10-10 00:20:34 +03:00
Aleksey Midenkov
75ba5c815d MDEV-16210 FK constraints on versioned tables use historical rows, which may cause constraint violation
Constraint check is done on secondary index update.
F.ex. DELETE does row_upd_sec_index_entry() and checks constraints in
row_upd_check_references_constraints(). UPDATE is optimized for the
case when order is not changed (node->cmpl_info & UPD_NODE_NO_ORD_CHANGE)
and doesn't do row_upd_sec_index_entry(), so it doesn't check constraints.

Since for versioned DELETE we do UPDATE actually, but expect behaviour
of DELETE in terms of constraints, we should deny this optimization to
get constraints checked.

Fix wrong referenced table check when versioned DELETE inserts history in parent
table. Set check_ref to false in this case.

Removed unused dup_chk_only argument for row_ins_sec_index_entry() and
added check_ref argument.

MDEV-18057 fix was superseded by this fix and reverted.

foreign.test:
All key_type combinations: pk, unique, sec(ondary).
2019-10-10 00:20:34 +03:00
Aleksey Midenkov
6684989801 versioning test suite fixes
Preparation for MDEV-16210:

replace.test:
key_type combinations: PK and UNIQUE.

foreign.test:
Preparation for key_type combinations.

Other fixes:

* Merged versioning.update2 into versioning.update;
* Removed test2 database and done individual drop instead.
2019-10-10 00:20:34 +03:00
Nikita Malyavin
f6a7730c45 MDEV-16490: It's possible to make a system versioned table without any versioning field
* do not allow versioned table to be without versioned (non-system) fields
* prohibit changing field versioning, when removing table versioning
* handle CREATE...SELECT as well
2019-09-09 20:14:47 +03:00
Monty
a071e0e029 Merge branch '10.2' into 10.3 2019-09-03 13:17:32 +03:00
Aleksey Midenkov
b96e4424fb MDEV-17613 MIN/MAX Optimization (Select tables optimized away) does not work
Current easy fix is not possible, because SELECT clones ha_partition
and then closes the clone which leads to unclosed transaction in
partitions we forcely prune out. We cound solve this by closing these
partitions (and release from transaction) in
change_partitions_to_open() at versioning conditions stage, but this
is problematic because table lock is acquired for each partition at
open stage and therefore must be released when we close partition
handler in change_partitions_to_open(). More details in MDEV-20376.

This should change after MDEV-20250 where mechanism of opening
partitions will be improved.

This reverts commit cdbac54df0.
2019-08-21 16:08:31 +03:00
Aleksey Midenkov
a20f6f9853 MDEV-20336 Assertion bitmap_is_set(read_partitions) upon SELECT FOR UPDATE from versioned table
Exclude SELECT and INSERT SELECT from vers_set_hist_part(). We cannot
likewise exclude REPLACE SELECT because it may REPLACE into itself
(and REPLACE generates history).

INSERT also does not generate history, but we have history
modification setting which might be interfered.
2019-08-14 17:32:19 +03:00
Aleksey Midenkov
39db116562 MDEV-18862 Unfortunate error message upon attempt to drop system versioning
Fix error code.
2019-08-14 11:58:58 +03:00
Aleksey Midenkov
0b74c8832d MDEV-19127 Assertion `row_start_field' failed in vers_prepare_keys upon ALTER TABLE
Prevent conflicting clauses at parser level.

Clear HA_VERSIONED_TABLE flag for DROP SYSTEM VERSIONING (for the sake
of strictness).
2019-08-11 12:32:08 +03:00
Aleksey Midenkov
638e78853f MDEV-18862 Unfortunate error message upon attempt to drop system versioning
Special case for DROP PERIOD when system fields are implicit.
2019-08-11 12:32:08 +03:00
Aleksey Midenkov
5851e668d7 MDEV-19304 Segfault in ALTER TABLE after UPDATE for SIMULTANEOUS_ASSIGNMENT
For MODE_SIMULTANEOUS_ASSIGNMENT it is required to return back field
offsets from record[1] to record[0]. 'continue' in warning branch did
skip of rfield->move_field_offset() call.
2019-08-11 12:32:08 +03:00
Aleksey Midenkov
98758b52b3 MDEV-20068 History partition rotation is not done under LOCK TABLES
Wrong value F_WRLCK for thr_lock_type.
2019-08-11 12:32:08 +03:00
Aleksey Midenkov
cdbac54df0 MDEV-17613 MIN/MAX Optimization (Select tables optimized away) does not work
Prune to now-partition when there is no FOR SYSTEM_TIME clause.
2019-08-11 12:32:08 +03:00
Aleksey Midenkov
00a254cc06 MDEV-20186 Wrong result or Assertion on INSERT after DELETE HISTORY
Fix reinitialize vers_write on table reopen.
2019-07-26 18:21:23 +03:00
Aleksey Midenkov
f6d0d309fa MDEV-19814 Assertion `update->n_fields < ulint(table->n_cols + table->n_v_cols)' on DELETE HISTORY
Turn off versioned_write for DELETE HISTORY.
2019-07-25 21:11:15 +03:00
Aleksey Midenkov
b0dd048edd MDEV-19175 Server crashes in ha_partition::vers_can_native upon INSERT DELAYED into versioned partitioned table
Uninitialized LEX::part_info on ha_partition::vers_can_native().
2019-07-09 10:01:54 +03:00
Aleksey Midenkov
3ffa06bc20 Tests: versioning suite fix when no test_versioning plugin 2019-07-09 10:01:54 +03:00
Sergei Golubchik
95d783af62 fix versioning.simple for embedded 2019-06-15 22:32:09 +02:00