Commit graph

2978 commits

Author SHA1 Message Date
Alexey Botchkov
e9fd327ee3 MDEV-17399 Add support for JSON_TABLE.
The specific table handler for the table functions was introduced,
and used to implement JSON_TABLE.
2021-04-21 10:21:43 +04:00
Marko Mäkelä
d2e2d32933 Merge 10.5 into 10.6 2021-04-14 12:32:27 +03:00
Marko Mäkelä
6c3e860cbf Merge 10.4 into 10.5 2021-04-14 11:35:39 +03:00
Marko Mäkelä
5008171b05 Merge 10.3 into 10.4 2021-04-14 10:33:59 +03:00
Aleksey Midenkov
77ffbbca49 MDEV-25172 Wrong error message for ADD COLUMN .. AS ROW START
Handle one more condition in fix_alter_info() for non-versioned table
and produce ER_VERS_NOT_VERSIONED error.
2021-03-31 21:25:41 +03:00
Aleksey Midenkov
0c99e6e9a6 MDEV-22562 Assertion `next_insert_id == 0' upon UPDATE on system-versioned table
Don't update autoinc counter on history row insert. Uniqueness is kept
due to merge with row_end.
2021-03-31 21:25:36 +03:00
Marko Mäkelä
2ad61c6782 Merge 10.5 into 10.6 2021-03-29 16:16:12 +03:00
Marko Mäkelä
e8b7fceb82 MDEV-24302: RESET MASTER hangs
Starting with MariaDB 10.5, roughly after MDEV-23855 was fixed,
we are observing sporadic hangs during the execution of the
RESET MASTER statement. We are hoping to fix the hangs with these
changes, but due to the rather infrequent occurrence of the hangs
and our inability to reliably reproduce the hangs, we cannot be
sure of this.

What we do know is that innodb_force_recovery=2 (or a larger setting)
will prevent srv_master_callback (the former srv_master_thread) from
running. In that mode, periodic log flushes would never occur and
RESET MASTER could hang indefinitely. That is demonstrated by the new
test case that was developed by Andrei Elkin. We fix this case by
implementing a special case for it.

This also includes some code cleanup and renames of misleadingly
named code. The interface has nothing to do with log checkpoints in
the storage engine; it is only about requesting log writes to be
persistent.

handlerton::commit_checkpoint_request,
commit_checkpoint_notify_ha(): Remove the unused parameter hton.

log_requests.start: Replaces pending_checkpoint_list.
log_requests.end: Replaces pending_checkpoint_list_end.
log_requests.mutex: Replaces pending_checkpoint_mutex.

log_flush_notify_and_unlock(), log_flush_notify(): Replaces
innobase_mysql_log_notify().  The new implementation should be
functionally equivalent to the old one.

innodb_log_flush_request(): Replaces innobase_checkpoint_request().
Implement a fast path for common cases, and reduce the mutex hold time.
POSSIBLE FIX OF THE HANG: We will invoke commit_checkpoint_notify_ha()
for the current request if it is already satisfied, as well as invoke
log_flush_notify_and_unlock() for any satisfied requests.

log_write(): Invoke log_flush_notify() when the write is already durable.
This was missing WITH_PMEM when the log is in persistent memory.

Reviewed by: Vladislav Vaintroub
2021-03-29 15:16:23 +03:00
Varun Gupta
f691d9865b MDEV-7317: Make an index ignorable to the optimizer
This feature adds the functionality of ignorability for indexes.
Indexes are not ignored be default.

To control index ignorability explicitly for a new index,
use IGNORE or NOT IGNORE as part of the index definition for
CREATE TABLE, CREATE INDEX, or ALTER TABLE.

Primary keys (explicit or implicit) cannot be made ignorable.

The table INFORMATION_SCHEMA.STATISTICS get a new column named IGNORED that
would store whether an index needs to be ignored or not.
2021-03-04 22:50:00 +05:30
Sergei Golubchik
25d9d2e37f Merge branch 'bb-10.4-release' into bb-10.5-release 2021-02-15 16:43:15 +01:00
Sergei Golubchik
00a313ecf3 Merge branch 'bb-10.3-release' into bb-10.4-release
Note, the fix for "MDEV-23328 Server hang due to Galera lock conflict resolution"
was null-merged. 10.4 version of the fix is coming up separately
2021-02-12 17:44:22 +01:00
Sergei Golubchik
60ea09eae6 Merge branch '10.2' into 10.3 2021-02-01 13:49:33 +01:00
Sergei Golubchik
0d8bd7cc3a MDEV-18428 Memory: If transactional=0 is specified in CREATE TABLE, it is not possible to ALTER TABLE
* be strict in CREATE TABLE, just like in ALTER TABLE, because
  CREATE TABLE, just like ALTER TABLE, can be rolled back for any engine
* but don't auto-convert warnings into errors for engine warnings
  (handler::create) - this matches ALTER TABLE behavior
* and not when creating a default record, these errors are handled
  specially (and replaced with ER_INVALID_DEFAULT)
* always issue a Note when a non-unique key is truncated, because it's
  not a Warning that can be converted to an Error. Before this commit
  it was a Note for blobs and a Warning for all other data types.
2021-01-11 21:54:47 +01:00
Sergei Golubchik
9b750dcbd8 MDEV-23536 Race condition between KILL and transaction commit
Server part:
  kill_handlerton() was accessing thd->ha_data[] for some other thd,
  while it could be concurrently modified by its owner thd.

  protect thd->ha_data[] modifications with a mutex.
  require this mutex when accessing thd->ha_data[] from kill_handlerton.

InnoDB part:
  on close_connection, detach trx from thd before freeing the trx
2021-01-11 21:54:47 +01:00
Oleksandr Byelkin
02e7bff882 Merge commit '10.4' into 10.5 2021-01-06 10:53:00 +01:00
Oleksandr Byelkin
478b83032b Merge branch '10.3' into 10.4 2020-12-25 09:13:28 +01:00
Oleksandr Byelkin
25561435e0 Merge branch '10.2' into 10.3 2020-12-23 19:28:02 +01:00
Rucha Deodhar
74223c33d1 MDEV-23209: Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())'
failed in Diagnostics_area::set_ok_status on INSERT

Analysis: Error is not returned when strict mode is enabled and value is
truncated because double is outside range.
Fix: Return HA_ERR_AUTOINC_ERANGE if the error was reported when double is
outside range.
2020-12-15 13:00:24 +05:30
Sergei Petrunia
6859e80df7 MDEV-24351: S3, same-backend replication: Dropping a table on master...
..causes error on slave.
Cause: if the master doesn't have the frm file for the table,
DROP TABLE code will call ha_delete_table_force() to drop the table
in all available storage engines.
The issue was that this code path didn't check for
HTON_TABLE_MAY_NOT_EXIST_ON_SLAVE flag for the storage engine,
and so did not add "... IF EXISTS" to the statement that's written
to the binary log.  This can cause error on the slave when it tries to
drop a table that's already gone.
2020-12-08 17:58:22 +03:00
Marko Mäkelä
6a1e655cb0 Merge 10.4 into 10.5 2020-12-02 18:29:49 +02:00
Marko Mäkelä
589cf8dbf3 Merge 10.3 into 10.4 2020-12-01 19:51:14 +02:00
Sergei Golubchik
00f54b56b1 cleanup: RAII helper for changing thd->count_cuted_rows 2020-11-25 22:19:59 +01:00
Nikita Malyavin
f244b499e7 handler: move row change start signal down after the checks 2020-11-02 14:21:08 +10:00
Nikita Malyavin
afca976885 MDEV-22639 Assertion failed in ha_check_overlaps upon multi-table update
After Sergei's cleanup this assertion is not actual anymore -- we can't
predict if the handler was used for lookup, especially in multi-update
scenario.

`position(old_data)` is made earlier in `ha_check_overlaps`, therefore it
is guaranteed that we compare right refs.
2020-11-02 14:11:43 +10:00
Nikita Malyavin
d543363f25 MDEV-22714 Assertion failed upon multi-update on table WITHOUT OVERLAPS
The problem here was that ha_check_overlaps internally uses ha_index_read,
which in case of fail overwrites table->status. Even though the handlers
are different, they share a common table, so the value is anyway spoiled.
This is bad, and table->status is badly designed and overweighted by
functionality, but nothing can be done with it, since the code related to
this logic is ancient and it's impossible to extract it with normal effort.

So let's just save and restore the value in ha_update_row before and after
the checks.

Other operations like INSERT and simple UPDATE are not in risk, since they
don't use this table->status approach.
DELETE does not do any unique checks, so it's also safe.
2020-11-02 14:11:42 +10:00
Marko Mäkelä
1657b7a583 Merge 10.4 to 10.5 2020-10-22 17:08:49 +03:00
Marko Mäkelä
46957a6a77 Merge 10.3 into 10.4 2020-10-22 13:27:18 +03:00
Marko Mäkelä
e3d692aa09 Merge 10.2 into 10.3 2020-10-22 08:26:28 +03:00
Daniele Sciascia
fdf87973cb MDEV-23081 Stray XA transactions at startup, with wsrep_on=OFF
Change xarecover_handlerton so that transaction with WSREP prefixed
xids are rolled back when Galera is disabled.

Reviewd-by: Jan Lindström <jan.lindstrom@mariadb.com>
2020-10-21 16:29:07 +03:00
Marko Mäkelä
620ea816ad Merge 10.1 into 10.2 2020-10-21 14:02:04 +03:00
Monty
71d263a198 MDEV-23691 S3 storage engine: delayed slave can drop the table
This commit fixed the problems with S3 after the "DROP TABLE FORCE" changes.
It also fixes all failing replication S3 tests.

A slave is delayed if it is trying to execute replicated queries on a
table that is already converted to S3 by the master later in the binlog.

Fixes for replication events on S3 tables for delayed slaves:
- INSERT and INSERT ... SELECT and CREATE TABLE are ignored but written
  to the binary log.   UPDATE & DELETE will be fixed in a future commit.

Other things:
- On slaves with --s3-slave-ignore-updates set, allow S3 tables to be
  opened in read-write mode. This was done to be able to
  ignore-but-replicate queries like insert.  Without this change any
  open of an S3 table failed with 'Table is read only' which is too
  early to be able to replicate the original query.
- Errors are now printed if handler::extra() call fails in
  wait_while_tables_are_used().
- Error message for row changes are changed from HA_ERR_WRONG_COMMAND
  to HA_ERR_TABLE_READONLY.
- Disable some maria_extra() calls for S3 tables. This could cause
  S3 tables to fail in some cases.
- Added missing thr_lock_delete() to ma_open() in case of failure.
- Removed from mysql_prepare_insert() the not needed argument 'table'.
2020-10-21 03:09:29 +03:00
Aleksey Midenkov
9b46d8e5c4 MDEV-23968 CREATE TEMPORARY TABLE .. LIKE (system versioned table) returns error if unique index is defined in the table
- Remove row_start/row_end from keys in fix_create_like();

- Disable manual adding of implicit row_start/row_end to indexes on
  CREATE TABLE. INVISIBLE_SYSTEM fields are unoperable by user;

- Fix memory leak on allocation of Key_part_spec.
2020-10-20 10:49:54 +03:00
Sergei Petrunia
3e807d255e MDEV-23938: innodb row_search_idx_cond_check handle ICP_ABORTED_BY_USER
- row_search_mvcc() should return DB_INTERRUPTED when it got killed.
- Add a syncpoint for the ICP check.
- Add test coverage for killed-during-ICP-check scenario

Backport of MDEV-22761 fixes for ICP from 10.4 commits:
* a6f956488c
* c03885cd9c

XtraDB was fixed in deb3b9a174

Reviewer: Daniel Black
2020-10-16 09:44:03 +11:00
Sergei Petrunia
c03885cd9c MDEV-22761: innodb row_search_idx_cond_check handle CHECK_ABORTED_BY_USER
Part :
- row_search_mvcc() should return DB_INTERRUPTED when it got
- Move the sync point from innodb internals to
  handler_rowid_filter_check() where other storage engines can use
  it too
- Add a similar syncpoint for the ICP check.
- Add a bigger test and test coverage for Rowid Filter with MyISAM
- Add test coverage for killed-during-ICP-check scenario
2020-10-14 15:14:46 +03:00
Sujatha
25ede13611 Merge branch '10.4' into 10.5 2020-09-29 16:59:36 +05:30
Monty
16ea692ed4 MDEV-23586 Mariabackup: GTID saved for replication in 10.4.14 is wrong
MDEV-21953 deadlock between BACKUP STAGE BLOCK_COMMIT and parallel
replication

Fixed by partly reverting MDEV-21953 to put back MDL_BACKUP_COMMIT locking
before log_and_order.

The original problem for MDEV-21953 was that while a thread was waiting in
for another threads to commit in 'log_and_order', it had the
MDL_BACKUP_COMMIT lock. The backup thread was waiting to get the
MDL_BACKUP_WAIT_COMMIT lock, which blocks all new MDL_BACKUP_COMMIT locks.
This causes a deadlock as the waited-for thread can never get past the
MDL_BACKUP_COMMIT lock in ha_commit_trans.

The main part of the bug fix is to release the MDL_BACKUP_COMMIT lock while
a thread is waiting for other 'previous' threads to commit. This ensures
that no transactional thread keeps MDL_BACKUP_COMMIT while waiting, which
ensures that there are no deadlocks anymore.
2020-09-25 13:07:03 +03:00
Oleksandr Byelkin
48b5777ebd Merge branch '10.4' into 10.5 2020-08-04 17:24:15 +02:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Oleksandr Byelkin
c32f71af7e Merge branch '10.2' into 10.3 2020-08-03 13:41:29 +02:00
Rucha Deodhar
97f7bfcebc MDEV-21017: Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())'
failed or late ER_PERIOD_FIELD_WRONG_ATTRIBUTES upon attempt to create
existing table

Analysis: Error state is not stored when field is checked in
Table_period_info::check_field()
Fix: Store error state by setting res to true.
2020-08-03 10:44:14 +05:30
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Monty
ce699df905 thd->m_transaction_psi was not properly cleared for new connections
This happend when using XA transactions. I also added some extra asserts
to ensure that m_transactions are properly cleared.

Other things:
- Removed set_time() from THD::init_for_queries() as dispatch_command()
  is already doing that.
- Removed duplicate init_for_queries() from prepare_new_connection_state().
  The init_for_queries() functions should only be called once per
  connection.
2020-07-23 10:54:33 +03:00
Ian Gilfillan
d2982331a6 Code comment spellfixes 2020-07-22 23:18:12 +02:00
Marko Mäkelä
4ec032b492 Merge 10.4 into 10.5 2020-07-21 17:33:16 +03:00
Monty
fc48c8ff4c MDEV-21953 deadlock between BACKUP STAGE BLOCK_COMMIT and parallel repl.
The issue was:
T1, a parallel slave worker thread, is waiting for another worker thread to
commit. While waiting, it has the MDL_BACKUP_COMMIT lock.
T2, working for mariabackup, is doing BACKUP STAGE BLOCK_COMMIT and blocks
all commits.
This causes a deadlock as the thread T1 is waiting for can't commit.

Fixed by moving locking of MDL_BACKUP_COMMIT from ha_commit_trans() to
commit_one_phase_2()

Other things:
- Added a new argument to ha_comit_one_phase() to signal if the
  transaction was a write transaction.
- Ensured that ha_maria::implicit_commit() is always called under
  MDL_BACKUP_COMMIT. This code is not needed in 10.5
- Ensure that MDL_Request values 'type' and 'ticket' are always
  initialized. This makes it easier to check the state of the MDL_Request.
- Moved thd->store_globals() earlier in handle_rpl_parallel_thread() as
  thd->init_for_queries() could use a MDL that could crash if store_globals
  where not called.
- Don't call ha_enable_transactions() in THD::init_for_queries() as this
  is both slow (uses MDL locks) and not needed.
2020-07-21 12:42:42 +03:00
Sergei Golubchik
478591608e optimize ha_delete_table_force
first try discovering engines, then the rest.
otherwise every DROP TABLE non_existent; will do
lots of i/o trying to remove .MYI/.MYD/.MAI/.MAD/.CSV/etc files

this matches the old behavior where DROP TABLE always tried to discover
the table before dropping.
2020-07-04 01:44:47 +02:00
Sergei Golubchik
6c52931680 replace HTON_AUTOMATIC_DELETE_TABLE with return -1 from drop_table() 2020-07-04 01:44:47 +02:00
Sergei Golubchik
7c2ba9e9d7 MDEV-11412 Ensure that table is truly dropped when using DROP TABLE
don't do table discovery on DROP. DROP falls back to "force"
approach when a table isn't found and will try to drop in all
engines anyway. That is, trying to discover in all engines before
the drop is redundant and may be expensive.
2020-07-04 01:44:47 +02:00
Sergei Golubchik
529b6dffe9 small cleanup 2020-07-04 01:44:46 +02:00
Sergei Golubchik
35f566db8d cleanup: make dd_frm_type to work as documented
remove redundant argument, return all possible enum values
2020-07-04 01:44:46 +02:00