Commit graph

23284 commits

Author SHA1 Message Date
Aleksey Midenkov
f2a944516e Merge 10.4 into 10.5 2020-05-15 17:13:35 +03:00
Jan Lindström
523d67a272 MDEV-22494 : Galera assertion lock_sys.mutex.is_owned() at lock_trx_handle_wait_low
Problem was that trx->lock.was_chosen_as_wsrep_victim variable was
not set back to false after it was set true.

wsrep_thd_bf_abort
	Add assertions for correct mutex status and take necessary
	mutexes before calling thd->awake_no_mutex().

innobase_rollback_trx()
	Reset trx->lock.was_chosen_as_wsrep_victim

wsrep_abort_slave_trx()
	Removed unused function.

wsrep_innobase_kill_one_trx()
	Added function comment, removed unnecessary parameters
	and added debug assertions to enforce correct usage. Added
	more debug output to help out on error analysis.

wsrep_abort_transaction()
	Added debug assertions and removed unused variables.

trx0trx.h
	Removed assert_trx_is_free macro and replaced it with
	assert_freed() member function.

trx_create()
	Use above assert_free() and initialize wsrep variables.

trx_free()
	Use assert_free()

trx_t::commit_in_memory()
	Reset lock.was_chosen_as_wsrep_victim

trx_rollback_for_mysql()
	Reset trx->lock.was_chosen_as_wsrep_victim

Add test case galera_bf_kill
2020-05-15 09:04:02 +03:00
Marko Mäkelä
b30a013142 Merge 10.4 into 10.5 2020-05-13 14:25:06 +03:00
Marko Mäkelä
38f6c47f8a Merge 10.3 into 10.4 2020-05-13 12:52:57 +03:00
Marko Mäkelä
15fa70b840 Merge 10.2 into 10.3 2020-05-13 11:45:05 +03:00
Marko Mäkelä
6bc4444d7c Merge 10.1 into 10.2 2020-05-13 11:12:31 +03:00
Vlad Lesin
218d20ffe3 MDEV-22398: mariabackup.innodb_xa_rollback fails on repeat
Flush LSN to system tablespace on innodb shutdown if XA is rolled back by
mariabackup.
2020-05-12 18:44:36 +03:00
Marko Mäkelä
0e6a5786d4 Cleanup: Remove InnoDB wrappers of thd_charset(), thd_query_safe()
innobase_get_charset(), innobase_get_stmt_safe(): Remove.
It is more efficient and readable to invoke thd_charset()
and thd_query_safe() directly, without a non-inlined wrapper function.
2020-05-12 10:15:44 +03:00
Marko Mäkelä
ba3d58ad4c MDEV-22523 index->rtr_ssn.mutex is wasting memory
As part of the SPATIAL INDEX implementation in InnoDB,
dict_index_t was expanded by a rtr_ssn_t field. There are only
3 operations for this field, all protected by rtr_ssn_t::mutex:

* btr_cur_search_to_nth_level() stores the least significant 32 bits
of the 64-bit value that is stored in the index root page.
(This would better be done when the table is opened for the
very first time.)
* rtr_get_new_ssn_id() increments the value by 1.
* rtr_get_current_ssn_id() reads the current value.

All these operations can be implemented equally safely by using
atomic memory access operations.
2020-05-11 14:23:37 +03:00
Sergei Golubchik
13038e4705 Merge branch '10.4' into 10.5 2020-05-09 20:43:36 +02:00
Petr Vaněk
4ae778bbec innodb: add space between thread name and "to exit" text 2020-05-09 15:33:57 +02:00
Marko Mäkelä
72c7b4eb4c MDEV-16678 fixup: Remove orphaned DBUG_EXECUTE_IF
In commit ea37b14409
we removed the MDEV-16222 test case but forgot to remove
the debug instrumentation.
2020-05-08 18:31:34 +03:00
Marko Mäkelä
1887b5ae87 MDEV-22501 Various issues when using --innodb-data-file-size-debug=-1
Let us limit the maximum value of the debug parameter
innodb_data_file_size to 256 MiB. It is only being used
in the test innodb.log_data_file_size, and the size
of the system tablespace should never exceed some 70 MiB
in ./mtr. Thus, 256 MiB should be a reasonable limit.

The fact that negative values that are passed to unsigned parameters
wrap around to the maximum value appears to be a regression due to
commit 18ef02b04d
and has been filed as bug MDEV-22219.
2020-05-08 13:27:57 +03:00
Marko Mäkelä
0dee57c6f5 MDEV-19344 innodb.innodb-change-buffer-recovery fails
The test was incompatible with ./mtr --repeat=2 until
commit 2d6719d7ee
fixed that.

It turns out that the failing assertion that we disabled in
commit 3db94d2403
is bogus and can fail when the change buffer is emptied
during the last batch of crash recovery. The reason for this
is the condition around the page_create_empty() call in
page_cur_delete_rec(). The condition was removed in MariaDB
Server 10.5 as part of MDEV-12353, in
commit 7ae21b18a6 and
commit f8a9f90667.

The bug that the assertion aimed to catch is MDEV-22497, which
was fixed in commit 26aab96ecf.
2020-05-07 21:01:22 +03:00
Marko Mäkelä
26aab96ecf MDEV-22497 [ERROR] InnoDB: Unable to purge a record
The InnoDB insert buffer was upgraded in MySQL 5.5 into a change
buffer that also covers delete-mark and delete (purge) operations.

There is an important constraint for delete operations: a B-tree
leaf page must not become empty unless the entire tree becomes empty,
consisting of an empty root page. Because change buffer merges only
occur on a single leaf page at a time, delete operations must not be
buffered if it is possible that the last record of the page could be
deleted. (In that case, we would refuse to use the change buffer, and
if we really delete the last record, we would shrink the index tree.)

The function ibuf_get_volume_buffered_hash() is part of our insurance
that the page would not become empty. It is supposed to map each
buffered INSERT or DELETE_MARK record payload into a hash value.
We will only count each such record as a distinct key if there is no
hash collision. DELETE operations will always decrement the predicted
number fo records in the page.

Due to a bug in the function, we would actually compute the hash value
not only on the record payload, but also on some following bytes,
in case the record contains NULL values. In MySQL Bug #61104, we had
some examples of this dating back to 2012. But back then, we failed to
reproduce the bug, and in commit d84c95579b
we simply demoted the hard assertion to a message printout and a debug
assertion failure.

ibuf_get_volume_buffered_hash(): Correctly compute the hash value
of the payload bytes only. Note: we will consider
('foo','bar'),(NULL,'foobar'),('foob','ar') to be equal, but this
is not a problem, because in case of a hash collision, we could
also consider ('boo','far') to be equal, and underestimate the number
of records in the page, leading to refusing to buffer a DELETE.
2020-05-07 20:44:33 +03:00
Marko Mäkelä
4a5be2e94e MDEV-22495 Assertion ...status != buf_page_t::FREED in ibuf_read_merge_pages()
ibuf_read_merge_pages(): Request a possibly freed page.
The change buffer is discarded lazily for freed pages either
by this function or when buf_page_create() reuses a page.

buf_page_get_low(): Relax a debug assertion.
Do not attempt change buffer merge on freed pages.

ibuf_merge_or_delete_for_page(): Assert that the page state is NORMAL.
INIT_ON_FLUSH is not possible, because in that case buf_page_create()
should have removed any buffered changes for the page.

buf_page_get_gen(): Apply buffered changes also in the case when
we can avoid reading the page based on buffered redo log records.
This addresses a hard-to-reproduce scenario that was broken in
commit 6697135c6d.
2020-05-07 17:57:03 +03:00
Marko Mäkelä
18a62eb76d MDEV-21133 follow-up: Use fil_page_get_type()
Let us use the common accessor function fil_page_get_type()
instead of accessing the page header field FIL_PAGE_TYPE directly.
2020-05-07 17:15:34 +03:00
Sergei Petrunia
8d85715d50 MDEV-21794: Optimizer flag rowid_filter leads to long query
Rowid Filter check is just like Index Condition Pushdown check: before
we check the filter, we must check if we have walked out of the range
we are scanning. (If we did, we should return, and not continue the scan).

Consequences of this:
- Rowid filtering doesn't work for keys that have partially-covered
  blob columns (just like Index Condition Pushdown)
- The rowid filter function has three return values: CHECK_POS (passed)
  CHECK_NEG (filtered out), CHECK_OUT_OF_RANGE.

All of the above is implemented in this patch
2020-05-07 12:27:17 +02:00
Marko Mäkelä
ba573c4736 MDEV-21133 follow-up: More my_assume_aligned hints
fsp0pagecompress.h: Remove.
Invoke fil_page_get_type() and FSP_FLAGS_GET_PAGE_COMPRESSION_LEVEL
directly.

log_block_get_flush_bit(), log_block_set_flush_bit():
Access the byte directly.

dict_sys_read_row_id(): Remove (unused function).
2020-05-07 12:25:00 +03:00
Marko Mäkelä
7bcaa541aa Merge 10.4 into 10.5 2020-05-05 21:16:22 +03:00
Eugene Kosov
36b8ac2c0d remove double std::map lookup 2020-05-05 21:15:21 +03:00
Sergei Golubchik
67aaf51cf9 cleanup: ha_external_unlock() helper
as mentioned in f9f33b85be and generally to make it
easier to talk about
2020-05-05 19:41:12 +02:00
Marko Mäkelä
2c3c851d2c Merge 10.3 into 10.4 2020-05-05 20:33:10 +03:00
Marko Mäkelä
474290540a MDEV-22465: DROP indexed COLUMN is wrongly claimed to be ALGORITHM=INSTANT
ha_innobase::check_if_supported_inplace_alter(): Do not allow
ALGORITHM=INSTANT for operations that avoid a table rebuild
but involve dropping (or creating) secondary indexes.
2020-05-05 20:32:14 +03:00
Eugene Kosov
89ff4176c1 MDEV-22437 make THR_THD* variable thread_local
Now all access goes through _current_thd() and set_current_thd()
functions.

Some functions like THD::store_globals() can not fail now.
2020-05-05 18:13:31 +03:00
Kentoku SHIBA
90aad47dd9 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
Add test cases.
2020-05-05 22:45:17 +09:00
Kentoku SHIBA
69925c0d2e MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
Add test cases.
2020-05-05 22:43:45 +09:00
Marko Mäkelä
b9f177f66a MDEV-11254 cleanup: Remove buf_page_t::write_size
commit 6495806e59 removed all reads
of buf_page_t::write_size. Let us remove the field altogether.
2020-05-05 08:54:33 +03:00
Marko Mäkelä
64488a6f2d Cleanup: Remove global functions or redundant parameters
fil_flush_file_spaces(): Remove the constant parameter.

buf_flush_start(), buf_flush_end(): Use static linkage.

fil_page_get_prev(), fil_page_get_next(), fil_addr_is_null(): Remove.
2020-05-05 07:06:37 +03:00
Marko Mäkelä
97261cbb7a Merge 10.4 into 10.5 2020-05-04 22:07:18 +03:00
Marko Mäkelä
1ed98782e7 MDEV-22452: Fix cmake -DWITH_WSREP=OFF
commit 5e7e7153b4 accidentally
broke the build without WSREP, by misplacing an #endif.
2020-05-04 22:06:31 +03:00
Marko Mäkelä
8648b9bed8 Merge 10.2 into 10.3 2020-05-04 22:06:00 +03:00
Marko Mäkelä
1cccd3c7cc MDEV-7962: Fix cmake WITH_WSREP=OFF
commit d467bb7e5e accidentally
broke the build without WSREP, by misplacing an #endif.
2020-05-04 22:01:26 +03:00
Marko Mäkelä
3f65ce5781 Merge 10.1 into 10.2 2020-05-04 19:00:21 +03:00
Marko Mäkelä
d467bb7e5e MDEV-7962 post-push fixes
This is a partial backport of
commit 5e7e7153b4 from 10.4.

assert_trx_is_free(): Assert !is_wsrep().

trx_init(): Do not initialize trx->wsrep, because it must have been
initialized already.

trx_commit_in_memory(): Invoke wsrep_commit_ordered(). This call
was being skipped, because the transaction object had already been
freed to the pool.

trx_rollback_for_mysql(), innobase_commit_low(),
innobase_rollback_trx(): Always reset trx->wsrep.
2020-05-04 18:57:55 +03:00
Marko Mäkelä
4337a3b5f9 Merge 10.4 into 10.5 2020-05-04 18:43:00 +03:00
Marko Mäkelä
5e7e7153b4 MDEV-22452: Missing call to wsrep_commit_ordered in trx_t::commit()
This is a follow-up fix to the changes that were made in MDEV-7962.

assert_trx_is_free(): Assert !is_wsrep().

trx_init(): Do not initialize trx->wsrep, because it must have been
initialized already.

trx_t::commit_in_memory(): Invoke wsrep_commit_ordered(). This call
was being skipped, because the transaction object had already been
freed to the pool.

trx_rollback_for_mysql(), innobase_commit_low(),
innobase_close_connection(): Always reset trx->wsrep.
2020-05-04 18:31:30 +03:00
Oleksandr Byelkin
7fb73ed143 Merge branch '10.2' into 10.3 2020-05-04 16:47:11 +02:00
Thirunarayanan Balathandayuthapani
a1eb151327 MDEV-22410 Change the error when table schema mismatch happens
- InnoDB gives "table is marked as crashed and should be repaired" when
table schema mismatch.
2020-05-04 19:11:39 +05:30
Thirunarayanan Balathandayuthapani
42bba9782b MDEV-22446 InnoDB aborts while adding instant column for discarded tablespace
- Instant alter should change the metadata alone when table is
discarded. It shouldn't try to add metadata record in clustered index.
Also make the clustered index to non-instant format.
2020-05-04 16:23:34 +05:30
Thirunarayanan Balathandayuthapani
2748c4993c MDEV-19092 Server crash when renaming the column when
FOREIGN_KEY_CHECKS is disabled

- Referenced index can be null While renaming the referenced column name.
In that case, rename the referenced column name in dict_foreign_t and
find the equivalent referenced index.
2020-05-04 14:33:45 +05:30
Oleksandr Byelkin
ca091e6372 Merge branch '10.1' into 10.2 2020-05-02 08:44:17 +02:00
Oleksandr Byelkin
4fc8961d49 Merge remote-tracking branch 'connect/10.1' into 10.1 2020-04-30 18:57:45 +02:00
Oleksandr Byelkin
da1fbcb665 Merge branch 'merge-tokudb-5.6' into 10.1 2020-04-30 18:56:55 +02:00
Oleksandr Byelkin
15db581ecb 5.6.47-87.0 2020-04-30 18:29:36 +02:00
Oleksandr Byelkin
23c6fb3e62 Merge branch '5.5' into 10.1 2020-04-30 17:36:41 +02:00
Eugene Kosov
7f9dc0d84a split log_t::buf into two buffers
Maybe this patch will help catch problems like buffer overflow.

log_t::first_in_use: removed

log_t::buf: this is where mtr_t are supposed to append data
log_t::flush_buf: this is from server writes to a file

Those two buffers are std::swap()ped when some thread is gonna write
to a file
2020-04-30 11:56:16 +03:00
Marko Mäkelä
f544a712c8 Cleanup: Reduce que_thr_t, que_fork_t, trx_lock_t size
que_thr_t::magic_n: Remove. Access to freed data is best caught
by AddressSanitizer.

que_thr_t::start_running(): Replaces que_thr_move_to_run_state_for_mysql()
and que_thr_move_to_run_state(), which were identical non-inline functions.

que_thr_t::stop_no_error(): Replaces que_thr_stop_for_mysql_no_error().

que_fork_t::n_active_thrs, trx_lock_t::n_active_thrs: Make debug-only.

que_fork_t::set_active(bool active): Update n_active_thrs.
2020-04-30 10:43:43 +03:00
Marko Mäkelä
3e6722d88d Cleanup: More trx_t member functions
trx_t::rollback(): Renamed from trx_rollback_to_savepoint().

trx_t::rollback_low(): Renamed from trx_rollback_to_savepoint_low().

fts_sql_commit(): Defined as an alias of trx_commit_for_mysql().
fts_sql_rollback(): Defined as an alias of trx_t::rollback().

fts_rename_aux_tables_to_hex_format(): Fix the error handling
that likely never worked because we failed to roll back the
first transaction.
2020-04-29 16:00:17 +03:00
Marko Mäkelä
496d0372ef Merge 10.4 into 10.5 2020-04-29 15:40:51 +03:00