Commit graph

6599 commits

Author SHA1 Message Date
Marko Mäkelä
0b9cee2cbf Merge 10.2 into 10.3 2019-10-18 09:05:27 +03:00
Marko Mäkelä
fa929f7cdf Simplify row_undo_ins_remove_sec_low()
Reduce the scope of some variables, remove a goto and a redundant
assertion.

For B-tree secondary indexes, this function can remove a delete-marked
purgeable record, in case a row rollback of the INSERT was initiated
due to an error in an earlier secondary index.
2019-10-17 17:12:23 +03:00
Marko Mäkelä
b027830232 MDEV-20850 Merge new release of InnoDB 5.7.28 to 10.2 2019-10-17 17:08:58 +03:00
Marko Mäkelä
fa32d28f2f MDEV-20852 BtrBulk is unnecessarily holding dict_index_t::lock
The BtrBulk class, which was introduced in MySQL 5.7, is by design
the exclusive writer to an index. It is therefore unnecessary to
acquire the dict_index_t::lock in that code.

Holding the dict_index_t::lock would unnecessarily block other threads
(SQL connections and the InnoDB purge threads) from buffering concurrent
modifications to being-created secondary indexes.

This fix is motivated by a change in MySQL 5.7.28:
Bug #29008298 MYSQLD CRASHES ITSELF WHEN CREATING INDEX
mysql/mysql-server@f9fb96c20f

PageBulk::init(), PageBulk::latch(): Never acquire m_index->lock.

PageBulk::storeExt(): Remove some pointer indirection, and improve
a debug assertion that seems to prove that some code is redundant.

BtrBulk::pageCommit(): Assert that m_index->lock is not being held.

btr_blob_log_check_t: Do not acquire m_index->lock if
m_op == BTR_STORE_INSERT_BULK. Add UNIV_UNLIKELY hints around
that condition.

btr_store_big_rec_extern_fields(): Allow index->lock not to be held
while op == BTR_STORE_INSERT_BULK. Add UNIV_UNLIKELY hints around
that condition.
2019-10-17 14:04:07 +03:00
Aleksey Midenkov
7ae0be25a6 MDEV-20812 Unexpected ER_ROW_IS_REFERENCED_2 upon DELETE from versioned table with FK
MDEV-16210 original case was wrongly allowed versioned DELETE from
referenced table where reference is by non-primary key. InnoDB UPDATE
has optimization for new rows not changing its clustered index
position. In this case InnoDB doesn't update all secondary indexes and
misses the one holding the referenced key. The fix was to disable this
optimization for versioned DELETE. In case of versioned DELETE we
forcely update all secondary indexes and therefore check them for
constraints.

But the above fix raised another problem with versioned DELETE on
foreign table side. In case when there was no corresponding record in
referenced table (illegal foreign reference can be done with "set
foreign_key_checks=off") there was spurious constraint check (because
versioned DELETE is actually UPDATE) and hence the operation failed
with constraint error.

MDEV-16210 tried to fix the above problem by checking foreign table
instead of referenced table and that at least was illegal.

Constraint check is done by row_ins_check_foreign_constraint() no
matter what kind of table is checked, referenced or foreign
(controlled by check_ref argument).

Referenced table is checked by row_upd_check_references_constraints().
Foreign table is checked by row_ins_check_foreign_constraints().

Current fix rolls back the wrong fix for the above problem and
disables referenced table check for DELETE on foreign side by
introducing `check_foreign` argument which when set to *false* skips
row_ins_check_foreign_constraints() call.
2019-10-15 13:29:18 +03:00
Marko Mäkelä
f989c0ce66 MDEV-20813: Do not rotate keys for unallocated pages
fil_crypt_rotate_page(): Skip the key rotation for pages that carry 0
in FIL_PAGE_TYPE. This avoids not only unnecessary writes, but also
failures of the recently added debug assertion in
buf_flush_init_for_writing() that the FIL_PAGE_TYPE should be nonzero.

Note: the debug assertion can fail if the file was originally created
before MySQL 5.5. In old InnoDB versions, FIL_PAGE_TYPE was only
initialized for B-tree pages, to FIL_PAGE_INDEX. For any other pages,
the field could be garbage, including FIL_PAGE_INDEX. In MariaDB 10.2
and later, buf_flush_init_for_writing() would initialize the
FIL_PAGE_TYPE on such old pages, but only after passing the debug
assertion that insists that pages have a nonzero FIL_PAGE_TYPE.
Thus, the debug assertion at the start of buf_flush_init_for_writing()
can fail when upgrading from very old debug files. This assertion is
only present in debug builds, not release builds.
2019-10-14 17:26:21 +03:00
Marko Mäkelä
fa6c606257 MDEV-20814 Assertion index->is_instant() failed on trivial upgrade from 10.1
rec_init_offsets(): Relax the assertion that was added in
commit 01f45becd1
to catch ROW_FORMAT=REDUNDANT records that have fewer fields
than expected.

This assertion would fail when accessing the records of the
built-in InnoDB table SYS_INDEXES. The column MERGE_THRESHOLD
had been effectively instantly added in MariaDB Server 10.2
(and MySQL 5.7), but is_instant() does not hold for that index.
Relax the assertion, so that it will not fail in this case.
2019-10-12 21:58:34 +03:00
Marko Mäkelä
bb450b1fed Merge 10.2 into 10.3 2019-10-12 15:38:58 +03:00
Marko Mäkelä
361e8284f3 MDEV-20813 Assertion failure in buf_flush_init_for_writing() for innodb_immediate_scrub_data_uncompressed=ON
The assertion that was added in
commit c0c003beb4
to augment the fix of MDEV-20805 turns out to be invalid when
innodb_immediate_scrub_data_uncompressed is enabled.
In this mode, fsp_init_file_page() will be invoked on data pages
that have been freed, causing writes of almost-all-zero pages.

btr_page_free(): Adjust the comment.

buf_flush_init_for_writing(): Disable the assertion with a note
that it should be re-enabled in MDEV-15528.
2019-10-12 15:28:55 +03:00
Marko Mäkelä
8e3d85e112 Merge 10.2 into 10.3 2019-10-12 06:34:09 +03:00
Marko Mäkelä
38736928e7 Fix -std=c++98 -Wzero-length-array
This is another follow-up fix to
commit b393e2cb0c
which turned out to be still broken.

Replace the C++11 keyword 'constexpr' with #define.

debug_sync_t::str: Remove the zero-length array.
Replace sync->str with reinterpret_cast<char*>(&sync[1]).
2019-10-11 21:26:16 +03:00
Marko Mäkelä
1e1b53ccfd After-merge fix: Correct an assertion
During IMPORT TABLESPACE, we do invoke
buf_flush_init_for_writing() with block==NULL and newest_lsn!=0.
2019-10-11 21:24:48 +03:00
Marko Mäkelä
966d97b5f9 Merge 10.1 into 10.2 2019-10-11 18:38:18 +03:00
Marko Mäkelä
1fd1ef25c2 Fix CMAKE_BUILD_TYPE=Debug
Remove unused variables and type mismatch that was introduced
in commit b393e2cb0c

Also, fix a typo in the documentation of the parameter, and
update the test.
2019-10-11 18:36:08 +03:00
Marko Mäkelä
c0c003beb4 MDEV-20805 follow-up: Catch writes of bogus pages
buf_flush_init_for_writing(): Assert that FIL_PAGE_TYPE is set
except when creating a new data file with a dummy first page.

buf_dblwr_create(): Ensure that FIL_PAGE_TYPE on all pages
will be initialized. Reset buf_dblwr_being_created at the end.
2019-10-11 15:32:04 +03:00
Marko Mäkelä
cbfd6882f4 Merge 5.5 into 10.1 2019-10-11 15:19:55 +03:00
Marko Mäkelä
ea61b79694 MDEV-20805 ibuf_add_free_page() is not initializing FIL_PAGE_TYPE first
In the function recv_parse_or_apply_log_rec_body() there are debug checks
for validating the state of the page when redo log records are being
applied. Most notably, FIL_PAGE_TYPE should be set before anything else
is being written to the page.

ibuf_add_free_page(): Set FIL_PAGE_TYPE before performing any other changes.
2019-10-11 14:12:36 +03:00
Nikita Malyavin
350e46a8b5 MDEV-18546 ASAN heap-use-after-free in innobase_get_computed_value / row_purge
the bug was already fixed in MDEV-17005, so now only test is added
2019-10-11 17:02:39 +10:00
Nikita Malyavin
b393e2cb0c add innodb_debug_sync var to support DEBUG_SYNC from purge threads 2019-10-11 17:02:39 +10:00
Marko Mäkelä
4cdb72f237 MDEV-19783: Relax an assertion
btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
if the metadata record exists, index->is_instant() must hold.
The assertion of commit 01f45becd1
could fail during innobase_instant_try().
2019-10-10 21:22:38 +03:00
Marko Mäkelä
01f45becd1 MDEV-19783: Add more assertions
btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
the metadata record exists if and only if index->is_instant().

page_validate(): Correct the wording of a message.

rec_init_offsets(): Assert that whenever a record is in "instant ALTER"
format, index->is_instant() must hold.
2019-10-10 20:40:26 +03:00
Marko Mäkelä
7f84e3ad75 Merge 10.2 into 10.3 2019-10-10 20:38:44 +03:00
Marko Mäkelä
6d7a826953 MDEV-20788: Bogus assertion failure for PAGE_FREE list
In MDEV-11369 (instant ADD COLUMN) in MariaDB Server 10.3,
we introduced the hidden metadata record that must be the
first record in the clustered index if and only if
index->is_instant() holds.

To catch MDEV-19783, in
commit ed0793e096 and
commit 99dc40d6ac
we added some assertions to find cases where
the metadata record is missing while it should not be, or a
record exists when it should not. Those assertions were invalid
when traversing the PAGE_FREE list. That list can contain anything;
we must only be able to determine the successor and the size of
each garbage record in it.

page_validate(), page_simple_validate_old(), page_simple_validate_new():
Do not invoke page_rec_get_next_const() for traversing the PAGE_FREE
list, but instead use a lower-level accessor that does not attempt to
validate the REC_INFO_MIN_REC_FLAG.

page_copy_rec_list_end_no_locks(),
page_copy_rec_list_start(), page_delete_rec_list_start():
Add assertions.

btr_page_get_split_rec_to_left(): Remove a redundant return value,
and make the output parameter the return value.

btr_page_get_split_rec_to_right(), btr_page_split_and_insert(): Clean up.
2019-10-10 20:29:30 +03:00
Aleksey Midenkov
75ba5c815d MDEV-16210 FK constraints on versioned tables use historical rows, which may cause constraint violation
Constraint check is done on secondary index update.
F.ex. DELETE does row_upd_sec_index_entry() and checks constraints in
row_upd_check_references_constraints(). UPDATE is optimized for the
case when order is not changed (node->cmpl_info & UPD_NODE_NO_ORD_CHANGE)
and doesn't do row_upd_sec_index_entry(), so it doesn't check constraints.

Since for versioned DELETE we do UPDATE actually, but expect behaviour
of DELETE in terms of constraints, we should deny this optimization to
get constraints checked.

Fix wrong referenced table check when versioned DELETE inserts history in parent
table. Set check_ref to false in this case.

Removed unused dup_chk_only argument for row_ins_sec_index_entry() and
added check_ref argument.

MDEV-18057 fix was superseded by this fix and reverted.

foreign.test:
All key_type combinations: pk, unique, sec(ondary).
2019-10-10 00:20:34 +03:00
Marko Mäkelä
6fde0073bf Rename log_make_checkpoint_at() to log_make_checkpoint()
The function was always called with lsn=LSN_MAX.
Remove that redundant parameter.

Spotted by Thirunarayanan Balathandayuthapani.
2019-10-09 18:47:14 +03:00
Marko Mäkelä
892378fb9d Merge 10.2 into 10.3 2019-10-09 13:25:11 +03:00
Thirunarayanan Balathandayuthapani
c65cb244b3 MDEV-19335 Remove buf_page_t::encrypted
The field buf_page_t::encrypted was added in MDEV-8588.
It was made mostly redundant in MDEV-12699. Remove the field.
2019-10-09 13:13:12 +03:00
Marko Mäkelä
24232ec12c Merge 10.1 into 10.2 2019-10-09 08:30:23 +03:00
Eugene Kosov
ed0793e096 MDEV-19783: Add more REC_INFO_MIN_REC_FLAG checks
btr_cur_pessimistic_delete(): code changed in a way that allows
to put more REC_INFO_MIN_REC_FLAG assertions inside btr_set_min_rec_mark().
Without that change tests innodb.innodb-table-online,
innodb.temp_table_savepoint and innodb_zip.prefix_index_liftedlimit fail.

Removed basically duplicated page_zip_validate() calls
which fails because of temporary(!) invariant violation.
That fixed innodb_zip.wl5522_debug_zip and
innodb_zip.prefix_index_liftedlimit
2019-10-09 08:29:26 +03:00
Eugene Kosov
99dc40d6ac MDEV-19783 Random crashes and corrupt data in INSTANT-added columns
The bug affects MariaDB Server 10.3 or later, but it makes sense
to improve CHECK TABLE in earlier versions already.

page_validate(): Check REC_INFO_MIN_REC_FLAG in the records.
This allows CHECK TABLE to catch more bugs.
2019-10-09 08:29:26 +03:00
Marko Mäkelä
d480d28f4f Add page_has_prev(), page_has_next(), page_has_siblings()
Until now, InnoDB inefficiently compared the aligned fields
FIL_PAGE_PREV, FIL_PAGE_NEXT to the byte-order-agnostic value FIL_NULL.

This is a backport of 32170f8c6d
from MariaDB Server 10.3.
2019-10-09 08:29:26 +03:00
Marko Mäkelä
d95f96ad1b Merge 5.5 into 10.1 2019-10-08 12:43:37 +03:00
Marko Mäkelä
db9a4d928d Remove orphan declaration buf_flush_wait_batch_end_wait_only()
The function was declared but not defined in
commit 9d6d1902e0
2019-10-07 17:18:10 +03:00
Sergey Vojtovich
5b2fa078e8 Cleanup mman.h includes
As it is included from my_global.h already.
2019-10-02 20:21:30 +04:00
Daniel Black
716c748f97 MDEV-20684: innodb/query cache use madvise CORE/NOCORE on FreeBSD
This applies to large allocations.

This maps to the way Linux does it in MDEV-10814 except FreeBSD uses
different constants.

Adjust error string to match to implementation.

Tested on FreeBSD-12.0
2019-10-02 20:00:05 +04:00
Alexander Barkov
7e44c455f4 Merge remote-tracking branch 'origin/10.2' into 10.3 2019-10-01 09:37:40 +04:00
Marko Mäkelä
46b785262b Fix -Wunused for CMAKE_BUILD_TYPE=RelWithDebInfo
For release builds, do not declare unused variables.

unpack_row(): Omit a debug-only variable from WSREP diagnostic message.

create_wsrep_THD(): Fix -Wmaybe-uninitialized for the PSI_thread_key.
2019-09-30 12:49:53 +03:00
Marko Mäkelä
2911a9a693 Merge 10.2 into 10.3 2019-09-27 15:56:15 +03:00
Thirunarayanan Balathandayuthapani
c76873f23d MDEV-20688 Recovery crashes after unnecessarily reading a corrupted page
The test encryption.innodb-redo-badkey was accidentally disabled
until commit 23657a2101 enabled
it recently. Once it was enabled, it started failing randomly.

recv_recover_corrupt_page(): Do not assume that any redo log exists
for the page. A page may be unnecessarily read by read-ahead.
When noting the corruption, reset recv_addr->state to RECV_PROCESSED,
so that even if the same page is re-read again, we will only
decrement recv_sys->n_addrs once.
2019-09-27 17:46:10 +05:30
Marko Mäkelä
d874cdeccc dict_load_table(): Remove constant parameter cached=true
Spotted by Thirunarayanan Balathandayuthapani.
2019-09-27 14:29:22 +03:00
Marko Mäkelä
718fcee0a3 Reduce rw_lock_debug_mutex contention
rw_lock_own(), rw_lock_own_flagged(): Traverse the rw_lock_t::debug_list
only after quickly checking if the thread is holding X-latch or SX-latch.
2019-09-27 14:22:59 +03:00
Marko Mäkelä
4ec0c346b8 Remove a useless large test, and add a debug assertion
The test innodb_fts.fulltext_table_evict was only creating 1000 tables
with fulltext indexes, only to check that no tables with fulltext
indexes are being evicted.

The reason why tables containing fulltext indexes cannot be evicted is
that fts_optimize_init() invokes dict_table_prevent_eviction().
2019-09-27 14:05:39 +03:00
Marko Mäkelä
3e4931cdf3 MDEV-20675 Crash in SHOW ENGINE INNODB STATUS with innodb_force_recovery=5
lock_print_info::operator(): Do not dereference purge_sys.query in case
it is NULL. We would not initialize purge_sys if innodb_force_recovery
is set to 5 or 6.

The test case will be added by merge from 10.2.
2019-09-26 13:18:22 +03:00
Marko Mäkelä
7ae290c757 Merge 10.2 into 10.3 2019-09-24 13:39:31 +03:00
Marko Mäkelä
ef701bfd07 Remove the unused function btr_page_get()
btr_block_get(): Remove #ifdef around the definition
2019-09-24 13:29:23 +03:00
Marko Mäkelä
c016ea660e Merge 10.2 into 10.3 2019-09-23 10:25:34 +03:00
Marko Mäkelä
60cb5559a9 MDEV-17614 post-fix: Remove dead dup_chk_only=true code.
The parameter dup_chk_only was always passed as a constant false.
Remove the parameter and the dead code related to it.
2019-09-23 08:29:39 +03:00
Thirunarayanan Balathandayuthapani
f94d9ab9f8 MDEV-20483 Follow-up fix
At commit, trx->lock.table_locks (which is a cache of trx_locks) can
consist of NULL pointers. Add a debug assertion for that, and clear
the vector.
2019-09-18 20:20:04 +05:30
Marko Mäkelä
bb4214272a Merge 10.1 into 10.2 2019-09-18 16:24:48 +03:00
Marko Mäkelä
a624b99f91 Remove an unused declaration 2019-09-18 16:10:03 +03:00