Commit graph

22379 commits

Author SHA1 Message Date
Marko Mäkelä
55c75b6bb3 Merge 10.3 into 10.4 2019-10-12 06:50:12 +03:00
Marko Mäkelä
8e3d85e112 Merge 10.2 into 10.3 2019-10-12 06:34:09 +03:00
Marko Mäkelä
38736928e7 Fix -std=c++98 -Wzero-length-array
This is another follow-up fix to
commit b393e2cb0c
which turned out to be still broken.

Replace the C++11 keyword 'constexpr' with #define.

debug_sync_t::str: Remove the zero-length array.
Replace sync->str with reinterpret_cast<char*>(&sync[1]).
2019-10-11 21:26:16 +03:00
Marko Mäkelä
1e1b53ccfd After-merge fix: Correct an assertion
During IMPORT TABLESPACE, we do invoke
buf_flush_init_for_writing() with block==NULL and newest_lsn!=0.
2019-10-11 21:24:48 +03:00
Marko Mäkelä
966d97b5f9 Merge 10.1 into 10.2 2019-10-11 18:38:18 +03:00
Marko Mäkelä
1fd1ef25c2 Fix CMAKE_BUILD_TYPE=Debug
Remove unused variables and type mismatch that was introduced
in commit b393e2cb0c

Also, fix a typo in the documentation of the parameter, and
update the test.
2019-10-11 18:36:08 +03:00
Marko Mäkelä
c0c003beb4 MDEV-20805 follow-up: Catch writes of bogus pages
buf_flush_init_for_writing(): Assert that FIL_PAGE_TYPE is set
except when creating a new data file with a dummy first page.

buf_dblwr_create(): Ensure that FIL_PAGE_TYPE on all pages
will be initialized. Reset buf_dblwr_being_created at the end.
2019-10-11 15:32:04 +03:00
Marko Mäkelä
cbfd6882f4 Merge 5.5 into 10.1 2019-10-11 15:19:55 +03:00
Marko Mäkelä
ea61b79694 MDEV-20805 ibuf_add_free_page() is not initializing FIL_PAGE_TYPE first
In the function recv_parse_or_apply_log_rec_body() there are debug checks
for validating the state of the page when redo log records are being
applied. Most notably, FIL_PAGE_TYPE should be set before anything else
is being written to the page.

ibuf_add_free_page(): Set FIL_PAGE_TYPE before performing any other changes.
2019-10-11 14:12:36 +03:00
Nikita Malyavin
350e46a8b5 MDEV-18546 ASAN heap-use-after-free in innobase_get_computed_value / row_purge
the bug was already fixed in MDEV-17005, so now only test is added
2019-10-11 17:02:39 +10:00
Nikita Malyavin
b393e2cb0c add innodb_debug_sync var to support DEBUG_SYNC from purge threads 2019-10-11 17:02:39 +10:00
Marko Mäkelä
09afd3da1a Merge 10.3 into 10.4 2019-10-10 21:30:40 +03:00
Marko Mäkelä
4cdb72f237 MDEV-19783: Relax an assertion
btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
if the metadata record exists, index->is_instant() must hold.
The assertion of commit 01f45becd1
could fail during innobase_instant_try().
2019-10-10 21:22:38 +03:00
Marko Mäkelä
01f45becd1 MDEV-19783: Add more assertions
btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
the metadata record exists if and only if index->is_instant().

page_validate(): Correct the wording of a message.

rec_init_offsets(): Assert that whenever a record is in "instant ALTER"
format, index->is_instant() must hold.
2019-10-10 20:40:26 +03:00
Marko Mäkelä
7f84e3ad75 Merge 10.2 into 10.3 2019-10-10 20:38:44 +03:00
Marko Mäkelä
6d7a826953 MDEV-20788: Bogus assertion failure for PAGE_FREE list
In MDEV-11369 (instant ADD COLUMN) in MariaDB Server 10.3,
we introduced the hidden metadata record that must be the
first record in the clustered index if and only if
index->is_instant() holds.

To catch MDEV-19783, in
commit ed0793e096 and
commit 99dc40d6ac
we added some assertions to find cases where
the metadata record is missing while it should not be, or a
record exists when it should not. Those assertions were invalid
when traversing the PAGE_FREE list. That list can contain anything;
we must only be able to determine the successor and the size of
each garbage record in it.

page_validate(), page_simple_validate_old(), page_simple_validate_new():
Do not invoke page_rec_get_next_const() for traversing the PAGE_FREE
list, but instead use a lower-level accessor that does not attempt to
validate the REC_INFO_MIN_REC_FLAG.

page_copy_rec_list_end_no_locks(),
page_copy_rec_list_start(), page_delete_rec_list_start():
Add assertions.

btr_page_get_split_rec_to_left(): Remove a redundant return value,
and make the output parameter the return value.

btr_page_get_split_rec_to_right(), btr_page_split_and_insert(): Clean up.
2019-10-10 20:29:30 +03:00
Marko Mäkelä
0f7732d1d1 MDEV-19335 adjustment for innodb_checksum_algorithm=full_crc32
When MDEV-12026 introduced innodb_checksum_algorithm=full_crc32 in
MariaDB 10.4, it accidentally added a dependency on buf_page_t::encrypted.
Now that the flag has been removed, we must adjust the page-read routine.

buf_page_io_complete(): When the full_crc32 page checksum matches but the
tablespace ID in the page does not match after decrypting, we should
declare it a decryption failure and suppress the page dump output and
any attempts to re-read the page.
2019-10-10 15:24:14 +03:00
Aleksey Midenkov
545c545206 Fix compilation 2 (GCC 9)
Fixed warning: -Woverloaded-virtual for GCC 9 (Clang treats it differently)

Caused by c9cba59749
2019-10-10 13:37:02 +03:00
Marko Mäkelä
c11e5cdd12 Merge 10.3 into 10.4 2019-10-10 11:19:25 +03:00
Aleksey Midenkov
3c78d1b640 Fix Mroonga compilation
Fixed warnings: -Woverloaded-virtual, -Winconsistent-missing-override

Caused by c9cba59749
2019-10-10 11:13:05 +03:00
Aleksey Midenkov
75ba5c815d MDEV-16210 FK constraints on versioned tables use historical rows, which may cause constraint violation
Constraint check is done on secondary index update.
F.ex. DELETE does row_upd_sec_index_entry() and checks constraints in
row_upd_check_references_constraints(). UPDATE is optimized for the
case when order is not changed (node->cmpl_info & UPD_NODE_NO_ORD_CHANGE)
and doesn't do row_upd_sec_index_entry(), so it doesn't check constraints.

Since for versioned DELETE we do UPDATE actually, but expect behaviour
of DELETE in terms of constraints, we should deny this optimization to
get constraints checked.

Fix wrong referenced table check when versioned DELETE inserts history in parent
table. Set check_ref to false in this case.

Removed unused dup_chk_only argument for row_ins_sec_index_entry() and
added check_ref argument.

MDEV-18057 fix was superseded by this fix and reverted.

foreign.test:
All key_type combinations: pk, unique, sec(ondary).
2019-10-10 00:20:34 +03:00
Marko Mäkelä
6fde0073bf Rename log_make_checkpoint_at() to log_make_checkpoint()
The function was always called with lsn=LSN_MAX.
Remove that redundant parameter.

Spotted by Thirunarayanan Balathandayuthapani.
2019-10-09 18:47:14 +03:00
Marko Mäkelä
892378fb9d Merge 10.2 into 10.3 2019-10-09 13:25:11 +03:00
Thirunarayanan Balathandayuthapani
c65cb244b3 MDEV-19335 Remove buf_page_t::encrypted
The field buf_page_t::encrypted was added in MDEV-8588.
It was made mostly redundant in MDEV-12699. Remove the field.
2019-10-09 13:13:12 +03:00
Oleksandr Byelkin
896b869685 MDEV-19238 Mariadb spider - crashes on where null
(fix and explanation came with MDEV-20753 (duplicate of this bug))
2019-10-09 08:55:00 +02:00
Oleksandr Byelkin
b7408be0c3 MDEV-20753: Sequence with limit 0 crashes server
Do not try to push down conditions to engine if query was resolved without tables (and so the engine).
2019-10-09 08:55:00 +02:00
Marko Mäkelä
24232ec12c Merge 10.1 into 10.2 2019-10-09 08:30:23 +03:00
Eugene Kosov
ed0793e096 MDEV-19783: Add more REC_INFO_MIN_REC_FLAG checks
btr_cur_pessimistic_delete(): code changed in a way that allows
to put more REC_INFO_MIN_REC_FLAG assertions inside btr_set_min_rec_mark().
Without that change tests innodb.innodb-table-online,
innodb.temp_table_savepoint and innodb_zip.prefix_index_liftedlimit fail.

Removed basically duplicated page_zip_validate() calls
which fails because of temporary(!) invariant violation.
That fixed innodb_zip.wl5522_debug_zip and
innodb_zip.prefix_index_liftedlimit
2019-10-09 08:29:26 +03:00
Eugene Kosov
99dc40d6ac MDEV-19783 Random crashes and corrupt data in INSTANT-added columns
The bug affects MariaDB Server 10.3 or later, but it makes sense
to improve CHECK TABLE in earlier versions already.

page_validate(): Check REC_INFO_MIN_REC_FLAG in the records.
This allows CHECK TABLE to catch more bugs.
2019-10-09 08:29:26 +03:00
Marko Mäkelä
d480d28f4f Add page_has_prev(), page_has_next(), page_has_siblings()
Until now, InnoDB inefficiently compared the aligned fields
FIL_PAGE_PREV, FIL_PAGE_NEXT to the byte-order-agnostic value FIL_NULL.

This is a backport of 32170f8c6d
from MariaDB Server 10.3.
2019-10-09 08:29:26 +03:00
Marko Mäkelä
6bc75a4253 MDEV-15528 preparation: Remove a constant parameter
fsp_free_seg_inode(): Remove the constant parameter log=true.
Only fsp_free_page() could also be called with log=false.

The dead code was introduced in
commit 304ae942f7
by me, and spotted by Thirunarayanan Balathandayuthapani.
2019-10-08 13:05:35 +03:00
Marko Mäkelä
d95f96ad1b Merge 5.5 into 10.1 2019-10-08 12:43:37 +03:00
Marko Mäkelä
db9a4d928d Remove orphan declaration buf_flush_wait_batch_end_wait_only()
The function was declared but not defined in
commit 9d6d1902e0
2019-10-07 17:18:10 +03:00
Eugene Kosov
5e65c67cfc fix clang warning 2019-10-03 17:45:36 +03:00
Sergey Vojtovich
5b2fa078e8 Cleanup mman.h includes
As it is included from my_global.h already.
2019-10-02 20:21:30 +04:00
Daniel Black
716c748f97 MDEV-20684: innodb/query cache use madvise CORE/NOCORE on FreeBSD
This applies to large allocations.

This maps to the way Linux does it in MDEV-10814 except FreeBSD uses
different constants.

Adjust error string to match to implementation.

Tested on FreeBSD-12.0
2019-10-02 20:00:05 +04:00
Alexander Barkov
dc588e3d3f Merge remote-tracking branch 'origin/10.3' into 10.4 2019-10-01 10:45:52 +04:00
Alexander Barkov
7e44c455f4 Merge remote-tracking branch 'origin/10.2' into 10.3 2019-10-01 09:37:40 +04:00
Robert Bindar
576a5f091e MDEV-20647 Fix and enable SphinxSE tests 2019-09-30 15:47:09 +03:00
Marko Mäkelä
46b785262b Fix -Wunused for CMAKE_BUILD_TYPE=RelWithDebInfo
For release builds, do not declare unused variables.

unpack_row(): Omit a debug-only variable from WSREP diagnostic message.

create_wsrep_THD(): Fix -Wmaybe-uninitialized for the PSI_thread_key.
2019-09-30 12:49:53 +03:00
Marko Mäkelä
9b5cdeeb0f Merge 10.3 into 10.4 2019-09-27 16:26:53 +03:00
Marko Mäkelä
ea2b19dee6 MDEV-20117: Fix another scenario
Thanks to Eugene Kosov for noting that the fix is incomplete.
It turns out that on instant DROP/reorder column (MDEV-15562),
we must always write the metadata record, even though the table
was empty. Alternatively, we should guarantee that all undo
log records for the table have been purged. (Attempting to do
that by updating table_id leads to other problems; see
commit 1b31d8852c00b4bab6e6fe179b97db45ccb8d535.)

It would be tempting to remove dict_index_t::clear_instant_alter()
altogether, but it turns that we need that when the instant ALTER TABLE
operation of a first-time DROP COLUMN is being rolled back.

innobase_instant_try(): Clarify a comment. Purge never calls
dict_index_t::clear_instant_alter(), but it may invoke
dict_index_t::clear_instant_add(). On first-time instant DROP/reorder,
always write a metadata record, even if the table is empty.
2019-09-27 16:01:55 +03:00
Marko Mäkelä
2911a9a693 Merge 10.2 into 10.3 2019-09-27 15:56:15 +03:00
Thirunarayanan Balathandayuthapani
c76873f23d MDEV-20688 Recovery crashes after unnecessarily reading a corrupted page
The test encryption.innodb-redo-badkey was accidentally disabled
until commit 23657a2101 enabled
it recently. Once it was enabled, it started failing randomly.

recv_recover_corrupt_page(): Do not assume that any redo log exists
for the page. A page may be unnecessarily read by read-ahead.
When noting the corruption, reset recv_addr->state to RECV_PROCESSED,
so that even if the same page is re-read again, we will only
decrement recv_sys->n_addrs once.
2019-09-27 17:46:10 +05:30
Marko Mäkelä
d874cdeccc dict_load_table(): Remove constant parameter cached=true
Spotted by Thirunarayanan Balathandayuthapani.
2019-09-27 14:29:22 +03:00
Marko Mäkelä
718fcee0a3 Reduce rw_lock_debug_mutex contention
rw_lock_own(), rw_lock_own_flagged(): Traverse the rw_lock_t::debug_list
only after quickly checking if the thread is holding X-latch or SX-latch.
2019-09-27 14:22:59 +03:00
Marko Mäkelä
4ec0c346b8 Remove a useless large test, and add a debug assertion
The test innodb_fts.fulltext_table_evict was only creating 1000 tables
with fulltext indexes, only to check that no tables with fulltext
indexes are being evicted.

The reason why tables containing fulltext indexes cannot be evicted is
that fts_optimize_init() invokes dict_table_prevent_eviction().
2019-09-27 14:05:39 +03:00
Marko Mäkelä
ca9e0089d5 MDEV-19740: Fix GCC 9.2.1 -Wmaybe-uninitialized on AMD64
For CMAKE_BUILD_TYPE=Debug, the default MYSQL_MAINTAINER_MODE=AUTO
implies -Werror along with other flags in cmake/maintainer.cmake,
which would break the debug builds when CMAKE_CXX_FLAGS include -O2.

This fix includes a backport of 6dd3f24090
from MariaDB 10.3.
2019-09-27 10:43:23 +03:00
Marko Mäkelä
1f4ee3fa5a MDEV-20117 Assertion 0 failed in row_sel_get_clust_rec_for_mysql
The crash scenario is as follows:

(1) A non-empty table exists.
(2) MDEV-15562 instant ADD/DROP/reorder has been invoked.
(3) Some purgeable undo log exists for the table.
(4) The table becomes empty, containing not even any delete-marked records,
only containing the hidden metadata record that was added in (2).
(5) An instant ADD/DROP/reorder column is executed, and the table
is emptied and the (2) metadata removed.
(6) Purge processes an undo log record from (3), which will refer to
a non-existent clustered index field, because the metadata that
was created in (2) was remoeved in (5).

We fix this by adjusting step (5) so that we will never remove the
MDEV-15562-style metadata record. Removing the MDEV-11369 metadata
record (instant ADD COLUMN to the end of the table) is completely
fine at any time when the table becomes empty, because
dict_index_t::n_fields will remain unchanged.

innobase_instant_try(): Never remove the MDEV-15562 metadata record.

page_cur_delete_rec(): Do not reset FIL_PAGE_TYPE when the
MDEV-15562 metadata record is being removed as part of
btr_cur_pessimistic_update() invoked by innobase_instant_try().
2019-09-26 19:45:10 +03:00
Marko Mäkelä
bb5afc7ceb Merge 10.3 into 10.4 2019-09-26 16:56:02 +03:00