Commit graph

6744 commits

Author SHA1 Message Date
Marko Mäkelä
44298e4dea Merge 10.2 into 10.3
Also, clean up the test innodb_gis.geometry a little further.
2020-03-20 18:12:17 +02:00
Marko Mäkelä
b8b3edff13 MDEV-21549 IMPORT TABLESPACE fails to adjust all tablespace ID in root pages
After MDEV-12353, the consistency check that I originally added for
commit 1b9fe0bbac
(InnoDB Plugin for MySQL 5.1) started randomly failing.

It turns out that the IMPORT TABLESPACE code was always incorrect:
it did not update the (redundantly stored) tablespace ID
in index tree root pages. It only does that for page headers
and BLOB pointers.

PageConverter::update_index_page(): Update the tablespace ID
in the BTR_SEG_TOP and BTR_SEG_LEAF of index root pages.
2020-03-20 15:53:04 +02:00
Marko Mäkelä
6960e9ed24 MDEV-21983: Crash on DROP/RENAME TABLE after DISCARD TABLESPACE
fil_delete_tablespace(): Remove the unused parameter drop_ahi,
and add the parameter if_exists=false. We want to suppress
error messages if we know that the tablespace has been discarded.

dict_table_rename_in_cache(): Pass the new parameter to
fil_delete_tablespace(), that is, do not complain about
missing tablespace if the tablespace has been discarded.

row_make_new_pathname(): Declare as static.

row_drop_table_for_mysql(): Tolerate !table->data_dir_path
when the tablespace has been discarded.

row_rename_table_for_mysql(): Skip part of the RENAME TABLE
when fil_space_get_first_path() returns NULL.
2020-03-19 14:23:47 +02:00
Marko Mäkelä
9fd692aeca MDEV-13626: Clean up the buffer pool resizing tests from MySQL 5.7
buf_pool_resize(): Simplify the fault injection
for innodb.buf_pool_resize_oom.

innodb.buf_pool_resize_oom: Use a small buffer pool.

innodb.innodb_buffer_pool_load_now: Make use of the sequence engine,
to avoid creating explicit InnoDB record locks. Clean up the
accesses to information_schema.innodb_buffer_page_lru.
2020-03-19 13:01:20 +02:00
Thirunarayanan Balathandayuthapani
09e8707d90 MDEV-21826 Recovery failure : loop of Read redo log up to LSN
- This issue is caused by MDEV-19176
(bba59abb03).
- Problem is that there is miscalculation of available memory during
recovery if innodb_buffer_pool_instances > 1.
- Ignore the buffer pool instance while calculating available_memory
- Removed recv_n_pool_free_frames variable and use buf_pool_get_n_pages()
instead.
2020-03-18 15:25:28 +05:30
Marko Mäkelä
ab0034a789 MDEV-20370 Crash after OPTIMIZE TABLE on TEMPORARY TABLE
Temporary tables are typically short-lived, and temporary tables
are assumed to be accessed only by the thread that is handling
the owning connection. Hence, they must not be subject to
defragmenting.

ha_innobase::optimize(): Do not add temporary tables to
the defragment_table() queue.
2020-03-17 16:28:16 +02:00
Petr Vaněk
b1d7ba472e innodb: fix typo in function description
Function os_file_get_last_error_low returns error number +
OS_FILE_ERROR_MAX (which is currently set to 200) for unknown numbers to
this program.
2020-03-16 17:36:16 +01:00
Marko Mäkelä
5fe87ac413 Merge 10.2 into 10.3 2020-03-13 12:31:55 +02:00
Marko Mäkelä
2e8b0c56a0 MDEV-21933 INFORMATION_SCHEMA.INNODB_SYS_TABLESPACES accesses SYS_DATAFILES
All tablespace metadata is buffered in fil_system. There is a LRU
mechanism, but that only controls the opening and closing of
fil_node_t::handle.

It is much more efficient and less error-prone to access data file names
by looking up the fil_space_t object rather than by essentially joining
each row with an access to SYS_DATAFILES via the InnoDB internal SQL parser.

dict_get_first_path(): Declare static. The function may only be needed
when loading or updating the data dictionary. Also, change a condition
in order to avoid a bogus GCC 10 -Wstringop-overflow warning for
mem_strdupl() about len==ULINT_UNDEFINED.

i_s_sys_tablespaces_fill_table(): Do not access other InnoDB internal
dictionary tables than SYS_TABLESPACES.
2020-03-13 08:07:02 +02:00
Marko Mäkelä
32904dc5fa Merge 10.1 into 10.2 2020-03-13 07:20:36 +02:00
Eugene Kosov
5257bcfc7a InnoDB: improve error message for checksum mismatch 2020-03-12 14:47:45 +03:00
Eugene Kosov
7f36300df5 MDEV-21918 improve page_zip_verify_checksum()
actually, page_zip_verify_checksum() generally allows all-zeroes
checksums because our CRC32 checksum is something like
crc_1 ^ crc_2 ^ crc_3

Also, all zeroes page is considered correct.

As a side effect fix nasty reinterpret_cast<> UB

Also, since c0f47a4a58 innodb_checksum_algorithm=full_crc32
exists which computes CRC32 in one go (without bitwise arithmetic)
2020-03-11 18:02:12 +03:00
Eugene Kosov
df88e7cefa fix typedef-related warning and cleanup using namespace std 2020-03-11 16:27:37 +03:00
Oleksandr Byelkin
3c9bc0ce19 Merge branch '10.2' into 10.3 2020-03-11 14:05:41 +01:00
Marko Mäkelä
02343c4a54 MDEV-19740: Correct a type mismatch WITH_INNODB_EXTRA_DEBUG 2020-03-10 15:47:52 +02:00
Oleksandr Byelkin
58b70dc136 Merge branch '10.2' into 10.3 2020-02-10 20:34:16 +01:00
Oleksandr Byelkin
594282a534 Merge branch '10.1' into 10.2 2020-02-10 14:31:39 +01:00
Marko Mäkelä
e568dc9723 Remove unused SRV_MASTER_PURGE_INTERVAL
The symbol SRV_MASTER_PURGE_INTERVAL became unused in
mysql/mysql-server@42f3691958
when separate purge threads were introduced in MySQL 5.6.5.
2020-02-08 18:58:28 +02:00
Thirunarayanan Balathandayuthapani
3be751d5b9 MDEV-21608 Assertion `n_ext == dtuple_get_n_ext(dtuple)' failed during updation of PK
- n_ext value may be less than dtuple_get_n_ext(dtuple) when PK is being
updated and new record inherits the externally stored fields from
delete mark old record.
2020-02-07 16:01:31 +05:30
Thirunarayanan Balathandayuthapani
80da232576 MDEV-21563 FTS thread aborts during shutdown
- Added the test case in innodb_fts suite
- Updated copyright year in row0mysql.cc
2020-02-07 15:22:23 +05:30
Thirunarayanan Balathandayuthapani
280bf17829 MDEV-21563 FTS thread aborts during shutdown
Problem:
=======
After discarding the table, fts_optimize_thread aborts during shutdown.
InnoDB fails to remove the table from fts_optimize_wq and it leads to
the fts_optimize_thread to lookup for the auxiliary table and fails.

Fix:
====
While discarding the fts table, remove the table from fts_optimize_wq.
2020-02-06 20:42:29 +05:30
Marko Mäkelä
ebbc572b82 MDEV-12121: Clean up WITH_INNODB_AHI=OFF
buf_flush_or_remove_pages(): Only define BUF_LRU_DROP_SEARCH_SIZE
and dependent code when the adaptive hash index has been enabled.
2020-02-06 10:50:04 +02:00
Marko Mäkelä
a9d1324867 Cleanup: Remove mem_block_t::magic_n and mem_block_validate()
Use of freed memory is better caught by AddressSanitizer,
especially with ASAN_POISON_MEMORY_REGION that is aliased
by MEM_NOACCESS and UNIV_MEM_FREE.
2020-02-03 12:34:08 +02:00
Marko Mäkelä
37b9734c06 MDEV-21636 information_schema.innodb_mutexes.name column is not populated
The column INFORMATION_SCHEMA.INNODB_MUTEXES.NAME is not populated ever since
commit 2e814d4702 applied the InnoDB changes from
MySQL 5.7.9 to MariaDB Server 10.2.2.

Since the same commit, the view is only providing information about
rw_lock_t, not any mutexes.

For now, let us convert the source code file name and line number of
the rw_lock_t creation into a name. A better option in the future might
be to store the information somewhere where it can be looked up by
mysql_pfs_key_t, and possibly to remove the CREATE_FILE and CREATE_LINE
columns.
2020-02-03 12:34:08 +02:00
Eugene Kosov
bd36a4ca12 introduce HASH_REPLACE() for hash_table_t
HASH_REPLACE(): allows to not travel through linked list twice
when HASH_INSERT() happens right after HASH_DELETE()
2020-01-31 22:14:18 +08:00
Marko Mäkelä
5ff66fb0b9 Merge 10.2 into 10.3 2020-01-31 11:37:12 +02:00
Marko Mäkelä
2daf3b14fe Merge 10.1 into 10.2 2020-01-31 10:53:56 +02:00
Marko Mäkelä
0b36c27e0c MDEV-20307: Remove a useless debug check to save stack space
fil_space_encrypt(): Remove the debug check that decrypts the
just encrypted page. We are exercising the decryption of encrypted
pages enough via --suite=encryption,mariabackup. It is a waste of
computing resources to decrypt every page immediately after encrypting it.

The redundant check had been added in
commit 2bedc3978b (MDEV-9931).
2020-01-31 10:06:55 +02:00
Thirunarayanan Balathandayuthapani
afe9caa82d MDEV-21564 Assert in trx_purge_add_update_undo_to_history during shutdown
- dict_table_try_drop_aborted() can generate undo logs. So it should
be avoided after shutdown of background threads.
2020-01-30 17:54:49 +05:30
Thirunarayanan Balathandayuthapani
a134ec3736 MDEV-21550 Assertion `!table->fts->in_queue' failed in fts_optimize_remove_table
Problem:
=======
  The problem is that InnoDB doesn't add the table in fts slots if drop table fails. InnoDB marks the table is in fts slots while processing sync message. So the consecutive alter statement assumes that table is in queue and tries to remove it. But InnoDB can't find the table in fts_slots.

Solution:
=========
  i)  Removal of in_queue in fts_t while processing the fts sync message.
  ii) Add the table to fts_slots when drop table fails.
2020-01-28 18:21:00 +05:30
Oleksandr Byelkin
ceda5f724f Merge branch '10.2' into 10.3 2020-01-24 14:16:20 +01:00
Marko Mäkelä
1d12bff42c MDEV-20775: page_zip_validate() failure due to AUTO_INCREMENT
cmake -DWITH_INNODB_EXTRA_DEBUG:BOOL=ON
was broken ever since commit 8777458a6e
(MDEV-6076 Persistent AUTO_INCREMENT for InnoDB).

There is a race condition between page reads that call
page_zip_validate() (while holding clustered index root page S-latch)
and writes that update PAGE_ROOT_AUTO_INC
(with buf_block_t::lock SX-latch, compatible with S-latch).

page_zip_validate_low(): Skip the PAGE_ROOT_AUTO_INC field on
clustered index root pages in order to avoid false positives.
2020-01-23 16:25:46 +02:00
Marko Mäkelä
6373ec3ec7 Merge 10.2 into 10.3 2020-01-18 16:56:16 +02:00
Marko Mäkelä
02af6278fb InnoDB 5.6.47 and XtraDB 5.6.46-86.2
The only change is a change of the version number.
In MySQL 5.6.46, the copyright comments in a number of files were changed
in mysql/mysql-server@f1a006ece7
but there was no functional change to InnoDB code.
This was also reflected by XtraDB. We are not changing the copyright
comments in MariaDB Server for now.

Between MySQL 5.6.46 and 5.6.47, InnoDB was not changed at all.

Actually, we had forgotten to update the InnoDB version number to
5.6.46. With this change, we are updating InnoDB
from 5.6.45 to 5.6.47 and XtraDB from 5.6.45-86.1 to 5.6.46-86.2.
2020-01-17 17:39:20 +02:00
Marko Mäkelä
7b70cbd838 MDEV-21499 Merge new release of InnoDB 5.7.29 to 10.2 2020-01-17 16:24:40 +02:00
Nikša Skeledžija
c25a0662b3 Bug - GCC 9.2.1 REPORTS A NEW WARNING FOR OS_FILE_GET_PARENT_DIR
- Fixed a warning visible in optimized build related to calling
memcpy with length parameters larger than ptrdiff_t max.

rb#23333 approved by Annamalai Gurusami <annamalai.gurusami@oracle.com>
2020-01-17 16:23:13 +02:00
Marko Mäkelä
08b0b2b6fb MDEV-21513: Avoid some crashes in ALTER TABLE...IMPORT TABLESPACE
IndexPurge::next(): Replace btr_pcur_move_to_next_user_rec()
with some equivalent code that performs sanity checks without
killing the server. Perform some additional sanity checks as well.

This change is motivated by
mysql/mysql-server@48de4d74f4
which unnecessarily introduces storage overhead to btr_pcur_t
and uses a test case that injects a fault somewhere else,
not in the code path that was modified.
2020-01-17 16:22:13 +02:00
Marko Mäkelä
457ce97ef2 MDEV-21512 InnoDB may hang due to SPATIAL INDEX
MySQL 5.7.29 includes the following fix:
Bug  INNODB: A LONG SEMAPHORE WAIT
mysql/mysql-server@5cdbb22b51

There is no test case. It seems that the problem could occur when
a spatial index is large and peculiar enough so that multiple R-tree
leaf pages will have the exactly same maximum bounding rectangle (MBR).

The commit message suggests that the hang can occur when R-tree
non-leaf pages are being merged, which should only be possible
during transaction rollback or the purge of transaction history,
when the R-tree index is at least 2 levels high and very many records
are being deleted. The message says that a comparison result that two
spatial index node pointer records are equal will cause an infinite loop
in rtr_page_copy_rec_list_end_no_locks(). Hence, we must include the
child page number in the comparison to be consistent with
mysql/mysql-server@2e11fe0e15.

We fix this bug in a simpler way, involving fewer code changes.

cmp_rec_rec(): Renamed from cmp_rec_rec_with_match().
Assert that rec2 always resides in an index page.
Treat non-leaf spatial index pages specially.
2020-01-17 14:27:29 +02:00
Marko Mäkelä
c3695b4058 MDEV-21511: Remove unnecessary code
Now that we will be invoking dtuple_get_n_ext() instead of
letting btr_push_update_extern_fields() update an already
calculated value, it is unnecessary to calculate the n_ext
upfront.

row_rec_to_index_entry(), row_rec_to_index_entry_low():
Remove the output parameter n_ext.
2020-01-17 14:27:29 +02:00
Marko Mäkelä
5838b52743 MDEV-21511 Wrong estimate of affected BLOB columns in update
During update, rollback, or MVCC read, we may miscalculate
the number of off-page columns, and thus the size of the
clustered index record. The function btr_push_update_extern_fields()
is mostly redundant, because the off-page columns would also be
moved by row_upd_index_replace_new_col_val(), which is invoked
via row_upd_index_replace_new_col_vals().

btr_push_update_extern_fields(): Remove.

This is based on
mysql/mysql-server@1fa475b85d
which refines a fix for a recovery bug fix
mysql/mysql-server@ce0a1e85e2
in MySQL 5.7.5.

No test case was provided by Oracle.
Some of the changed code is being covered by the existing test
innodb.blob-crash.
2020-01-17 14:27:28 +02:00
Marko Mäkelä
3e38d15585 MDEV-21509 Possible hang during purge of history, or rollback
WL#6326 in MariaDB 10.2.2 introduced a potential hang on purge or rollback
when an index tree is being shrunk by multiple levels.

This fix is based on
mysql/mysql-server@f2c5852630
with the main difference that our version of the test case uses
DEBUG_SYNC instrumentation on ROLLBACK, not on purge.

btr_cur_will_modify_tree(): Simplify the check further.
This is the actual bug fix.

row_undo_mod_remove_clust_low(), row_undo_mod_clust(): Add DEBUG_SYNC
instrumentation for the test case.
2020-01-17 14:27:28 +02:00
Sergei Petrunia
e709eb9bf7 Merge branch '10.2' into 10.3
# Conflicts:
#	mysql-test/suite/galera/r/MW-388.result
#	mysql-test/suite/galera/t/MW-388.test
#	mysql-test/suite/innodb/r/truncate_inject.result
#	mysql-test/suite/innodb/t/truncate_inject.test
#	mysql-test/suite/rpl/r/rpl_stop_slave.result
#	mysql-test/suite/rpl/t/rpl_stop_slave.test
#	sql/sp_head.cc
#	sql/sp_head.h
#	sql/sql_lex.cc
#	sql/sql_yacc.yy
#	storage/xtradb/buf/buf0dblwr.cc
2020-01-17 00:46:40 +03:00
Sergei Petrunia
b04429434a Merge branch '10.1' into 10.2
# Conflicts:
#	sql/sp_head.cc
#	sql/sql_select.cc
#	sql/sql_trigger.cc
2020-01-17 00:24:17 +03:00
Marko Mäkelä
bde7e0ba6e MDEV-21500 Server hang when using simulated AIO
The write-heavy test innodb_zip.wl6501_scale_1 timed out on
10.2 60d7011c5f for me.
Out of os_aio_n_segments=6, 5 are waiting for an event in
os_aio_simulated_handler(). One thread is waiting for a
write to complete in buf_dblwr_add_to_batch(), but that
would never happen, because nothing is waking up the simulated AIO
handler threads.

This hang appears to have been introduced in MySQL 5.6.12
in mysql/mysql-server@26cfde776c.
2020-01-16 11:40:21 +02:00
Eugene Kosov
56529a7d7f MDEV-21454 Show actual mismatching values in mismatch error messages from row_import::match_table_columns()
Patch by Hartmut Holzgraefe
2020-01-10 22:50:19 +07:00
Eugene Kosov
cba9ed1279 fix compilation 2020-01-08 20:18:50 +07:00
Marko Mäkelä
0b4ae6724f Fixup MDEV-21429: Correct a definition
INNOBASE_ALTER_NOVALIDATE: Remove the set of operations
INNOBASE_ONLINE_CREATE that was accidentally included in the
definition.

In the merge of 82187a1221 to 10.3
(in commit eda719793a) the flags
were defined correctly.

This bug was caught by the test innodb_zip.index_large_prefix.
2020-01-07 14:38:21 +02:00
Marko Mäkelä
eda719793a Merge 10.2 into 10.3 2020-01-07 12:14:35 +02:00
Marko Mäkelä
82187a1221 MDEV-21429 TRUNCATE and OPTIMIZE are being refused due to "row size too large"
By default (innodb_strict_mode=ON), InnoDB attempts to guarantee
at DDL time that any INSERT to the table can succeed.
MDEV-19292 recently revised the "row size too large" check in InnoDB.
The check still is somewhat inaccurate;
that should be addressed in MDEV-20194.

Note: If a table contains multiple long string columns so that each column
is part of a column prefix index, then an UPDATE that attempts to modify
all those columns at once may fail, because the undo log record might
not fit in a single undo log page (of innodb_page_size). In the worst case,
the undo log record would grow by about 3KiB of for each updated column.

The DDL-time check (since the InnoDB Plugin for MySQL 5.1) is optional
in the sense that when the maximum B-tree record size or undo log
record size would be exceeded, the DML operation will fail and the
transaction will be properly rolled back.

create_table_info_t::row_size_is_acceptable(): Add the parameter
'bool strict' so that innodb_strict_mode=ON can be overridden during
TRUNCATE, OPTIMIZE and ALTER TABLE...FORCE (when the storage format
is not changing).

create_table_info_t::create_table(): Perform a sloppy check for
TRUNCATE TABLE (create_fk=false).

prepare_inplace_alter_table_dict(): Perform a sloppy check for
simple operations.

trx_is_strict(): Remove. The function became unused in
commit 98694ab0cb (MDEV-20949).
2020-01-07 11:02:12 +02:00
Eugene Kosov
fd899b3bbd Lets add another intrusive double linked list!
Features:
* STL-like interface
* Fast modification: no branches on insertion or deletion
* Fast iteration: one pointer dereference and one pointer comparison
* Your class can be a part of several lists

Modeled after std::list<T> but currently has fewer methods (not complete yet)

For even more performance it's possible to customize list with templates so
it won't have size counter variable or won't NULLify unlinked node.

How existing lists differ?

No existing lists support STL-like interface.

I_List:
* slower iteration (one more branch on iteration)
* element can't be a part of two lists simultaneously

I_P_List:
* slower modification (branches, except for the fastest push_back() case)
* slower iteration (one more branch on iteration)

UT_LIST_BASE_NODE_T:
* slower modification (branches)

Three UT_LISTs were replaced: two in fil_system_t and one in dyn_buf_t.
2020-01-04 13:39:14 +07:00