Commit graph

22142 commits

Author SHA1 Message Date
Marko Mäkelä
5ff66fb0b9 Merge 10.2 into 10.3 2020-01-31 11:37:12 +02:00
Marko Mäkelä
2daf3b14fe Merge 10.1 into 10.2 2020-01-31 10:53:56 +02:00
Marko Mäkelä
0b36c27e0c MDEV-20307: Remove a useless debug check to save stack space
fil_space_encrypt(): Remove the debug check that decrypts the
just encrypted page. We are exercising the decryption of encrypted
pages enough via --suite=encryption,mariabackup. It is a waste of
computing resources to decrypt every page immediately after encrypting it.

The redundant check had been added in
commit 2bedc3978b (MDEV-9931).
2020-01-31 10:06:55 +02:00
Thirunarayanan Balathandayuthapani
afe9caa82d MDEV-21564 Assert in trx_purge_add_update_undo_to_history during shutdown
- dict_table_try_drop_aborted() can generate undo logs. So it should
be avoided after shutdown of background threads.
2020-01-30 17:54:49 +05:30
Vicențiu Ciorbaru
07e34cddb6 MDEV-14330: move tokudb manpages to right packages
Move tokuftdump and tokuft_logprint man pages to storage/tokudb.

The man pages are now part of tokudb-engine cmake component. This change
is mostly for RPM & DEB based packaging generated through CMake & CPack.

Debian upstream already handles this change via the custom scripts in debian/
2020-01-28 18:27:18 +02:00
Thirunarayanan Balathandayuthapani
a134ec3736 MDEV-21550 Assertion `!table->fts->in_queue' failed in fts_optimize_remove_table
Problem:
=======
  The problem is that InnoDB doesn't add the table in fts slots if drop table fails. InnoDB marks the table is in fts slots while processing sync message. So the consecutive alter statement assumes that table is in queue and tries to remove it. But InnoDB can't find the table in fts_slots.

Solution:
=========
  i)  Removal of in_queue in fts_t while processing the fts sync message.
  ii) Add the table to fts_slots when drop table fails.
2020-01-28 18:21:00 +05:30
Oleksandr Byelkin
ceda5f724f Merge branch '10.2' into 10.3 2020-01-24 14:16:20 +01:00
Marko Mäkelä
7aa443ca7d Remove an unused tokuvalgrind script
This is the only symlink in the repository. Symlinks can cause
trouble when using file systems or operating systems that do not
support them.

Also remove the unused file DartConfig.cmake that refers to the script.
2020-01-23 16:31:25 +02:00
Marko Mäkelä
1d12bff42c MDEV-20775: page_zip_validate() failure due to AUTO_INCREMENT
cmake -DWITH_INNODB_EXTRA_DEBUG:BOOL=ON
was broken ever since commit 8777458a6e
(MDEV-6076 Persistent AUTO_INCREMENT for InnoDB).

There is a race condition between page reads that call
page_zip_validate() (while holding clustered index root page S-latch)
and writes that update PAGE_ROOT_AUTO_INC
(with buf_block_t::lock SX-latch, compatible with S-latch).

page_zip_validate_low(): Skip the PAGE_ROOT_AUTO_INC field on
clustered index root pages in order to avoid false positives.
2020-01-23 16:25:46 +02:00
Vlad Lesin
4e7f3fb833 MDEV-14183: aria_pack segfaults in compress_maria_file
Column definition order in st_maria_share::columndef can differ from
order of fields in record(see also st_maria_share::column_nr,
st_maria_columndef::column_nr, _ma_column_nr_write(),
_ma_column_nr_read()). This was not taken into account in aria_pack
tool.

The fix is to initialize elements of HUFF_COUNTS array in the correct
order.
2020-01-21 17:06:31 +03:00
Oleksandr Byelkin
ade89fc898 Merge branch '10.2' into 10.3 2020-01-21 09:11:14 +01:00
Oleksandr Byelkin
3a1716a7e7 Merge branch '10.1' into 10.2 2020-01-20 16:15:05 +01:00
Oleksandr Byelkin
d07664498b Merge remote-tracking branch 'connect/10.2' into 10.2 2020-01-20 16:10:30 +01:00
Oleksandr Byelkin
7993f893b8 Merge branch 'merge-tokudb-5.6' into 10.1 2020-01-19 14:37:25 +01:00
Oleksandr Byelkin
6cb208107e 5.6.46-86.2 2020-01-19 14:08:35 +01:00
Oleksandr Byelkin
10eacd5ff7 Merge branch 'merge-perfschema-5.6' into 10.1 2020-01-19 13:11:45 +01:00
Oleksandr Byelkin
3aff3f3679 5.6.47 2020-01-19 12:52:07 +01:00
Oleksandr Byelkin
5ce6ec3fb5 Merge remote-tracking branch 'connect/10.1' into 10.1 2020-01-19 12:27:44 +01:00
Marko Mäkelä
6373ec3ec7 Merge 10.2 into 10.3 2020-01-18 16:56:16 +02:00
Marko Mäkelä
02af6278fb InnoDB 5.6.47 and XtraDB 5.6.46-86.2
The only change is a change of the version number.
In MySQL 5.6.46, the copyright comments in a number of files were changed
in mysql/mysql-server@f1a006ece7
but there was no functional change to InnoDB code.
This was also reflected by XtraDB. We are not changing the copyright
comments in MariaDB Server for now.

Between MySQL 5.6.46 and 5.6.47, InnoDB was not changed at all.

Actually, we had forgotten to update the InnoDB version number to
5.6.46. With this change, we are updating InnoDB
from 5.6.45 to 5.6.47 and XtraDB from 5.6.45-86.1 to 5.6.46-86.2.
2020-01-17 17:39:20 +02:00
Marko Mäkelä
7b70cbd838 MDEV-21499 Merge new release of InnoDB 5.7.29 to 10.2 2020-01-17 16:24:40 +02:00
Nikša Skeledžija
c25a0662b3 Bug #30499288 - GCC 9.2.1 REPORTS A NEW WARNING FOR OS_FILE_GET_PARENT_DIR
- Fixed a warning visible in optimized build related to calling
memcpy with length parameters larger than ptrdiff_t max.

rb#23333 approved by Annamalai Gurusami <annamalai.gurusami@oracle.com>
2020-01-17 16:23:13 +02:00
Marko Mäkelä
08b0b2b6fb MDEV-21513: Avoid some crashes in ALTER TABLE...IMPORT TABLESPACE
IndexPurge::next(): Replace btr_pcur_move_to_next_user_rec()
with some equivalent code that performs sanity checks without
killing the server. Perform some additional sanity checks as well.

This change is motivated by
mysql/mysql-server@48de4d74f4
which unnecessarily introduces storage overhead to btr_pcur_t
and uses a test case that injects a fault somewhere else,
not in the code path that was modified.
2020-01-17 16:22:13 +02:00
Marko Mäkelä
457ce97ef2 MDEV-21512 InnoDB may hang due to SPATIAL INDEX
MySQL 5.7.29 includes the following fix:
Bug #30287668 INNODB: A LONG SEMAPHORE WAIT
mysql/mysql-server@5cdbb22b51

There is no test case. It seems that the problem could occur when
a spatial index is large and peculiar enough so that multiple R-tree
leaf pages will have the exactly same maximum bounding rectangle (MBR).

The commit message suggests that the hang can occur when R-tree
non-leaf pages are being merged, which should only be possible
during transaction rollback or the purge of transaction history,
when the R-tree index is at least 2 levels high and very many records
are being deleted. The message says that a comparison result that two
spatial index node pointer records are equal will cause an infinite loop
in rtr_page_copy_rec_list_end_no_locks(). Hence, we must include the
child page number in the comparison to be consistent with
mysql/mysql-server@2e11fe0e15.

We fix this bug in a simpler way, involving fewer code changes.

cmp_rec_rec(): Renamed from cmp_rec_rec_with_match().
Assert that rec2 always resides in an index page.
Treat non-leaf spatial index pages specially.
2020-01-17 14:27:29 +02:00
Marko Mäkelä
c3695b4058 MDEV-21511: Remove unnecessary code
Now that we will be invoking dtuple_get_n_ext() instead of
letting btr_push_update_extern_fields() update an already
calculated value, it is unnecessary to calculate the n_ext
upfront.

row_rec_to_index_entry(), row_rec_to_index_entry_low():
Remove the output parameter n_ext.
2020-01-17 14:27:29 +02:00
Marko Mäkelä
5838b52743 MDEV-21511 Wrong estimate of affected BLOB columns in update
During update, rollback, or MVCC read, we may miscalculate
the number of off-page columns, and thus the size of the
clustered index record. The function btr_push_update_extern_fields()
is mostly redundant, because the off-page columns would also be
moved by row_upd_index_replace_new_col_val(), which is invoked
via row_upd_index_replace_new_col_vals().

btr_push_update_extern_fields(): Remove.

This is based on
mysql/mysql-server@1fa475b85d
which refines a fix for a recovery bug fix
mysql/mysql-server@ce0a1e85e2
in MySQL 5.7.5.

No test case was provided by Oracle.
Some of the changed code is being covered by the existing test
innodb.blob-crash.
2020-01-17 14:27:28 +02:00
Marko Mäkelä
3e38d15585 MDEV-21509 Possible hang during purge of history, or rollback
WL#6326 in MariaDB 10.2.2 introduced a potential hang on purge or rollback
when an index tree is being shrunk by multiple levels.

This fix is based on
mysql/mysql-server@f2c5852630
with the main difference that our version of the test case uses
DEBUG_SYNC instrumentation on ROLLBACK, not on purge.

btr_cur_will_modify_tree(): Simplify the check further.
This is the actual bug fix.

row_undo_mod_remove_clust_low(), row_undo_mod_clust(): Add DEBUG_SYNC
instrumentation for the test case.
2020-01-17 14:27:28 +02:00
Sergei Petrunia
e709eb9bf7 Merge branch '10.2' into 10.3
# Conflicts:
#	mysql-test/suite/galera/r/MW-388.result
#	mysql-test/suite/galera/t/MW-388.test
#	mysql-test/suite/innodb/r/truncate_inject.result
#	mysql-test/suite/innodb/t/truncate_inject.test
#	mysql-test/suite/rpl/r/rpl_stop_slave.result
#	mysql-test/suite/rpl/t/rpl_stop_slave.test
#	sql/sp_head.cc
#	sql/sp_head.h
#	sql/sql_lex.cc
#	sql/sql_yacc.yy
#	storage/xtradb/buf/buf0dblwr.cc
2020-01-17 00:46:40 +03:00
Sergei Petrunia
b04429434a Merge branch '10.1' into 10.2
# Conflicts:
#	sql/sp_head.cc
#	sql/sql_select.cc
#	sql/sql_trigger.cc
2020-01-17 00:24:17 +03:00
Marko Mäkelä
bde7e0ba6e MDEV-21500 Server hang when using simulated AIO
The write-heavy test innodb_zip.wl6501_scale_1 timed out on
10.2 60d7011c5f for me.
Out of os_aio_n_segments=6, 5 are waiting for an event in
os_aio_simulated_handler(). One thread is waiting for a
write to complete in buf_dblwr_add_to_batch(), but that
would never happen, because nothing is waking up the simulated AIO
handler threads.

This hang appears to have been introduced in MySQL 5.6.12
in mysql/mysql-server@26cfde776c.
2020-01-16 11:40:21 +02:00
Olivier Bertrand
8ff3eb417c - Fix MDEV-21450
Failed compile when XML table type is not supported.
  Was because XMLDEF was unconditionally called from REST table.
  modified:   storage/connect/tabrest.cpp

- Make cmake less verbose
  modified:   storage/connect/CMakeLists.txt

- Hide Switch_to_definer_security_ctx not defined for 10.1 and 10.0
  modified:   storage/connect/ha_connect.cc
2020-01-12 19:59:07 +01:00
Olivier Bertrand
85f2217cc8 - Update grant tests for new MariaDB version 10.1.44
modified:   storage/connect/mysql-test/connect/disabled.def
  modified:   storage/connect/mysql-test/connect/r/grant.result
  modified:   storage/connect/mysql-test/connect/r/ini_grant.result
  modified:   storage/connect/mysql-test/connect/r/mysql_grant.result
  modified:   storage/connect/mysql-test/connect/r/xml2_grant.result
  modified:   storage/connect/mysql-test/connect/r/xml_grant.result
  modified:   storage/connect/mysql-test/connect/t/grant.inc
  modified:   storage/connect/mysql-test/connect/t/grant.test
  modified:   storage/connect/mysql-test/connect/t/grant2.test
  modified:   storage/connect/mysql-test/connect/t/ini_grant.test
  modified:   storage/connect/mysql-test/connect/t/mysql_grant.test
2020-01-12 00:52:46 +01:00
Olivier Bertrand
54449161f8 Merge with last MariaDB version 2020-01-11 18:34:57 +01:00
Olivier Bertrand
98f70fa26b - Fix MDEV-21450
Failed compile when XML table type is not supported.
  Was because XMLDEF was unconditionally called from REST table.
  modified:   storage/connect/tabrest.cpp
2020-01-11 16:05:39 +01:00
Eugene Kosov
56529a7d7f MDEV-21454 Show actual mismatching values in mismatch error messages from row_import::match_table_columns()
Patch by Hartmut Holzgraefe
2020-01-10 22:50:19 +07:00
Olivier Bertrand
0b624debdf Update to version 1.07 (as for MariaDB 10.2) 2020-01-09 16:10:25 +01:00
Eugene Kosov
cba9ed1279 fix compilation 2020-01-08 20:18:50 +07:00
Marko Mäkelä
0b4ae6724f Fixup MDEV-21429: Correct a definition
INNOBASE_ALTER_NOVALIDATE: Remove the set of operations
INNOBASE_ONLINE_CREATE that was accidentally included in the
definition.

In the merge of 82187a1221 to 10.3
(in commit eda719793a) the flags
were defined correctly.

This bug was caught by the test innodb_zip.index_large_prefix.
2020-01-07 14:38:21 +02:00
Marko Mäkelä
eda719793a Merge 10.2 into 10.3 2020-01-07 12:14:35 +02:00
Marko Mäkelä
82187a1221 MDEV-21429 TRUNCATE and OPTIMIZE are being refused due to "row size too large"
By default (innodb_strict_mode=ON), InnoDB attempts to guarantee
at DDL time that any INSERT to the table can succeed.
MDEV-19292 recently revised the "row size too large" check in InnoDB.
The check still is somewhat inaccurate;
that should be addressed in MDEV-20194.

Note: If a table contains multiple long string columns so that each column
is part of a column prefix index, then an UPDATE that attempts to modify
all those columns at once may fail, because the undo log record might
not fit in a single undo log page (of innodb_page_size). In the worst case,
the undo log record would grow by about 3KiB of for each updated column.

The DDL-time check (since the InnoDB Plugin for MySQL 5.1) is optional
in the sense that when the maximum B-tree record size or undo log
record size would be exceeded, the DML operation will fail and the
transaction will be properly rolled back.

create_table_info_t::row_size_is_acceptable(): Add the parameter
'bool strict' so that innodb_strict_mode=ON can be overridden during
TRUNCATE, OPTIMIZE and ALTER TABLE...FORCE (when the storage format
is not changing).

create_table_info_t::create_table(): Perform a sloppy check for
TRUNCATE TABLE (create_fk=false).

prepare_inplace_alter_table_dict(): Perform a sloppy check for
simple operations.

trx_is_strict(): Remove. The function became unused in
commit 98694ab0cb (MDEV-20949).
2020-01-07 11:02:12 +02:00
Eugene Kosov
fd899b3bbd Lets add another intrusive double linked list!
Features:
* STL-like interface
* Fast modification: no branches on insertion or deletion
* Fast iteration: one pointer dereference and one pointer comparison
* Your class can be a part of several lists

Modeled after std::list<T> but currently has fewer methods (not complete yet)

For even more performance it's possible to customize list with templates so
it won't have size counter variable or won't NULLify unlinked node.

How existing lists differ?

No existing lists support STL-like interface.

I_List:
* slower iteration (one more branch on iteration)
* element can't be a part of two lists simultaneously

I_P_List:
* slower modification (branches, except for the fastest push_back() case)
* slower iteration (one more branch on iteration)

UT_LIST_BASE_NODE_T:
* slower modification (branches)

Three UT_LISTs were replaced: two in fil_system_t and one in dyn_buf_t.
2020-01-04 13:39:14 +07:00
Oleksandr Byelkin
7753a29064 Merge branch '10.2' into 10.3 2020-01-03 13:44:16 +01:00
Marko Mäkelä
4a012ce2f4 Post-fix for MDEV-12253: Remove redundant log writes
The 8 bytes at FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION
will be overwritten at page flush, in buf_page_encrypt_before_write(),
ever since commit 765a43605a
(MariaDB 10.1.23, 10.2.6, 10.3.1).

btr_store_big_rec_extern_fields(): Remove useless writes to the
page header (and to the redo log) for ROW_FORMAT=COMPRESSED BLOB pages.
2019-12-30 18:12:55 +02:00
Marko Mäkelä
02e3006957 MDEV-21405 Assertion failed on instant ADD COLUMN
btr_cur_pessimistic_insert(): Relax a too strict debug assertion that
would fail when the function is invoked by btr_cur_pessimistic_update()
during innobase_add_instant_try(), that is, when updating the hidden
metadata record during a subsequent ADD COLUMN operation involves
splitting the leftmost clustered index leaf page.

This is a partial backport of 301bd62b25
from 10.4.
2019-12-30 10:08:18 +02:00
Nikita Malyavin
4923604ee2 MDEV-18865 Assertion `t->first->versioned_by_id()' failed in innodb_prepare_commit_versioned
Cause:
* row_start != 0 treated as it exists. Probably, possible row permutations had not been taken in mind.

Solution:
* Checking both row_start and row_end is correct, so versioned() function is used
2019-12-29 12:16:04 +02:00
Marko Mäkelä
5ab70e7f68 Merge 10.2 into 10.3 2019-12-27 15:14:48 +02:00
Marko Mäkelä
16bce0f6fe Cleanup: Remove dict_delete_tablespace_and_datafiles()
The function was only called by innobase_drop_tablespace(),
which was removed in commit 494e4b99a4
and added in commit 2e814d4702.
2019-12-27 11:23:28 +02:00
Eugene Kosov
cbc170adf2 MDEV-21362 do something with -fno-builtin-memcmp for rem0cmp.cc
innodb.cmake: restrict -fno-builtin-memcmp for GCC versions older that 4.6
where optimization issue was solved.
2019-12-24 17:03:27 +08:00
Thirunarayanan Balathandayuthapani
90ba87cb9e MDEV-19176 Reduce the memory usage during recovery
- post-push to fix the compilation issue
2019-12-23 15:56:57 +05:30
Thirunarayanan Balathandayuthapani
bba59abb03 MDEV-19176 Reduce the memory usage during recovery
- Moved the recv_sys->heap memory condition inside recv_parse_log_recs().
So that, InnoDB can mark the status as STORE_NO earlier.

- InnoDB uses one third of buffer pool chunk size for reading the redo
log records. In that case, we can avoid the scenario where buffer ran
out of memory issue during recovery.
2019-12-23 15:51:02 +05:30