Commit graph

1640 commits

Author SHA1 Message Date
Marko Mäkelä
f69a3b2e92 After-merge fix for commit d4df7bc9b1
The merge omitted some InnoDB and XtraDB conflict resolutions,
most notably, failing to merge the fix of MDEV-12173.

ibuf_merge_or_delete_for_page(), lock_rec_block_validate():
Invoke fil_space_acquire_silent() instead of fil_space_acquire().
This fixes MDEV-12173.

wsrep_debug, wsrep_trx_is_aborting(): Removed unused declarations.

_fil_io(): Remove. Instead, declare default parameters for the XtraDB
fil_io().

buf_read_page_low(): Declare default parameters, and clean up some
callers.

os_aio(): Correct the macro that is defined when !UNIV_PFS_IO.
2018-02-02 19:57:59 +02:00
Sergei Golubchik
d4df7bc9b1 Merge branch 'github/10.0' into 10.1 2018-02-02 10:09:44 +01:00
Jan Lindström
2749d25096 MDEV-13499: Backing up table that "doesn't exist in engine" cause crash in mariabackup when using encryption
Problem was that there is intentional crah when .ibd file does
not found. In mariabackup case we should avoid this crash.
2018-01-29 14:23:22 +02:00
Marko Mäkelä
706ed8552d Revert "MDEV-6928: Add trx pointer to struct mtr_t"
This reverts commit 3486135bb5.

The commit comment ended in the words: "This is needed later."
Apparently the "later" never arrived.
2018-01-29 11:05:17 +02:00
Vicențiu Ciorbaru
12c42bd2c7 Remove xtradb "fragmentation-statistics" patches
The patches are implemented using server changes in THD. We are not
interested in having special code for certain storage engines, if
possible.
2018-01-24 20:21:58 +02:00
Vicențiu Ciorbaru
a82bb5d316 Merge branch 'merge-xtradb-5.6' into 10.0 2018-01-24 20:20:11 +02:00
Vicențiu Ciorbaru
d833bb65d5 Merge remote-tracking branch '5.5' into 10.0 2018-01-24 12:29:31 +02:00
Marko Mäkelä
8637931f11 Add ASAN instrumentation (and more strict Valgrind) to InnoDB
mem_heap_free_heap_top(): Remove UNIV_MEM_ASSERT_W() and unpoison
the memory region first, because part of it may have been poisoned
by an earlier mem_heap_free_top() call.
Poison the address range at the end.

mem_heap_block_free(): Poison the address range at the end.

UNIV_MEM_ASSERT_AND_ALLOC(): Replace with UNIV_MEM_ALLOC().
We want to keep the address ranges poisoned (unaccessible) as
long as possible.

UNIV_MEM_ASSERT_AND_FREE(): Replace with UNIV_MEM_FREE().
2018-01-23 20:34:05 +02:00
Marko Mäkelä
70a9b12de9 Silence -Wimplicit-fallthrough 2018-01-23 18:08:55 +02:00
Vicențiu Ciorbaru
cd33250d2a 5.6.38-83.0 2018-01-23 18:04:34 +02:00
Marko Mäkelä
431607237d MDEV-12173 "Error: trying to do an operation on a dropped tablespace"
InnoDB is issuing a 'noise' message that is not a sign of abnormal
operation. The only issuers of it are the debug function
lock_rec_block_validate() and the change buffer merge.
While the error should ideally never occur in transactional locking,
we happen to know that DISCARD TABLESPACE and TRUNCATE TABLE and
possibly DROP TABLE are breaking InnoDB table locks.

When it comes to the change buffer merge, the message simply is useless
noise. We know perfectly well that a tablespace can be dropped while a
change buffer merge is pending. And the code is prepared to handle that,
which is demonstrated by the fact that whenever the message was issued,
InnoDB did not crash.

fil_inc_pending_ops(): Remove the parameter print_err.
2018-01-22 16:58:13 +02:00
Sergei Golubchik
03eb15933d improve ASAN instrumentation: InnoDB/XtraDB 2018-01-22 11:39:54 +01:00
Marko Mäkelä
906ce0962d MDEV-7049 MySQL#74585 - InnoDB: Failing assertion: *mbmaxlen < 5 in file ha_innodb.cc line 1904
InnoDB limited the maximum number of bytes per character to 4.
But, the filename character set that was introduced in MySQL 5.1
uses up to 5 bytes per character.

To allow InnoDB tables to be created with wider characters, let
us split the mbminmaxlen fields into mbminlen, mbmaxlen, and increase
the limit to 7 bytes per character. This will increase the payload size
of dtype_t and dict_col_t by one bit. The storage size will be unchanged
(54 bits and 77 bits will use the same number of bytes as the
previous sizes 53 and 76 bits).
2018-01-22 11:18:10 +02:00
Sergei Golubchik
fafdac3365 Merge branch 'merge/merge-xtradb-5.5' into 5.5 2018-01-18 17:56:28 +01:00
Marko Mäkelä
5e87f49a99 Make row_mysql_table_id_reassign() static 2018-01-18 09:29:49 +02:00
Sergei Golubchik
85a5e58d71 5.5.58-38.10 2018-01-17 17:28:33 +01:00
Marko Mäkelä
c15b3d2d41 Merge 10.0 into 10.1 2018-01-11 10:44:05 +02:00
Marko Mäkelä
4c1479545d Merge 5.5 into 10.0 2018-01-11 10:16:52 +02:00
Marko Mäkelä
bdcd7f79e4 MDEV-14916 InnoDB reports warning for "Purge reached the head of the history list"
The warning was originally added in
commit c67663054a
(MySQL 4.1.12, 5.0.3) to trace claimed undo log corruption that
was analyzed in https://lists.mysql.com/mysql/176250
on November 9, 2004.

Originally, the limit was 20,000 undo log headers or transactions,
but in commit 9d6d1902e0
in MySQL 5.5.11 it was increased to 2,000,000.

The message can be triggered when the progress of purge is prevented
by a long-running transaction (or just an idle transaction whose
read view was started a long time ago), by running many transactions
that UPDATE or DELETE some records, then starting another transaction
with a read view, and finally by executing more than 2,000,000
transactions that UPDATE or DELETE records in InnoDB tables. Finally,
when the oldest long-running transaction is completed, purge would
run up to the next-oldest transaction, and there would still be more
than 2,000,000 transactions to purge.

Because the message can be triggered when the database is obviously
not corrupted, it should be removed. Heavy users of InnoDB should be
monitoring the "History list length" in SHOW ENGINE INNODB STATUS;
there is no need to spam the error log.
2018-01-11 09:55:10 +02:00
Marko Mäkelä
a9c55c0059 MDEV-13814 Extra logging when innodb_log_archive=ON
Backport the fix from 10.0.33 to 5.5, in case someone compiles XtraDB
with -DUNIV_LOG_ARCHIVE
2018-01-10 10:21:52 +02:00
Marko Mäkelä
a408e881cf MDEV-14174 crash on start with innodb-track-changed-pages
The XtraDB option innodb_track_changed_pages causes
the function log_group_read_log_seg() to be invoked
even when recv_sys==NULL, leading to the SIGSEGV.

This regression was caused by
MDEV-11027 InnoDB log recovery is too noisy
2018-01-10 09:27:01 +02:00
Jan Lindström
07aa985979 MDEV-14776: InnoDB Monitor output generated by specific error is flooding error logs
innodb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor.

xtradb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor. Remove code that does not seem to
	be used.

innodb-lru-force-no-free-page.test
	New test case to force produce desired error message.
2018-01-09 12:48:31 +02:00
Marko Mäkelä
29b6e809a9 Merge 10.0 into 10.1 2018-01-08 14:51:20 +02:00
Marko Mäkelä
c903ba2f1e MDEV-13205 InnoDB: Failing assertion: !dict_index_is_online_ddl(index) upon ALTER TABLE
dict_foreign_find_index(): Ignore incompletely created indexes.
After a failed ADD UNIQUE INDEX, an incompletely created index
could be left behind until the next ALTER TABLE statement.
2018-01-08 14:26:55 +02:00
Marko Mäkelä
16d308e21d MDEV-14874 innodb_encrypt_log corrupts the log when the LSN crosses 32-bit boundary
This bug affects both writing and reading encrypted redo log in
MariaDB 10.1, starting from version 10.1.3 which added support for
innodb_encrypt_log. That is, InnoDB crash recovery and Mariabackup
will sometimes fail when innodb_encrypt_log is used.

MariaDB 10.2 or Mariabackup 10.2 or later versions are not affected.

log_block_get_start_lsn(): Remove. This function would cause trouble if
a log segment that is being read is crossing a 32-bit boundary of the LSN,
because this function does not allow the most significant 32 bits of the
LSN to change.

log_blocks_crypt(), log_encrypt_before_write(), log_decrypt_after_read():
Add the parameter "lsn" for the start LSN of the block.

log_blocks_encrypt(): Remove (unused function).
2018-01-08 09:44:40 +02:00
Marko Mäkelä
21470de148 Merge 10.0 into 10.1 2018-01-04 20:42:29 +02:00
Marko Mäkelä
4496fd71f4 Fix a truncation warning introduced in MDEV-12323 2018-01-04 20:39:00 +02:00
Marko Mäkelä
016caa3d20 Merge 10.0 into 10.1 2018-01-02 21:57:22 +02:00
Marko Mäkelä
51e4650ed0 Merge 5.5 into 10.0 2018-01-02 21:52:46 +02:00
Marko Mäkelä
20fab71b14 Follow-up to MDEV-14799: Remove bogus debug assertions
trx_undo_rec_get_partial_row(): When the PRIMARY KEY includes a
column prefix of an externally stored column, the already parsed
part of the undo log record may contain a reference to
an off-page column. This is the case in the bug58912 test in
innodb.innodb.
2018-01-02 21:41:39 +02:00
Marko Mäkelä
d384ead0f0 MDEV-14799 After UPDATE of indexed columns, old values will not be purged from secondary indexes
This is a regression caused by MDEV-14051 'Undo log record is too big.'

Purge in the secondary index is wrongly skipped in
row_purge_upd_exist_or_extern() because node->row only does not contain all
indexed columns.

trx_undo_rec_get_partial_row(): Add the parameter for node->update
so that the updated columns will be copied from the initial part
of the undo log record.
2018-01-02 19:11:10 +02:00
Marko Mäkelä
9d7c0882fa Merge 10.0 into 10.1 2017-12-21 17:25:51 +02:00
Marko Mäkelä
0202e47274 MDEV-12827 Assertion failure when reporting duplicate key error in online table rebuild
row_log_table_apply_insert_low(), row_log_table_apply_update():
When reporting the error_key_num, only count the clustered index
if it corresponds to a key in the SQL layer.

The assertion failure was probably introduced by the (incomplete)
MySQL 5.6.28 bug fix
Bug #21364096 THE BOGUS DUPLICATE KEY ERROR IN ONLINE DDL
WITH INCORRECT KEY NAME
which we are improving.

Side note: the fix was incorrectly merged to MySQL 5.7.10;
incorrect key names will continue to be reported in MySQL 5.7.
2017-12-21 17:19:13 +02:00
Marko Mäkelä
1cf28964f5 MDEV-6247 post-fix: Re-enable some debug assertions
These assertions were disabled in MariaDB 10.1.1 in
commit df4dd593f2
with a bogus comment referring to the function wsrep_fake_trx_id()
that was introduced in the very same commit.
2017-12-21 10:19:49 +02:00
Vicențiu Ciorbaru
e3d89652e5 Merge branch '10.0' into 10.1 2017-12-20 13:30:05 +02:00
Marko Mäkelä
f7f5c710e4 Correct a function comment
The comment became stale in commit 9f57e595b4
which removed the parameter "flags".
2017-12-20 09:21:08 +02:00
Jan Lindström
ed7e4b68ed DEV-14701: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M
Problem was that crypt_data->min_key_version is not a reliable way
to detect is tablespace encrypted and could lead that in first page
of the second (page 192 and similarly for other files if more configured)
system tablespace file used key_version is replaced with zero leading
a corruption as in next startup page is though to be corrupted.
Note that crypt_data->min_key_version is updated only after all
pages from tablespace have been processed (i.e. key rotation is done)
and flushed.

fil_write_flushed_lsn
        Use crypt_data->should_encrypt() instead.
2017-12-19 16:45:10 +02:00
Marko Mäkelä
09c5bbf471 Merge 10.0 into 10.1 2017-12-18 20:05:50 +02:00
Marko Mäkelä
40088bfc7e MDEV-13407 innodb.drop_table_background failed in buildbot with "Tablespace for table exists"
The InnoDB background DROP TABLE queue is something that we should
really remove, but are unable to until we remove dict_operation_lock
so that DDL and DML operations can be combined in a single transaction.

Because the queue is not persistent, it is not crash-safe. In stable
versions of MariaDB, we can only try harder to drop all enqueued
tables before server shutdown.

row_mysql_drop_t::table_id: Replaces table_name.

row_drop_tables_for_mysql_in_background():
Do not remove the entry from the list as long as the table exists.
In this way, the table should eventually be dropped.
2017-12-18 19:51:44 +02:00
Alexander Barkov
c1e5fef05d MDEV-14008 Assertion failing: `!is_set() || (m_status == DA_OK_BULK && is_bulk_op()) 2017-12-18 11:25:38 +04:00
Marko Mäkelä
7be5b6f0e6 Merge 10.0 into 10.1 2017-12-13 23:04:30 +02:00
Marko Mäkelä
9d76b27498 Follow-up fix for MDEV-12352: Plug a memory leak
trx_rollback_active(): When aborting the rollback, free the query graph.
2017-12-13 23:02:37 +02:00
Marko Mäkelä
2fe990df9f Fix the grammar of an error message 2017-12-13 20:41:32 +02:00
Marko Mäkelä
46305b006b Merge 10.0 into 10.1 2017-12-13 19:12:16 +02:00
Marko Mäkelä
b1977a39de MDEV-12323 Rollback progress log messages during crash recovery are intermixed with unrelated log messages
trx_roll_must_shutdown(): During the rollback of recovered transactions,
report progress and check if the rollback should be interrupted because
of a pending shutdown.

trx_roll_max_undo_no, trx_roll_progress_printed_pct: Remove, along with
the messages that were interleaved with other messages.
2017-12-13 18:56:22 +02:00
Marko Mäkelä
08d0ea1fcf Follow-up to MDEV-11027: Use recv_sys_t::report() 2017-12-13 18:53:46 +02:00
Marko Mäkelä
b46fa627ca MDEV-12352 InnoDB shutdown should not be blocked by a large transaction rollback
row_undo_step(), trx_rollback_active(): Abort the rollback of a
recovered ordinary transaction if fast shutdown has been initiated.

trx_rollback_resurrected(): Convert an aborted-rollback transaction
into a fake XA PREPARE transaction, so that fast shutdown can proceed.
2017-12-13 18:02:09 +02:00
Marko Mäkelä
6559ba71a5 MDEV-13797 InnoDB may hang if shutdown is initiated soon after startup while rolling back recovered incomplete transactions
trx_rollback_resurrected(): If shutdown was initiated, fake all
remaining active transactions to XA PREPARE state, so that shutdown
can proceed. Also, make the parameter "all" an output that will be
assigned to FALSE in this case.

trx_rollback_or_clean_recovered(): Remove the shutdown check
(it was moved to trx_rollback_resurrected()).

trx_undo_free_prepared(): Relax assertions.
2017-12-13 16:30:38 +02:00
Marko Mäkelä
ce07d09fd3 Merge 10.0 into 10.1 2017-12-12 19:28:26 +02:00
Jan Lindström
e66bb57267 MDEV-12837: WSREP: BF lock wait long
This is 10.1 version where no merge error exists.

wsrep_on_check
        New check function. Galera can't be enabled
        if innodb-lock-schedule-algorithm=VATS.

innobase_kill_query
        In Galera async kill we could own lock mutex.

innobase_init
        If Variance-Aware-Transaction-Sheduling Algorithm (VATS) is
        used on Galera we refuse to start InnoDB.

Changed innodb-lock-schedule-algorithm as read-only parameter
as it was designed to be.

lock_rec_other_has_expl_req,
lock_rec_other_has_conflicting,
lock_rec_lock_slow
lock_table_other_has_incompatible
lock_rec_insert_check_and_lock

        Change pointer to conflicting lock to normal pointer as this
        pointer contents could be changed later.
2017-12-09 11:20:46 +02:00