Commit graph

1750 commits

Author SHA1 Message Date
Marko Mäkelä
e9842de20c Merge 10.1 into 10.2 2018-01-11 12:05:57 +02:00
Marko Mäkelä
c15b3d2d41 Merge 10.0 into 10.1 2018-01-11 10:44:05 +02:00
Marko Mäkelä
4c1479545d Merge 5.5 into 10.0 2018-01-11 10:16:52 +02:00
Marko Mäkelä
bdcd7f79e4 MDEV-14916 InnoDB reports warning for "Purge reached the head of the history list"
The warning was originally added in
commit c67663054a
(MySQL 4.1.12, 5.0.3) to trace claimed undo log corruption that
was analyzed in https://lists.mysql.com/mysql/176250
on November 9, 2004.

Originally, the limit was 20,000 undo log headers or transactions,
but in commit 9d6d1902e0
in MySQL 5.5.11 it was increased to 2,000,000.

The message can be triggered when the progress of purge is prevented
by a long-running transaction (or just an idle transaction whose
read view was started a long time ago), by running many transactions
that UPDATE or DELETE some records, then starting another transaction
with a read view, and finally by executing more than 2,000,000
transactions that UPDATE or DELETE records in InnoDB tables. Finally,
when the oldest long-running transaction is completed, purge would
run up to the next-oldest transaction, and there would still be more
than 2,000,000 transactions to purge.

Because the message can be triggered when the database is obviously
not corrupted, it should be removed. Heavy users of InnoDB should be
monitoring the "History list length" in SHOW ENGINE INNODB STATUS;
there is no need to spam the error log.
2018-01-11 09:55:10 +02:00
Marko Mäkelä
a9c55c0059 MDEV-13814 Extra logging when innodb_log_archive=ON
Backport the fix from 10.0.33 to 5.5, in case someone compiles XtraDB
with -DUNIV_LOG_ARCHIVE
2018-01-10 10:21:52 +02:00
Marko Mäkelä
a408e881cf MDEV-14174 crash on start with innodb-track-changed-pages
The XtraDB option innodb_track_changed_pages causes
the function log_group_read_log_seg() to be invoked
even when recv_sys==NULL, leading to the SIGSEGV.

This regression was caused by
MDEV-11027 InnoDB log recovery is too noisy
2018-01-10 09:27:01 +02:00
Jan Lindström
07aa985979 MDEV-14776: InnoDB Monitor output generated by specific error is flooding error logs
innodb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor.

xtradb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor. Remove code that does not seem to
	be used.

innodb-lru-force-no-free-page.test
	New test case to force produce desired error message.
2018-01-09 12:48:31 +02:00
Marko Mäkelä
d8eef0f611 Merge 10.1 into 10.2 2018-01-08 16:49:31 +02:00
Marko Mäkelä
29b6e809a9 Merge 10.0 into 10.1 2018-01-08 14:51:20 +02:00
Marko Mäkelä
c903ba2f1e MDEV-13205 InnoDB: Failing assertion: !dict_index_is_online_ddl(index) upon ALTER TABLE
dict_foreign_find_index(): Ignore incompletely created indexes.
After a failed ADD UNIQUE INDEX, an incompletely created index
could be left behind until the next ALTER TABLE statement.
2018-01-08 14:26:55 +02:00
Marko Mäkelä
9ede569260 Merge 10.1 into 10.2 2018-01-08 09:54:40 +02:00
Marko Mäkelä
16d308e21d MDEV-14874 innodb_encrypt_log corrupts the log when the LSN crosses 32-bit boundary
This bug affects both writing and reading encrypted redo log in
MariaDB 10.1, starting from version 10.1.3 which added support for
innodb_encrypt_log. That is, InnoDB crash recovery and Mariabackup
will sometimes fail when innodb_encrypt_log is used.

MariaDB 10.2 or Mariabackup 10.2 or later versions are not affected.

log_block_get_start_lsn(): Remove. This function would cause trouble if
a log segment that is being read is crossing a 32-bit boundary of the LSN,
because this function does not allow the most significant 32 bits of the
LSN to change.

log_blocks_crypt(), log_encrypt_before_write(), log_decrypt_after_read():
Add the parameter "lsn" for the start LSN of the block.

log_blocks_encrypt(): Remove (unused function).
2018-01-08 09:44:40 +02:00
Marko Mäkelä
c8e6364407 Merge branch 10.1 into 10.2 2018-01-04 20:47:34 +02:00
Marko Mäkelä
21470de148 Merge 10.0 into 10.1 2018-01-04 20:42:29 +02:00
Marko Mäkelä
4496fd71f4 Fix a truncation warning introduced in MDEV-12323 2018-01-04 20:39:00 +02:00
Marko Mäkelä
d361401bc2 Merge 10.1 into 10.2, with some MDEV-14799 fixups
trx_undo_page_report_modify(): For SPATIAL INDEX, keep logging
updated off-page columns twice, so that
the minimum bounding rectangle (MBR) will be logged.
Avoiding the redundant logging would require larger changes
to the undo log format.

row_build_index_entry_low(): Handle SPATIAL_UNKNOWN more robustly,
by refusing to purge the record from the spatial index.
We can get this code when processing old undo log from 10.2.10 or
10.2.11 (the releases affected by MDEV-14799, which was a regression
from MDEV-14051).
2018-01-03 11:56:24 +02:00
Marko Mäkelä
016caa3d20 Merge 10.0 into 10.1 2018-01-02 21:57:22 +02:00
Marko Mäkelä
51e4650ed0 Merge 5.5 into 10.0 2018-01-02 21:52:46 +02:00
Marko Mäkelä
20fab71b14 Follow-up to MDEV-14799: Remove bogus debug assertions
trx_undo_rec_get_partial_row(): When the PRIMARY KEY includes a
column prefix of an externally stored column, the already parsed
part of the undo log record may contain a reference to
an off-page column. This is the case in the bug58912 test in
innodb.innodb.
2018-01-02 21:41:39 +02:00
Marko Mäkelä
d384ead0f0 MDEV-14799 After UPDATE of indexed columns, old values will not be purged from secondary indexes
This is a regression caused by MDEV-14051 'Undo log record is too big.'

Purge in the secondary index is wrongly skipped in
row_purge_upd_exist_or_extern() because node->row only does not contain all
indexed columns.

trx_undo_rec_get_partial_row(): Add the parameter for node->update
so that the updated columns will be copied from the initial part
of the undo log record.
2018-01-02 19:11:10 +02:00
Vicențiu Ciorbaru
985d2d393c Merge remote-tracking branch 'origin/10.1' into 10.2 2017-12-22 12:23:39 +02:00
Marko Mäkelä
9d7c0882fa Merge 10.0 into 10.1 2017-12-21 17:25:51 +02:00
Marko Mäkelä
0202e47274 MDEV-12827 Assertion failure when reporting duplicate key error in online table rebuild
row_log_table_apply_insert_low(), row_log_table_apply_update():
When reporting the error_key_num, only count the clustered index
if it corresponds to a key in the SQL layer.

The assertion failure was probably introduced by the (incomplete)
MySQL 5.6.28 bug fix
Bug #21364096 THE BOGUS DUPLICATE KEY ERROR IN ONLINE DDL
WITH INCORRECT KEY NAME
which we are improving.

Side note: the fix was incorrectly merged to MySQL 5.7.10;
incorrect key names will continue to be reported in MySQL 5.7.
2017-12-21 17:19:13 +02:00
Marko Mäkelä
1cf28964f5 MDEV-6247 post-fix: Re-enable some debug assertions
These assertions were disabled in MariaDB 10.1.1 in
commit df4dd593f2
with a bogus comment referring to the function wsrep_fake_trx_id()
that was introduced in the very same commit.
2017-12-21 10:19:49 +02:00
Vicențiu Ciorbaru
e3d89652e5 Merge branch '10.0' into 10.1 2017-12-20 13:30:05 +02:00
Marko Mäkelä
f7f5c710e4 Correct a function comment
The comment became stale in commit 9f57e595b4
which removed the parameter "flags".
2017-12-20 09:21:08 +02:00
Marko Mäkelä
8d70097c21 Merge 10.1 to 10.2
Follow-up fix to MDEV-14008: Let Field_double::val_uint() silently
return 0 on error
2017-12-19 16:48:28 +02:00
Jan Lindström
ed7e4b68ed DEV-14701: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M
Problem was that crypt_data->min_key_version is not a reliable way
to detect is tablespace encrypted and could lead that in first page
of the second (page 192 and similarly for other files if more configured)
system tablespace file used key_version is replaced with zero leading
a corruption as in next startup page is though to be corrupted.
Note that crypt_data->min_key_version is updated only after all
pages from tablespace have been processed (i.e. key rotation is done)
and flushed.

fil_write_flushed_lsn
        Use crypt_data->should_encrypt() instead.
2017-12-19 16:45:10 +02:00
Marko Mäkelä
09c5bbf471 Merge 10.0 into 10.1 2017-12-18 20:05:50 +02:00
Marko Mäkelä
40088bfc7e MDEV-13407 innodb.drop_table_background failed in buildbot with "Tablespace for table exists"
The InnoDB background DROP TABLE queue is something that we should
really remove, but are unable to until we remove dict_operation_lock
so that DDL and DML operations can be combined in a single transaction.

Because the queue is not persistent, it is not crash-safe. In stable
versions of MariaDB, we can only try harder to drop all enqueued
tables before server shutdown.

row_mysql_drop_t::table_id: Replaces table_name.

row_drop_tables_for_mysql_in_background():
Do not remove the entry from the list as long as the table exists.
In this way, the table should eventually be dropped.
2017-12-18 19:51:44 +02:00
Alexander Barkov
c1e5fef05d MDEV-14008 Assertion failing: `!is_set() || (m_status == DA_OK_BULK && is_bulk_op()) 2017-12-18 11:25:38 +04:00
Marko Mäkelä
ece9c54e10 Merge 10.1 into 10.2 2017-12-14 08:40:01 +02:00
Marko Mäkelä
7be5b6f0e6 Merge 10.0 into 10.1 2017-12-13 23:04:30 +02:00
Marko Mäkelä
9d76b27498 Follow-up fix for MDEV-12352: Plug a memory leak
trx_rollback_active(): When aborting the rollback, free the query graph.
2017-12-13 23:02:37 +02:00
Marko Mäkelä
2fe990df9f Fix the grammar of an error message 2017-12-13 20:41:32 +02:00
Marko Mäkelä
46305b006b Merge 10.0 into 10.1 2017-12-13 19:12:16 +02:00
Marko Mäkelä
b1977a39de MDEV-12323 Rollback progress log messages during crash recovery are intermixed with unrelated log messages
trx_roll_must_shutdown(): During the rollback of recovered transactions,
report progress and check if the rollback should be interrupted because
of a pending shutdown.

trx_roll_max_undo_no, trx_roll_progress_printed_pct: Remove, along with
the messages that were interleaved with other messages.
2017-12-13 18:56:22 +02:00
Marko Mäkelä
08d0ea1fcf Follow-up to MDEV-11027: Use recv_sys_t::report() 2017-12-13 18:53:46 +02:00
Marko Mäkelä
b46fa627ca MDEV-12352 InnoDB shutdown should not be blocked by a large transaction rollback
row_undo_step(), trx_rollback_active(): Abort the rollback of a
recovered ordinary transaction if fast shutdown has been initiated.

trx_rollback_resurrected(): Convert an aborted-rollback transaction
into a fake XA PREPARE transaction, so that fast shutdown can proceed.
2017-12-13 18:02:09 +02:00
Marko Mäkelä
6559ba71a5 MDEV-13797 InnoDB may hang if shutdown is initiated soon after startup while rolling back recovered incomplete transactions
trx_rollback_resurrected(): If shutdown was initiated, fake all
remaining active transactions to XA PREPARE state, so that shutdown
can proceed. Also, make the parameter "all" an output that will be
assigned to FALSE in this case.

trx_rollback_or_clean_recovered(): Remove the shutdown check
(it was moved to trx_rollback_resurrected()).

trx_undo_free_prepared(): Relax assertions.
2017-12-13 16:30:38 +02:00
Marko Mäkelä
ce07d09fd3 Merge 10.0 into 10.1 2017-12-12 19:28:26 +02:00
Jan Lindström
e66bb57267 MDEV-12837: WSREP: BF lock wait long
This is 10.1 version where no merge error exists.

wsrep_on_check
        New check function. Galera can't be enabled
        if innodb-lock-schedule-algorithm=VATS.

innobase_kill_query
        In Galera async kill we could own lock mutex.

innobase_init
        If Variance-Aware-Transaction-Sheduling Algorithm (VATS) is
        used on Galera we refuse to start InnoDB.

Changed innodb-lock-schedule-algorithm as read-only parameter
as it was designed to be.

lock_rec_other_has_expl_req,
lock_rec_other_has_conflicting,
lock_rec_lock_slow
lock_table_other_has_incompatible
lock_rec_insert_check_and_lock

        Change pointer to conflicting lock to normal pointer as this
        pointer contents could be changed later.
2017-12-09 11:20:46 +02:00
Marko Mäkelä
51c73a431f Merge 10.1 into 10.2 2017-12-07 08:17:50 +02:00
Marko Mäkelä
447931c6ab Post-fix for MDEV-14587
dict_stats_process_entry_from_defrag_pool(): Release the mutex
2017-12-07 08:14:49 +02:00
Marko Mäkelä
1d526f31fb Merge 10.1 into 10.2 2017-12-05 14:23:57 +02:00
Marko Mäkelä
63cbb98275 MDEV-14587 dict_stats_process_entry_from_defrag_pool() fails to call dict_table_close() when index==NULL
dict_stats_process_entry_from_defrag_pool(): Simplify the logic,
and always call dict_table_close() when dict_table_open() returned
a non-NULL handle.
2017-12-05 13:25:09 +02:00
Vladislav Vaintroub
9cefffdab1 Merge remote-tracking branch 'origin/10.1' into 10.2
# Conflicts:
#	storage/innobase/include/os0file.h
2017-11-24 17:30:26 +00:00
Vladislav Vaintroub
40756c9151 Fix Windows build with -DPLUGIN_PERFSCHEMA=NO 2017-11-24 16:55:20 +00:00
Marko Mäkelä
f1cc6e3874 Merge 10.1 into 10.2 2017-11-24 17:17:16 +02:00
wlad
b6d72ed44d MDEV-14283 : Fix Solaris 10 build.
- introduce system check for posix_memalign (not available on Solaris 10)
- Disable dtrace probes, to fix weird link errors in mariabackup
2017-11-21 21:14:06 +01:00