Commit graph

19402 commits

Author SHA1 Message Date
Sergei Petrunia
c481fc9ca7 Change MyRocks maturity from Alpha to Beta 2018-01-12 15:58:34 +03:00
Sergei Petrunia
d32f5be307 MDEV-14372: Fix and enable rocksdb.information_schema test
- Make Rdb_binlog_manager::unpack_value to not have a stack overrun
  when it is reading invalid data (which it currently does as we in
  MariaDB do not store binlog coordinates under BINLOG_INFO_INDEX_NUMBER,
  see comments in MDEV-14892 for details).
- We may need to store these coordinates in the future, so instead of
  removing the call of this function, let's make it work properly for
  all possible inputs.
2018-01-12 15:58:34 +03:00
Marko Mäkelä
e9842de20c Merge 10.1 into 10.2 2018-01-11 12:05:57 +02:00
Marko Mäkelä
c15b3d2d41 Merge 10.0 into 10.1 2018-01-11 10:44:05 +02:00
Marko Mäkelä
4c1479545d Merge 5.5 into 10.0 2018-01-11 10:16:52 +02:00
Marko Mäkelä
bdcd7f79e4 MDEV-14916 InnoDB reports warning for "Purge reached the head of the history list"
The warning was originally added in
commit c67663054a
(MySQL 4.1.12, 5.0.3) to trace claimed undo log corruption that
was analyzed in https://lists.mysql.com/mysql/176250
on November 9, 2004.

Originally, the limit was 20,000 undo log headers or transactions,
but in commit 9d6d1902e0
in MySQL 5.5.11 it was increased to 2,000,000.

The message can be triggered when the progress of purge is prevented
by a long-running transaction (or just an idle transaction whose
read view was started a long time ago), by running many transactions
that UPDATE or DELETE some records, then starting another transaction
with a read view, and finally by executing more than 2,000,000
transactions that UPDATE or DELETE records in InnoDB tables. Finally,
when the oldest long-running transaction is completed, purge would
run up to the next-oldest transaction, and there would still be more
than 2,000,000 transactions to purge.

Because the message can be triggered when the database is obviously
not corrupted, it should be removed. Heavy users of InnoDB should be
monitoring the "History list length" in SHOW ENGINE INNODB STATUS;
there is no need to spam the error log.
2018-01-11 09:55:10 +02:00
Marko Mäkelä
dfde5ae912 MDEV-14130 InnoDB messages should not refer to the MySQL 5.7 manual
Replace most occurrences of the REFMAN macro. For some pages there
is no replacement yet.
2018-01-10 13:53:44 +02:00
Marko Mäkelä
d1cf9b167c MDEV-14909 MariaDB 10.2 refuses to start up after clean shutdown of MariaDB 10.3
recv_log_recover_10_3(): Determine if a log from MariaDB 10.3 is clean.

recv_find_max_checkpoint(): Allow startup with a clean 10.3 redo log.

srv_prepare_to_delete_redo_log_files(): When starting up with a 10.3 log,
display a "Downgrading redo log" message instead of "Upgrading".
2018-01-10 13:18:02 +02:00
Marko Mäkelä
a9c55c0059 MDEV-13814 Extra logging when innodb_log_archive=ON
Backport the fix from 10.0.33 to 5.5, in case someone compiles XtraDB
with -DUNIV_LOG_ARCHIVE
2018-01-10 10:21:52 +02:00
Marko Mäkelä
a408e881cf MDEV-14174 crash on start with innodb-track-changed-pages
The XtraDB option innodb_track_changed_pages causes
the function log_group_read_log_seg() to be invoked
even when recv_sys==NULL, leading to the SIGSEGV.

This regression was caused by
MDEV-11027 InnoDB log recovery is too noisy
2018-01-10 09:27:01 +02:00
Jan Lindström
07aa985979 MDEV-14776: InnoDB Monitor output generated by specific error is flooding error logs
innodb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor.

xtradb/buf_LRU_get_free_block
	Add debug instrumentation to produce error message about
	no free pages. Print error message only once and do not
	enable innodb monitor. Remove code that does not seem to
	be used.

innodb-lru-force-no-free-page.test
	New test case to force produce desired error message.
2018-01-09 12:48:31 +02:00
Marko Mäkelä
075f61a1d4 Revert part of commit fec844aca8
row_insert_for_mysql(): Remove some duplicated code
2018-01-09 11:30:36 +02:00
Marko Mäkelä
d8eef0f611 Merge 10.1 into 10.2 2018-01-08 16:49:31 +02:00
Marko Mäkelä
29b6e809a9 Merge 10.0 into 10.1 2018-01-08 14:51:20 +02:00
Marko Mäkelä
c903ba2f1e MDEV-13205 InnoDB: Failing assertion: !dict_index_is_online_ddl(index) upon ALTER TABLE
dict_foreign_find_index(): Ignore incompletely created indexes.
After a failed ADD UNIQUE INDEX, an incompletely created index
could be left behind until the next ALTER TABLE statement.
2018-01-08 14:26:55 +02:00
Marko Mäkelä
899c5899be MLOG-13101 Debug assertion failed in recv_parse_or_apply_log_rec_body()
recv_parse_or_apply_log_rec_body(): Tolerate MLOG_4BYTES for
dummy-writing the FIL_PAGE_SPACE_ID, written by fil_crypt_rotate_page().
2018-01-08 13:00:04 +02:00
Marko Mäkelä
8099941b46 MDEV-13487 Assertion failure in rec_get_trx_id()
rec_get_trx_id(): Because rec is not necessarily residing in
a buffer pool page (it could be an old version of a clustered index
record, allocated from heap), remove the debug assertions that
depend on page_align(rec).
2018-01-08 13:00:04 +02:00
Marko Mäkelä
9ede569260 Merge 10.1 into 10.2 2018-01-08 09:54:40 +02:00
Marko Mäkelä
16d308e21d MDEV-14874 innodb_encrypt_log corrupts the log when the LSN crosses 32-bit boundary
This bug affects both writing and reading encrypted redo log in
MariaDB 10.1, starting from version 10.1.3 which added support for
innodb_encrypt_log. That is, InnoDB crash recovery and Mariabackup
will sometimes fail when innodb_encrypt_log is used.

MariaDB 10.2 or Mariabackup 10.2 or later versions are not affected.

log_block_get_start_lsn(): Remove. This function would cause trouble if
a log segment that is being read is crossing a 32-bit boundary of the LSN,
because this function does not allow the most significant 32 bits of the
LSN to change.

log_blocks_crypt(), log_encrypt_before_write(), log_decrypt_after_read():
Add the parameter "lsn" for the start LSN of the block.

log_blocks_encrypt(): Remove (unused function).
2018-01-08 09:44:40 +02:00
Marko Mäkelä
9c9db1cbe2 MDEV-14059 Work around a problem exposed by InnoDB GIS debug check
row_sel_get_clust_rec_for_mysql(): Look up the page from the
buffer pool, similar to how MySQL 5.7 does it.
2018-01-05 12:10:16 +02:00
Marko Mäkelä
c8e6364407 Merge branch 10.1 into 10.2 2018-01-04 20:47:34 +02:00
Marko Mäkelä
21470de148 Merge 10.0 into 10.1 2018-01-04 20:42:29 +02:00
Marko Mäkelä
4496fd71f4 Fix a truncation warning introduced in MDEV-12323 2018-01-04 20:39:00 +02:00
Marko Mäkelä
218dbf68b8 MDEV-14058 InnoDB Assertion failure !leaf on rem0rec.cc line 566 on test innodb_gis.rtree_recovery
The function rtr_update_mbr_field_in_place() is generating
MLOG_REC_UPDATE_IN_PLACE or MLOG_COMP_REC_UPDATE_IN_PLACE records
on non-leaf pages, even though MLOG_WRITE_STRING would perfectly
suffice for updating a fixed-length data field.

btr_cur_parse_update_in_place(): If flags==7, the record may be
from rtr_update_mbr_field_in_place(), and we must check if the
page is a leaf page. Otherwise, assume that it is.

btr_cur_update_in_place(): Assert that the page is a leaf page.
2018-01-04 19:35:53 +02:00
Monty
5e0b13d173 Fixed wrong arguments to printf and related functions
Other things, mainly to get
create_mysqld_error_find_printf_error tool to work:

- Added protection to not include mysqld_error.h twice
- Include "unireg.h" instead of "mysqld_error.h" in server
- Added protection if ER_XX messages are already defined
- Removed wrong calls to my_error(ER_OUTOFMEMORY) as
  my_malloc() and my_alloc will do this automatically
- Added missing %s to ER_DUP_QUERY_NAME
- Removed old and wrong calls to my_strerror() when using
  MY_ERROR_ON_RENAME (wrong merge)
- Fixed deadlock error message from Galera. Before the extra
  information given to ER_LOCK_DEADLOCK was missing because
  ER_LOCK_DEADLOCK doesn't provide any extra information.

I kept #ifdef mysqld_error_find_printf_error_used in sql_acl.h
to make it easy to do this kind of check again in the future
2018-01-04 16:24:09 +02:00
Marko Mäkelä
fcde91114d Merge 10.1 into 10.2 2018-01-03 20:42:09 +02:00
Marko Mäkelä
1e89c86dd7 Merge 10.0 into 10.1 2018-01-03 20:41:34 +02:00
Marko Mäkelä
8ac1982fcc Merge 5.5 into 10.0 2018-01-03 20:40:41 +02:00
Marko Mäkelä
84c9c8b2e9 Silence some -Wimplicit-fallthrough by proper spelling 2018-01-03 15:01:17 +02:00
Marko Mäkelä
d361401bc2 Merge 10.1 into 10.2, with some MDEV-14799 fixups
trx_undo_page_report_modify(): For SPATIAL INDEX, keep logging
updated off-page columns twice, so that
the minimum bounding rectangle (MBR) will be logged.
Avoiding the redundant logging would require larger changes
to the undo log format.

row_build_index_entry_low(): Handle SPATIAL_UNKNOWN more robustly,
by refusing to purge the record from the spatial index.
We can get this code when processing old undo log from 10.2.10 or
10.2.11 (the releases affected by MDEV-14799, which was a regression
from MDEV-14051).
2018-01-03 11:56:24 +02:00
Marko Mäkelä
016caa3d20 Merge 10.0 into 10.1 2018-01-02 21:57:22 +02:00
Marko Mäkelä
51e4650ed0 Merge 5.5 into 10.0 2018-01-02 21:52:46 +02:00
Varun Gupta
b2115ce235 Rocksdb fails to build when the source and build directory are not the same. This is a follow
up fix for MDEV-12458
2018-01-02 21:50:40 +02:00
Marko Mäkelä
20fab71b14 Follow-up to MDEV-14799: Remove bogus debug assertions
trx_undo_rec_get_partial_row(): When the PRIMARY KEY includes a
column prefix of an externally stored column, the already parsed
part of the undo log record may contain a reference to
an off-page column. This is the case in the bug58912 test in
innodb.innodb.
2018-01-02 21:41:39 +02:00
Marko Mäkelä
d384ead0f0 MDEV-14799 After UPDATE of indexed columns, old values will not be purged from secondary indexes
This is a regression caused by MDEV-14051 'Undo log record is too big.'

Purge in the secondary index is wrongly skipped in
row_purge_upd_exist_or_extern() because node->row only does not contain all
indexed columns.

trx_undo_rec_get_partial_row(): Add the parameter for node->update
so that the updated columns will be copied from the initial part
of the undo log record.
2018-01-02 19:11:10 +02:00
Monty
322c637c1c Fixed compiler warnings about possible uninitialized variables 2018-01-01 14:10:22 +02:00
Varun Gupta
7703095d2e MDEV-12458: Variable and log records to indicate RocksDB version are missing
Added a system variabe rocsdb_git_hash to MyRocks which tell us the version of RocksDB being used
2017-12-31 21:58:50 +02:00
Vicențiu Ciorbaru
9aeb5d01d6 Merge remote-tracking branch 'origin/10.1' into bb-10.2-vicentiu 2017-12-28 19:27:00 +02:00
Monty
14de2ad3cb Removed curr_bitmap_pos++ from possible macro 2017-12-28 11:47:03 +02:00
Sergei Golubchik
7e4c185c77 MDEV-14272 Mariadb crashes with signal 11 when using federatedx engine and galera
cherry-pick e6ce97a592
2017-12-28 02:13:25 +01:00
Sergei Golubchik
c881d82c93 cleanup: ha_myisam::data_file_name and index_file_name
don't allocate them on THD::mem_root on every init(HA_STATUS_CONST) call,
do it once in open() (because they don't change) on TABLE::mem_root
(so they stay valid until the table is closed)
2017-12-25 15:18:21 +01:00
Sergei Petrunia
068b1d0eb8 Merge branch '10.2' into bb-10.2-mariarocks 2017-12-23 13:32:54 +03:00
Vicențiu Ciorbaru
985d2d393c Merge remote-tracking branch 'origin/10.1' into 10.2 2017-12-22 12:23:39 +02:00
Sergei Petrunia
8c3a1b15e6 Fix a typo in previous commit 2017-12-22 13:22:59 +03:00
Sergei Petrunia
5426f6e30f Better comments: explain why slave_use_idempotent_for_recovery is disabled 2017-12-22 02:30:32 +03:00
Sergei Petrunia
ab0a7e13b2 More fixes in MyRocks' rocksdb_rpl testsuite
- Make my.cnf to include rpl_1slave_base.cnf (needed for tests that
  actually use replication, i.e. need a functioning slave)
- Adjust and enable singledelete_idempotent_table.test
- More edits in disabled.def
2017-12-22 02:21:27 +03:00
Sergei Petrunia
aed781ef7f Cleanup in storage/rocksdb/mysql-test/rocksdb_rpl/t/disabled.def 2017-12-22 01:13:43 +03:00
Sergei Petrunia
9c28fd7a33 Merge branch '10.2' into bb-10.2-mariarocks 2017-12-21 23:34:49 +03:00
Marko Mäkelä
9d7c0882fa Merge 10.0 into 10.1 2017-12-21 17:25:51 +02:00
Marko Mäkelä
0202e47274 MDEV-12827 Assertion failure when reporting duplicate key error in online table rebuild
row_log_table_apply_insert_low(), row_log_table_apply_update():
When reporting the error_key_num, only count the clustered index
if it corresponds to a key in the SQL layer.

The assertion failure was probably introduced by the (incomplete)
MySQL 5.6.28 bug fix
Bug #21364096 THE BOGUS DUPLICATE KEY ERROR IN ONLINE DDL
WITH INCORRECT KEY NAME
which we are improving.

Side note: the fix was incorrectly merged to MySQL 5.7.10;
incorrect key names will continue to be reported in MySQL 5.7.
2017-12-21 17:19:13 +02:00