Commit graph

11661 commits

Author SHA1 Message Date
Marko Mäkelä
0c6514eece MDEV-19781: Skip the test for embedded server 2019-07-02 21:33:54 +03:00
Marko Mäkelä
7f1e1309bb MDEV-12626: Import innodb_buffer_pool_dump_pct adjusted for MDEV-11454 2019-07-02 15:24:23 +03:00
Marko Mäkelä
6bb922e58e MDEV-13626: Import and adjust innodb.blob-crash 2019-07-02 15:18:12 +03:00
Thirunarayanan Balathandayuthapani
85d0a1955f MDEV-19914 Server startup fails while dropping garbage encrypted tablespace if innodb_encryption_threads > 0
- Avoiding accessing encryption thread mutex before initiating
the encryption threads
2019-07-01 15:21:17 +05:30
Thirunarayanan Balathandayuthapani
ed6da51f3e MDEV-19869 Port innodb_fts.fulltext2 from mysql to mariadb.
- Ported mysql Bug#20597981 test case to mariadb-10.2
- InnoDB never used fts_doc_id_in_read_set. Basically it tells
innodb to read the fts_doc_id from the index record itself.
2019-07-01 13:46:56 +05:30
Thirunarayanan Balathandayuthapani
723a4b1d78 MDEV-17228 Encrypted temporary tables are not encrypted
- Introduce a new variable called innodb_encrypt_temporary_tables which is
a boolean variable. It decides whether to encrypt the temporary tablespace.
- Encrypts the temporary tablespace based on full checksum format.
- Introduced a new counter to track encrypted and decrypted temporary
tablespace pages.
- Warnings issued if temporary table creation has conflict value with
innodb_encrypt_temporary_tables
- Added a new test case which reads and writes the pages from/to temporary
tablespace.
2019-06-28 19:07:59 +05:30
Thirunarayanan Balathandayuthapani
e4a0dbfb4a MDEV-19781 Add page id matching check in innochecksum tool
Added the condition in innochecksum tool to check page id mismatch.
This could catch the write corruption caused by InnoDB.

Added the debug insert inside fil_io() to check whether it writes
the page to wrong offset.
2019-06-28 18:58:52 +05:30
Monty
f7a4a8719b MDEV-14996 kill during FLUSH TABLES FOR EXPORT causes assert 2019-06-27 20:57:25 +03:00
Marko Mäkelä
92feac53a6 MDEV-19886 InnoDB returns misleading ER_NO_SUCH_TABLE_IN_ENGINE
A fix in MySQL 5.7.6 was not completely merged to MariaDB:
Bug#19419026 WHEN A TABLESPACE IS NOT FOUND, DO NOT REPORT "TABLE NOT FOUND"
2019-06-27 15:39:04 +03:00
Monty
7a2958f456 MDEV-17576 Assertion in maria_extra upon ALTER on table with triggers and locks
The problem was that the code in maria_extra assumed that there could be
only one table open when doing maria_extra(MA_FORCE_REOPEN)
However in the case of triggers, there can be multiple copies of
the table open.

Fixed by removing assert.
2019-06-25 16:17:29 +03:00
Marko Mäkelä
41e02d149e Remove the stray test innodb_zip.16k
This was accidentally broken in the parent commit.
2019-06-25 15:55:55 +03:00
Marko Mäkelä
a12fb5d0eb Replace innodb_zip.16k with innodb_zip.page_size
Also, move part of the test back to innodb.innodb_mysql
and another part to a new test innodb.purge.

Last but not least, merge the tests innodb_zip.4k and innodb_zip.8k
to innodb_zip.page_size.
2019-06-24 17:07:20 +03:00
Jan Lindström
c631bd7f10 Disable galera.MW-388. 2019-06-19 11:22:36 +03:00
Michael Widenius
8acbf9c1f9 MDEV-19595 fixed
The test cases for the MDEV found several independent bugs
in MariaDB server and Aria:
- If a temporary table was marked as crashed, it could never
  be deleted.
- Opening of a crashed temporary table gave an error message
  but the error was never forwarded to the caller which caused
  an assert() in my_ok()
- init_read_record() did mmap of all temporary tables, which is
  probably not a good idea as this area can potentially be
  very big. Changed code to only mmap internal temporary tables.
- mmap-ed tables where not unmapped in case of repair/optimize
  which caused bad data in table and crashes if the original
  table files where replaced with new ones (as the old mmap
  was still in place). Fixed by removing the mmap in case
  of repair.
- Cleaned up usage of code that disabled mmap in Aria
2019-06-19 00:35:44 +03:00
Michael Widenius
c8b5fa4afc MDEV-19055 Failures with temporary tables and Aria
There was two separate problems:
- Aria pagecache didn't properly handle re-reading of blocks
  that have given errors before (this triggered an assert)
- temporary tables that where opened several times where
  not properly closed in ALTER, REPAIR or OPTIMIZE table

Other things
- Added a couple of asserts that will make it easier to
  find problems like this in the future.
2019-06-17 17:50:08 +03:00
Sergei Petrunia
2b0eb352b3 Trivial test result update after fix for MDEV-19771 2019-06-16 12:12:00 +03:00
Michael Widenius
c02d6164fb MDEV-19771 REPLACE on table with virtual_field can cause crash
Fixes also MDEV-17837

Problem was that we did not ignore warnings from virtual fields when
updated virtual fields for to-be-replaced row.
2019-06-15 14:54:21 +03:00
Thirunarayanan Balathandayuthapani
e9145aab44 MDEV-19435 buf_fix_count > 0 for corrupted page when it exits the LRU list
Problem:
=========
One of the purge thread access the corrupted page and tries to remove from
LRU list. In the mean time, other purge threads are waiting for same page
in buf_wait_for_read(). Assertion(buf_fix_count == 0) fails for the
purge thread which tries to remove the page from LRU list.

Solution:
========
- Set the page id as FIL_NULL to indicate the page is corrupted before
removing the block from LRU list. Acquire hash lock for the particular
page id and wait for the other threads to release buf_fix_count
for the block.

- Added the error check for btr_cur_open() in row_search_on_row_ref().
2019-06-13 16:13:51 +03:00
Jan Lindström
371a8a6615 Galera test cleanup. 2019-06-13 13:18:54 +03:00
Marko Mäkelä
06be8cd38f Clean up the test innodb.innodb-64k-crash
Before killing the server, ensure that the incomplete state of
the transaction will be made durable and will be applied and
rolled back on recovery, so that each time, roughly the same
amount of work will be done.

Remove DML statements after the recovery, and execute
CHECK TABLE instead.
2019-06-12 19:18:47 +03:00
Marko Mäkelä
1d31bed212 Merge 10.1 into 10.2 2019-06-12 19:12:00 +03:00
Marko Mäkelä
56c60b2fc5 MDEV-16111 encryption.innodb_lotoftables failed in buildbot with wrong result
Remove the test, because it easily fails with a result difference.
Analysis by Thirunarayanan Balathandayuthapani:

By default, innodb_encrypt_tables=0.
1) Test case creates 100 tables in innodb_encrypt_1.
2) creates another 100 unencrypted tables (encryption=off) in innodb_encrypt_2
3) creates another 100 encrypted tables (encryption=on) in innodb_encrypt_3
4) enabling innodb_encrypt_tables=1 and checking that only
100 encrypted tables exist. (already we have 100 in dictionary)
5) opening all tables again (no idea why)
6) After that, set innodb_encrypt_tables=0 and wait for 100 tables
to be decrypted (already we have 100 unencrypted tables)
7) dropping all databases

Sporadic failure happens because after step 4, it could encrypt the
normal table too, because innodb_encryption_threads=4.

This test was added in MDEV-9931, which was about InnoDB startup being
slow due to all .ibd files being opened. There have been a number of
later fixes to this problem. Currently the latest one is
commit cad56fbaba, in which some tests
(in particular the test innodb.alter_kill) could fail if all InnoDB
.ibd files are read during startup. That could make this test redundant.

Let us remove the test, because it is big, slow, unreliable, and
does not seem to reliably catch the problem that all files are being
read on InnoDB startup.
2019-06-12 19:08:49 +03:00
Marko Mäkelä
4bbd8be482 Merge 10.1 into 10.2 2019-06-12 10:30:01 +03:00
Jan Lindström
34b38ad726 MDEV-19736: Galera test failure on
Remove unneeded select to provider name. Provider can have different
names and can be located on different directory on different
environments.
2019-06-12 08:24:30 +03:00
Thirunarayanan Balathandayuthapani
bb5d04c9b8 MDEV-19695 Import tablespace doesn't work with ROW_FORMAT=COMPRESSED encrypted tablespace
Problem:
=======
fil_iterate() writes imported tablespace page0 as it is to discarded
tablespace. Space id wasn't even changed. While opening the tablespace,
tablespace fails with space id mismatch error.

Fix:
====
fil_iterate() copies the page0 with discarded space id to imported
tablespace.
2019-06-06 12:54:34 +05:30
Zicheng Huang
d7c8423a3d fix MDEV-18750: failed to flashback large-size binlog file
fix MDEV-18750: failed to flashback large-size binlog file

fix mysqlbinlog flashback failure caused by reading io_cache without MY_FULL_IO flag

fix MDEV-18750: mysqlbinlog flashback failure on large binlog
2019-06-05 13:56:27 +02:00
Sergei Golubchik
aa83b9cf4f update a test result, followup fae6539ef7 2019-06-03 15:15:20 +03:00
Marko Mäkelä
1d0c27412c MDEV-19541: Suppress an error also on Windows 2019-05-29 13:03:32 +03:00
Sujatha
b347396181 MDEV-11094: Blackhole table updates on slave fail when row annotation is enabled
Problem:
=======
rpl_blackhole.test fails when executed with following options
mysqld=--binlog_annotate_row_events=1, mysqld=--replicate_annotate_row_events=1

Test output:
------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
rpl.rpl_blackhole_bug 'mix'              [ pass ]    791
rpl.rpl_blackhole_bug 'row'              [ fail ]
Replicate_Wild_Ignore_Table
Last_Errno	1032
Last_Error	Could not execute Update_rows_v1 event on table test.t1; Can't find
record in 't1', Error_code: 1032; handler error HA_ERR_END_OF_FILE; the event's
master log master-bin.000001, end_log_pos 1510

Analysis:
=========
Enabling "replicate_annotate_row_events" on slave, Tells the slave to write
annotate rows events received from the master to its own binary log. The
received annotate events are applied after the Gtid event as shown below.
thd->query() will be set to the actual query received from the master, through
annotate event. Annotate_rows event should not be deleted after the event is
applied as the thd->query will be used to generate new Annotate_rows event
during applying the subsequent Rows events. After the last Rows event has been
applied, the saved Annotate_rows event (if any) will be deleted.

In balckhole engine all the DML operations are noops as they donot store any
data. They simply return success without doing any operation. But the existing
strictly expects thd->query() to be 'NULL' to identify that row based
replication is in use. This assumption will fail when row annotations are
enabled as the query is not 'NULL'. Hence various row based operations like
'update', 'delete', 'index lookup' will fail when row annotations are enabled.

Fix:
===
Extend the row based replication check to include row annotations as well.
i.e Either the thd->query() is NULL or thd->query() points to query and row
annotations are in use.
2019-05-29 15:18:52 +05:30
Marko Mäkelä
6eefeb6fea MDEV-19541: Avoid infinite loop of reading a corrupted page
row_search_mvcc(): Duplicate the logic of btr_pcur_move_to_next()
so that an infinite loop can be avoided when advancing to the next
page fails due to a corrupted page.
2019-05-29 11:20:56 +03:00
Marko Mäkelä
eeee1832d7 Speed up buildbot by requiring --big-test for some slow tests 2019-05-29 08:28:15 +03:00
Marko Mäkelä
642ddc3131 MDEV-19541: Add a forgotten test case
Also, --skip-innodb-buffer-pool-load-at-startup to avoid a crash
in buf_load() due to loading pages that we are corrupting intentionally.
2019-05-29 08:14:49 +03:00
Marko Mäkelä
1ca75ae1c8 MDEV-19587 innodb_force_recovery=5 crash on DROP SCHEMA
At higher levels of innodb_force_recovery, the InnoDB transaction
subsystem will not be set up at all.

At slightly lower levels, recovered transactions will not be rolled back,
and DDL operations could hang due to locks being held at all.

Let us consistently refuse all writes if the predicate
high_level_read_only holds. We failed to refuse DROP TABLE
and DROP DATABASE. (Refusing DROP TABLE is a partial backport
from MDEV-19570 in the 10.5 branch.)
2019-05-28 16:43:02 +03:00
Marko Mäkelä
d59e15bdb9 Merge 10.1 into 10.2 2019-05-28 15:56:24 +03:00
Thirunarayanan Balathandayuthapani
79b46ab2a6 MDEV-19541 InnoDB crashes when trying to recover a corrupted page
- Don't apply redo log for the corrupted page when innodb_force_recovery > 0.
- Allow the table to be dropped when index root page is
corrupted when innodb_force_recovery > 0.
2019-05-28 11:55:02 +03:00
Andrei Elkin
aaf53ea0b6 MDEV-17948 Assertion `thd_killed(thd) || !m_active_tranxs ..
Simulation of a big-sized event in rpl.rpl_semi_sync_skip_repl did not clean
up after itself so screw the last binlog event offset which could jump
backwards.
The test is refined to rotate a binlog file with simulation and use the next
one for logics of the test incl master-slave synchonization.
2019-05-24 17:30:35 +03:00
Monty
3a871c39a9 Fixed monitor.test to handle statistics >= 10 2019-05-21 09:23:31 +03:00
Alexey Botchkov
71ee69c81c MDEV-17456 Malicious SUPER user can possibly change audit log configuration without leaving traces.
thread_pool_server_audit.result fixed.
2019-05-20 17:45:32 +04:00
Alexey Botchkov
d4e9a50e88 MDEV-17456 Malicious SUPER user can possibly change audit log configuration without leaving traces.
Fix for the SET GLOBAL server_audit_loggin=on; added.
2019-05-19 23:50:23 +04:00
Jan Lindström
395ce1dcb3 MDEV-16021: galera mtr test galera_evs_suspect_timeout crashed
Crash was timeout crash. Add correct waits for connections, wsrep
sync waits and auto increment offset save and restore.
2019-05-17 18:30:34 +03:00
Jan Lindström
cd87e4e134 MDEV-13549: Timeout in wait_condition.inc for PROCESSLIST
Use wsrep sync wait instead of unnecessary waits and
correct slave setting.
2019-05-17 18:30:34 +03:00
Jan Lindström
bc511443b1 MDEV-17061: Test failure on galera.galera_gcs_fc_limit
Remove unnecessary sleeps and fix wait_condition to use
wsrep_flow_control_paused i.e. we wait until flow control
pauses a transaction on master.
2019-05-17 18:30:28 +03:00
Sergei Golubchik
fae6539ef7 restore the correct test result 2019-05-17 16:56:22 +02:00
Alexey Botchkov
cd16d6d518 MDEV-13992 Implement JSON_MERGE_PATCH.
JSON_MERGE_PATCH implemented. Added JSON_MERGE_PRESERVE as a synonim for
the JSON_MERGE.
2019-05-17 11:53:58 +04:00
Jan Lindström
c84f390df2 MDEV-16021: galera mtr test galera_evs_suspect_timeout crashed
Crash was timeout crash. Add correct waits for connections, wsrep
sync waits and auto increment offset save and restore.
2019-05-17 08:29:45 +03:00
Jan Lindström
61469b3a3b MDEV-13549: Timeout in wait_condition.inc for PROCESSLIST
Use wsrep sync wait instead of unnecessary waits and
correct slave setting.
2019-05-17 08:29:45 +03:00
Jan Lindström
579c1a8c20 MDEV-17061: Test failure on galera.galera_gcs_fc_limit
Remove unnecessary sleeps and fix wait_condition to use
wsrep_flow_control_paused i.e. we wait until flow control
pauses a transaction on master.
2019-05-17 08:29:15 +03:00
Marko Mäkelä
796486d19b MDEV-19485: Add a test case
The bug was introduced in MariaDB 10.4.0 by
commit 0e5a4ac253
but it is good to have a regression test for this scenario
in all applicable MariaDB versions.

Cover the purge of an undo log record that was written before
the completion of ADD SPATIAL INDEX.
2019-05-16 14:17:22 +03:00
Sujatha
43bbf88dcb MDEV-19158: MariaDB 10.2.22 is writing duplicate entries into binary log
Problem:
========
We have a Master/Master Setup on two servers, but are only writing to one of
those servers (so it is essentially Master/Slave) We upgraded from 10.1.* to
10.2.22 last week and starting with the upgrade, we are getting duplicate key
errors on the slave. BINLOG=mixed.

Analysis:
=========
This issue happens with LOCK TABLES and binlog_format=MIXED combination. When an
UNSAFE statement is encountered in 'MIXED' mode, it is logged in the form of
'ROW' format. For all the tables that are part of LOCK TABLES list their table maps
are written into the binary log. For each table in the list a check is
done to see if 'check_table_binlog_row_based_done' flag is set or not. If it is not set
a check process is initiated to see if table qualifies for row based binary
logging or not and 'check_table_binlog_row_based_done' is set. This flag will be
cleared at the time of closing thread tables.

But there can be special cases where the LOCK TABLES contains more number of
tables but the unsafe query is actually using subset of tables from LOCK TABLES
list.

For example: LOCK TABLES locks t1,t2,t3 but the unsafe statement makes use of
only two tables t1,t3. In this case the 'check_table_binlog_row_based_done' flag
is enabled for table 't2' while writing table map, but 'close_thread_tables'
function call will not reset this flag. Since the flag is not cleared for table
't2' even a safe statement which used t2 will be logged in the form of row based
format.

This leads to an assert on debug builds and causes duplicate entries in release
builds. In release builds a statement is logged in the form of both ROW and
STATEMENT format. This causes the slave to fail with duplicate key error.

Fix:
===
During 'close_thread_tables' when LOCK TABLE modes are active "ha_reset" is done
for all the tables which were part of current statement. As mentioned in the
example 'ha_reset' is called for tables 't1' and 't3'. This will clear the
'check_table_binlog_row_based_done' flag. At this point add a check for the rest
of the tables to see if 'check_table_binlog_row_based_done' is enabled or not.
If enabled clear the flag.
2019-05-14 16:06:55 +05:30
Sujatha
d0d663f3db Merge branch '10.1' into 10.2 2019-05-14 16:05:09 +05:30