Commit graph

9673 commits

Author SHA1 Message Date
Marko Mäkelä
5044dae239 Merge 10.0 into 10.1 2017-01-10 14:30:11 +02:00
Marko Mäkelä
4b05d60e62 Make encryption.innodb_lotoftables more robust.
Perform a slow shutdown at the start of the test, and create all
InnoDB tables with STATS_PERSISTENT=0, so that any I/O related to
background tasks (change buffer merge, purge, persistent statistics)
should be eliminated.
2017-01-09 09:32:44 +02:00
Marko Mäkelä
59ea6456c3 Minor cleanup of innodb.innodb-change-buffer-recovery
This should be a non-functional change. I was unable to repeat
MDEV-11626 innodb.innodb-change-buffer-recovery fails for xtradb
and cannot determine the reason for the failure without having access
to the files.

The repeatability of MDEV-11626 should not be affected by these changes.
2017-01-09 09:28:00 +02:00
Monty
eed319b6fb MDEV-11317: ! is_set()' or !is_set() || (m_status == DA_OK_BULK && is_bulk_op())' fails in Diagnostics_area::set_ok_status on CREATE OR REPLACE with ARCHIVE table
Problem was with deleting non existing .frm file for a storage engine that
doesn't have .frm files (yet)

Fixed by not giving an error for non existing .frm files for storage engines
that are using discovery
Fixed also valgrind supression related to the given test case
2017-01-08 17:51:36 +02:00
Marko Mäkelä
384f4d1e36 Post-push fix for MDEV-11556: Make the debug variable UINT.
Sometimes innodb_data_file_size_debug was reported as INT UNSIGNED
instead of BIGINT UNSIGNED. Make it uint instead of ulong to get
a more deterministic result.
2017-01-07 15:27:59 +02:00
Marko Mäkelä
8049d2e9d9 Merge 10.0 into 10.1 2017-01-05 20:32:15 +02:00
Marko Mäkelä
ffb38c9771 MDEV-8139 Fix scrubbing tests
encryption.innodb_scrub: Clean up. Make it also cover ROW_FORMAT=COMPRESSED,
removing the need for encryption.innodb_scrub_compressed.
Add a FIXME comment saying that we should create a secondary index, to
demonstrate that also undo log pages get scrubbed. Currently that is
not working!

Also clean up encryption.innodb_scrub_background, but keep it disabled,
because the background scrubbing does not work reliably.

Fix both tests so that if something is not scrubbed, the test will be
aborted, so that the data files will be preserved. Allow the tests to
run on Windows as well.
2017-01-05 00:20:17 +02:00
Elena Stepanova
9bf92706d1 MDEV-8518 rpl.sec_behind_master-5114 fails sporadically in buildbot
- fix the test to avoid false-negatives before MDEV-5114 patch;
- fix the race condition which made the test fail on slow builders
2017-01-04 14:50:10 +02:00
Sachin Setiya
d9a1a201aa MDEV-11016 wsrep_node_is_ready() check is too strict
Problem:-
  The condition that checks for node readiness is too strict as it does
  not allow SELECTs even if these selects do not access any tables.
    For example,if we run
       SELECT 1;
    OR
       SELECT @@max_allowed_packet;
Solution:-
  We need not to report this error when all_tables(lex->query_tables)
  is NULL:
2017-01-03 10:10:58 +05:30
Sachin Setiya
d02a77bc5f MDEV-11636 Extra persistent columns on slave always gets NULL in RBR
Problem:- In replication if slave has extra persistent column then these
column are not computed while applying write-set from master.

Solution:- While applying row events from server, we will generate values
for extra persistent columns.
2017-01-01 16:45:44 +05:30
Sachin Setiya
2f5670dc26 MDEV-11636 Extra persistent columns on slave always gets NULL in RBR
Problem:- In replication if slave has extra persistent column then these
column are not computed while applying write-set from master.

Solution:- While applying row events from server, we will generate values
for extra persistent columns.
2017-01-01 11:37:30 +05:30
Marko Mäkelä
8451e09073 MDEV-11556 InnoDB redo log apply fails to adjust data file sizes
fil_space_t::recv_size: New member: recovered tablespace size in pages;
0 if no size change was read from the redo log,
or if the size change was implemented.

fil_space_set_recv_size(): New function for setting space->recv_size.

innodb_data_file_size_debug: A debug parameter for setting the system
tablespace size in recovery even when the redo log does not contain
any size changes. It is hard to write a small test case that would
cause the system tablespace to be extended at the critical moment.

recv_parse_log_rec(): Note those tablespaces whose size is being changed
by the redo log, by invoking fil_space_set_recv_size().

innobase_init(): Correct an error message, and do not require a larger
innodb_buffer_pool_size when starting up with a smaller innodb_page_size.

innobase_start_or_create_for_mysql(): Allow startup with any initial
size of the ibdata1 file if the autoextend attribute is set. Require
the minimum size of fixed-size system tablespaces to be 640 pages,
not 10 megabytes. Implement innodb_data_file_size_debug.

open_or_create_data_files(): Round the system tablespace size down
to pages, not to full megabytes, (Our test truncates the system
tablespace to more than 800 pages with innodb_page_size=4k.
InnoDB should not imagine that it was truncated to 768 pages
and then overwrite good pages in the tablespace.)

fil_flush_low(): Refactored from fil_flush().

fil_space_extend_must_retry(): Refactored from
fil_extend_space_to_desired_size().

fil_mutex_enter_and_prepare_for_io(): Extend the tablespace if
fil_space_set_recv_size() was called.

The test case has been successfully run with all the
innodb_page_size values 4k, 8k, 16k, 32k, 64k.
2016-12-30 09:52:24 +02:00
Marko Mäkelä
f493e395b0 Make the test work with any innodb_page_size. 2016-12-30 09:51:11 +02:00
Jan Lindström
283e9cf4cb MDEV-11656: 'Data structure corruption' IMPORT TABLESPACE doesn't work for encrypted InnoDB tables if space_id changed
Problem was that for encryption we use temporary scratch area for
reading and writing tablespace pages. But if page was not really
decrypted the correct updated page was not moved to scratch area
that was then written. This can happen e.g. for page 0 as it is
newer encrypted even if encryption is enabled and as we write
the contents of old page 0 to tablespace it contained naturally
incorrect space_id that is then later noted and error message
was written. Updated page with correct space_id was lost.

If tablespace is encrypted we use additional
temporary scratch area where pages are read
for decrypting readptr == crypt_io_buffer != io_buffer.

Destination for decryption is a buffer pool block
block->frame == dst == io_buffer that is updated.
Pages that did not require decryption even when
tablespace is marked as encrypted are not copied
instead block->frame is set to src == readptr.

If tablespace was encrypted we copy updated page to
writeptr != io_buffer. This fixes above bug.

For encryption we again use temporary scratch area
writeptr != io_buffer == dst
that is then written to the tablespace

(1) For normal tables src == dst ==  writeptr
ut_ad(!encrypted && !page_compressed ?
	src == dst && dst == writeptr + (i * size):1);
(2) For page compressed tables src == dst == writeptr
ut_ad(page_compressed && !encrypted ?
	src == dst && dst == writeptr + (i * size):1);
(3) For encrypted tables src != dst != writeptr
ut_ad(encrypted ?
	src != dst && dst != writeptr + (i * size):1);
2016-12-28 16:32:45 +02:00
Jan Lindström
55eb7120a0 MDEV-11218: encryption.innodb_encryption_discard_import failed in buildbot
Try to stabilize test cases. These test behave badly when run in certain order.
2016-12-22 14:02:51 +02:00
Sachin Setiya
9e032d6150 MDEV-11490 Galera_3nodes test suite does not suppress Warnings.
Problem:- While running individual tests of Galera_3nodes ,
  We get warnings like '[Warning] WSREP: Could not open state file
  for reading: '. And because of this individual tests fails.

  Solution:- We change suite.pm of Galera_3nodes to supress these warnings.
2016-12-21 09:37:58 +05:30
Nirbhay Choubey
75ab65aece Fix failing galera tests. 2016-12-20 15:31:18 -05:00
Marko Mäkelä
195241e125 Port the test innodb.doublewrite from MySQL 5.7. 2016-12-20 15:03:56 +02:00
Sachin Setiya
d93bbcad36 MDEV-11479 Improved wsrep_dirty_reads
Updated sysvars_wsrep.result file.
2016-12-14 20:13:36 +05:30
Sachin Setiya
0c79de2419 MDEV-11479 Improved wsrep_dirty_reads
Tasks:-
         Changes in wsrep_dirty_reads variable
         1.) Global + Session scope (Current: session-only)
         2.) Can be set using command line.
         3.) Allow all commands that do not change data (besides SELECT)
         4.) Allow prepared Statements that do not change data
         5.) Works with wsrep_sync_wait enabled
2016-12-14 09:30:43 +05:30
Sachin Setiya
25a9a3da34 Revert "MDEV-11016 wsrep_node_is_ready() check is too strict"
This reverts commit 7ed5563bbe.
2016-12-14 08:39:36 +05:30
Sergei Golubchik
2f20d297f8 Merge branch '10.0' into 10.1 2016-12-11 09:53:42 +01:00
Vicențiu Ciorbaru
eb4f2e063c MDEV-11533: Roles with trailing white spaces are not cleared correctly
Role names with trailing whitespaces are truncated in length as of
956e92d908 to fix MDEV-8609. The problem
is that the code that creates role mappings expects the string to be null
terminated.

Add the null terminator to account for that as well. In the future
the rest of the code can be cleaned up to never assume c style strings
but only LEX_STRINGS.
2016-12-10 22:23:25 +02:00
Sergei Golubchik
3e8155c637 Merge branch '5.5' into 10.0 2016-12-09 16:33:48 +01:00
Elena Stepanova
8e702bce66 MDEV-11504 binlog_encryption.encrypted_master_switch_to_unencrypted fails sporadically in buildbot
The reason is a simple race condition. Initially the test was meant to synchronize with master
before showing tables, but it turned out that the slave IO thread should fail by this point,
and synchronization was removed along with a server bugfix. Now added an intermediate sync
instead, to make sure that slave has replicated events before the point of failure
2016-12-08 17:05:01 +02:00
Alexey Botchkov
1d702ff07c MDEV-8329 MariaDB crashes when replicate_wild_ignore_table is set to NULL.
Rpl_filter::parse_filter_rule() made NULL-safe.
2016-12-07 14:42:08 +04:00
Sergei Golubchik
74d52ded72 fix binlog_encryption.binlog_incident test 2016-12-06 09:45:51 +01:00
Sergei Golubchik
b5aa0f437f MDEV-11319 mysqlbinlog crashes or fails with out of memory while reading some encrypted binlogs
support encrypted binlogs. Not decryption, but at least recognizing
that event are encrypted and prining them as such
2016-12-06 09:45:50 +01:00
Sergei Golubchik
952856c810 MDEV-11288 Server crashes in Binlog_crypt_data::init trying to feed encrypted log without decryption capabilities 2016-12-06 09:45:50 +01:00
Elena Stepanova
611f91605a MDEV-9038 Binlog encryption tests
- created binlog_encryption test suite and added it to the default list
- moved some tests from rpl, binlog and multisource suites to extra
  so that they could be re-used in different suites
- made minor changes in include files
2016-12-05 20:19:01 +02:00
Marko Mäkelä
9199d72759 MDEV-11233 CREATE FULLTEXT INDEX with a token longer than 127 bytes
crashes server

This bug is the result of merging the Oracle MySQL follow-up fix
BUG#22963169 MYSQL CRASHES ON CREATE FULLTEXT INDEX
without merging the base bug fix:
Bug#79475 Insert a token of 84 4-bytes chars into fts index causes
server crash.

Unlike the above mentioned fixes in MySQL, our fix will not change
the storage format of fulltext indexes in InnoDB or XtraDB
when a character encoding with mbmaxlen=2 or mbmaxlen=3
and the length of a word is between 128 and 84*mbmaxlen bytes.
The Oracle fix would allocate 2 length bytes for these cases.

Compatibility with other MySQL and MariaDB releases is ensured by
persisting the used maximum length in the SYS_COLUMNS table in the
InnoDB data dictionary.

This fix also removes some unnecessary strcmp() calls when checking
for the legacy default collation my_charset_latin1
(my_charset_latin1.name=="latin1_swedish_ci").

fts_create_one_index_table(): Store the actual length in bytes.
This metadata will be written to the SYS_COLUMNS table.

fts_zip_initialize(): Initialize only the first byte of the buffer.
Actually the code should not even care about this first byte, because
the length is set as 0.

FTX_MAX_WORD_LEN: Define as HA_FT_MAXCHARLEN * 4 aka 336 bytes,
not as 254 bytes.

row_merge_create_fts_sort_index(): Set the actual maximum length of the
column in bytes, similar to fts_create_one_index_table().

row_merge_fts_doc_tokenize(): Remove the redundant parameter word_dtype.
Use the actual maximum length of the column. Calculate the extra_size
in the same way as row_merge_buf_encode() does.
2016-12-05 15:25:59 +02:00
Elena Stepanova
ead6d0de02 Follow-up for MDEV-9451 - fix XtraDB rdiff files 2016-12-05 03:13:28 +02:00
Elena Stepanova
b0754ad45e Follow-up for MDEV-11429 - fix result files for embedded and 32-bit tests 2016-12-05 03:11:42 +02:00
Sergei Golubchik
7f2fd34500 MDEV-11231 Server crashes in check_duplicate_key on CREATE TABLE ... SELECT
be consistent and don't include the table name into the error message,
no other CREATE TABLE error does it.

(the crash happened, because thd->lex->query_tables was NULL)
2016-12-04 01:59:35 +01:00
Sergei Golubchik
c5ef621599 Merge branch 'merge/merge-tokudb-5.6' into 10.0 2016-12-04 01:59:08 +01:00
Sergei Golubchik
e4a0d75a0b import a test case from percona-server-5.6.34-79.1 2016-12-04 01:35:57 +01:00
Sergei Golubchik
f640527e65 typo fixed: s/MSYQL/MYSQL/ 2016-12-03 22:02:00 +01:00
Marko Mäkelä
f1b80d8ef1 MDEV-11236 Failing assertion: state == TRX_STATE_NOT_STARTED
trx_state_eq(): Add the parameter bool relaxed=false, to
allow trx->state==TRX_STATE_NOT_STARTED where a different
state is expected, if an error has been reported.

trx_release_savepoint_for_mysql(): Pass relaxed=true to
trx_state_eq(). That is, allow the transaction to be idle
when ROLLBACK TO SAVEPOINT is attempted after an error
has been reported to the client.
2016-12-02 16:25:47 +02:00
Monty
97b21a1953 MDEV-10759 Fix Aria to support 2-byte collation IDs
- Used same fix as for MyISAM: High level collation byte stored in unused
  bit_end position.
- Moved language from header to base_info
- Removed unused bit_end part in HA_KEY_SEG
2016-12-02 14:05:16 +02:00
Monty
2996f9aa88 MDEV-11429 Increase number of max table_open_cache instances
Increase max number of possible table_open_cache instances from 512K
to 1024K. This only affects user who are trying to set the variable over
the old limit.

Delete not used test table_open_cache_instances_basic
(Need to be added back and rewritten in 10.2)
2016-12-02 14:05:16 +02:00
sensssz
2fd3af4483 MDEV-11168: InnoDB: Failing assertion: !other_lock || wsrep_thd_is_BF(lock->trx->mysql_thd, FALSE) || wsrep_thd_is_BF(other_lock->trx->mysql_thd, FALSE)
Merged pull request:
Fix error in lock_has_higher_priority #266
https://github.com/MariaDB/server/pull/266

Added test case.
2016-12-02 11:08:18 +02:00
Vicențiu Ciorbaru
e99990c631 MDEV-10744: Roles are not fully case sensitive
Due to the collation used on the roles_mapping_hash, key comparison
would work in a case-insensitive manner. This is incorrect from the
roles mapping perspective. Make use of a case-sensitive collation for that hash,
the same one used for the acl_roles hash.
2016-12-01 15:01:33 +01:00
Jan Lindström
b209bc3eed MDEV-10427: innodb.innodb-wl5522-debug-zip fails sporadically in buildbot
Test intentionally crashes the server, thus corrupted pages possible.
2016-11-29 09:01:46 +02:00
Jan Lindström
dbdef41a59 MDEV-10686: innodb_zip.innodb_prefix_index_liftedlimit failed with timeout in buildbot
Test moved to big_test and not run with valgrind because of timeout.
2016-11-29 08:41:45 +02:00
Jan Lindström
9f31949b64 MDEV-10739: encryption.innodb-page_encryption_compression fails with timeout on valgrind
Test moved to big_test and not run on valgrind. Test heavy especially on debug builds.
2016-11-29 08:35:51 +02:00
SachinSetiya
ee3c99dfc7 Merge branch 'bb-mdev-11016' into 10.1 2016-11-22 16:17:05 +05:30
Jan Lindström
af05becb4c MDEV-10771: Test innodb_defragment_fill_factor does not work correctly
Added more columns to stabilice the test case.
2016-11-16 13:52:41 +02:00
Kristian Nielsen
bccd0b5e0e Merge branch 'mdev10863' into 10.1 2016-11-15 13:10:21 +01:00
Kristian Nielsen
717f212840 MDEV-10863: parallel replication tries to continue from wrong position
This occured when the SQL thread (but not the IO thread) stops while
GTID and parallel replication are used with multiple domain ids in the
GTID position, and is restarted.

In this case, the SQL needs to start some way back in the relay log,
applying or skipping events within each replication domain as
appropriate.

The SQL threads starts at the beginning of an old relay log file, and
this position may be in the middle of an event group. The bug was that
such partial event group could be re-applied, causing replication
corruption.

This patch fixes the issue, by making sure to skip any initial events
that were part of an earlier (already applied) event group.
2016-11-04 12:33:42 +01:00
Nirbhay Choubey
cbfb3f9cb1 Move disabled galera tests to galera/disabled.def 2016-11-03 19:30:02 -04:00