Commit graph

177637 commits

Author SHA1 Message Date
Vladislav Vaintroub
e292d1a800 Avoid noisy Clang 7 warning about unused variable.
Patch by Eugene Kosov.
2019-01-19 14:01:09 +01:00
Varun Gupta
2061e00c20 MDEV-14440: Assertion `inited==RND' failed in handler::ha_rnd_end
In the function QUICK_RANGE_SELECT::init_ror_merged_scan we create a seperate handler if the handler in
head->file cannot be reused. The flag free_file tells us if we have a seperate handler or not.
There are cases where you might create a handler and then there might be a failure(running ALTER)
and then we have to revert the handler back to the original one. The code does that
but it does not reset the flag 'free_file' in this case.
Also backported f2c418079d.
2019-01-18 23:36:47 +05:30
Marko Mäkelä
1d72db45a8 MDEV-18237 InnoDB: Unable to drop FTS index aux table and further errors (possibly bogus)
row_drop_table_for_mysql(): Fix a regression introduced in MDEV-16515.
Similar to the follow-up fixes MDEV-16647 and MDEV-17470, we must make
the internal tables of FULLTEXT INDEX immune to kills, to avoid noise
and resource leakage on DROP TABLE or ALTER TABLE. (Orphan internal tables
would be dropped at the next InnoDB startup only.)
2019-01-18 06:46:39 +02:00
Marko Mäkelä
c1aae37087 Re-record results for the changed max_value of table_definition_cache 2019-01-17 10:11:02 +02:00
Thirunarayanan Balathandayuthapani
ad376a05fa MDEV-18279 MLOG_FILE_WRITE_CRYPT_DATA fails to set the crypt_data->type for tablespace.
Problem:
========
MLOG_FILE_WRITE_CRYPT_DATA redo log fails to apply type for
the crypt_data present in the space. While processing the double-write
buffer pages, page fails to decrypt. It leads to warning message.

Fix:
====
Set the type while parsing MLOG_FILE_WRITE_CRYPT_DATA redo log.
If type and length is of invalid type then mark it as corrupted.
2019-01-17 13:10:45 +05:30
Marko Mäkelä
a84be48e00 Update ,32bit.rdiff 2019-01-17 09:39:20 +02:00
Marko Mäkelä
71eb762611 Merge 10.0 into 10.1 2019-01-17 06:40:24 +02:00
Daniel Black
19a7656fb1 safemalloc: warn, flush after fprintf
Corrects 94d722b6a4
2019-01-17 01:13:47 +04:00
Alexander Kuleshov
78f62e9079 remove duplicated paragraph from mysql_install_db.sh
Signed-off-by: Alexander Kuleshov <kuleshovmail@gmail.com>
2019-01-16 23:55:59 +04:00
Natanael Copa
459d6da869 MDEV-18269 - fix off-by-one bug in unittest
Fix the off-by-one overflow which was introduced with commit
b0fd06a6f2 (MDEV-15670 - unit.my_atomic failed in buildbot with
Signal 11 thrown)

Closes #1098.
2019-01-16 23:41:57 +04:00
Thirunarayanan Balathandayuthapani
038785e1f8 MDEV-18183 Server startup fails while dropping garbage encrypted tablespace
- There is no need to wait for crypt thread to stop accessing space while
dropping the garbage encrypted tablespace during recover.
2019-01-16 23:02:39 +05:30
Oleksandr Byelkin
db469b6907 MDEV-17475: Increase maximum possible value for table_definition_cache to match table_open_cache
Allow table definition cache be bigger than open table cache (due to problem with VIEWs and prepared statements).
2019-01-16 15:01:52 +01:00
Oleksandr Byelkin
1ecccb509c MDEV-17085: CHECKSUM TABLE EXTENDED does not work correctly
The problem was in calculating of the mask to clear unused null bits in case of using full byte.
2019-01-16 13:57:22 +01:00
Vladislav Vaintroub
235374aee3 MDEV-18254 upgrade HeidiSQL to 9.5 2019-01-15 18:44:03 +01:00
Marko Mäkelä
12f362c333 MDEV-18233 Moving the hash_node_t to improve locality of reference 2019-01-15 14:53:27 +02:00
Eugene Kosov
e0633f25e8 MDEV-18243 incorrect ASAN instrumentation
Poisoning memory after munmap() and friends is totally incorrect as this memory
could be anything.

os_mem_free_large(): remove memory poisoning
2019-01-15 15:32:18 +03:00
mkaruza
69328c31ed MDEV-18176 Galera test failure on galera.galera_gtid_slave_sst_rsync
If galera.galera_gtid_slave_sst_rsync is repeated more than once it will fail due incorrect GTID position. After stopping SLAVE node reset also GTID_SLAVE_POS variable.
2019-01-15 13:29:11 +02:00
Marko Mäkelä
71e9f0d123 MDEV-17797 Add ASAN poisoning for mem_heap_t
The merge commit d833bb65d5 did not
correctly merge the commit 03eb15933d.

Closes #948
2019-01-15 12:12:59 +02:00
Vladislav Vaintroub
7d3161def8 MDEV-18225 Avoid use of LOCK_prepared_stmt_count mutex in Statement_map destructo
This mutex can be freed when server shuts down (when thread_count goes down to 0)
, but it is still used inside THD::~THD() when Statement_map is destroyed.

The fix is to call Statement_map::reset() at the point where thread_count
is still positive, and avoid locking LOCK_prepared_stmt_count in THD
destructor.
2019-01-15 10:28:00 +01:00
Eugene Kosov
a06a3e4670 MDEV-18233 Moving the hash_node_t to improve locality of reference
When performing a hash search via HASH_SEARCH we first look at a key of a node
and then at its pointer to the next node in chain. If we have those in one cache
line instead of a two we reduce memory reads.

I found dict_table_t, fil_space_t and buf_page_t suitable for such improvement.
2019-01-14 22:14:56 +03:00
Marko Mäkelä
b4c471099d
Merge pull request #973 from tempesta-tech/tt-10.0-MDEV-16499-virtual-innodb
MDEV-16499 ER_NO_SUCH_TABLE_IN_ENGINE followed by "Please drop the table and recreate" upon adding FULLTEXT key to table with virtual column
2019-01-14 14:06:26 +02:00
FaramosCZ
7372fe4da1 xb_process_datadir(): Fix resource leaks
Closes #983, #984
2019-01-14 11:02:04 +02:00
Vladislav Vaintroub
dc42b3c4d9 Backport MDEV-17504 to 5.5
mysql_install_db.exe should not remove datadir, if it was not created by
it.
2019-01-11 01:45:09 +01:00
Jan Lindström
1d56d875fe MDEV-15740: InnoDB does not flush redo log when it shoul
During database recovery, a transaction with wsrep XID is
recovered from InnoDB in prepared state. However, when the
transaction is looked up with trx_get_trx_by_xid() in
innobase_commit_by_xid(), trx->xid gets cleared in
trx_get_trx_by_xid_low() and commit time serialization history
write does not update the wsrep XID in trx sys header for
that recovered trx. As a result the transaction gets
committed during recovery but the wsrep position does not
get updated appropriately.

As a fix, we preserve trx->xid for Galera over transaction
commit in recovery phase.

Fix authored by: Teemu Ollakka (GaleraCluster) and Marko Mäkelä.

	modified:   mysql-test/suite/galera/disabled.def
	modified:   mysql-test/suite/galera/r/galera_gcache_recover_full_gcache.result
	modified:   mysql-test/suite/galera/r/galera_gcache_recover_manytrx.result
	modified:   mysql-test/suite/galera/t/galera_gcache_recover_full_gcache.test
	modified:   mysql-test/suite/galera/t/galera_gcache_recover_manytrx.test
	modified:   storage/innobase/trx/trx0trx.cc
	modified:   storage/xtradb/trx/trx0trx.cc
2019-01-07 12:12:30 +02:00
Varun Gupta
d0d0f88f2c MDEV-13784: query causes seg fault
When we have a nested subquery then a subquery that was a dependent subquery
may change to an independent one when we optimizer the inner subqueries.
This is handled st_select_lex::optimize_unflattened_subqueries.
Currently a subquery that was changed to independent from dependent after optimization
phase incorrectly shows dependent in the output of Explain, this happens because we
don't update used_tables for the WHERE clause, ON clause, etc after the optimization phase.
2019-01-06 23:15:25 +05:30
Marko Mäkelä
7158edcba3 MDEV-18129 Backup fails for encrypted tables: mariabackup: Database page corruption detected at page 1
If an encrypted table is created during backup, then
mariabackup --backup could wrongly fail.

This caused a failure of the test mariabackup.huge_lsn once on buildbot.

This is due to the way how InnoDB creates .ibd files. It would first
write a dummy page 0 with no encryption information. Due to this,
xb_fil_cur_open() could wrongly interpret that the table is not encrypted.
Subsequently, page_is_corrupted() would compare the computed page
checksum to the wrong checksum. (There are both "before" and "after"
checksums for encrypted pages.)

To work around this problem, we introduce a Boolean option
--backup-encrypted that is enabled by default. With this option,
Mariabackup will assume that a nonzero key_version implies that the
page is encrypted. We need this option in order to be able to copy
encrypted tables from MariaDB 10.1 or 10.2, because unencrypted pages
that were originally created before MySQL 5.1.48 could contain nonzero
garbage in the fields that were repurposed for encryption.

Later, MDEV-18128 would clean up the way how .ibd files are created,
to remove the need for this option.

page_is_corrupted(): Add missing const qualifiers, and do not check
space->crypt_data unless --skip-backup-encrypted has been specified.

xb_fil_cur_read(): After a failed page read, output a page dump.
2019-01-03 16:46:38 +02:00
Sergei Golubchik
3ba3f81ae0 Merge branch '10.0' into 10.1 2019-01-03 09:56:24 +01:00
Sergei Golubchik
b87eb04f77 Merge branch '5.5' into 10.0 2019-01-03 00:20:53 +01:00
Sergei Golubchik
884caeafba fix RHEL8 "ambiguous python shebang" build failures 2019-01-02 19:32:05 +01:00
Sergei Golubchik
32150d2513 compilation warning on Windows 2019-01-02 19:28:48 +01:00
Sergei Golubchik
2450fd67ed fix the test for 2019 2019-01-02 12:03:15 +01:00
Sergei Golubchik
aeefd26ecb Merge branch '10.0' into 10.1 2018-12-29 23:44:45 +01:00
Marko Mäkelä
50c9469be8 MDEV-18105 Mariabackup fails to copy encrypted InnoDB system tablespace if LSN>4G
This is a regression caused by
commit 8c43f96388
that was part of the MDEV-12112 fixes.

page_is_corrupted(): Never interpret page_no=0 as encrypted.
2018-12-29 22:59:20 +02:00
Eugene Kosov
802ce9672f MDEV-18041 Database corruption after renaming a prefix-indexed column
This is a regression after MDEV-13671.

The bug is related to key part prefix lengths wich are stored in SYS_FIELDS.
Storage format is not obvious and was handled incorrectly which led to data
dictionary corruption.

SYS_FIELDS.POS actually contains prefix length too in case if any key part
has prefix length.

innobase_rename_column_try(): fixed prefixes handling

Tests for prefixed indexes added too.

Closes #1063
2018-12-29 22:57:05 +02:00
Marko Mäkelä
68143c8905 MDEV-17470: Fix the test for --embedded 2018-12-29 10:57:26 +02:00
Vladislav Vaintroub
ed66acb291 Silence LeakSanitizer by default in mariabackup, so that phanthom "leaks"
would not hide more interesting information, like invalid memory accesses.


some "leaks" are expected
- partly this is due to weird options parsing, that runs twice, and
does not free memory after the first run.
- also we do not mind to exit()  whenever it makes sense, without full
cleanup.
2018-12-29 02:06:19 +01:00
Eugene Kosov
c5a5eaa9a9 MDEV-17470 Orphan temporary files after interrupted ALTER cause InnoDB: Operating system error number 17 and eventual fatal error 71
Orphan #sql* tables may remain after ALTER TABLE
was interrupted by timeout or KILL or client disconnect.

This is a regression caused by MDEV-16515.

Similar to temporary tables (MDEV-16647), we had better ignore the
KILL when dropping the original table in the final part of ALTER TABLE.

Closes #1020
2018-12-28 17:05:48 +02:00
Marko Mäkelä
b74eb5a5fe row_drop_table_for_mysql(): Correct a parameter to innobase_format_name()
This fixes a regression that was introduced in MySQL 5.6.6
in an error handling code path, in the following change:

commit 024f363d6b5f09b20d1bba411af55be95c7398d3
Author: kevin.lewis@oracle.com <>
Date:   Fri Jun 15 09:01:42 2012 -0500

    Bug #14169459 INNODB; DROP TABLE DOES NOT DELETE THE IBD FILE
    FOR A TEMPORARY TABLE.
2018-12-28 12:28:16 +02:00
Rakshit Kumar
9ad1663f78 Grammatical errors of README-wsrep fixed. (#915)
Docs grammar fixed
2018-12-22 13:34:50 +04:00
Marko Mäkelä
40a094e4a8 Relax a too tight suppression
When using a wrong key, all encrypted pages will look corrupted,
and occasionally the test may access other pages than
the clustered index root page.
2018-12-21 09:40:36 +02:00
Vladislav Vaintroub
9f4a4cb401 Cleanup recent mariabackup validation patches.
- Refactor code to isolate page validation in page_is_corrupted() function.

- Introduce --extended-validation parameter(default OFF) for mariabackup
--backup to enable decryption of encrypted uncompressed pages during
backup.

- mariabackup would still always check checksum on encrypted data,
it is needed to detect  partially written pages.
2018-12-20 14:31:18 +01:00
Marko Mäkelä
ed36fc353f MDEV-18025: Detect corrupted innodb_page_compression=zlib pages
In MDEV-13103, I made a mistake in the error handling of
page_compressed=1 decryption when the default
innodb_compression_algorithm=zlib is used.
Due to this mistake, with certain versions of zlib,
MariaDB would fail to detect a corrupted page.

The problem was uncovered by the following tests:
mariabackup.unencrypted_page_compressed
mariabackup.encrypted_page_compressed
2018-12-20 13:33:09 +02:00
Sergei Golubchik
8634f7e528 Merge branch '5.5' into 10.0 2018-12-20 09:15:01 +01:00
Sergei Golubchik
8ede9b3ae5 MDEV-17975 Assertion ! is_set()' or !is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failed upon REVOKE under LOCK TABLE
open_grant_tables() returns -1/0/1, where -1 is an error, while 1 is not.
Don't store it's return value in bool
2018-12-20 08:39:54 +01:00
Sachin
f16d4d4c6e MDEV-17720 slave_ddl_exec_mode=IDEMPOTENT does not handle DROP DATABASE
Relevant if exists flag are added for create database and drop database.
2018-12-19 23:47:26 +05:30
Marko Mäkelä
dd72d7d561 MDEV-18025: Improve test case and consistency checks
Write a test case that computes valid crc32 checksums for
an encrypted page, but zeroes out the payload area, so
that the checksum after decryption fails.

xb_fil_cur_read(): Validate the page number before trying
any checksum calculation or decrypting or decompression.
Also, skip zero-filled pages. For page_compressed pages,
ensure that the FIL_PAGE_TYPE was changed. Also, reject
FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED if no decryption was attempted.
2018-12-19 15:45:35 +02:00
Varun Gupta
7e606a2d5c MDEV-17589: Stack-buffer-overflow with indexed varchar (utf8) field
Create a new constant MAX_DATA_LENGTH_FOR_KEY.
Replace the value of MAX_KEY_LENGTH to also include the LENGTH and NULL BYTES
of a field.
2018-12-19 10:38:46 +05:30
Varun Gupta
da4efd56aa Backported MDEV-11196(e4d10e09cf) and MDEV-10360(8a8ba1949b) to 10.0 2018-12-19 10:38:29 +05:30
Marko Mäkelä
1b471face8 MDEV-18025: Apply the fix to XtraDB and adjust tests
The fix was accidentally only applied to InnoDB, and encryption tests
were not adjusted.
2018-12-18 16:24:52 +02:00
Sergey Vojtovich
b0fd06a6f2 MDEV-15670 - unit.my_atomic failed in buildbot with Signal 11 thrown
Workaround glibc bug: https://sourceware.org/bugzilla/show_bug.cgi?id=20116
by making unittest threads joinable. It makes code better anyway.
2018-12-18 17:20:15 +04:00