Commit graph

18663 commits

Author SHA1 Message Date
Eugene Kosov
28cb041754 MDEV-18662 ib_wqueue_t has a data race
ib_wqueue_is_empty(): protect ib_list_is_empty() call

Closes #1202
2019-02-21 12:23:47 +02:00
Monty
346e460896 Fixed bug in macro _ma_mark_page_with_transid()
By pure chance the macro worked in the cases it was used, but
better to get this fixed!
2019-02-19 10:51:34 +02:00
Marko Mäkelä
98e185ee37 MDEV-18630 Uninitialised value in FOREIGN KEY error message
dict_create_foreign_constraints_low(): Clean up the way in
which the error messages are initialized, and ensure that
the table name is always initialized.
2019-02-18 21:42:58 +02:00
Marko Mäkelä
be25414828 MDEV-18016: Cover the no-rebuild case, and remove a bogus debug assertion
The code path where the table was not being rebuilt during ALTER TABLE
was not covered by the test. Add coverage, and remove the debug assertion
that could fail in this case.
2019-02-11 11:38:18 +02:00
Marko Mäkelä
5eb3e4d83c MDEV-15798 Mutex leak on accessing INFORMATION_SCHEMA.INNODB_MUTEXES
i_s_innodb_mutexes_fill_table(): Use the C++ RAII pattern
to ensure that the mutexes are released if an OK()
macro returns from the function prematurely.
2019-02-05 17:03:41 +02:00
Marko Mäkelä
955c7b3222 MDEV-16896 encryption.innodb-checksum-algorithm crashes
buf_page_is_corrupted(): Read the global variable srv_checksum_algorithm
only once in order to avoid a race condition when
SET GLOBAL innodb_checksum_algorithm=...;
is being executed concurrently with this function.
2019-02-03 17:09:20 +02:00
Marko Mäkelä
213ece2f2e Merge 10.1 into 10.1
This is joint work with Oleksandr Byelkin.
2019-02-02 13:00:15 +02:00
Marko Mäkelä
c1e1764fc4 Fix embedded innodb_plugin after 560799ebd8
wsrep_certification_rules: Define as a weak global symbol.
While there are separate _embedded.a for statically
linked storage engine plugins, there is only one ha_innodb.so
which is supposed to work with both values of WITH_WSREP.

The merge from 10.0-galera introduced a reference to a global
variable that is only defined when the server is built WITH_WSREP.
We must define that symbol as weak global, so that when
a dynamically linked InnoDB or XtraDB is used with the embedded
server (which never includes write-set replication patches),
the variable will be read as 0, instead of causing a failure to
load the InnoDB or XtraDB plugin.
2019-02-02 12:49:04 +02:00
Marko Mäkelä
6051842cd0 Minor clean-up 2019-02-01 09:13:18 +02:00
Oleksandr Byelkin
560799ebd8 Merge branch '10.0-galera' into 10.1 2019-01-31 09:34:34 +01:00
Marko Mäkelä
1522ee2949 MDEV-18016: Assertion failure on ALTER TABLE after foreign_key_checks=0
ha_innobase::commit_inplace_alter_table(): Do not crash if
innobase_update_foreign_cache() returns an error. It can return
an error on ALTER TABLE if an inconsistent FOREIGN KEY constraint
was created earlier when SET foreign_key_checks=0 was in effect.
Instead, report a warning to the client that constraints cannot
be loaded.
2019-01-29 15:20:26 +02:00
Marko Mäkelä
6699cac0bf MDEV-18256 Duplicated call to dict_foreign_remove_from_cache()
ha_innobase::prepare_inplace_alter_table(): Filter out duplicates
from ha_alter_info->alter_info->drop_list.elements.
2019-01-29 15:20:26 +02:00
Marko Mäkelä
5e06ee41a4 MDEV-18222: Duplicated call to dict_foreign_remove_from_cache()
innobase_rename_column_try(): Declare fk_evict as std::set
instead of std::list, in order to filter out duplicates.
2019-01-29 15:20:26 +02:00
Oleksandr Byelkin
f877f6b49d Fix xtradb version after merge 2019-01-29 11:50:07 +01:00
Oleksandr Byelkin
c991939bab MariaDB detect incorrect table name 2019-01-29 09:34:08 +01:00
Oleksandr Byelkin
8c2f3e0c16 Fix detection of version in tokudb 2019-01-28 20:17:54 +01:00
Oleksandr Byelkin
5cdb3fb25e Merge branch 'merge-tokudb-5.6' into 10.0 2019-01-28 20:15:57 +01:00
Oleksandr Byelkin
724b09d5e7 Version fix after merge 2019-01-28 15:42:16 +01:00
Oleksandr Byelkin
94b68b35f4 Reverting part of da34c7de5d that was already fixed by MDEV-17531 by Marko 2019-01-28 15:39:27 +01:00
Oleksandr Byelkin
959f7415bd Merge branch 'merge-xtradb-5.6' into 10.0 2019-01-28 15:37:01 +01:00
Olivier Bertrand
e6fcd72309 Squashed commit of connect/10.0:
commit 6a6a1f37798
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Fri Jan 4 12:31:52 2019 +0100

    - Fix a few bug mainly concerning discovery and call from OEM
      (and prepare new table types)

      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/tabjson.h
      modified:   storage/connect/tabxml.cpp
      modified:   storage/connect/tabxml.h

    - Fix wrong line estimate
      modified:   storage/connect/mysql-test/connect/r/part_table.result
      modified:   storage/connect/mysql-test/connect/t/part_table.test

commit bd7d2e912d9
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Tue Dec 4 23:35:09 2018 +0100

    Fix wrong version number

commit 4933680e7ab
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Sun Dec 2 00:25:05 2018 +0100

    - Make PlugSubAlloc to be exportable
      Suppress unused parameter from PlugSubSet
      modified:   storage/connect/global.h
      modified:   storage/connect/plugutil.cpp
      modified:   storage/connect/jsonudf.cpp
      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/user_connect.cc

    - Fix a bug making column catalog XML tables fail
      modified:   storage/connect/tabxml.cpp

    - Comment out wrong message
      modified:   storage/connect/ha_connect.cc

    - Update error message when sorting an ODBC table fails
      modified:   storage/connect/tabodbc.cpp

    - Add error message when gettting an address
      from an OEM fails.
      modified:   storage/connect/reldef.cpp

    - Make some modifications useful for OEM module writting
      Export discovery functions for CSV, JDBC and XML
      Remove unuseful include from tabjson.h
      Move TDBXML::data_charset function from header file to source
      modified:   storage/connect/tabfmt.h
      modified:   storage/connect/tabjson.h
      modified:   storage/connect/tabxml.cpp
      modified:   storage/connect/tabxml.h

    - Update test result
      modified:   storage/connect/mysql-test/connect/r/jdbc_oracle.result
2019-01-25 10:06:16 +01:00
Oleksandr Byelkin
13802fef83 5.6.42-84.2 2019-01-24 17:31:13 +01:00
Oleksandr Byelkin
3262afc6c5 5.6.42-84.2 2019-01-24 16:48:39 +01:00
Marko Mäkelä
3b6d2efcb1 Merge 10.0 into 10.1 2019-01-23 14:34:23 +02:00
Marko Mäkelä
52d13036d8 MDEV-17933 slow server status - dict_sys_get_size()
dict_sys_get_size(): Replace the time-consuming loop with
a crude estimate that can be computed without holding any mutex.

Even before dict_sys->size was removed in MDEV-13325,
not all memory allocations by the InnoDB data dictionary cache
were being accounted for. One example is foreign key constraints.
Another example is virtual column metadata, starting with 10.2.
2019-01-23 13:44:20 +02:00
Sergei Golubchik
2a0f1d6132 Bug#28867993: POSSIBLE ISSUE WITH MYSQL SERVER RESTART
on startup innodb is checking whether files "ib_logfileN"
(for N from 1 to 100) exist, and whether they're readable.
A non-existent file aborted the scan.
A directory instead of a file made InnoDB to fail.

Now it treats "directory exists" as "file doesn't exist".
2019-01-23 12:39:03 +01:00
Marko Mäkelä
31d592ba7d MDEV-18349 InnoDB file size changes are not safe when file system crashes
When InnoDB is invoking posix_fallocate() to extend data files, it
was missing a call to fsync() to update the file system metadata.
If file system recovery is needed, the file size could be incorrect.

When the setting innodb_flush_method=O_DIRECT_NO_FSYNC
that was introduced in MariaDB 10.0.11 (and MySQL 5.6) is enabled,
InnoDB would wrongly skip fsync() after extending files.

Furthermore, the merge commit d8b45b0c00
inadvertently removed XtraDB error checking for posix_fallocate()
which this fix is restoring.

fil_flush(): Add the parameter bool metadata=false to request that
fil_buffering_disabled() be ignored.

fil_extend_space_to_desired_size(): Invoke fil_flush() with the
extra parameter. After successful posix_fallocate(), invoke
os_file_flush(). Note: The bookkeeping for fil_flush() would not be
updated the posix_fallocate() code path, so the "redundant"
fil_flush() should be a no-op.
2019-01-23 12:18:00 +02:00
Marko Mäkelä
942a6bd009 MDEV-18349 InnoDB file size changes are not safe when file system crashes
fil_extend_space_to_desired_size(): Invoke fsync() after posix_fallocate()
in order to durably extend the file in a crash-safe file system.
2019-01-23 09:51:06 +02:00
Marko Mäkelä
1d72db45a8 MDEV-18237 InnoDB: Unable to drop FTS index aux table and further errors (possibly bogus)
row_drop_table_for_mysql(): Fix a regression introduced in MDEV-16515.
Similar to the follow-up fixes MDEV-16647 and MDEV-17470, we must make
the internal tables of FULLTEXT INDEX immune to kills, to avoid noise
and resource leakage on DROP TABLE or ALTER TABLE. (Orphan internal tables
would be dropped at the next InnoDB startup only.)
2019-01-18 06:46:39 +02:00
Thirunarayanan Balathandayuthapani
ad376a05fa MDEV-18279 MLOG_FILE_WRITE_CRYPT_DATA fails to set the crypt_data->type for tablespace.
Problem:
========
MLOG_FILE_WRITE_CRYPT_DATA redo log fails to apply type for
the crypt_data present in the space. While processing the double-write
buffer pages, page fails to decrypt. It leads to warning message.

Fix:
====
Set the type while parsing MLOG_FILE_WRITE_CRYPT_DATA redo log.
If type and length is of invalid type then mark it as corrupted.
2019-01-17 13:10:45 +05:30
Marko Mäkelä
71eb762611 Merge 10.0 into 10.1 2019-01-17 06:40:24 +02:00
Thirunarayanan Balathandayuthapani
038785e1f8 MDEV-18183 Server startup fails while dropping garbage encrypted tablespace
- There is no need to wait for crypt thread to stop accessing space while
dropping the garbage encrypted tablespace during recover.
2019-01-16 23:02:39 +05:30
Marko Mäkelä
12f362c333 MDEV-18233 Moving the hash_node_t to improve locality of reference 2019-01-15 14:53:27 +02:00
Eugene Kosov
e0633f25e8 MDEV-18243 incorrect ASAN instrumentation
Poisoning memory after munmap() and friends is totally incorrect as this memory
could be anything.

os_mem_free_large(): remove memory poisoning
2019-01-15 15:32:18 +03:00
Marko Mäkelä
71e9f0d123 MDEV-17797 Add ASAN poisoning for mem_heap_t
The merge commit d833bb65d5 did not
correctly merge the commit 03eb15933d.

Closes #948
2019-01-15 12:12:59 +02:00
Eugene Kosov
a06a3e4670 MDEV-18233 Moving the hash_node_t to improve locality of reference
When performing a hash search via HASH_SEARCH we first look at a key of a node
and then at its pointer to the next node in chain. If we have those in one cache
line instead of a two we reduce memory reads.

I found dict_table_t, fil_space_t and buf_page_t suitable for such improvement.
2019-01-14 22:14:56 +03:00
Marko Mäkelä
b4c471099d
Merge pull request #973 from tempesta-tech/tt-10.0-MDEV-16499-virtual-innodb
MDEV-16499 ER_NO_SUCH_TABLE_IN_ENGINE followed by "Please drop the table and recreate" upon adding FULLTEXT key to table with virtual column
2019-01-14 14:06:26 +02:00
Jan Lindström
1d56d875fe MDEV-15740: InnoDB does not flush redo log when it shoul
During database recovery, a transaction with wsrep XID is
recovered from InnoDB in prepared state. However, when the
transaction is looked up with trx_get_trx_by_xid() in
innobase_commit_by_xid(), trx->xid gets cleared in
trx_get_trx_by_xid_low() and commit time serialization history
write does not update the wsrep XID in trx sys header for
that recovered trx. As a result the transaction gets
committed during recovery but the wsrep position does not
get updated appropriately.

As a fix, we preserve trx->xid for Galera over transaction
commit in recovery phase.

Fix authored by: Teemu Ollakka (GaleraCluster) and Marko Mäkelä.

	modified:   mysql-test/suite/galera/disabled.def
	modified:   mysql-test/suite/galera/r/galera_gcache_recover_full_gcache.result
	modified:   mysql-test/suite/galera/r/galera_gcache_recover_manytrx.result
	modified:   mysql-test/suite/galera/t/galera_gcache_recover_full_gcache.test
	modified:   mysql-test/suite/galera/t/galera_gcache_recover_manytrx.test
	modified:   storage/innobase/trx/trx0trx.cc
	modified:   storage/xtradb/trx/trx0trx.cc
2019-01-07 12:12:30 +02:00
Sergei Golubchik
3ba3f81ae0 Merge branch '10.0' into 10.1 2019-01-03 09:56:24 +01:00
Sergei Golubchik
884caeafba fix RHEL8 "ambiguous python shebang" build failures 2019-01-02 19:32:05 +01:00
Sergei Golubchik
32150d2513 compilation warning on Windows 2019-01-02 19:28:48 +01:00
Sergei Golubchik
aeefd26ecb Merge branch '10.0' into 10.1 2018-12-29 23:44:45 +01:00
Eugene Kosov
802ce9672f MDEV-18041 Database corruption after renaming a prefix-indexed column
This is a regression after MDEV-13671.

The bug is related to key part prefix lengths wich are stored in SYS_FIELDS.
Storage format is not obvious and was handled incorrectly which led to data
dictionary corruption.

SYS_FIELDS.POS actually contains prefix length too in case if any key part
has prefix length.

innobase_rename_column_try(): fixed prefixes handling

Tests for prefixed indexes added too.

Closes #1063
2018-12-29 22:57:05 +02:00
Eugene Kosov
c5a5eaa9a9 MDEV-17470 Orphan temporary files after interrupted ALTER cause InnoDB: Operating system error number 17 and eventual fatal error 71
Orphan #sql* tables may remain after ALTER TABLE
was interrupted by timeout or KILL or client disconnect.

This is a regression caused by MDEV-16515.

Similar to temporary tables (MDEV-16647), we had better ignore the
KILL when dropping the original table in the final part of ALTER TABLE.

Closes #1020
2018-12-28 17:05:48 +02:00
Marko Mäkelä
b74eb5a5fe row_drop_table_for_mysql(): Correct a parameter to innobase_format_name()
This fixes a regression that was introduced in MySQL 5.6.6
in an error handling code path, in the following change:

commit 024f363d6b5f09b20d1bba411af55be95c7398d3
Author: kevin.lewis@oracle.com <>
Date:   Fri Jun 15 09:01:42 2012 -0500

    Bug #14169459 INNODB; DROP TABLE DOES NOT DELETE THE IBD FILE
    FOR A TEMPORARY TABLE.
2018-12-28 12:28:16 +02:00
Marko Mäkelä
ed36fc353f MDEV-18025: Detect corrupted innodb_page_compression=zlib pages
In MDEV-13103, I made a mistake in the error handling of
page_compressed=1 decryption when the default
innodb_compression_algorithm=zlib is used.
Due to this mistake, with certain versions of zlib,
MariaDB would fail to detect a corrupted page.

The problem was uncovered by the following tests:
mariabackup.unencrypted_page_compressed
mariabackup.encrypted_page_compressed
2018-12-20 13:33:09 +02:00
Marko Mäkelä
1b471face8 MDEV-18025: Apply the fix to XtraDB and adjust tests
The fix was accidentally only applied to InnoDB, and encryption tests
were not adjusted.
2018-12-18 16:24:52 +02:00
Thirunarayanan Balathandayuthapani
171271edf8 MDEV-18025 Mariabackup fails to detect corrupted page_compressed=1 tables
Problem:
=======
Mariabackup seems to fail to verify the pages of compressed tables.
The reason is that both fil_space_verify_crypt_checksum() and
buf_page_is_corrupted() will skip the validation for compressed pages.

Fix:
====
Mariabackup should call fil_page_decompress() for compressed and encrypted
compressed page. After that, call buf_page_is_corrupted() to
check the page corruption.
2018-12-18 18:07:17 +05:30
Marko Mäkelä
84f119f25c MDEV-12112/MDEV-12114: Relax strict_innodb, strict_none
Starting with commit 6b6987154a
the encrypted page checksum is only computed with crc32.
Encrypted data pages that were written earlier can contain
other checksums, but new ones will only contain crc32.

Because of this, it does not make sense to implement strict
checks of innodb_checksum_algorithm for other than strict_crc32.

fil_space_verify_crypt_checksum(): Treat strict_innodb as innodb
and strict_none as none. That is, allow a match from any of the
algorithms none, innodb, crc32. (This is how it worked before the
second MDEV-12112 fix.)

Thanks to Thirunarayanan Balathandayuthapani for pointing this out.
2018-12-18 09:52:28 +02:00
Marko Mäkelä
51a1fc737c Fix a compiler warning
fil_space_verify_crypt_checksum(): Add a dummy return statement
in case memory is corrupted and innodb_checksum_algorithm has
an invalid value.
2018-12-17 22:35:22 +02:00