Commit graph

181218 commits

Author SHA1 Message Date
Oleksandr Byelkin
c1caada886 MDEV-16278: Missing DELETE operation in COM_STMT_BULK_STMT
Allow array binding for DELETE, test it.
2018-12-14 10:47:10 +01:00
Marko Mäkelä
e3dda3d95e MDEV-17989 InnoDB: Failing assertion: dict_tf2_is_valid(flags, flags2)
With innodb_default_row_format=redundant, InnoDB would crash when
using table options that are incompatible with ROW_FORMAT=REDUNDANT.

create_table_info_t::m_default_row_format: Cache the value of
innodb_default_row_format.

create_table_info_t::check_table_options(): Validate ROW_TYPE_DEFAULT
with m_default_row_format.

create_table_info_t::innobase_table_flags(): Use the
cached m_default_row_format.

create_table_info_t: Never read m_form->s->row_type.
Use m_create_info->row_type instead.

dict_tf_set(): Never set invalid flags for ROW_FORMAT=REDUNDANT.

ha_innobase::truncate(): Set info.row_type based on the ROW_FORMAT
of the current table.
2018-12-13 21:25:12 +02:00
Marko Mäkelä
1a780eefc9 MDEV-17958 Make bug-endian innodb_checksum_algorithm=crc32 optional
In MySQL 5.7, it was noticed that files are not portable between
big-endian and little-endian processor architectures
(such as SPARC and x86), because the original implementation of
innodb_checksum_algorithm=crc32 was not byte order agnostic.

A byte order agnostic implementation of innodb_checksum_algorithm=crc32
was only added to MySQL 5.7, not backported to 5.6. Consequently,
MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C
implementation that works incorrectly on big-endian architectures,
and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C
implementation from MySQL 5.7.

MySQL 5.7 introduced a "legacy crc32" variant that is functionally
equivalent to the big-endian version of the original crc32 implementation.
Thanks to this variant, old data files can be transferred from big-endian
systems to newer versions.

Introducing new variants of checksum algorithms (without introducing
new names for them, or something on the pages themselves to identify
the algorithm) generally is a bad idea, because each checksum algorithm
is like a lottery ticket. The more algorithms you try, the more likely
it will be for the checksum to match on a corrupted page.

So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32,
and MariaDB 10.2.2 inherited this weakening.

We introduce a build option that together with MDEV-17957
makes innodb_checksum_algorithm=strict_crc32 strict again
by only allowing one variant of the checksum to match.

WITH_INNODB_BUG_ENDIAN_CRC32: A new cmake option for enabling the
bug-compatible "legacy crc32" checksum. This is only enabled on
big-endian systems by default, to facilitate an upgrade from
MariaDB 10.0 or 10.1. Checked by #ifdef INNODB_BUG_ENDIAN_CRC32.

ut_crc32_byte_by_byte: Remove (unused function).

legacy_big_endian_checksum: Remove. This variable seems to have
unnecessarily complicated the logic. When the weakening is enabled,
we must always fall back to the buggy checksum.

buf_page_check_crc32(): A helper function to compute one or
two CRC-32C variants.
2018-12-13 17:57:18 +02:00
Marko Mäkelä
2e5aea4bab Merge 10.1 into 10.2 2018-12-13 15:47:38 +02:00
Marko Mäkelä
621041b676 Merge 10.0 into 10.1
Also, apply the MDEV-17957 changes to encrypted page checksums,
and remove error message output from the checksum function,
because these messages would be useless noise when mariabackup
is retrying reads of corrupted-looking pages, and not that
useful during normal server operation either.

The error messages in fil_space_verify_crypt_checksum()
should be refactored separately.
2018-12-13 13:37:21 +02:00
Marko Mäkelä
8e613458e1 Fix cmake -DWITH_PARTITION_STORAGE_ENGINE:BOOL=OFF
This is a backport of a part of
commit 18455ec3f1
from 10.1.
2018-12-13 12:37:40 +02:00
Marko Mäkelä
5ab91f5914 Remove space before #ifdef 2018-12-13 12:15:27 +02:00
Thirunarayanan Balathandayuthapani
5f5e73f1fe MDEV-17957 Make Innodb_checksum_algorithm stricter for strict_* values
Problem:

  Innodb_checksum_algorithm checks for all checksum algorithm to
validate the page checksum even though the algorithm is specified as
strict_crc32, strict_innodb, strict_none.

Fix:

   Remove the checks for all checksum algorithm to validate the page
checksum if the algo is specified as strict_* values.
2018-12-13 12:06:14 +02:00
Varun Gupta
ce1669af12 Fix compile error when building without the partition engine 2018-12-13 10:03:09 +05:30
Sergei Golubchik
ad3346dddf add more dbug helpers for gdb 2018-12-12 20:22:52 +01:00
Sergei Golubchik
541500295a debian install/upgrade fixes
1. don't run full mysql_upgrade on every server restart,
   use --version-check to do it only once

2. fix syslog tag name in the postinst script, don't pretend
   mysqld_safe generated all these messages. Auto-detect the version
   to simplify maintenance
2018-12-12 20:19:56 +01:00
Sergei Golubchik
b58f28725b Merge branch '5.5' into 10.0 2018-12-12 20:19:06 +01:00
Marko Mäkelä
c913cd2b66 MDEV-17885 TRUNCATE on temporary table causes ER_GET_ERRNO
ha_innobase::truncate(): Because CREATE TEMPORARY TABLE
allows invalid table options when innodb_file_per_table=1,
do allow them also in TRUNCATE for temporary tables.
2018-12-12 16:31:34 +02:00
Jiaye Wu
9eadef013e Fix UNICODE issue of dlerror
Current implementation is conflicting. If UNICODE is defined, FormatMessage() will be FormatMessageW(), and variable win_errormsg with type char can not be passed to it, which should be changed to TCHAR instead. Since we don't use UNICODE here, we can use FormatMessageA() directly to avoid conversion error.
```
my_global.h(1092): error C2664: 'DWORD FormatMessageW(D
WORD,LPCVOID,DWORD,DWORD,LPWSTR,DWORD,va_list *)' : cannot convert argument 5 from 'char [2048]' to 'LPWSTR'
```
2018-12-12 12:36:28 +01:00
Marko Mäkelä
91173f9863 fts_is_charset_cjk(): Avoid referencing global symbols
References to global symbols prevent InnoDB from being built as a
dynamic plugin on Windows.

Refer to CHARSET_INFO::number, because that is what InnoDB is already
persistently storing in its data dictionary.
2018-12-12 13:30:40 +02:00
Marko Mäkelä
e0aebf5cf1 MDEV-17815 Assertion failed in btr_node_ptr_max_size for CHAR(0)
btr_node_ptr_max_size(): Treat CHAR(0) from SQL as a special case.
The InnoDB internal SQL parser maps the type "CHAR" to DATA_VARCHAR,
but MariaDB does allow CHAR(0) with an empty value, and does enforce
the length limitation.
2018-12-12 13:21:05 +02:00
Marko Mäkelä
db1210f939 Merge 10.1 into 10.2 2018-12-12 12:13:43 +02:00
Marko Mäkelä
f77f8f6d1a Merge 10.0 into 10.1 2018-12-12 10:48:53 +02:00
Eugene Kosov
d956709b4b MDEV-17833 ALTER TABLE is not enforcing prefix index size limit
ha_innobase::prepare_inplace_alter_table(): check max column length for every
index in a table, not just added in this particular ALTER TABLE with ADD INDEX ones.
2018-12-11 22:03:44 +03:00
Marko Mäkelä
0d7cf06af5 MDEV-17938 ALTER TABLE reports ER_TABLESPACE_EXISTS after failed ALTER TABLE
There was a race condition in the error handling of ALTER TABLE when
the table contains FULLTEXT INDEX.

During the error handling of an erroneous ALTER TABLE statement,
when InnoDB would drop the internally created tables for FULLTEXT INDEX,
it could happen that one of the hidden tables was being concurrently
accessed by a background thread. Because of this, InnoDB would defer
the drop operation to the background.

However, related to MDEV-13564 backup-safe TRUNCATE TABLE and its
prerequisite MDEV-14585, we had to make the background drop table queue
crash-safe by renaming the table to a temporary name before enqueueing it.
This renaming was introduced in a follow-up of the MDEV-13407 fix.
As part of this rename operation, we were unnecessarily parsing the
current SQL statement, because the same rename operation could also be
executed as part of ALTER TABLE via ha_innobase::rename_table().

If an ALTER TABLE statement was being refused due to incorrectly formed
FOREIGN KEY constraint, then it could happen that the renaming of the hidden
internal tables for FULLTEXT INDEX could also fail, triggering a host of
error log messages, and causing a subsequent table-rebuilding ALTER TABLE
operation to fail due to the tablespace already existing.

innobase_rename_table(), row_rename_table_for_mysql(): Add the parameter
use_fk for suppressing the parsing of FOREIGN KEY constraints. It
will only be passed as use_fk=true by ha_innobase::rename_table(),
which can be invoked as part of ALTER TABLE...ALGORITHM=COPY.
2018-12-10 11:44:39 +02:00
Varun Gupta
4886d14827 MDEV-17032: Estimates are higher for partitions of a table with @@use_stat_tables= PREFERABLY
The problem here is EITS statistics does not calculate statistics for the partitions of the table.
So a temporary solution would be to not read EITS statistics for partitioned tables.

Also disabling reading of EITS for columns that participate in the partition list of a table.
2018-12-07 19:59:45 +05:30
Marko Mäkelä
53440e2dda MDEV-17923: Fix the pointer arithmetics
&sys_buf + DATA_TRX_ID_LEN is not &(sys_buf + DATA_TRX_ID_LEN).
2018-12-07 15:31:43 +02:00
Marko Mäkelä
52778e2e3e After-merge fix 2018-12-07 15:13:39 +02:00
Marko Mäkelä
5e5deabdbc Merge 10.1 into 10.2 2018-12-07 13:41:10 +02:00
Marko Mäkelä
ecd3a7e00d Merge 10.0 into 10.1 2018-12-07 13:17:32 +02:00
Marko Mäkelä
12b1ba195c MDEV-17904 Crash in fts_is_sync_needed() after failed ALTER or CREATE TABLE
create_table_def(), ha_innobase::create(): Defer fts_optimize_add_table()
until after the table has been successfully created.
2018-12-07 13:12:39 +02:00
Marko Mäkelä
49a50a19a1 MDEV-17923 Assertion failed in trx_undo_page_report_modify after CREATE FULLTEXT INDEX
row_fts_merge_insert(): Correctly initialize DB_ROLL_PTR to a safe value
that will not be dereferenced by MVCC.
2018-12-07 11:54:03 +02:00
Vladislav Vaintroub
2a2e8ea8fe MDEV-17917 MTR: fixed race conditions in perfschema.socket_connect, main.connect 2018-12-06 21:46:14 +01:00
Sergei Golubchik
6491c591b2 Merge branch '10.0' into 10.1 2018-12-06 15:08:42 +01:00
Sergei Golubchik
daca7e70d7 MDEV-17898 FLUSH PRIVILEGES crashes server with segfault
merge_role_db_privileges() was remembering pointers into Dynamic_array
acl_dbs, and later was using them, while pushing more elements into the
array. But pushing can cause realloc, and it can invalidate all pointers.

Fix: remember and use indexes of elements, not pointers.
2018-12-06 14:22:07 +01:00
Sergei Golubchik
eed0013bed correct order of arguments for Dynamic_array<>::CMP_FUNC2 2018-12-06 14:22:07 +01:00
Sergei Golubchik
8a37ce0767 cleanup: DYNAMIC_ARRAY -> Dynamic_array<ACL_DB> acl_dbs 2018-12-06 14:22:07 +01:00
Varun Gupta
17e8570285 Added a testcase for mdev-17734 2018-12-05 19:31:25 +05:30
Marko Mäkelä
5ec9b88e11 Disable a frequently failing test 2018-12-04 15:30:04 +02:00
Thirunarayanan Balathandayuthapani
157d3c3bc1 MDEV-17432 Assertion `lock_trx_has_sys_table_locks(trx) == 0' failed upon ALTER TABLE .. ADD FOREIGN KEY
- Fixed innodb.table_flags test case in release build.
2018-12-03 15:57:21 +05:30
Thirunarayanan Balathandayuthapani
f2c7972a3d MDEV-17432 Assertion `lock_trx_has_sys_table_locks(trx) == 0' failed upon ALTER TABLE .. ADD FOREIGN KEY
- This is a regression of commit b26e603aeb. While dropping
the incompletely created table, InnoDB shouldn't consider that operation as non-atomic one.
2018-12-03 01:12:04 +05:30
Igor Babaev
46960365b1 MDEV-17871 Crash when running explain with CTE
When the with clause of a query contains a recursive CTE that is not used
then processing of EXPLAIN for this query does not require optimization
of the unit specifying this CTE. In this case if 'derived' is the
TABLE_LIST object created for this CTE then derived->derived_result is NULL
and any assignment to derived->derived_result->table causes a crash.
After fixing this problem in the code of st_select_lex_unit::prepare()
EXPLAIN for such a query worked without crashes. Yet an execution
plan for the recursive CTE appeared there. The cause of this problem was
an incorrect condition used in JOIN::save_explain_data_intern() that
determined whether CTE was to be optimized or not. A similar condition was
used in select_describe() and this patch has corrected it as well.
2018-12-01 15:06:04 -08:00
Marko Mäkelä
3e5162d814 Re-disable a failing test 2018-11-30 15:54:21 +02:00
Jan Lindström
7826b9b983 Fix syntax error on galera/disabled.def file 2018-11-29 17:11:19 +02:00
Jan Lindström
b4d102e828 MDEV-17810: Improve error printout when decryption fails or we identify page as both encrypted and unencrypted
fil_space_verify_crypt_checksum
	Print out both stored checksum and calculated checksums
	for encrypted page and unencrypted page asumptions.
2018-11-29 17:11:19 +02:00
Jan Lindström
33fdb443ea Fix xtrabackup SST tests by using innodb-safe-truncate=OFF.
Disable tests that do not yet pass.
2018-11-29 17:11:19 +02:00
Marko Mäkelä
447e493179 Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
Marko Mäkelä
be998bfdc5 MDEV-17859: Clean up the FOREIGN KEY handling
dict_create_add_foreigns_to_dictionary(): Do not commit the transaction.
The operation can still fail in dict_load_foreigns(), and we want
to be able to roll back the transaction.

create_table_info_t::create_table(): Never reset m_drop_before_rollback,
and never commit the transaction. We use a single point of rollback
in ha_innobase::create(). Merge the logic from
row_table_add_foreign_constraints().
2018-11-29 09:16:48 +02:00
Sergei Golubchik
32b7d456d5 mysqltest: use a dynamically growing command buffer 2018-11-28 20:18:49 +01:00
Marko Mäkelä
b26e603aeb MDEV-17859 Operating system errors in file operations after failed CREATE
This is a regression due to MDEV-17816.

When creating a table fails, we must roll back the dictionary
transaction. Because the rollback may rename tables, and because
InnoDB lacks proper undo logging for CREATE operations, we must
drop the incompletely created table before rolling back the
transaction, which could include a RENAME operation.
But, we must not blindly drop the table by name; after all,
the operation could have failed because another table by the
same name already existed.

create_table_info_t::m_drop_before_rollback: A flag that is set
if the table needs to be dropped before transaction rollback.

create_table_info_t::create_table(): Remove some duplicated
error handling.

ha_innobase::create(): On error, only drop the table if it was
actually created.
2018-11-28 15:17:56 +02:00
Sergei Petrunia
0485e51935 MDEV-13155: XA recovery not supported for RocksDB
Added a testcase
2018-11-28 12:39:36 +03:00
Marko Mäkelä
4a92165ff0 Remove unused mem_heap_allocator
The code became unused in commit 10590dd39c.
2018-11-28 00:52:30 +02:00
Marko Mäkelä
e82e216e37 MDEV-17849 Undo tablespace truncation recovery fails to shrink file
fil_space_t::add(): Replaces fil_node_create(), fil_node_create_low().
Let the caller pass fil_node_t::handle, to avoid having to close and
re-open files.

fil_node_t::read_page0(): Refactored from fil_node_open_file().
Read the first page of a data file.

fil_node_open_file(): Open the file only once.

srv_undo_tablespace_open(): Set the file handle for the opened
undo tablespace. This should ensure that ut_ad(file->is_open())
no longer fails in recv_add_trim().

xtrabackup_backup_func(): Remove some dead code.

xb_fil_cur_open(): Open files only if needed. Undo tablespaces
should already have been opened.
2018-11-27 14:49:39 +02:00
Marko Mäkelä
eb6364619f Remove the redundant variable fil_n_file_opened 2018-11-27 14:30:39 +02:00
Marko Mäkelä
b9824074a6 MDEV-17851 Assertion failure srv_undo_tablespaces > 1
trx_assign_rseg_low(): Skip dedicated undo tablespaces if
innodb_undo_tablespaces=0 has been set after the database was created.
2018-11-27 14:02:24 +02:00