Commit graph

183944 commits

Author SHA1 Message Date
Marko Mäkelä
f6e16bdc62 Merge 10.2 into 10.3 2018-12-13 21:58:35 +02:00
Marko Mäkelä
e3dda3d95e MDEV-17989 InnoDB: Failing assertion: dict_tf2_is_valid(flags, flags2)
With innodb_default_row_format=redundant, InnoDB would crash when
using table options that are incompatible with ROW_FORMAT=REDUNDANT.

create_table_info_t::m_default_row_format: Cache the value of
innodb_default_row_format.

create_table_info_t::check_table_options(): Validate ROW_TYPE_DEFAULT
with m_default_row_format.

create_table_info_t::innobase_table_flags(): Use the
cached m_default_row_format.

create_table_info_t: Never read m_form->s->row_type.
Use m_create_info->row_type instead.

dict_tf_set(): Never set invalid flags for ROW_FORMAT=REDUNDANT.

ha_innobase::truncate(): Set info.row_type based on the ROW_FORMAT
of the current table.
2018-12-13 21:25:12 +02:00
Marko Mäkelä
1a780eefc9 MDEV-17958 Make bug-endian innodb_checksum_algorithm=crc32 optional
In MySQL 5.7, it was noticed that files are not portable between
big-endian and little-endian processor architectures
(such as SPARC and x86), because the original implementation of
innodb_checksum_algorithm=crc32 was not byte order agnostic.

A byte order agnostic implementation of innodb_checksum_algorithm=crc32
was only added to MySQL 5.7, not backported to 5.6. Consequently,
MariaDB Server versions 10.0 and 10.1 only contain the CRC-32C
implementation that works incorrectly on big-endian architectures,
and MariaDB Server 10.2.2 got the byte-order agnostic CRC-32C
implementation from MySQL 5.7.

MySQL 5.7 introduced a "legacy crc32" variant that is functionally
equivalent to the big-endian version of the original crc32 implementation.
Thanks to this variant, old data files can be transferred from big-endian
systems to newer versions.

Introducing new variants of checksum algorithms (without introducing
new names for them, or something on the pages themselves to identify
the algorithm) generally is a bad idea, because each checksum algorithm
is like a lottery ticket. The more algorithms you try, the more likely
it will be for the checksum to match on a corrupted page.

So, essentially MySQL 5.7 weakened innodb_checksum_algorithm=crc32,
and MariaDB 10.2.2 inherited this weakening.

We introduce a build option that together with MDEV-17957
makes innodb_checksum_algorithm=strict_crc32 strict again
by only allowing one variant of the checksum to match.

WITH_INNODB_BUG_ENDIAN_CRC32: A new cmake option for enabling the
bug-compatible "legacy crc32" checksum. This is only enabled on
big-endian systems by default, to facilitate an upgrade from
MariaDB 10.0 or 10.1. Checked by #ifdef INNODB_BUG_ENDIAN_CRC32.

ut_crc32_byte_by_byte: Remove (unused function).

legacy_big_endian_checksum: Remove. This variable seems to have
unnecessarily complicated the logic. When the weakening is enabled,
we must always fall back to the buggy checksum.

buf_page_check_crc32(): A helper function to compute one or
two CRC-32C variants.
2018-12-13 17:57:18 +02:00
Marko Mäkelä
2e5aea4bab Merge 10.1 into 10.2 2018-12-13 15:47:38 +02:00
Marko Mäkelä
621041b676 Merge 10.0 into 10.1
Also, apply the MDEV-17957 changes to encrypted page checksums,
and remove error message output from the checksum function,
because these messages would be useless noise when mariabackup
is retrying reads of corrupted-looking pages, and not that
useful during normal server operation either.

The error messages in fil_space_verify_crypt_checksum()
should be refactored separately.
2018-12-13 13:37:21 +02:00
Marko Mäkelä
8e613458e1 Fix cmake -DWITH_PARTITION_STORAGE_ENGINE:BOOL=OFF
This is a backport of a part of
commit 18455ec3f1
from 10.1.
2018-12-13 12:37:40 +02:00
Marko Mäkelä
5ab91f5914 Remove space before #ifdef 2018-12-13 12:15:27 +02:00
Thirunarayanan Balathandayuthapani
5f5e73f1fe MDEV-17957 Make Innodb_checksum_algorithm stricter for strict_* values
Problem:

  Innodb_checksum_algorithm checks for all checksum algorithm to
validate the page checksum even though the algorithm is specified as
strict_crc32, strict_innodb, strict_none.

Fix:

   Remove the checks for all checksum algorithm to validate the page
checksum if the algo is specified as strict_* values.
2018-12-13 12:06:14 +02:00
Varun Gupta
ce1669af12 Fix compile error when building without the partition engine 2018-12-13 10:03:09 +05:30
Sergei Golubchik
ad3346dddf add more dbug helpers for gdb 2018-12-12 20:22:52 +01:00
Sergei Golubchik
541500295a debian install/upgrade fixes
1. don't run full mysql_upgrade on every server restart,
   use --version-check to do it only once

2. fix syslog tag name in the postinst script, don't pretend
   mysqld_safe generated all these messages. Auto-detect the version
   to simplify maintenance
2018-12-12 20:19:56 +01:00
Sergei Golubchik
b58f28725b Merge branch '5.5' into 10.0 2018-12-12 20:19:06 +01:00
Marko Mäkelä
c913cd2b66 MDEV-17885 TRUNCATE on temporary table causes ER_GET_ERRNO
ha_innobase::truncate(): Because CREATE TEMPORARY TABLE
allows invalid table options when innodb_file_per_table=1,
do allow them also in TRUNCATE for temporary tables.
2018-12-12 16:31:34 +02:00
Marko Mäkelä
839cf16bb2 Merge 10.2 into 10.3 2018-12-12 13:46:06 +02:00
Jiaye Wu
9eadef013e Fix UNICODE issue of dlerror
Current implementation is conflicting. If UNICODE is defined, FormatMessage() will be FormatMessageW(), and variable win_errormsg with type char can not be passed to it, which should be changed to TCHAR instead. Since we don't use UNICODE here, we can use FormatMessageA() directly to avoid conversion error.
```
my_global.h(1092): error C2664: 'DWORD FormatMessageW(D
WORD,LPCVOID,DWORD,DWORD,LPWSTR,DWORD,va_list *)' : cannot convert argument 5 from 'char [2048]' to 'LPWSTR'
```
2018-12-12 12:36:28 +01:00
Marko Mäkelä
91173f9863 fts_is_charset_cjk(): Avoid referencing global symbols
References to global symbols prevent InnoDB from being built as a
dynamic plugin on Windows.

Refer to CHARSET_INFO::number, because that is what InnoDB is already
persistently storing in its data dictionary.
2018-12-12 13:30:40 +02:00
Marko Mäkelä
e0aebf5cf1 MDEV-17815 Assertion failed in btr_node_ptr_max_size for CHAR(0)
btr_node_ptr_max_size(): Treat CHAR(0) from SQL as a special case.
The InnoDB internal SQL parser maps the type "CHAR" to DATA_VARCHAR,
but MariaDB does allow CHAR(0) with an empty value, and does enforce
the length limitation.
2018-12-12 13:21:05 +02:00
Sergei Golubchik
dce2cc1c6a fix handler test failures on s390x
keyinfo->name is a LEX_CSTRING

also, fix the location comment (that applied to ext_key_part_map)
2018-12-12 11:35:21 +01:00
Marko Mäkelä
db1210f939 Merge 10.1 into 10.2 2018-12-12 12:13:43 +02:00
Sergei Golubchik
56d3a0e73b MDEV-17967 Add a solution of the 8 queens problem to the regression test for CTE 2018-12-12 10:39:17 +01:00
Marko Mäkelä
f77f8f6d1a Merge 10.0 into 10.1 2018-12-12 10:48:53 +02:00
Eugene Kosov
d956709b4b MDEV-17833 ALTER TABLE is not enforcing prefix index size limit
ha_innobase::prepare_inplace_alter_table(): check max column length for every
index in a table, not just added in this particular ALTER TABLE with ADD INDEX ones.
2018-12-11 22:03:44 +03:00
Vladislav Vaintroub
ac31ff6275 MDEV-15760 - don't build mariabackup with -DPLUGIN_INNOBASE=DYNAMIC 2018-12-10 23:31:54 +01:00
Varun Gupta
8aef7f2bb9 MDEV-17778: Alter table leads to a truncation warning with ANALYZE command
Alter statement changed the THD structure by setting the value to FIELD_CHECK_WARN
and then not resetting it back. This led ANALYZE to throw a warning which previously
it didn't.
2018-12-10 18:27:13 +05:30
Marko Mäkelä
a72516348b MDEV-17938: ALTER TABLE error handling accesses freed memory
This regression was introduced in MDEV-11369 instant ADD COLUMN.

prepare_inplace_alter_table_dict(): Avoid dereferencing ctx->new_table
after row_create_table_for_mysql() returns a failure code, because
it will have freed the memory.
2018-12-10 13:02:04 +02:00
Marko Mäkelä
1d18665e0b Merge 10.2 into 10.3 2018-12-10 12:28:31 +02:00
Marko Mäkelä
0d7cf06af5 MDEV-17938 ALTER TABLE reports ER_TABLESPACE_EXISTS after failed ALTER TABLE
There was a race condition in the error handling of ALTER TABLE when
the table contains FULLTEXT INDEX.

During the error handling of an erroneous ALTER TABLE statement,
when InnoDB would drop the internally created tables for FULLTEXT INDEX,
it could happen that one of the hidden tables was being concurrently
accessed by a background thread. Because of this, InnoDB would defer
the drop operation to the background.

However, related to MDEV-13564 backup-safe TRUNCATE TABLE and its
prerequisite MDEV-14585, we had to make the background drop table queue
crash-safe by renaming the table to a temporary name before enqueueing it.
This renaming was introduced in a follow-up of the MDEV-13407 fix.
As part of this rename operation, we were unnecessarily parsing the
current SQL statement, because the same rename operation could also be
executed as part of ALTER TABLE via ha_innobase::rename_table().

If an ALTER TABLE statement was being refused due to incorrectly formed
FOREIGN KEY constraint, then it could happen that the renaming of the hidden
internal tables for FULLTEXT INDEX could also fail, triggering a host of
error log messages, and causing a subsequent table-rebuilding ALTER TABLE
operation to fail due to the tablespace already existing.

innobase_rename_table(), row_rename_table_for_mysql(): Add the parameter
use_fk for suppressing the parsing of FOREIGN KEY constraints. It
will only be passed as use_fk=true by ha_innobase::rename_table(),
which can be invoked as part of ALTER TABLE...ALGORITHM=COPY.
2018-12-10 11:44:39 +02:00
Kristian Nielsen
705fd4e943 Fix another random failure in rpl.rpl_gtid_crash 2018-12-08 22:54:34 +01:00
Marko Mäkelä
2fd0acd30f Fix the 64-bit Windows build 2018-12-08 00:06:07 +02:00
Kristian Nielsen
a02cac47f6 Fix an occational test failure in rpl.rpl_gtid_crash 2018-12-07 17:20:31 +01:00
Varun Gupta
4886d14827 MDEV-17032: Estimates are higher for partitions of a table with @@use_stat_tables= PREFERABLY
The problem here is EITS statistics does not calculate statistics for the partitions of the table.
So a temporary solution would be to not read EITS statistics for partitioned tables.

Also disabling reading of EITS for columns that participate in the partition list of a table.
2018-12-07 19:59:45 +05:30
Marko Mäkelä
21069c528e Merge 10.2 into 10.3 2018-12-07 15:39:34 +02:00
Marko Mäkelä
53440e2dda MDEV-17923: Fix the pointer arithmetics
&sys_buf + DATA_TRX_ID_LEN is not &(sys_buf + DATA_TRX_ID_LEN).
2018-12-07 15:31:43 +02:00
Marko Mäkelä
52778e2e3e After-merge fix 2018-12-07 15:13:39 +02:00
Marko Mäkelä
5e5deabdbc Merge 10.1 into 10.2 2018-12-07 13:41:10 +02:00
Marko Mäkelä
ecd3a7e00d Merge 10.0 into 10.1 2018-12-07 13:17:32 +02:00
Marko Mäkelä
12b1ba195c MDEV-17904 Crash in fts_is_sync_needed() after failed ALTER or CREATE TABLE
create_table_def(), ha_innobase::create(): Defer fts_optimize_add_table()
until after the table has been successfully created.
2018-12-07 13:12:39 +02:00
Marko Mäkelä
49a50a19a1 MDEV-17923 Assertion failed in trx_undo_page_report_modify after CREATE FULLTEXT INDEX
row_fts_merge_insert(): Correctly initialize DB_ROLL_PTR to a safe value
that will not be dereferenced by MVCC.
2018-12-07 11:54:03 +02:00
Vladislav Vaintroub
2a2e8ea8fe MDEV-17917 MTR: fixed race conditions in perfschema.socket_connect, main.connect 2018-12-06 21:46:14 +01:00
Sergei Golubchik
6491c591b2 Merge branch '10.0' into 10.1 2018-12-06 15:08:42 +01:00
Sergei Golubchik
daca7e70d7 MDEV-17898 FLUSH PRIVILEGES crashes server with segfault
merge_role_db_privileges() was remembering pointers into Dynamic_array
acl_dbs, and later was using them, while pushing more elements into the
array. But pushing can cause realloc, and it can invalidate all pointers.

Fix: remember and use indexes of elements, not pointers.
2018-12-06 14:22:07 +01:00
Sergei Golubchik
eed0013bed correct order of arguments for Dynamic_array<>::CMP_FUNC2 2018-12-06 14:22:07 +01:00
Sergei Golubchik
8a37ce0767 cleanup: DYNAMIC_ARRAY -> Dynamic_array<ACL_DB> acl_dbs 2018-12-06 14:22:07 +01:00
Varun Gupta
17e8570285 Added a testcase for mdev-17734 2018-12-05 19:31:25 +05:30
Marko Mäkelä
5ec9b88e11 Disable a frequently failing test 2018-12-04 15:30:04 +02:00
Marko Mäkelä
b6f203984b Merge 10.2 into 10.3 2018-12-04 13:18:14 +02:00
Thirunarayanan Balathandayuthapani
157d3c3bc1 MDEV-17432 Assertion `lock_trx_has_sys_table_locks(trx) == 0' failed upon ALTER TABLE .. ADD FOREIGN KEY
- Fixed innodb.table_flags test case in release build.
2018-12-03 15:57:21 +05:30
Thirunarayanan Balathandayuthapani
f2c7972a3d MDEV-17432 Assertion `lock_trx_has_sys_table_locks(trx) == 0' failed upon ALTER TABLE .. ADD FOREIGN KEY
- This is a regression of commit b26e603aeb. While dropping
the incompletely created table, InnoDB shouldn't consider that operation as non-atomic one.
2018-12-03 01:12:04 +05:30
Igor Babaev
46960365b1 MDEV-17871 Crash when running explain with CTE
When the with clause of a query contains a recursive CTE that is not used
then processing of EXPLAIN for this query does not require optimization
of the unit specifying this CTE. In this case if 'derived' is the
TABLE_LIST object created for this CTE then derived->derived_result is NULL
and any assignment to derived->derived_result->table causes a crash.
After fixing this problem in the code of st_select_lex_unit::prepare()
EXPLAIN for such a query worked without crashes. Yet an execution
plan for the recursive CTE appeared there. The cause of this problem was
an incorrect condition used in JOIN::save_explain_data_intern() that
determined whether CTE was to be optimized or not. A similar condition was
used in select_describe() and this patch has corrected it as well.
2018-12-01 15:06:04 -08:00
Marko Mäkelä
3e5162d814 Re-disable a failing test 2018-11-30 15:54:21 +02:00