Commit graph

1688 commits

Author SHA1 Message Date
Marko Mäkelä
f3718a112a MDEV-18220: Backport some code from MariaDB 10.2
fts_get_table_name(): Output to a caller-allocated buffer.

fts_get_table_name_prefix(): Use the lower-overhead allocation
ut_malloc() instead of mem_alloc().

This is based on mysql/mysql-server@d1584b9f38
in MySQL 5.7.4.
2019-05-10 07:57:01 +03:00
Sergey Vojtovich
d0b73fb8d3 MDEV-16060 - InnoDB: Failing assertion: ut_strcmp(index->name, key->name)
A sequel to 9180e86 and 149b754.

ALTER TABLE ... ADD FOREIGN KEY may crash if parent table is updated
concurrently.

Block FK parent table updates even earlier, before intermediate child
table is created.

Use proper charset info for my_casedn_str() and don't update original
identifiers so that lower_cast_table_names == 2 is honoured.
2019-05-09 11:13:44 +04:00
Marko Mäkelä
1cd31bc132 Bug#28573894 ALTER PARTITIONED TABLE ADD AUTO_INCREMENT DIFF RESULT DEPENDING ON ALGORITHM
For partitioned table, ensure that the AUTO_INCREMENT values will
be assigned from the same sequence. This is based on the following
change in MySQL 5.6.44:

commit aaba359c13d9200747a609730dafafc3b63cd4d6
Author: Rahul Malik <rahul.m.malik@oracle.com>
Date:   Mon Feb 4 13:31:41 2019 +0530

    Bug#28573894 ALTER PARTITIONED TABLE ADD AUTO_INCREMENT DIFF RESULT DEPENDING ON ALGORITHM

    Problem:
    When a partition table is in-place altered to add an auto-increment column,
    then its values are starting over for each partition.

    Analysis:
    In the case of in-place alter, InnoDB is creating a new sequence object
    for each partition. It is default initialized. So auto-increment columns
    start over for each partition.

    Fix:
    Assign old sequence of the partition to the sequence of next partition
    so it won't start over.

    RB#21148
    Reviewed by Bin Su <bin.x.su@oracle.com>
2019-04-25 14:12:45 +03:00
Marko Mäkelä
9e7bcb05d4 Clean up ib_sequence::m_max_value
Correctly document the usage of m_max_value. Remove the const
qualifier, so that the implicit assignment operator can be used.
Make all members of ib_sequence private, and add an accessor
member function max_value().
2019-04-25 14:12:45 +03:00
Marko Mäkelä
c0fca2863b Fix -Wnonnull-compare
InnoDB and XtraDB had redundant assertions for checking that
function parameters that were declared as nonnull were not NULL.
2019-04-03 09:46:49 +03:00
Marko Mäkelä
23eeecd68f MDEV-19111 Unused field INFORMATION_SCHEMA.INNODB_TABLESPACES_SCRUBBING.ROTATING_OR_FLUSHING
The MDEV-11738/MDEV-11581 fix was supposed to add the column
ROTATING_OR_FLUSHING to the INFORMATION_SCHEMA table
INNODB_TABLESPACES_ENCRYPTION, but it also added that column to
INNODB_TABLESPACES_SCRUBBING in InnoDB (not XtraDB).

The extra column was never initialized. We will remove it,
because key rotation has nothing to do with the scrubbing of
tablespace data.
2019-04-01 10:32:03 +03:00
Marko Mäkelä
d0116e10a5 Revert MDEV-18464 and MDEV-12009
This reverts commit 21b2fada7a
and commit 81d71ee6b2.

The MDEV-18464 change introduces a few data race issues. Contrary to
the documentation, the field trx_t::victim is not always being protected
by lock_sys_t::mutex and trx_t::mutex. Most importantly, it seems
that KILL QUERY could wrongly avoid acquiring both mutexes when
invoking lock_trx_handle_wait_low(), in case another thread had
already set trx->victim=true.

We also revert MDEV-12009, because it should depend on the MDEV-18464
fix being present.
2019-03-28 12:39:50 +02:00
Jan Lindström
21b2fada7a MDEV-18464: Port kill_one_trx fixes from 10.4 to 10.1
Pushed the decision for innodb transaction and system
locking down to lock0lock.cc level. With this,
we can avoid releasing these mutexes for executions
where these mutexes were acquired upfront.

This patch will also fix BF aborting of native threads, e.g.
threads which have declared wsrep_on=OFF. Earlier, we have
used, for innodb trx locks, was_chosen_as_deadlock_victim
flag, for marking inodb transactions, which are victims for
wsrep BF abort. With native threads (wsrep_on==OFF), re-using
was_chosen_as_deadlock_victim flag may lead to inteference
with real deadlock, and to deal with this, the patch has added new
flag for marking wsrep BF aborts only: victim=true

Similar way if replication decides to abort one of the threads
we mark victim by: victim=true

innobase_kill_query
	Remove lock sys and trx mutex handling.

wsrep_innobase_kill_one_trx
	Mark victim trx with victim=true

trx0trx.h
	Remove trx_abort_t type and abort type variable from
	trx struct. Add victim variable to trx.

wsrep_kill_victim
	Remove abort_type

lock_report_waiters_to_mysql
	Take also trx mutex and mark trx as a victim for
	replication abort.

lock_trx_handle_wait_low
	New low level function to check whether the transaction
	has already been rolled back because it was selected as
	a deadlock victim, or if it has to wait then cancel
	the wait lock.

lock_trx_handle_wait
	If transaction is not marked as victim take lock sys
	and trx mutex before calling lock_trx_handle_wait_low
	and release them after that.

row_search_for_mysql
	Remove lock sys and trx mutex taking and releasing.

trx_rollback_to_savepoint_for_mysql_low
trx_commit_in_memory
	Clean up victim variable.
2019-03-28 07:40:03 +02:00
Sergei Golubchik
f97d879bf8 cmake: re-enable -Werror in the maintainer mode
now we can afford it. Fix -Werror errors. Note:
* old gcc is bad at detecting uninit variables, disable it.
* time_t is int or long, cast it for printf's
2019-03-27 22:51:37 +01:00
Marko Mäkelä
e3adf96aeb MDEV-13818 CREATE INDEX leaks memory if running out of undo log space
row_merge_create_index_graph(): Relay the internal state
from dict_create_index_step(). Our caller should free the index
only if it was not copied, added to the cache, and freed.

row_merge_create_index(): Free the index template if it was
not added to the cache. This is a safer variant of the logic
that was introduced in 65070beffd in 10.2.

prepare_inplace_alter_table_dict(): Add additional fault injection
to exercise a code path where we have already added an index
to the cache.
2019-03-07 15:35:55 +02:00
Marko Mäkelä
91e4f00389 MDEV-18732 InnoDB: ALTER IGNORE returns error for NULL
Only starting with MariaDB 10.3.8 (MDEV-16365), InnoDB can actually
handle ALTER IGNORE TABLE correctly when introducing a NOT NULL
attribute to a column that contains a NULL value. Between
MariaDB Server 10.0 and 10.2, we would incorrectly return an error
for ALTER IGNORE TABLE when the column contains a NULL value.
2019-03-04 15:16:27 +02:00
Marko Mäkelä
e39d6e0c53 MDEV-18601 Can't create table with ENCRYPTED=DEFAULT when innodb_default_encryption_key_id!=1
The problem with the InnoDB table attribute encryption_key_id is that it is
not being persisted anywhere in InnoDB except if the table attribute
encryption is specified and is something else than encryption=default.
MDEV-17320 made it a hard error if encryption_key_id is specified to be
anything else than 1 in that case.

Ideally, we would always persist encryption_key_id in InnoDB. But, then we
would have to be prepared for the case that when encryption is being enabled
for a table whose encryption_key_id attribute refers to a non-existing key.

In MariaDB Server 10.1, our best option remains to not store anything
inside InnoDB. But, instead of returning the error that MDEV-17320
introduced, we should merely issue a warning that the specified
encryption_key_id is going to be ignored if encryption=default.

To improve the situation a little more, we will issue a warning if
SET [GLOBAL|SESSION] innodb_default_encryption_key_id is being set
to something that does not refer to an available encryption key.

Starting with MariaDB Server 10.2, thanks to MDEV-5800, we could open the
table definition from InnoDB side when the encryption is being enabled,
and actually fix the root cause of what was reported in MDEV-17320.
2019-02-28 23:20:31 +02:00
Julius Goryavsky
243f829c1c MDEV-9519: Data corruption will happen on the Galera cluster size change
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519
2019-02-25 11:19:07 +02:00
Marko Mäkelä
be25414828 MDEV-18016: Cover the no-rebuild case, and remove a bogus debug assertion
The code path where the table was not being rebuilt during ALTER TABLE
was not covered by the test. Add coverage, and remove the debug assertion
that could fail in this case.
2019-02-11 11:38:18 +02:00
Marko Mäkelä
5eb3e4d83c MDEV-15798 Mutex leak on accessing INFORMATION_SCHEMA.INNODB_MUTEXES
i_s_innodb_mutexes_fill_table(): Use the C++ RAII pattern
to ensure that the mutexes are released if an OK()
macro returns from the function prematurely.
2019-02-05 17:03:41 +02:00
Marko Mäkelä
213ece2f2e Merge 10.1 into 10.1
This is joint work with Oleksandr Byelkin.
2019-02-02 13:00:15 +02:00
Oleksandr Byelkin
560799ebd8 Merge branch '10.0-galera' into 10.1 2019-01-31 09:34:34 +01:00
Marko Mäkelä
1522ee2949 MDEV-18016: Assertion failure on ALTER TABLE after foreign_key_checks=0
ha_innobase::commit_inplace_alter_table(): Do not crash if
innobase_update_foreign_cache() returns an error. It can return
an error on ALTER TABLE if an inconsistent FOREIGN KEY constraint
was created earlier when SET foreign_key_checks=0 was in effect.
Instead, report a warning to the client that constraints cannot
be loaded.
2019-01-29 15:20:26 +02:00
Marko Mäkelä
6699cac0bf MDEV-18256 Duplicated call to dict_foreign_remove_from_cache()
ha_innobase::prepare_inplace_alter_table(): Filter out duplicates
from ha_alter_info->alter_info->drop_list.elements.
2019-01-29 15:20:26 +02:00
Marko Mäkelä
5e06ee41a4 MDEV-18222: Duplicated call to dict_foreign_remove_from_cache()
innobase_rename_column_try(): Declare fk_evict as std::set
instead of std::list, in order to filter out duplicates.
2019-01-29 15:20:26 +02:00
Marko Mäkelä
71eb762611 Merge 10.0 into 10.1 2019-01-17 06:40:24 +02:00
Marko Mäkelä
b4c471099d
Merge pull request #973 from tempesta-tech/tt-10.0-MDEV-16499-virtual-innodb
MDEV-16499 ER_NO_SUCH_TABLE_IN_ENGINE followed by "Please drop the table and recreate" upon adding FULLTEXT key to table with virtual column
2019-01-14 14:06:26 +02:00
Sergei Golubchik
3ba3f81ae0 Merge branch '10.0' into 10.1 2019-01-03 09:56:24 +01:00
Sergei Golubchik
32150d2513 compilation warning on Windows 2019-01-02 19:28:48 +01:00
Sergei Golubchik
aeefd26ecb Merge branch '10.0' into 10.1 2018-12-29 23:44:45 +01:00
Eugene Kosov
802ce9672f MDEV-18041 Database corruption after renaming a prefix-indexed column
This is a regression after MDEV-13671.

The bug is related to key part prefix lengths wich are stored in SYS_FIELDS.
Storage format is not obvious and was handled incorrectly which led to data
dictionary corruption.

SYS_FIELDS.POS actually contains prefix length too in case if any key part
has prefix length.

innobase_rename_column_try(): fixed prefixes handling

Tests for prefixed indexes added too.

Closes #1063
2018-12-29 22:57:05 +02:00
Marko Mäkelä
f77f8f6d1a Merge 10.0 into 10.1 2018-12-12 10:48:53 +02:00
Eugene Kosov
d956709b4b MDEV-17833 ALTER TABLE is not enforcing prefix index size limit
ha_innobase::prepare_inplace_alter_table(): check max column length for every
index in a table, not just added in this particular ALTER TABLE with ADD INDEX ones.
2018-12-11 22:03:44 +03:00
Marko Mäkelä
ecd3a7e00d Merge 10.0 into 10.1 2018-12-07 13:17:32 +02:00
Marko Mäkelä
12b1ba195c MDEV-17904 Crash in fts_is_sync_needed() after failed ALTER or CREATE TABLE
create_table_def(), ha_innobase::create(): Defer fts_optimize_add_table()
until after the table has been successfully created.
2018-12-07 13:12:39 +02:00
Eugene Kosov
e8bb94ccc8 MDEV-16499 [10.1] ER_NO_SUCH_TABLE_IN_ENGINE followed by "Please drop the table and recreate" upon adding FULLTEXT key to table with virtual column
There was an incorrect check for MariaDB and InnoDB
tables fields count. Corruption was reported when there was no corruption.
Also, a warning message had incorrect field numbers for both MariaDB and InnoDB
tables.

ha_innobase::open(): fixed check and message
2018-11-30 01:15:30 +03:00
Marko Mäkelä
59c82dde09 Merge 10.0 into 10.1 2018-11-07 08:08:45 +02:00
Anel Husakovic
9c026273a9 Add implementation in .h and delete unneccessery printing 2018-11-06 12:07:26 +00:00
Jan Lindström
ef40018535 MDEV-17230: encryption_key_id from alter is ignored by encryption threads
Background: Used encryption key_id is stored to encryption metadata
i.e. crypt_data that is stored on page 0 of the tablespace of the
table. crypt_data is created only if implicit encryption/not encryption
is requested i.e. ENCRYPTED=[YES|NO] table option is used
fil_create_new_single_table_tablespace on fil0fil.cc.

Later if encryption is enabled all tables that use default encryption
mode (i.e. no encryption table option is set) are encrypted with
default encryption key_id that is 1. See fil_crypt_start_encrypting_space on
fil0crypt.cc.

ha_innobase::check_table_options()
	If default encryption is used and encryption is disabled, you may
        not use nondefault encryption_key_id as it is not stored anywhere.
2018-11-06 10:22:25 +02:00
Marko Mäkelä
bae21bfb5d Merge 10.0 into 10.1 2018-11-05 17:50:41 +02:00
Marko Mäkelä
db55b39fb2 Revert some InnoDB/XtraDB changes
The relevant InnoDB/XtraDB fixes up to 5.6.42 had already
been applied to MariaDB in commit 30c3d6db32.

Revert some changes that appeared in
the merge commit 87d852f102.
2018-11-05 16:47:14 +02:00
Marko Mäkelä
d63e198061 Merge 10.0 into 10.1 2018-11-05 12:15:17 +02:00
Eugene Kosov
03977e8273 MDEV-13671 InnoDB should use case-insensitive column name comparisons like the rest of the server
Problem affects INPLACE ALTER rename columns.

innobase_rename_column_try(): some strcmp() was replaced with my_strcasecmp(),
queries to update data dictionary was updated to not match column name case.
2018-11-05 11:59:59 +02:00
Jan Lindström
d6ee7ab1a1 Merge remote-tracking branch 'origin/10.0' into bb-10.0-galera 2018-10-31 08:46:37 +02:00
Sergei Golubchik
44f6f44593 Merge branch '10.0' into 10.1 2018-10-30 15:10:01 +01:00
Jan Lindström
b0fe082b36 Merge remote-tracking branch 'origin/5.5-galera' into 10.0-galera 2018-10-30 13:22:52 +02:00
Sergei Golubchik
87d852f102 Merge branch 'merge/merge-innodb-5.6' into 10.0 2018-10-28 01:22:18 +02:00
Sergei Golubchik
da34c7de5d 5.6.42 2018-10-27 21:05:16 +02:00
Thirunarayanan Balathandayuthapani
1dacd5f299 MDEV-12547: InnoDB FULLTEXT index has too strict innodb_ft_result_cache_limit max limit
- Backported the MYSQL_SYSVAR_SIZE_T to 10.0
- The parameter innodb_ft_result_cache_limit was only 32 bits wide
also on 64-bit systems. Make it size_t, so that it will be 64 bits
on 64-bit systems.
- Added a test case that show how innodb_ft_result_cache_limit variables
behaves in 32bit and 64 bit system.
2018-10-16 13:02:50 +05:30
Vasil Dimov
6aa578ec5a Generalize "bool shared/exclusive" argument to enum
wsrep_append_foreign_key() and wsrep_append_key() used to take a boolean
argument denoting whether the relevant certification key type is shared
(assuming it is exclusive if the argument is false). Change that
argument to the enum wsrep_key_type from wsrep_api.h, so that eventually
other types can also be passed (like WSREP_KEY_SEMI).

This is a non-functional change.

(cherry picked from commit 360bf36dbb9378b36ef57921c725a9505e19e0d9)
2018-10-10 13:02:25 +03:00
Oleksandr Byelkin
31081593aa Merge branch '11.0' into 10.1 2018-09-06 22:45:19 +02:00
Sergei Golubchik
d831cefb43 MDEV-16465 Invalid (old?) table or database name or hang in ha_innobase::delete_table and log semaphore wait upon concurrent DDL with foreign keys
Disable "Invalid (old?) table or database name" warning when
converting table names in InnoDB's get_foreign_key_info().

Because a name can be a temporary table name during the ALTER TABLE,
and some other thread can do SHOW CREATE TABLE for the other table
in the FK relationships _anytime_.
2018-09-04 09:49:53 +02:00
Sergei Golubchik
8bee7c16c8 compiler warnings (clang 4.0.1 on i386)
extra/mariabackup/fil_cur.cc:361:42: warning: format specifies type 'unsigned long' but the argument has type 'ib_int64_t' (aka 'long long') [-Wformat]
extra/mariabackup/fil_cur.cc:376:9: warning: format specifies type 'unsigned long' but the argument has type 'ib_int64_t' (aka 'long long') [-Wformat]
sql/handler.cc:6196:45: warning: format specifies type 'unsigned long' but the argument has type 'wsrep_trx_id_t' (aka 'unsigned long long') [-Wformat]
sql/log.cc:1681:16: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
sql/log.cc:1687:16: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
sql/wsrep_sst.cc:1388:86: warning: format specifies type 'long' but the argument has type 'wsrep_seqno_t' (aka 'long long') [-Wformat]
sql/wsrep_sst.cc:232:86: warning: format specifies type 'long' but the argument has type 'wsrep_seqno_t' (aka 'long long') [-Wformat]
storage/connect/filamdbf.cpp:450:47: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
storage/connect/filamdbf.cpp:970:47: warning: format specifies type 'short' but the argument has type 'int' [-Wformat]
storage/connect/inihandl.cpp:197:16: warning: address of array 'key->name' will always evaluate to 'true' [-Wpointer-bool-conversion]
storage/innobase/btr/btr0scrub.cc:151:17: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/innobase/buf/buf0buf.cc:5085:8: warning: nonnull parameter 'bpage' will evaluate to 'true' on first encounter [-Wpointer-bool-conversion]
storage/innobase/fil/fil0crypt.cc:2454:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/innobase/handler/ha_innodb.cc:18685:7: warning: format specifies type 'unsigned long' but the argument has type 'wsrep_trx_id_t' (aka 'unsigned long long') [-Wformat]
storage/innobase/row/row0mysql.cc:3319:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/innobase/row/row0mysql.cc:3327:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/maria/ma_norec.c:35:10: warning: implicit conversion from 'int' to 'my_bool' (aka 'char') changes value from 131 to -125 [-Wconstant-conversion]
storage/maria/ma_norec.c:42:10: warning: implicit conversion from 'int' to 'my_bool' (aka 'char') changes value from 131 to -125 [-Wconstant-conversion]
storage/maria/ma_test2.c:1009:12: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
storage/maria/ma_test2.c:1010:12: warning: format specifies type 'unsigned long' but the argument has type 'size_t' (aka 'unsigned int') [-Wformat]
storage/mroonga/ha_mroonga.cpp:9189:44: warning: use of logical '&&' with constant operand [-Wconstant-logical-operand]
storage/mroonga/vendor/groonga/lib/expr.c:4987:22: warning: comparison of constant -1 with expression of type 'grn_operator' is always false [-Wtautological-constant-out-of-range-compare]
storage/xtradb/btr/btr0scrub.cc:151:17: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/xtradb/buf/buf0buf.cc:5047:8: warning: nonnull parameter 'bpage' will evaluate to 'true' on first encounter [-Wpointer-bool-conversion]
storage/xtradb/fil/fil0crypt.cc:2454:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/xtradb/row/row0mysql.cc:3324:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
storage/xtradb/row/row0mysql.cc:3332:5: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
unittest/sql/mf_iocache-t.cc:120:35: warning: format specifies type 'unsigned long' but the argument has type 'int' [-Wformat]
unittest/sql/mf_iocache-t.cc:96:35: note: expanded from macro 'INFO_TAIL'
2018-09-04 09:19:48 +02:00
Sergei Golubchik
22bcfa011a cleanup: FOREIGN_KEY_INFO
instead of returning strings for CASCADE/RESTRICT
from every storage engine, use enum values

Backport of a3614d33e8
2018-09-04 08:37:44 +02:00
Marko Mäkelä
3b5d3cd68e Revert MDEV-9519 due to regressions
This reverts commit 75dfd4acb9.
2018-08-31 12:36:31 +03:00