Commit graph

21485 commits

Author SHA1 Message Date
Marko Mäkelä
9835f7b80f Merge 10.1 into 10.2 2019-03-04 16:46:58 +02:00
Marko Mäkelä
91e4f00389 MDEV-18732 InnoDB: ALTER IGNORE returns error for NULL
Only starting with MariaDB 10.3.8 (MDEV-16365), InnoDB can actually
handle ALTER IGNORE TABLE correctly when introducing a NOT NULL
attribute to a column that contains a NULL value. Between
MariaDB Server 10.0 and 10.2, we would incorrectly return an error
for ALTER IGNORE TABLE when the column contains a NULL value.
2019-03-04 15:16:27 +02:00
Olivier Bertrand
d421df7ea6 - Fix MDEV-18292: CONNECT Engine JDBC not able to issue
simple UPDATE statement from trigger or stored procedure
  Was not fixed when the same table was called several times
  with different modes. Fixed by checking if a new statement
  is compatible in the start_stmt function. It nows do the
  same checks than external_lock.
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/ha_connect.h

- typo
  modified:   storage/connect/user_connect.cc
2019-03-03 21:24:02 +01:00
Sergei Golubchik
4ca2079142 MDEV-18486 Database crash on a table with indexed virtual column
don't do anything special for stored generated columns
in MyISAM repair code.

add an assert that if there are virtual indexed columns, they
_must_ be beyond the file->s->base.reclength boundary
2019-03-01 13:23:34 -05:00
Marko Mäkelä
4a1c66290d MDEV-18775 Fix ALTER TABLE error handling for DROP INDEX
On an error (such as when an index cannot be dropped due to
FOREIGN KEY constraints), the field dict_index_t::to_be_dropped
was only being cleared in debug builds, even though the field
is available and being used also in non-debug builds.

This was a regression that was introduced by myself originally
in MySQL 5.7.6 and later merged to MariaDB 10.2.2, in
d39898de8e

An error manifested itself in the MariaDB Server 10.4 non-debug build,
involving instant ADD or DROP column. Because an earlier failed
ALTER TABLE operation incorrectly left the dict_index_t::to_be_dropped
flag set, the column pointers of the index fields would fail to be
adjusted for instant ADD or DROP column (MDEV-15562). The instant
ADD COLUMN in MariaDB Server 10.3 is unlikely to be affected by a
similar scenario, because dict_table_t::instant_add_column() in 10.3
is applying the transformations to all indexes, not skipping
to-be-dropped ones.
2019-03-01 12:16:12 +02:00
Marko Mäkelä
e39d6e0c53 MDEV-18601 Can't create table with ENCRYPTED=DEFAULT when innodb_default_encryption_key_id!=1
The problem with the InnoDB table attribute encryption_key_id is that it is
not being persisted anywhere in InnoDB except if the table attribute
encryption is specified and is something else than encryption=default.
MDEV-17320 made it a hard error if encryption_key_id is specified to be
anything else than 1 in that case.

Ideally, we would always persist encryption_key_id in InnoDB. But, then we
would have to be prepared for the case that when encryption is being enabled
for a table whose encryption_key_id attribute refers to a non-existing key.

In MariaDB Server 10.1, our best option remains to not store anything
inside InnoDB. But, instead of returning the error that MDEV-17320
introduced, we should merely issue a warning that the specified
encryption_key_id is going to be ignored if encryption=default.

To improve the situation a little more, we will issue a warning if
SET [GLOBAL|SESSION] innodb_default_encryption_key_id is being set
to something that does not refer to an available encryption key.

Starting with MariaDB Server 10.2, thanks to MDEV-5800, we could open the
table definition from InnoDB side when the encryption is being enabled,
and actually fix the root cause of what was reported in MDEV-17320.
2019-02-28 23:20:31 +02:00
Jan Lindström
f65f40bb35 Merge remote-tracking branch 'origin/10.1' into 10.2 2019-02-28 13:08:11 +02:00
Julius Goryavsky
50b3632fa4 MDEV-9519: Data corruption will happen on the Galera cluster size change
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519
2019-02-26 08:09:04 +02:00
Julius Goryavsky
2c734c980e MDEV-9519: Data corruption will happen on the Galera cluster size change
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519
2019-02-26 07:45:11 +02:00
Julius Goryavsky
243f829c1c MDEV-9519: Data corruption will happen on the Galera cluster size change
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519
2019-02-25 11:19:07 +02:00
Marko Mäkelä
15a20367fb buf_page_get_zip(): Deduplicate some code 2019-02-23 10:31:07 +02:00
Marko Mäkelä
2c8d9a4e59 MDEV-10813: Update buf_page_t::buf_fix_count outside mutex
Since MySQL 5.6.16 (and MariaDB Server 10.0.11), changes of
buf_page_t::buf_fix_count are atomic memory operations if
PAGE_ATOMIC_REF_COUNT is defined. Since MySQL 5.7
(and MariaDB Server 10.2.2), the field is always updated
by atomic memory operations.

In a few occurrences, updates of the counter were unnecessarily
surrounded by an acquisition and release of the block mutex
(buf_block_t::mutex or buf_pool_t::zip_mutex). Remove these
unnecessary mutex operations.
2019-02-22 22:56:22 +02:00
Eugene Kosov
28cb041754 MDEV-18662 ib_wqueue_t has a data race
ib_wqueue_is_empty(): protect ib_list_is_empty() call

Closes #1202
2019-02-21 12:23:47 +02:00
Marko Mäkelä
fc124778ea Merge 10.2 into 10.3 2019-02-19 17:41:13 +02:00
Thirunarayanan Balathandayuthapani
88b6dc4db5 MDEV-18639 Assertion failure upon attempt to start with lower number of undo tablespaces
trx_rseg_t::is_persistent(): Correct a bogus debug assertion.
2019-02-19 12:18:50 +02:00
Marko Mäkelä
af6fdc1307 Merge 10.1 into 10.2 2019-02-19 11:15:10 +02:00
Marko Mäkelä
ca76fc4a3a MDEV-18611: mariabackup silently ended during xtrabackup_copy_logfile()
log_group_read_log_seg(): Always return false when returning
before reading end_lsn.

xtrabackup_copy_logfile(): On error, indicate whether
a corrupt log record was encountered.

Only xtrabackup_copy_logfile() in Mariabackup cared about the
return value of the function. InnoDB crash recovery was not
affected by this bug.
2019-02-19 11:14:03 +02:00
Monty
346e460896 Fixed bug in macro _ma_mark_page_with_transid()
By pure chance the macro worked in the cases it was used, but
better to get this fixed!
2019-02-19 10:51:34 +02:00
Marko Mäkelä
e3f6ea5080 Merge 10.1 into 10.2 2019-02-18 22:21:02 +02:00
Marko Mäkelä
98e185ee37 MDEV-18630 Uninitialised value in FOREIGN KEY error message
dict_create_foreign_constraints_low(): Clean up the way in
which the error messages are initialized, and ensure that
the table name is always initialized.
2019-02-18 21:42:58 +02:00
Aleksey Midenkov
a4cd91c526 MDEV-18603 Debug option to disable InnoDB monitor threads
If --debug=d,innodb_skip_monitors is specified, skip the creation
of srv_error_monitor_thread and srv_monitor_thread. In this way,
interactive debugging sessions will not be interfered by messages
about a mutex or rw_lock being held for a long time.

Closes #1192
2019-02-18 08:19:22 +02:00
Alexey Botchkov
e8b6c15010 connect.xml.result fixed. 2019-02-13 23:26:23 +04:00
Daniel Black
ce0678f6cb hash (storage): hp_hashnr is local
Remove 'register' quantifer from args too.
2019-02-13 23:15:55 +04:00
Daniel Black
7fa67e391f heap: remove NEW_HASH_FUNCTION
This was added in 29fa72526a (in 2000) and nothing defines
NEW_HASH_FUNCTION. A low collision is the wrong tradeoff,
it should be focused on throughput.

d405bee058 shows the existing function
is pretty cpu intensive and needs to be improved as well.
2019-02-13 23:15:55 +04:00
Marko Mäkelä
4e7ee166a9 MDEV-18295 IMPORT TABLESPACE fails with instant-altered tables
When importing a tablespace, we must initialize dummy DEFAULT NULL
values for any instantly added columns in order to avoid a debug
assertion failure when PageConverter::update_records() invokes
rec_get_offsets(). Finally, when the operation completes, we must
evict and reload the table definition, so that the correct
default values for instantly added columns will be loaded.

ha_innobase::discard_or_import_tablespace(): On successful
IMPORT TABLESPACE, evict and reload the table definition,
so that btr_cur_instant_init() will load the correct metadata.

PageConverter::update_index_page(): Fill in dummy DEFAULT NULL values
for instantly added columns. These will be replaced upon the
completion of the operation by evicting and reloading the metadata.

row_discard_tablespace(): Invoke dict_table_t::remove_instant().
After DISCARD TABLESPACE, the table is no longer in "instant ALTER"
format, because there is no data file attached.
2019-02-11 14:42:48 +02:00
Marko Mäkelä
9e4f299404 Merge 10.1 into 10.2 2019-02-11 11:42:18 +02:00
Marko Mäkelä
be25414828 MDEV-18016: Cover the no-rebuild case, and remove a bogus debug assertion
The code path where the table was not being rebuilt during ALTER TABLE
was not covered by the test. Add coverage, and remove the debug assertion
that could fail in this case.
2019-02-11 11:38:18 +02:00
Sergei Golubchik
ca325a46d2 CONNECT: update test results 2019-02-10 00:21:43 +01:00
Sergei Golubchik
894f44b60b CONNECT: Windows paths
Followup for db8f0daeb4
2019-02-09 22:54:10 +01:00
Olivier Bertrand
dc8d1ada9f Fix xml result by setting eol as lf modified: storage/connect/mysql-test/connect/r/xml.result 2019-02-09 18:24:28 +01:00
Olivier Bertrand
ae88fe4519 Fix xml result mismatch modified: storage/connect/mysql-test/connect/r/xml.result 2019-02-09 17:03:20 +01:00
Olivier Bertrand
58dfdfc0dd - Fix GetTableName that returned wrong value under Windows
modified:   storage/connect/ha_connect.cc
2019-02-09 13:07:56 +01:00
Monty
d0799a0479 Removed compiler warnings from tokudb
- Backport from 10.4
2019-02-06 22:18:20 +02:00
Marko Mäkelä
e84dc567be Merge 10.1 into 10.2 2019-02-05 17:35:48 +02:00
Marko Mäkelä
5eb3e4d83c MDEV-15798 Mutex leak on accessing INFORMATION_SCHEMA.INNODB_MUTEXES
i_s_innodb_mutexes_fill_table(): Use the C++ RAII pattern
to ensure that the mutexes are released if an OK()
macro returns from the function prematurely.
2019-02-05 17:03:41 +02:00
Marko Mäkelä
625994b7cc MDEV-16982 Server crashes in mem_heap_dup upon DELETE from table with virtual columns
An uninitialized buffer is passed to row_sel_store_mysql_rec() but
InnoDB may not initialize everything. Looks like it's ok in most cases
but not always.
The partially initialized buffer was later passed to
ha_innobase::write_row() which reads random NULL bit values for
virtual columns and random stuff happens.

No test case for MariaDB 10.2 was found.
The test case for MariaDB 10.3 involves partitioning,
system versioning and the TRASH_ALLOC fill pattern 0xA5.
Test case depends very much on the number and layout of columns.
Think about 0xA5 byte for a NULL bit mask.

row_sel_store_mysql_rec(): always initialize virtual columns NULL bit

Closes #1144
2019-02-05 12:02:41 +02:00
Sergei Golubchik
db8f0daeb4 CONNECT: calculate table_name from the path
CONNECT used to compute table file path as
table->s->db_name + table->s->table_name
instead of using table->s->path. This was incorrect and now it breaks
for temporary tables during ALTER TABLE.

Temporarily "fix" it by making CONNECT to use what it always used
as a table name - the last component in the table->s->path.
2019-02-05 01:34:17 +01:00
Sergei Golubchik
78d5a764b2 compiler warnings 2019-02-05 01:34:17 +01:00
Oleksandr Byelkin
1ed1b7794f Merge remote-tracking branch 'connect/11.2' into 10.2 2019-02-04 16:09:42 +01:00
Marko Mäkelä
ab2458c61f Merge 10.2 into 10.3 2019-02-04 15:12:14 +02:00
Marko Mäkelä
a249e57b68 Merge 10.1 into 10.2
Temporarily disable a test for
commit 2175bfce3e
because fixing it in 10.2 requires updating libmariadb.
2019-02-03 17:22:05 +02:00
Marko Mäkelä
955c7b3222 MDEV-16896 encryption.innodb-checksum-algorithm crashes
buf_page_is_corrupted(): Read the global variable srv_checksum_algorithm
only once in order to avoid a race condition when
SET GLOBAL innodb_checksum_algorithm=...;
is being executed concurrently with this function.
2019-02-03 17:09:20 +02:00
Olivier Bertrand
a0e26599a3 - Fix MDEV-13136: enhance CREATE SERVER MyServerName
FOREIGN DATA WRAPPER to work with CONNECT engine
  modified:   storage/connect/tabjdbc.cpp

- Add a function to retrieve User variable value (DEVELOPMENT only)
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/jsonudf.cpp
  modified:   storage/connect/jsonudf.h
  modified:   storage/connect/tabjdbc.cpp
2019-02-03 15:19:04 +01:00
Marko Mäkelä
213ece2f2e Merge 10.1 into 10.1
This is joint work with Oleksandr Byelkin.
2019-02-02 13:00:15 +02:00
Marko Mäkelä
c1e1764fc4 Fix embedded innodb_plugin after 560799ebd8
wsrep_certification_rules: Define as a weak global symbol.
While there are separate _embedded.a for statically
linked storage engine plugins, there is only one ha_innodb.so
which is supposed to work with both values of WITH_WSREP.

The merge from 10.0-galera introduced a reference to a global
variable that is only defined when the server is built WITH_WSREP.
We must define that symbol as weak global, so that when
a dynamically linked InnoDB or XtraDB is used with the embedded
server (which never includes write-set replication patches),
the variable will be read as 0, instead of causing a failure to
load the InnoDB or XtraDB plugin.
2019-02-02 12:49:04 +02:00
Marko Mäkelä
081fd8bfa2 Merge 10.1 into 10.2 2019-02-02 11:40:02 +02:00
Thirunarayanan Balathandayuthapani
7c7161a1bd MDEV-18194 Incremental prepare tries to access page which is out of tablespace bounds
Problem:
=======
Mariabackup incremental prepare creates new tablespace when it encounter
new tablespace. It sets the intial size as FIL_IBD_FILE_INITIAL_SIZE (4).
But while applying redo log, it tries to access 5th page and then
it leads to out of tablespace error.

Fix:
===
While parsing the redo log record, track FSP_SIZE in recv_spaces for the
respective space id. Assign the recv_size for the tablespace when it
is loaded. Extend the tablespace depends on recv_size while applying
the redo log record.
2019-02-01 09:15:53 +02:00
Marko Mäkelä
6051842cd0 Minor clean-up 2019-02-01 09:13:18 +02:00
Thirunarayanan Balathandayuthapani
f669cecbe3 MDEV-18415 mariabackup.mdev-14447 test case fails with Table 'test.t' doesn't exist in engine
- Added retry logic if validation of first page fails with checksum
mismatch.
2019-02-01 08:53:50 +02:00
Oleksandr Byelkin
560799ebd8 Merge branch '10.0-galera' into 10.1 2019-01-31 09:34:34 +01:00