Commit graph

181604 commits

Author SHA1 Message Date
Marko Mäkelä
913e33e423 Merge 10.1 into 10.2
Rewrite the MDEV-13818 fix to prevent heap-use-after-free.

Add a test case for MDEV-18272.
2019-03-07 17:52:27 +02:00
Marko Mäkelä
e3adf96aeb MDEV-13818 CREATE INDEX leaks memory if running out of undo log space
row_merge_create_index_graph(): Relay the internal state
from dict_create_index_step(). Our caller should free the index
only if it was not copied, added to the cache, and freed.

row_merge_create_index(): Free the index template if it was
not added to the cache. This is a safer variant of the logic
that was introduced in 65070beffd in 10.2.

prepare_inplace_alter_table_dict(): Add additional fault injection
to exercise a code path where we have already added an index
to the cache.
2019-03-07 15:35:55 +02:00
Marko Mäkelä
1a5028f430 Fix the WITH_ASAN clang build of dynamic plugins 2019-03-07 14:32:20 +02:00
Marko Mäkelä
4c0f43f45a Merge 10.0 into 10.1 2019-03-07 12:27:42 +02:00
Marko Mäkelä
0a0eed8016 Merge 5.5 into 10.0 2019-03-07 12:04:22 +02:00
Marko Mäkelä
8024f8c6b8 MDEV-18272 InnoDB fails to rollback after exceeding FOREIGN KEY recursion depth
row_mysql_handle_errors(): Correct the wrong error handling for
the code DB_FOREIGN_EXCEED_MAX_CASCADE that was introduced in

c0923d396a

    commit 35f5429eda
    Author: Jimmy Yang <jimmy.yang@oracle.com>
    Date:   Wed Oct 6 06:55:34 2010 -0700

        Manual port Bug #Bug #54582 "stack overflow when opening many tables
        linked with foreign keys at once" from mysql-5.1-security to
        mysql-5.5-security again.

        rb://391 approved by Heikki

No known test case exists for repeating the bug before MariaDB 10.2.
The scenario should be that DB_FOREIGN_EXCEED_MAX_CASCADE is returned,
then InnoDB wrongly skips the rollback to the start of the current
row operation, and finally the SQL layer commits the transaction.
Normally the SQL layer would roll back either the entire transaction or
to the start of the statement. In the faulty scenario, InnoDB would
leave the transaction in an inconsistent state, and the SQL layer could
commit the transaction.
2019-03-07 11:57:14 +02:00
Galina Shalygina
2faefe5f7f MDEV-18383: Missing rows with pushdown condition defined with IF-function
using Item_cond

This bug is similar to the bug MDEV-16765.
It appears because of the wrong pushdown into HAVING clause while this
pushdown shouldn't be made at all.
This happens because function that checks if Item_cond can be pushed
always returns that it can be pushed.

To fix it new method Item_cond::excl_dep_on_table() was added.
2019-03-07 12:35:18 +03:00
Sergei Golubchik
57dd892ce8 fix memory leaks in mysql_client_test
This fixes main.mysql_client_test, main.mysql_client_test_comp,
main.mysql_client_test_nonblock failures
in ASAN_OPTIONS="abort_on_error=1" runs
2019-03-06 15:28:27 +01:00
Sergei Golubchik
2f742b571f MDEV-18376 Memory leak in main.mysqladmin
don't initialize mysql structure before it actually becomes needed.

This fixes main.mysqladmin failures
in ASAN_OPTIONS="abort_on_error=1" runs
2019-03-06 15:28:27 +01:00
Sergei Golubchik
65070beffd MDEV-13818 CREATE INDEX leaks memory if running out of undo log space
free already allocated indexes if row_merge_create_index() fails

This fixes innodb.alter_crash failure
in ASAN_OPTIONS="abort_on_error=1" runs
2019-03-06 15:28:27 +01:00
Sergei Golubchik
2b4027e633 mronga: fix a memory leak
use the correct delete operator

This fixes mroonga/storage.column_generated_stored_add_column failures
in ASAN_OPTIONS="abort_on_error=1" runs
2019-03-06 15:28:27 +01:00
Sergei Golubchik
5f105e756b MDEV-18625 ASAN unknown-crash in my_copy_fix_mb / ha_mroonga::storage_inplace_alter_table_add_column
disable inplace alter for adding stored generated columns.

This fixes mroonga/storage.column_generated_stored_add_column failures
in ASAN_OPTIONS="abort_on_error=1" runs

Also, add a test case that shows the bug without ASAN.
2019-03-06 15:28:27 +01:00
Sergei Golubchik
84645366c4 ASAN loves stack, give it some
fixes these test failures in ASAN builds (in 10.1 and 10.4):
* main.signal_demo3
* main.sp
* sys_vars.max_sp_recursion_depth_func
* mroonga/storage.foreign_key_delete_existent
* mroonga/storage.foreign_key_delete_nonexistent
* mroonga/storage.foreign_key_insert_existent
* mroonga/storage.foreign_key_update_existent
* mroonga/storage.foreign_key_update_nonexistent
* mroonga/storage.function_command_auto-escape
* mroonga/storage.function_command_select
* mroonga/storage.variable_enable_operations_recording_insert
2019-03-06 15:12:11 +01:00
Sergei Golubchik
5ce6fb59a0 disable LeakSanitizer for unit.dbug test 2019-03-06 15:12:11 +01:00
Marko Mäkelä
c155946c90 Merge 10.1 into 10.2 2019-03-06 15:15:59 +02:00
Alexander Barkov
26f0d72a3f A cleanup for MDEV-18333 Slow_queries count doesn't increase when slow_query_log is turned off
thd->lex->m_sql_cmd was not cleared between queries.
log_slow_query() could crash (when running mtr --ps) because of this.
2019-03-06 17:08:03 +04:00
Marko Mäkelä
485dcb07d1 MDEV-18637 Assertion `cache' failed in fts_init_recover_doc
I know no test case for this bug in 10.1. So a test case will be
committed separately in 10.2

fts_reset_get_doc(): properly initialize fts_get_doc_t::cache
2019-03-06 14:46:58 +02:00
Marko Mäkelä
4b5dc47f56 MDEV-18659: Revert a non-functional change
fts_fetch_index_words(): Restore the initialization len=0.
The test innodb_fts.create in 10.2 would end up in an infinite loop
if this assignment is removed, because a following iteration of the
while() loop would assign zip->zp->avail_in=len with the original value
instead of the 0 that was reset in the previous iteration.
2019-03-06 12:45:54 +02:00
Marko Mäkelä
b761211685 MDEV-18659: Fix string truncation/overflow in InnoDB and XtraDB
Fix the warnings issued by GCC 8 -Wstringop-truncation
and -Wstringop-overflow in InnoDB and XtraDB.

This work is motivated by Jan Lindström. The patch mainly differs
from his original one as follows:

(1) We remove explicit initialization of stack-allocated string buffers.
The minimum amount of initialization that is needed is a terminating
NUL character.
(2) GCC issues a warning for invoking strncpy(dest, src, sizeof dest)
because if strlen(src) >= sizeof dest, there would be no terminating
NUL byte in dest. We avoid this problem by invoking strncpy() with
a limit that is 1 less than the buffer size, and by always writing
NUL to the last byte of the buffer.
(3) We replace strncpy() with memcpy() or strcpy() in those cases
when the result is functionally equivalent.

Note: fts_fetch_index_words() never deals with len==UNIV_SQL_NULL.
This was enforced by an assertion that limits the maximum length
to FTS_MAX_WORD_LEN. Also, the encoding that InnoDB uses for
the compressed fulltext index is not byte-order agnostic, that is,
InnoDB data files that use FULLTEXT INDEX are not portable between
big-endian and little-endian systems.
2019-03-06 11:22:27 +02:00
Marko Mäkelä
b21930fb0f MDEV-18749: Uninitialized value upon ADD FULLTEXT INDEX
row_merge_create_fts_sort_index(): Initialize dict_col_t.

This fixes an access to uninitialized dict_col_t::ind when a debug
assertion in MariaDB 10.4 invokes is_dropped() in
rec_get_converted_size_comp_prefix_low(). Older MariaDB versions
seem to be unaffected by the uninitialized values, but it should
not hurt to initialize everything.
2019-03-06 10:37:43 +02:00
Marko Mäkelä
90f09ba8c2 MDEV-18819: Add a test case 2019-03-05 11:53:23 +02:00
Marko Mäkelä
730ed9907b After-merge fix: Add explicit type conversion
Lesson learned: A HA_TOPTION_SYSVAR that is bound to
MYSQL_THDVAR_UINT does not have the type uint, but ulonglong.
2019-03-04 18:38:42 +02:00
Marko Mäkelä
9835f7b80f Merge 10.1 into 10.2 2019-03-04 16:46:58 +02:00
Marko Mäkelä
74d648db12 Make a size_t-to-uint conversion explicit 2019-03-04 16:25:14 +02:00
Marko Mäkelä
4fbf86454f Fix Galera results 2019-03-04 16:25:08 +02:00
Marko Mäkelä
91e4f00389 MDEV-18732 InnoDB: ALTER IGNORE returns error for NULL
Only starting with MariaDB 10.3.8 (MDEV-16365), InnoDB can actually
handle ALTER IGNORE TABLE correctly when introducing a NOT NULL
attribute to a column that contains a NULL value. Between
MariaDB Server 10.0 and 10.2, we would incorrectly return an error
for ALTER IGNORE TABLE when the column contains a NULL value.
2019-03-04 15:16:27 +02:00
Alexander Barkov
19df45a705 MDEV-18333 Slow_queries count doesn't increase when slow_query_log is turned off 2019-03-04 13:49:15 +04:00
Sergei Golubchik
4ca2079142 MDEV-18486 Database crash on a table with indexed virtual column
don't do anything special for stored generated columns
in MyISAM repair code.

add an assert that if there are virtual indexed columns, they
_must_ be beyond the file->s->base.reclength boundary
2019-03-01 13:23:34 -05:00
Sergei Golubchik
6793736009 .gitignore 2019-03-01 12:41:05 -05:00
Sergei Golubchik
5e2d2053d8 bugfix: set mysql_real_data_home_len correctly
after mysql_real_data_home was updated in get_options()
2019-03-01 12:41:05 -05:00
Sergei Golubchik
8d47d9ed88 SSL test fixes
* fix CRL tests to work
* regenerate certificates to be at least 2048 bit
  (fixes buster and rhel8 in buildbot)
* update generate-ssl-cert.sh to generate crl files
* make all SSL tests to use certificates generated
  in generate-ssl-cert.sh, remove unused certificates

Backport from 10.4 9c60535f86
2019-03-01 12:41:05 -05:00
Sergei Golubchik
20043cf650 remove aws kms plugin from debian builds too 2019-03-01 12:40:43 -05:00
Sergei Golubchik
000c6cfc61 debian: more robust control file hacking
when deleting a package, delete up to the next empty line,
instead of relying on a package description being of
some fixed number of lines long
2019-03-01 12:40:43 -05:00
Oleksandr Byelkin
f2e1451740 Merge branch '10.0' into 10.1 2019-03-01 15:52:06 +01:00
Marko Mäkelä
4a1c66290d MDEV-18775 Fix ALTER TABLE error handling for DROP INDEX
On an error (such as when an index cannot be dropped due to
FOREIGN KEY constraints), the field dict_index_t::to_be_dropped
was only being cleared in debug builds, even though the field
is available and being used also in non-debug builds.

This was a regression that was introduced by myself originally
in MySQL 5.7.6 and later merged to MariaDB 10.2.2, in
d39898de8e

An error manifested itself in the MariaDB Server 10.4 non-debug build,
involving instant ADD or DROP column. Because an earlier failed
ALTER TABLE operation incorrectly left the dict_index_t::to_be_dropped
flag set, the column pointers of the index fields would fail to be
adjusted for instant ADD or DROP column (MDEV-15562). The instant
ADD COLUMN in MariaDB Server 10.3 is unlikely to be affected by a
similar scenario, because dict_table_t::instant_add_column() in 10.3
is applying the transformations to all indexes, not skipping
to-be-dropped ones.
2019-03-01 12:16:12 +02:00
Jan Lindström
003b507416 Merge remote-tracking branch 'origin/10.1' into 10.2 2019-03-01 09:17:19 +02:00
Oleksandr Byelkin
2d34713294 Increase the version 2019-02-28 23:40:49 +01:00
Marko Mäkelä
e39d6e0c53 MDEV-18601 Can't create table with ENCRYPTED=DEFAULT when innodb_default_encryption_key_id!=1
The problem with the InnoDB table attribute encryption_key_id is that it is
not being persisted anywhere in InnoDB except if the table attribute
encryption is specified and is something else than encryption=default.
MDEV-17320 made it a hard error if encryption_key_id is specified to be
anything else than 1 in that case.

Ideally, we would always persist encryption_key_id in InnoDB. But, then we
would have to be prepared for the case that when encryption is being enabled
for a table whose encryption_key_id attribute refers to a non-existing key.

In MariaDB Server 10.1, our best option remains to not store anything
inside InnoDB. But, instead of returning the error that MDEV-17320
introduced, we should merely issue a warning that the specified
encryption_key_id is going to be ignored if encryption=default.

To improve the situation a little more, we will issue a warning if
SET [GLOBAL|SESSION] innodb_default_encryption_key_id is being set
to something that does not refer to an available encryption key.

Starting with MariaDB Server 10.2, thanks to MDEV-5800, we could open the
table definition from InnoDB side when the encryption is being enabled,
and actually fix the root cause of what was reported in MDEV-17320.
2019-02-28 23:20:31 +02:00
Oleksandr Byelkin
7b5c63856b Merge branch '5.5' into 10.0 2019-02-28 21:50:00 +01:00
Oleksandr Byelkin
cb11b3fbe9 MDEV-17055: Server crashes in find_order_in_list upon 2nd (3rd) execution of SP with UPDATE
1. Always drop merged_for_insert flag on cleanup (there could be errors which prevent TABLE to be assigned)
2. Make more precise cleanup of select parts which was touched
2019-02-28 18:14:53 +01:00
Alexander Barkov
0ad598a00b A cleanup in derived table handling: removing duplicate code from st_select_lex::handle_derived()
st_select_lex::handle_derived() and mysql_handle_list_of_derived() had
exactly the same implementations.

- Adding a new method LEX::handle_list_of_derived() instead
- Removing public function mysql_handle_list_of_derived()
- Reusing LEX::handle_list_of_derived() in st_select_lex::handle_derived()
2019-02-28 18:13:28 +04:00
Jan Lindström
622e9e8a7a MDEV-18265: Replace deprecated variable debug to debug_dbug on Galera tests
Replaced debug to debug_dbug on 10.1 on galera suite. Nothing to do
in wsrep and galera_3nodes suites.
2019-02-28 13:31:57 +02:00
Jan Lindström
f65f40bb35 Merge remote-tracking branch 'origin/10.1' into 10.2 2019-02-28 13:08:11 +02:00
Jan Lindström
5a87e3ee87 Revert offending part of MDEV-9519: Data corruption will happen on the Galera cluster size change
This will allow test binlog.binlog_stm_binlog to pass more often.
Note that this is not a real fix to that test failure.
2019-02-28 09:29:19 +02:00
Alexander Barkov
cac14b9225 MDEV-17725 Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failed in Diagnostics_area::set_ok_status upon ALTER failing due to error from engine 2019-02-26 15:41:27 +04:00
Julius Goryavsky
2c734c980e MDEV-9519: Data corruption will happen on the Galera cluster size change
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519
2019-02-26 07:45:11 +02:00
Julius Goryavsky
243f829c1c MDEV-9519: Data corruption will happen on the Galera cluster size change
If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519
2019-02-25 11:19:07 +02:00
Alexander Barkov
80c3fd184d Backporting MDEV-15597 Add class Load_data_outvar and avoid using Item::STRING_ITEM for Item_user_var_as_out_param detection
This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
2019-02-23 17:43:59 +04:00
Alexander Barkov
8036ad541e Backporting MDEV-15497 Wrong empty value in a GEOMETRY column on LOAD DATA
This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
2019-02-23 17:43:59 +04:00
Alexander Barkov
0ef8848526 Backporting MDEV-14628 Wrong autoinc value assigned by LOAD XML in the NO_AUTO_VALUE_ON_ZERO mode
This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
2019-02-23 17:43:59 +04:00