Commit graph

195662 commits

Author SHA1 Message Date
Thirunarayanan Balathandayuthapani
19283c67c6 MDEV-28679 After upgrade to 10.7.3-1 with enabled data-at-rest encryption unable to restore dump file.
- InnoDB bulk insert fails to use encryption buffer for encrypting
the temporary log file. Declare the m_crypt_block, m_crypt_pfx in
row_merge_bulk_t to be used for encrypting the temporary file.
2022-07-26 11:25:56 +05:30
Thirunarayanan Balathandayuthapani
3808ffbcb5 MDEV-28242 Assertion `!check_foreigns' failed in trx_t::check_bulk_buffer
If transaction does bulk insert and disables the foreign_key_check
then InnoDB fails with the assert failure. InnoDB has strict
assertion that check_foreigns and unique_secondary_check
should be enabled if the transaction does bulk insert
in innodb_prepare_commit_versioned().
2022-07-26 11:25:56 +05:30
Yusuke Abe
48f3cf7570
MDEV-28795 Deprecate spider_bka_table_name_type 2022-07-25 09:33:11 +09:00
Monty
c3ddffe29c Fix mysqld--help.test
Disable printing of provider-... as these differ if
the modules are static
2022-07-09 16:51:55 +03:00
Alexander Barkov
380874549c MDEV-29062 Wrong result set metadata for a mix of INT+ENUM 2022-07-08 15:29:23 +04:00
Oleksandr Byelkin
a5f78505d7 MDEV-28838 password_reuse_check plugin mixes username and password
To prevent the problem of mixing user name and password and
host name and user name we add length of the hostname and user name
to the hash.
2022-07-06 12:23:31 +02:00
Alexander Barkov
c12192b1c6 MDEV-27015 Assertion `!is_null()' failed in FixedBinTypeBundle<FbtImpl>::Fbt FixedBinTypeBundle<FbtImpl>::Field_fbt::to_fbt()
The assuption that Field::is_null() is always false when
Field_fbt::val_native() or Field_fbt::to_fbt() are called
was wrong.

In some cases, e.g. when this helper Field method is called:

  inline String *val_str(String *str, const uchar *new_ptr)

we temporarily reset Field::ptr to some alternative record buffer
but don't reset null_ptr, so null_ptr still points to null flags
of the original record. In such cases it's meaningless to test
the original Field::null_ptr when Field::ptr is temporarily reset:
they don't relate to each other.

Removing the DBUG_ASSERT.
2022-07-05 12:11:14 +04:00
Marko Mäkelä
3dff84cd15 Merge 10.6 into 10.7 2022-07-01 17:45:29 +03:00
Marko Mäkelä
62a20f8047 Merge 10.5 into 10.6 2022-07-01 15:24:50 +03:00
Marko Mäkelä
b546913ba2 Valgrind: Disable tests that would often time out
Starting with 10.5, InnoDB crash recovery tests seem to time out
more easily under Valgrind, which emulates multiple threads by
interleaving them in a single operating system thread.

These tests will still be covered by
AddressSanitizer and MemorySanitizer.
2022-07-01 14:42:13 +03:00
Marko Mäkelä
f09687094c Merge 10.4 into 10.5 2022-07-01 14:42:02 +03:00
Vladislav Vaintroub
ba3354fca6 MDEV-28995 Sporadic Assertion on shutdown in threadpool_winsockets.cc
Remove the affected assert.
Wait for all AIO_buffer_cache::release_buffer() to finish before
AIO_buffer_cache::clear().
2022-07-01 13:04:44 +02:00
Marko Mäkelä
392ee571c1 Merge 10.3 into 10.4 2022-07-01 13:10:36 +03:00
Thirunarayanan Balathandayuthapani
990cde800a MDEV-28912 NON-UNIQUE FTS_DOC_ID index mistaken as FTS_DOC_ID_INDEX
- InnoDB mistakenly identifies the non-unique FTS_DOC_ID index as
FTS_DOC_ID_INDEX while loading the table. dict_load_indexes()
should check whether the index is unique before assigning
fts_doc_id_index
2022-07-01 13:03:52 +03:00
Marko Mäkelä
7c35ad16e3 MDEV-28389 fixup: Fix pre-GCC 10 -Wconversion
Before version 10, GCC would think that a right shift of an
unsigned char returns int. Let us explicitly cast that back,
to silence a bogus -Wconversion warning.
2022-07-01 13:02:43 +03:00
Marko Mäkelä
045771c050 Fix most clang-15 -Wunused-but-set-variable
Also, refactor trx_i_s_common_fill_table() to remove dead code.

Warnings about yynerrs in Bison-generated yyparse() will remain for now.
2022-07-01 09:48:36 +03:00
Marko Mäkelä
6dc1bc3a58 Fix clang-15 -Wdeprecated-non-prototype
K&R style function definitions are deprecated in all versions of C
and not supported in C2x.
2022-07-01 09:34:31 +03:00
Marko Mäkelä
90792e4a43 Merge 10.5 into 10.6 2022-06-30 19:41:07 +03:00
Thirunarayanan Balathandayuthapani
99de8cc028 MDEV-28919 Assertion `(((core_null) + 7) >> 3) == oindex.n_core_null_bytes || !not_redundant()' failed
- In case of discarded tablespace, InnoDB can't read the root page to
assign the n_core_null_bytes. Consecutive instant DDL fails because
of non-matching n_core_null_bytes.
2022-06-30 17:30:13 +05:30
Marko Mäkelä
a1267724cb MDEV-26293 InnoDB: Failing assertion: space->is_ready_to_close() ...
fil_space_t::acquire_low(): Introduce a parameter that specifies
which flags should be avoided. At all times, referenced() must not
be incremented if the STOPPING flag is set. When fil_system.mutex
is not being held by the current thread, the reference must not be
incremented if the CLOSING flag is set (unless NEEDS_FSYNC is set,
in fil_space_t::flush()).

fil_space_t::acquire(): Invoke acquire_low(STOPPING | CLOSING).
In this way, the reference count cannot be incremented after
fil_space_t::try_to_close() invoked fil_space_t::set_closing().

If the CLOSING flag was set, we must retry acquire_low() after
acquiring fil_system.mutex.

fil_space_t::prepare_acquired(): Replaces prepare(true).

fil_space_t::acquire_and_prepare(): Replaces prepare().
This basically retries fil_space_t::acquire() after
acquiring fil_system.mutex.
2022-06-30 14:28:16 +03:00
Marko Mäkelä
afe607dffa MSAN: Disable some slow tests 2022-06-30 13:00:58 +03:00
Thirunarayanan Balathandayuthapani
eb7e24932b MDEV-28806 Assertion `flag == 1' failure in row_build_index_entry_low upon concurrent ALTER and UPDATE
- During online ADD INDEX, InnoDB was incorrectly writing log for an
UPDATE that does not affect the being-created index.
2022-06-30 15:28:24 +05:30
Thirunarayanan Balathandayuthapani
e34f878139 MDEV-28706 Redundant InnoDB table fails during alter
- Redundant InnoDB table fails to set the flags2 while loading
the table. It leads to "Upgrade index name failure" during alter
operation. InnoDB should set the flags2 to FTS_AUX_HEX_NAME when
fts is being loaded
2022-06-30 11:19:49 +05:30
Sergei Golubchik
85c0f4d2de C/C 3.3 2022-06-29 15:39:06 +02:00
Marko Mäkelä
cac6f0a8c4 Merge 10.6 into 10.7 2022-06-29 16:17:14 +03:00
Marko Mäkelä
c1e3fc0e0d MDEV-28977: mariabackup.huge_lsn,strict_full_crc32 fails in 10.8
recv_sys_t::recover_deferred(): Hold the exclusive page latch until
the tablespace has been set up. Otherwise, the write of the page
may be lost due to non-existent tablespace. This race only affects
the recovery of the first page in a newly created tablespace.

This race condition was introduced in MDEV-24626.
2022-06-29 15:48:44 +03:00
Aleksey Midenkov
d89cac0884 MDEV-28567 MDL debug logging
Log MDL state transitions. Trace-friendly message
format. DBUG_LOCK_FILE replaced by thread-local storage.

Logged states legend:
  Seized   lock was acquired without waiting
  Waiting  lock is waiting
  Acquired lock was acquired after waiting
  Released lock was released
  Deadlock lock was aborted due to deadlock
  Timeout  lock was aborted due to timeout >0
  Nowait   lock was aborted due to zero timeout
  Killed   lock was aborted due to kill message
  OOM	   can not acquire because out of memory

Usage:
  mtr --mysqld=--debug=d,mdl,query:i:o,/tmp/mdl.log

Cleanup from garbage messages:
  sed -i -re \
  '/(mysql|performance_schema|sys|mtr)\// d; /MDL_BACKUP_/ d' \
  /tmp/mdl.log
2022-06-28 23:36:39 +03:00
Marko Mäkelä
2fa3ada072 Fix a sporadic failure of main.backup_locks
Ever since commit 9608773f75
the InnoDB persistent statistics are enabled on all InnoDB tables
by default. We must filter out any output that indicates that the
statistics tables are being internally accessed by InnoDB.
2022-06-28 20:58:45 +03:00
Monty
5e40934d24 MDEV-28897 Wrong table.get_ref_count() upon concurrent truncate and backup stage operation
The issue was that flush_tables() didn't take a MDL lock on cached
TABLE_SHARE before calling open_table() to do a HA_EXTRA_FLUSH call.
Most engines seams to have no issue with it, but apparantly this conflicts
with InnoDB in 10.6 when using TRUNCATE

Fixed by taking a MDL lock before trying to open the table in
flush_tables().

There is no test case as it hard to repeat the scheduling that causes
the error. I did run the test case in MDEV-28897 to verify
that the bug is fixed.
2022-06-28 15:57:41 +03:00
Alexander Barkov
efdbb3cf31 A cleanup for MDEV-25243 ASAN heap-use-after-free in Item_func_sp::execute_impl upon concurrent view DDL and I_S query with view and function
The test was reported to fail sporadicaly with this diff:

--- mysql-test/main/information_schema_tables.result
+++ mysql-test/main/information_schema_tables.reject
@@ -21,6 +21,8 @@
 disconnect con1;
 connection default;
 DROP VIEW IF EXISTS vv;
+Warnings:
+Note	4092	Unknown VIEW: 'test.vv'

in the "The originally reported non-deterministic test" part.
Disabling warnings around the DROP VIEW statement.
2022-06-28 13:34:58 +04:00
Marko Mäkelä
02a313dc56 MDEV-18976 fixup: encryption.innodb-redo-nokeys
This test failure is similar to encryption.innodb-redo-badkey,
which was fixed in commit 0f0a45b2dc.
2022-06-28 12:29:30 +03:00
Marko Mäkelä
ac0af4ec4a Merge 10.6 into 10.7 2022-06-28 08:34:12 +03:00
Nayuta Yanagisawa
dbd562787a MDEV-24343 Spider Left join failed Unknown column 't0.ID' in 'on clause'
The Spider mixes the comma join with other join types, and thus
ERROR 1054 occurs. This is well-known issue caused by the higher
precedence of JOIN over the comma (,).

We can fix the problem simply by using JOINs instead of commas.
2022-06-28 05:23:14 +09:00
Nayuta Yanagisawa
5375f0b495 MDEV-21310 AUTO_INCREMENT column throws range error on INSERT in partitioned table | Assertion `part_share->auto_inc_initialized || !can_use_for_auto_inc_init()' failed.
The bug is caused by a similar mechanism as MDEV-21027.

The function, check_insert_or_replace_autoincrement, failed to open
all the partitions on INSERT SELECT statements and it results in the
assertion error.
2022-06-28 01:48:55 +09:00
Nayuta Yanagisawa
a26700cca5 MDEV-28352 Spider: heap-use-after-free in ha_spider::lock_tables(), heap freed by spider_commit()
The heap-use-after-free is caused by the following mechanism:

  * In the execution of FLUSH TABLE WITH READ LOCK, the function
    spider_free_trx_conn() is called and the connections held by
    SPIDER_TRX::trx_conn_hash are freed.

  * Then, an instance of ha_spider maintains the freed connections
    because they are also referenced from ha_spider::conns.
    The ha_spider instance is kept in a lock structure until the
    corresponding table is unlocked.

  * Spider accesses ha_spider::conns on the implicit UNLOCK TABLE
    issued by BEGIN.

In the first place, when the connections have been freed, it means
that there are really no remote table locked by Spider.
Thus, there is no need for Spider to access ha_spider::cons on the
implicit UNLOCK TABLE.

We can fix the bug by removing the above mentioned access to
ha_spider::conns. We also modified spider_free_trx_conn() so that it
frees the connections only when no table is locked to reduce the
chance of another heap-use-after-free on ha_spider::conns.
2022-06-28 01:03:33 +09:00
Alexander Barkov
4a7e337e5f MDEV-28963 Incompatible data type assignment through SP vars is not consistent with columns 2022-06-27 19:52:48 +04:00
Nayuta Yanagisawa
c4bfb61803 MDEV-28479 fixup: Update spider/bg and spider/handler suites 2022-06-27 22:57:33 +09:00
Marko Mäkelä
1ae8160710 MDEV-26979 heap-use-after-free or SIGSEGV when accessing INNODB_SYS_TABLESTATS during DDL
i_s_dict_fill_sys_tablestats(): Read all fields of dict_table_t
while holding dict_sys.latch.

dict_sys_t::allow_eviction(): Remove.
2022-06-27 16:51:53 +03:00
Marko Mäkelä
20cf63fe8b Merge 10.5 into 10.6 2022-06-27 16:51:27 +03:00
Marko Mäkelä
773f1dad94 Merge 10.4 into 10.5 2022-06-27 16:17:02 +03:00
Marko Mäkelä
b922ae5fc9 Merge 10.3 into 10.4 2022-06-27 16:16:20 +03:00
Marko Mäkelä
f339ef3f97 MDEV-26577 InnoDB: Failing assertion: dict_tf2_is_valid(flags, flags2) during ADD COLUMN
prepare_inplace_alter_table_dict(): If the table will not be rebuilt,
preserve all of the original ROW_FORMAT, including the compressed
page size flags related to ROW_FORMAT=COMPRESSED.
2022-06-27 16:00:34 +03:00
Marko Mäkelä
a75ad73545 MDEV-28389 fixup: Fix compiler warnings
hex_to_ascii(): Add #if around the definition to avoid
clang -Wunused-function. Avoid GCC 5 -Wconversion with a cast.
2022-06-27 14:50:00 +03:00
Marko Mäkelä
39f45f6f89 MDEV-28950 Assertion `*err == DB_SUCCESS' failed in btr_page_split_and_insert
btr_root_raise_and_insert(), btr_lift_page_up(),
rtr_page_split_and_insert(): Reset DB_FAIL from a failure to
copy records on a ROW_FORMAT=COMPRESSED page to DB_SUCCESS
before retrying.

This fixes a regression that was introduced by
commit 0b47c126e3 (MDEV-13542).

btr_root_raise_and_insert(): Remove a redundant condition.
btr_page_split_and_insert() will invoke btr_page_split_and_insert()
if needed.
2022-06-27 12:32:03 +03:00
Alexander Barkov
0bed4d72c0 MDEV-28918 Implicit cast from INET6 UNSIGNED works differently on UPDATE vs ALTER
Now INSERT, UPDATE, ALTER statements involving incompatible data type pairs, e.g.:

    UPDATE TABLE t1 SET col_inet6=col_int;
    INSERT INTO t1 (col_inet6) SELECT col_in FROM t2;
    ALTER TABLE t1 MODIFY col_inet6 INT;

consistently return an error at the statement preparation time:

    ERROR HY000: Illegal parameter data types inet6 and int for operation 'SET'

and abort the statement before starting interating rows.

This error is the same with what is raised for queries like:
    SELECT col_inet6 FROM t1 UNION SELECT col_int FROM t2;
    SELECT COALESCE(col_inet6, col_int) FROM t1;

Before this change the error was caught only during the execution time,
when a Field_xxx::store_xxx() was called for the very firts row.
The behavior was not consistent between various statements and could do different things:
- abort the statement
- set a column to the data type default value (e.g. '::' for INET6)
- set a column to NULL

A typical old error was:

    ERROR 22007: Incorrect inet6 value: '1' for column `test`.`t1`.`a` at row 1

EXCEPTION:

Note, there is an exception: a multi-row INSERT..VALUES, e.g.:
    INSERT INTO t1 (col_a,col_b) VALUES (a1,b1),(a2,b2);
checks assignment compability at the preparation time for the very first row only:
    (col_a,col_b) vs (a1,b1)

Other rows are still checked at the execution time and return the old warnings
or errors in case of a failure. This is done because catching all rows at the
preparation time would change behavior significantly. So it still works
according to the STRICT_XXX_TABLES sql_mode flags and the table transaction ability.

This is too late to change this behavior in 10.7.
There is no a firm decision yet if a multi-row INSERT..VALUES
behavior will change in later versions.
2022-06-27 12:49:40 +04:00
Marko Mäkelä
7d92c9d212 Suppress a message that may be emitted on slow systems
On FreeBSD, tests run on persistent storage, and no asynchronous I/O
has been implemented. Warnings about 205-second waits on dict_sys.latch
may occur.
2022-06-27 11:03:52 +03:00
Marko Mäkelä
87bd79b1e7 Merge 10.5 into 10.6 2022-06-27 10:59:31 +03:00
Marko Mäkelä
ea847cbeaf Merge 10.4 into 10.5 2022-06-27 10:51:20 +03:00
Marko Mäkelä
03174cabd7 Fix GCC -Og -Wmaybe-uninitialized 2022-06-27 10:49:15 +03:00
Marko Mäkelä
dd7e9fb38a MDEV-28854 after-merge fix: Remove a test for MDEV-26583 2022-06-27 10:47:05 +03:00