Commit graph

16300 commits

Author SHA1 Message Date
Marko Mäkelä
f3785f099c Merge 10.6 into 10.7 2022-09-13 08:48:40 +03:00
Marko Mäkelä
0ccb95c2a9 Merge 10.5 into 10.6 2022-09-12 15:01:55 +03:00
Andrei
bc12478a9a MDEV-24660 MYSQL_BIN_LOG::cleanup(): Assertion `b->xid_count == 0'
The shutdown time assert was caused by untimely deactivation of
the binlog background thread and related structs destruction.
It could specifically occur when a transaction is replication unsafe
and has to be completed with a ROLLBACK event in binlog.

This gets fixed with the binlog background thread stop relocation
to a point and user transactions have been completed.
A test case is added to binlog.binlog_checkpoint which
also receives as a bonus a minor correction to reactivate a MDEV-4322 test
case that originally required a shutdown phase (that ceased to do).
2022-09-09 19:22:41 +03:00
Nayuta Yanagisawa
3133435134 Merge 10.6 into 10.7 2022-09-08 21:54:45 +09:00
Marko Mäkelä
d2e649aec2 MDEV-29440 InnoDB instant ALTER TABLE recovery must use READ UNCOMMITTED
In commit 8f8ba75855 (MDEV-27234)
the data dictionary recovery was changed to use READ COMMITTED
so that table-rebuild operations (OPTIMIZE TABLE, TRUNCATE TABLE,
some forms of ALTER TABLE) would be recovered correctly.

However, for operations that avoid a table rebuild thanks to
being able to instantly ADD, DROP or reorder columns, recovery
must use the READ UNCOMMITTED isolation level so that changes to
the hidden metadata record can be rolled back.

We will detect instant operations by detecting uncommitted changes
to SYS_COLUMNS in case there is no uncommitted change of SYS_TABLES.ID
for the table. In any table-rebuilding DDL operation, the SYS_TABLES.ID
(and likely also the table name) will be updated.

As part of rolling back the instant ALTER TABLE operation, after the
operation on the hidden metadata record has been rolled back, a rollback
of an INSERT into SYS_COLUMNS in row_undo_ins_remove_clust_rec() will
invoke trx_t::evict_table() to discard the READ UNCOMMITTED definition
of the table. After that, subsequent recovery steps will load and use
the correct table definition.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Matthias Leich
2022-09-08 14:57:50 +03:00
Vlad Lesin
0fa4dd0747 MDEV-29433 innodb.lock_delete_updated is unstable
Use suspend thread syncpoint instead of include/wait_condition.inc to
make sure DELETE created waiting lock before the next UPDATE begins
locking.
2022-09-08 11:48:13 +03:00
Andrei
5563202089 MDEV-29322 ASAN heap-use-after-free in Query_log_event::do_apply_event
The ASAN report was made in the parallel slave execution of a query
event and implicitly involved (so also parallelly run) Format-Description
event.
The Query actually had unexpected impossible dependency on a preceding
"old" FD whose instance got destructed, to cause the ASAN error.

The case is fixed with storing the FD's value into Query-log-event
at its instantiating on slave. The stored value is from the very
FD of the Query's original binlog so remains to be correct
at the query event applying.

The branch C. of a new rpl_parallel_29322.test also demonstrates
(may need few --repeat though) the bug in its simple form of the same
server version binlog.
2022-09-07 20:35:43 +03:00
Marko Mäkelä
bacaf2d4f4 MDEV-29342 Assertion failure in file que0que.cc line 728
Additional fixes for 10.6:

fts_sync_commit(): Release cache->lock also on rollback.

fts_sync_write_words(): Avoid a crash if an error occurs,
by stopping at the first error.

fts_add_doc_by_id(): Sync the doc id only after adding the doc id
to the cache.
2022-09-07 12:57:53 +03:00
Marko Mäkelä
0c0b697ae3 Merge 10.6 into 10.7 2022-09-07 08:56:06 +03:00
Marko Mäkelä
1985204044 Merge 10.5 into 10.6 2022-09-07 08:47:20 +03:00
Marko Mäkelä
38d36b59f9 Merge 10.4 into 10.5 2022-09-07 08:26:21 +03:00
Marko Mäkelä
c7ba237793 Merge 10.3 into 10.4 2022-09-07 08:08:59 +03:00
Thirunarayanan Balathandayuthapani
ac49b7a845 MDEV-29342 Assertion failure in file que0que.cc line 728
- During shutdown, InnoDB fts fails to update synced doc id
when there is only one doc id about to sync. While starting
the server, InnoDB fetches the already synced doc id from
config table. In the subsequent sync operation, InnoDB fails
with DB_DUPLICATE_KEY error.
2022-09-06 17:23:31 +05:30
Marko Mäkelä
027a9963b0 MDEV-13542 fixup: Allow purge to run in a test 2022-09-06 09:52:08 +03:00
Jan Lindström
5fdbb3a72e Merge 10.6 into 10.7 2022-09-05 14:55:47 +03:00
Brandon Nesterenko
47812017c6 MDEV-28530: Revoking privileges from a non-existing user on a master breaks replication on the slave in the presence of replication filters
Problem:
========
Replication can break while applying a query log event if its
respective command errors on the primary, but is ignored by the
replication filter within Grant_tables on the replica. The bug
reported by MDEV-28530 shows this with REVOKE ALL PRIVILEGES using a
non-existent user. The primary will binlog the REVOKE command with
an error code, and the replica will think the command executed with
success because the replication filter will ignore the command while
accessing the Grant_tables classes. When the replica performs an
error check, it sees the difference between the error codes, and
replication breaks.

Solution:
========
If the replication filter check done by Grant_tables logic ignores
the tables, reset thd->slave_expected_error to 0 so that
Query_log_event::do_apply_event() can be made aware that the
underlying query was ignored when it compares errors.

Note that this bug also effects DROP USER if not all users exist
in the provided list, and the patch fixes and tests this case.

Reviewed By:
============
andrei.elkin@mariadb.com
2022-09-03 13:04:44 +03:00
Thirunarayanan Balathandayuthapani
e27c3d5230 MDEV-29425 Buffer overflow in dict_index_t::col_info::add()
- During rollback of DDL, InnoDB should copy the collation
changed column into the index heap
2022-09-01 16:07:40 +05:30
Marko Mäkelä
40aa94df35 MDEV-29435 CHECK TABLE forgets to release latches after reporting failure
btr_validate_level(): Invoke mtr.commit() after a failure.
This omission was introduced in
commit 0b47c126e3 (MDEV-13542).
2022-09-01 10:40:27 +03:00
Marko Mäkelä
bdf62ece6c MDEV-29374 InnoDB recovery fails with "Data structure corruption"
recv_sys_t::free_corrupted_page(): Identify the corrupted page in
an error or warning message.

buf_page_free(): Just in case, register the page as modified.
This should already have been done in mtr_t::free() as part of
fseg_free_page_low().

mtr_t::memo_push(): Simplify a condition, so that when invoked
with MTR_MEMO_PAGE_X_MODIFY, we will do the right thing.

fseg_free_page_low(): Remove an accidentally added return statement
that prevented mtr_t::free() from being called. This fixes a regression
that was introduced in
commit 0b47c126e3 (MDEV-13542).
2022-08-31 17:52:16 +03:00
Marko Mäkelä
4f2dc716ee MDEV-13668 fixup: Remove test work-arounds 2022-08-31 15:58:52 +03:00
Marko Mäkelä
b86be02ecf Merge 10.6 into 10.7 2022-08-30 13:02:42 +03:00
Marko Mäkelä
f410974f0f Merge 10.5 into 10.6 2022-08-30 13:01:16 +03:00
Marko Mäkelä
29fa9bcee0 Merge 10.4 into 10.5 2022-08-30 12:29:04 +03:00
Marko Mäkelä
7e574eb52c Merge 10.3 into 10.4 2022-08-30 12:17:33 +03:00
Marko Mäkelä
57739ae94a MDEV-13888: innodb_fts.innodb_fts_plugin failed
Add ORDER BY to make the test deterministic.

Add FLUSH TABLES to avoid crash recovery warnings about the table
mysql.plugin. This tends to occur on Valgrind, where the server
shutdown could presumably time out, resulting in a forced kill.
2022-08-30 12:03:58 +03:00
Marko Mäkelä
422f3204ef MDEV-29409 Buffer overflow in my_wc_mb_filename() on RENAME TABLE
dict_table_rename_in_cache(), dict_table_get_highest_foreign_id():
Reserve sufficient space for the fkid[] buffer, and ensure that the
fkid[] will be NUL-terminated.

The fkid[] must accommodate both the database name (which is already
encoded in my_charset_filename) and the constraint name
(which must be converted to my_charset_filename) so that we can check
if it is in the format databasename/tablename_ibfk_1 (all encoded in
my_charset_filename).
2022-08-30 12:02:56 +03:00
Marko Mäkelä
b260903832 MDEV-29258 Failing assertion for name length on RENAME TABLE
trx_undo_page_report_rename(): Use the correct maximum length of
a table name. Both the database name and the table name can be up to
NAME_CHAR_LEN (64 characters) times 5 bytes per character in the
my_charset_filename encoding. They are not encoded in UTF-8!

fil_op_write_log(): Reserve the correct amount of log buffer for
a rename operation. The file name will be appended by
mlog_catenate_string().

rename_file_ext(): Reserve a large enough buffer for the file names.
2022-08-30 10:59:31 +03:00
Elena Stepanova
0d1de5e1d1 MDEV-29403 innodb.innodb_sys_semaphore_waits fails with wrong errno 5014
take into account C/C specific CR_ERR_NET_WRITE error
2022-08-28 21:35:21 +03:00
Alexander Barkov
b9c2ae505d MDEV-29356 Assertion `0' failed in Type_handler_row::Item_save_in_field on INSERT 2022-08-26 14:25:04 +04:00
tmokmss
827b049e1e MDEV-18873 Server crashes in Compare_identifiers::operator or in my_strcasecmp_utf8 upon ADD PERIOD IF NOT EXISTS with empty name
empty identifier specified as `` ends up with a NULL LEX_CSTRING::str in lexer.
This is not considered correct in upper layers, for example in Compare_identifiers::operator().
Empty column name is usually avoided by a check_column_name() call while parsing,
and period name matches the column name completely.
Hence, this fix uses the mentioned call for verification, too.
2022-08-26 12:40:58 +03:00
Daniel Black
79b58f1ca8 MDEV-23607 MariaBackup - align required GRANTS to cmd options
Since the 10.5 split of the privileges, the required GRANTs
for various mariabackup operations has changed.

In the addition of tests, a number of mappings where incorrect:

The option --lock-ddl-per-table didn't require connection admin.

The option --safe-slave-backup requires SLAVE MONITOR even without
the --no-lock option.
2022-08-26 11:52:53 +10:00
Marko Mäkelä
76bb671e42 Merge 10.5 into 10.6 2022-08-25 16:02:44 +03:00
Marko Mäkelä
9929301ecd Merge 10.4 into 10.5 2022-08-25 15:31:19 +03:00
Marko Mäkelä
851058a3e6 Merge 10.3 into 10.4 2022-08-25 15:17:20 +03:00
Vlad Lesin
8ff1096999 MDEV-29081 trx_t::lock.was_chosen_as_deadlock_victim race in lock_wait_end()
The issue is that trx_t::lock.was_chosen_as_deadlock_victim can be reset
before the transaction check it and set trx_t::error_state.

The fix is to reset trx_t::lock.was_chosen_as_deadlock_victim only in
trx_t::commit_in_memory(), which is invoked on full rollback. There is
also no need to have separate bit in
trx_t::lock.was_chosen_as_deadlock_victim to flag transaction it was
chosen as a victim of Galera conflict resolution, the same variable can be
used for both cases except debug build. For debug build we need to
distinguish deadlock and Galera's abort victims for debug checks. Also
there is no need to check for deadlock in lock_table_enqueue_waiting() for
Galera as the coresponding check presents in lock_wait().

Local variable "error_state" in lock_wait() was replaced with
trx->error_state, because before the replace
lock_sys_t::cancel<false>(trx, lock) and lock_sys.deadlock_check() could
change trx->error_state, which then could be overwritten with the local
"error_state" variable value.

The lock_wait_suspend_thread_enter DEBUG_SYNC point name is misleading,
because lock_wait_suspend_thread was eliminated in e71e613. It was renamed
to lock_wait_start.

Reviewed by: Marko Mäkelä, Jan Lindström.
2022-08-24 17:06:57 +03:00
Marko Mäkelä
bdd80e3fb1 Merge 10.6 into 10.7 2022-08-24 09:22:34 +03:00
Thirunarayanan Balathandayuthapani
5b4c832c7e MDEV-29314 Assertion `n_fields > n_cols' failed in dict_index_t::init_change_cols
- Newly created InnoDB fulltext index does have only one column
and doesn't associate with primary key fields during DDL.
init_change_cols() has strict assertion that number of fields
should be greater than number of collation change columns.
2022-08-23 19:40:53 +05:30
Thirunarayanan Balathandayuthapani
4feb9df105 MDEV-29282 atomic.rename_trigger fails occasionally
This reverts part of commit 212994f704
(MDEV-28974) and implements a better fix that works in that special case
while avoiding other failures.

fil_name_process(): Do not rename the tablespace in deferred_spaces;
it already contains the latest name for the space id.

deferred_spaces::create(): In mariadb-backup --prepare,
replace absolute data directory file path with short name
relative to the backup directory and store it as filename while
deferring tablespace file creation.
2022-08-23 08:47:59 +03:00
Brandon Nesterenko
8963d64ee8 MDEV-28294: set default role bypasses Replicate_Wild_Ignore_Table: mysql.%
Problem:
========
When replicating SET DEFAULT ROLE, the pre-update check (i.e. that
in set_var_default_role::check()) tries to validate the existence of
the given rules/user even when the targeted tables are ignored. When
previously issued CREATE USER/ROLE commands are ignored by the
replica because of the replication filtering rules, this results in
an error because the targeted data does not exist.

Solution:
========
Before checking that the given roles/user exist of a SET DEFAULT
ROLE command, first ensure that the mysql.user and
mysql.roles_mapping tables are not excluded by replication filters.

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>
Sergei Golubchik <serg@mariadb.com>
2022-08-22 18:23:56 +03:00
Marko Mäkelä
d65a2b7bde Merge 10.5 into 10.6 2022-08-22 14:02:43 +03:00
Marko Mäkelä
1d90d6874d Merge 10.4 into 10.5 2022-08-22 13:38:40 +03:00
Marko Mäkelä
36d173e523 Merge 10.3 into 10.4 2022-08-22 12:34:42 +03:00
Marko Mäkelä
6005f3c548 MDEV-25257 follow-up: Fix a test
Ever since commit 09177eadc3
the test innodb.row_format_redundant cannot work when the
data file was created with innodb_checksum_algorithm=full_crc32.

Backport of 10.5 commit a9d0bb12e6
2022-08-22 12:33:46 +03:00
Marko Mäkelä
fd0cd4801a MDEV-13013 fixup: Adjust a test 2022-08-22 12:32:47 +03:00
Marko Mäkelä
75c416d362 MDEV-24626 fixup: mariabackup.compress_qpress 2022-08-19 09:26:13 +03:00
Thirunarayanan Balathandayuthapani
32167225c7 MDEV-13013 InnoDB unnecessarily extends data files
- While creating a new InnoDB segment, allocates the extent
before allocating the inode or page allocation even though
the pages are present in fragment segment. This patch does
reserve the extent when InnoDB ran out of fragment pages
in the tablespace.
2022-08-17 11:08:49 +05:30
Thirunarayanan Balathandayuthapani
af552f2903 Disabling atomic.rename_trigger test case because of frequent
failures
2022-08-16 21:33:45 +05:30
Oleksandr Byelkin
65a963f755 Merge branch '10.6' into 10.7 2022-08-10 13:12:32 +02:00
Oleksandr Byelkin
c442e1ae21 Merge branch '10.5' into 10.6 2022-08-10 13:06:08 +02:00
Oleksandr Byelkin
1ac0bce36e Merge branch '10.4' into 10.5 2022-08-10 12:24:31 +02:00