Commit graph

194042 commits

Author SHA1 Message Date
Marko Mäkelä
003095e899 MDEV-26936 Recovery crash on rolling back DELETE FROM SYS_INDEXES
row_undo_mod_clust_low(): If we are in recovery and rolling back
a DELETE operation on the SYS_INDEXES table, and the
SYS_INDEXES.NAME starts with the magic byte 0xff
that identifies uncommitted ADD INDEX stubs, we must not
try to evict the table definition because such index stubs
would be skipped by dict_load_indexes() anyway.
2021-10-29 16:25:15 +03:00
Marko Mäkelä
37a4ea3f59 MDEV-25683 fixup: MSVC warning C4018: signed/unsigned mismatch 2021-10-29 16:25:11 +03:00
Oleksandr Byelkin
ad3e416e8d columnstore-6.2.1-1 2021-10-29 13:04:10 +02:00
Oleksandr Byelkin
facd9d524d Merge branch '10.5' into 10.6 2021-10-29 13:01:02 +02:00
Oleksandr Byelkin
1974df01d5 columnstore-5.6.3-2 2021-10-29 12:19:34 +02:00
Oleksandr Byelkin
1c1396f09c Merge branch '10.4' into 10.5 2021-10-29 10:21:52 +02:00
Marko Mäkelä
dbd6c6dc01 MDEV-25683 Atomic DDL: With innodb_force_recovery=3 InnoDB: Trying to load index but the index tree has been freed
The purpose of the parameter innodb_force_recovery is to allow some
data to be dumped from a corrupted database. Its values used to be
as follows:

innodb_force_recovery=0: normal (default)

innodb_force_recovery=1: ignore (skip log for) corrupted pages or
missing data files when applying the redo log

innodb_force_recovery=2: additionally, disable background tasks
(such as the purge of committed undo logs)

innodb_force_recovery=3: additionally, disable the rollback of
recovered incomplete (not committed or XA PREPARE) transactions

innodb_force_recovery=4: same as 3 (since MDEV-19514 in MariaDB 10.5)

innodb_force_recovery=5: additionally, do not process any undo log,
disallow any writes, and force READ UNCOMMITTED isolation level

innodb_force_recovery=6: additionally, pretend that ib_logfile0 does
not exist (prevent any recovery). Never use this!

The bad thing that happens with innodb_force_recovery=3 and
innodb_force_recovery=4 is that also the rollback of any recovered
DDL transaction will be skipped. This would break the DDL log recovery
that was introduced in MDEV-17567.

For one data directory sample, the DDL log recovery would hangs due to
a conflict on the InnoDB SYS_TABLES table, because the lock holder
transaction was not rolled back due to innodb_force_recovery=3.

Fix: Make innodb_force_recovery=3 skip the DML transaction rollback only,
and make innodb_force_recovery=4 (renamed to SRV_FORCE_NO_DDL_UNDO)
behave like innodb_force_recovery=3 used to (skip the rollback of all
recovered transactions, both DML and DDL).

Startup with innodb_force_recovery=4 will be unaffected by this change.
(There may be hangs, possibly preceded by messages about failing to
load an index.)

Side note: With innodb_force_recovery=5, any DDL log for InnoDB tables
will be essentially ignored by InnoDB, but the server will start up.
2021-10-29 10:20:58 +03:00
Vladislav Vaintroub
ea45f0ebfb MDEV-26925 - upgrade fails creating trigger in sysschema, if root user does not exist
Fix by removing the trigger. It does not do anything useful anyway.
2021-10-28 14:28:50 +02:00
Oleksandr Byelkin
e10838268e wolfssl v4.8.1-stable 2021-10-28 14:23:22 +02:00
Oleksandr Byelkin
89f69c62cf Merge branch '10.3' into 10.4 2021-10-28 13:57:15 +02:00
Oleksandr Byelkin
2ddea602ce Merge branch '10.2' into 10.3 2021-10-28 12:41:27 +02:00
Oleksandr Byelkin
b3cdf4168c fix depricated pthread_yield() for tokudb 2021-10-28 12:24:31 +02:00
Sergei Golubchik
1203b65849 compilation fixes for sys-devel/gcc-11.2.0:11
for example:

sql/sql_prepare.cc:5714:63: error: 'static void Ed_result_set::operator delete(void*, MEM_ROOT*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
2021-10-28 12:01:25 +02:00
Oleksandr Byelkin
99c893586c Merge remote-tracking branch 'connect/10.2' into 10.2 2021-10-28 10:30:36 +02:00
Vladislav Vaintroub
ff3274dd7c Fix message severity for "thread pool blocked" messages.
Those messages don't indicate errors, they should be normal warnings.
2021-10-28 09:59:24 +02:00
Marko Mäkelä
d8c6c53a06 Merge 10.5 into 10.6 2021-10-28 09:08:58 +03:00
Marko Mäkelä
a8ded39557 Merge 10.4 into 10.5 2021-10-28 08:48:36 +03:00
Marko Mäkelä
3a79e5fd31 Merge 10.3 into 10.4 2021-10-28 08:28:39 +03:00
Marko Mäkelä
657bcf928e Merge 10.2 into 10.3 2021-10-28 07:50:05 +03:00
Marko Mäkelä
563daec123 MDEV-26867: Update the InnoDB version number to 5.7.36
The InnoDB changes in MySQL 5.7.36 that were applicable to MariaDB
were covered by MDEV-26864, MDEV-26865, MDEV-26866.
2021-10-28 07:35:49 +03:00
Nikita Malyavin
1f5ca66e53 MDEV-26866 FOREIGN KEY…SET NULL corrupts an index on a virtual column
The initial test case for MySQL Bug #33053297 is based on
mysql/mysql-server@27130e2507.

innobase_get_field_from_update_vector is not a suitable function to fetch
updated row info, as well as parent table's update vector is not always
suitable. For instance, in case of DELETE it contains undefined data.

castade->update vector seems to be good enough to fetch all base columns
update data, and besides faster, and less error-prone.
2021-10-28 07:32:27 +03:00
Julius Goryavsky
7948a1dc53 MDEV-26914: Unreleased mutex in the exec_relay_log_event() function
In the replication-related code, in the exec_relay_log_event() (slave.cc)
function, where the "data_lock" mutex is captured, this mutex is then not
released on one of the early return branches within a specific insert for
WSREP, namely under the branch: "if (wsrep_before_statement(thd))". As a
result, the mutex remains captured, resulting in errors or hangs.

This commit fixes this issue, which is now showing up as intermittent
failures in mtr tests for galera and galera_sr suites.
2021-10-28 03:17:12 +02:00
Marko Mäkelä
1ad1d78981 MDEV-26779: Enable adaptive spinning on ARMv8 for lock_sys.wait_mutex
Similar to commit f7684f0ca5 (MDEV-26855)
we will try to enable the adaptive spinloop for lock_sys.wait_mutex
on ARMv8.

Enabling any form of spinloop for lock_sys.wait_mutex did not show a
significant improvement in our tests on AMD64.

Spinning can be argued to be a hack to reduce the impact on mutex
contention. It would be better to adjust the code to reduce
contention in the first place.
2021-10-27 17:21:19 +03:00
Marko Mäkelä
83dbf2c995 Merge 10.5 into 10.6 2021-10-27 13:47:58 +03:00
Marko Mäkelä
f7bd369973 Merge 10.4 into 10.5 2021-10-27 13:29:16 +03:00
Marko Mäkelä
772d6d347d MDEV-18543 fixup: Fix 32-bit builds 2021-10-27 13:28:16 +03:00
Sergei Petrunia
3a9967d757 Fix compile warning:
ha_rocksdb.h:459:15: warning: 'table_type' overrides a member
function but is not marked 'override' [-Winconsistent-missing-override]
2021-10-27 10:51:08 +03:00
Marko Mäkelä
d4a89b9262 Merge 10.5 into 10.6 2021-10-27 10:06:02 +03:00
Alexander Barkov
2ed148c8d7 MDEV-25402 Assertion `!str || str != Ptr' failed in String::copy
The assert inside String::copy() prevents copying from from "str"
if its own String::Ptr also points to the same memory.

The idea of the assert is that copy() performs memory reallocation,
and this reallocation can free (and thus invalidate) the memory pointed by Ptr,
which can lead to further copying from a freed memory.

The assert was incomplete: copy() can free the memory pointed by its Ptr
only if String::alloced is true!

If the String is not alloced, it is still safe to copy even from
the location pointed by Ptr.

This scenario demonstrates a safe copy():
  const char *tmp= "123";
  String str1(tmp, 3);
  String str2(tmp, 3);
  // This statement is safe:
  str2.copy(str1->ptr(), str1->length(), str1->charset(), cs_to, &errors);

Inside the copy() the parameter "str" is equal to String::Ptr in this example.
But it's still ok to reallocate the memory for str2, because str2
was a constant before the copy() call. Thus reallocation does not
make the memory pointed by str1->ptr() invalid.

Adjusting the assert condition to allow copying for constant strings.
2021-10-27 10:50:15 +04:00
Marko Mäkelä
44f9736e0b Merge 10.4 into 10.5 2021-10-27 09:48:22 +03:00
Marko Mäkelä
4b8340d899 Fix tests for PLUGIN_PARTITION=NO 2021-10-27 08:54:37 +03:00
Alexander Barkov
05a0eae335 MDEV-22380 Assertion `name.length == strlen(name.str)' failed .. w/optimizer_trace enabled
Adding 10.4 specific tests.
2021-10-27 07:21:34 +04:00
Alexander Barkov
7b75242918 Merge remote-tracking branch 'origin/10.3' into 10.4 2021-10-27 07:14:45 +04:00
Alexander Barkov
e97b785d76 MDEV-22380: Assertion `name.length == strlen(name.str)' failed ...
Also fixes:
MDEV-25399 Assertion `name.length == strlen(name.str)' failed in Item_func_sp::make_send_field

Also fixes a problem that in this scenario:

SET NAMES binary;
SELECT 'some not well-formed utf8 string';

the auto-generated column name copied the binary string value directly
to the Item name, without checking utf8 well-formedness.

After this change auto-generated column names work as follows:
- Zero bytes 0x00 are copied to the name using HEX notation
- In case of "SET NAMES binary", all bytes sequences that do not make
  well-formed utf8 characters are copied to the name using HEX notation.
2021-10-27 06:09:57 +04:00
Eugene Kosov
d74d95961a MDEV-18543 IMPORT TABLESPACE fails after instant DROP COLUMN
ALTER TABLE IMPORT doesn't properly handle instant alter metadata.
This patch makes IMPORT read, parse and apply instant alter metadata at the
very beginning of operation. So, cases when source table has some metadata
and destination table doesn't have it now works fine.

DISCARD already removes instant metadata so importing normal table into
instant table worked fine before this patch.

decrypt_decompress(): decrypts and decompresses page if needed

handle_instant_metadata(): this should be the first thing to read source
table. Basically, it applies instant metadata to a destination
dict_table_t object. This is the first thing to read FSP flags so
all possible checks of it were moved to this function.

PageConverter::update_index_page(): it doesn't now read instant metadata.
This logic were moved into handle_instant_metadata()

row_import::match_flags(): this is a first part row_import::match_schema().
As a separate function it's used by handle_instant_metadata().

fil_space_t::is_full_crc32_compressed(): added convenient function

ha_innobase::discard_or_import_tablespace(): do not reload table definition
to read instant metadata because handle_instant_metadata() does it better.
The reverted code was originally added in
4e7ee166a9

ANONYMOUS_VAR: this is a handy thing to use along with make_scope_exit()

full_crc32_import.test shows different results, because no
dict_table_close() and dict_table_open_on_id() happens.
Thus, SHOW CREATE TABLE shows a little bit older table definition.
2021-10-26 22:50:58 +06:00
Alexander Barkov
49098bfd49 MDEV-26732 Assertion `0' failed in Item::val_native
Also fixes MDEV-24619 Wrong result or Assertion `0' in Item::val_native / Type_handler_inet6::Item_val_native_with_conversion

Type_handler_inet6::create_item_copy() created a generic Item_copy_string,
which does not implement val_native() - it has a dummy implementation
with DBUG_ASSERT(0), which made the server crash.

Fix:

- Adding a new class Type_handler_inet6
  which implements val_native().
- Fixing Type_handler_inet6::create_item_copy()
  to make Item_copy_inet6 instead of Item_copy_string.
2021-10-26 18:04:17 +04:00
Diego Dupin
d062b69037 MDEV-26868: Session tracking flag in OK_PACKET
Take into account client capabilities.
2021-10-26 15:22:15 +02:00
Oleksandr Byelkin
1f70e4b00c pthread_yield() is depricated now, so use sched_yield() if possible. 2021-10-26 15:05:13 +02:00
Oleksandr Byelkin
1fb4537e6f Safemalloc typo fix found by clang. 2021-10-26 15:05:13 +02:00
Vladislav Vaintroub
2084800768 Try to fix appveyor to prevent occasional failing of mysql_client_test
Sometimes, although not often, it would timeout after 3 minutes.
2021-10-26 14:33:07 +02:00
Vladislav Vaintroub
6211c35549 MDEV-23391 Crash/assertion CREATE OR REPLACE TABLE AS SELECT under LOCK TABLE
Happens with Innodb engine.

Move unlock_locked_table() past drop_open_table(), and
rollback current statement, so that we can actually unlock the table.

Anything else results in assertions, in drop, or unlock, or in close_table.
2021-10-26 14:33:07 +02:00
Thirunarayanan Balathandayuthapani
81b8547697 MDEV-26902 Auxilary fts table evicts during DDL
MDEV-25702(commit 696de6d06c) should've
closed the fts table further. This patch closes the table after
finishing the bulk insert operation.
2021-10-26 16:01:00 +05:30
Marko Mäkelä
58fe6b47d4 MDEV-26903: Assertion ctx->trx->state == TRX_STATE_ACTIVE on DROP INDEX
rollback_inplace_alter_table(): Tolerate a case where the transaction
is not in an active state. If ha_innobase::commit_inplace_alter_table()
failed with a deadlock, the transaction would already have been
rolled back. This omission of error handling was introduced in
commit 1bd681c8b3 (MDEV-25506 part 3).

After commit c3c53926c4 (MDEV-26554)
it became easier to trigger DB_DEADLOCK during exclusive table lock
acquisition in ha_innobase::commit_inplace_alter_table().

lock_table_low(): Add DBUG injection "innodb_table_deadlock".
2021-10-26 09:54:37 +03:00
Alexey Botchkov
efedf3da68 MDEV-22711 Assertion `nr != 0' failed in handler::update_auto_increment.
DBUG_ASSERT removed as the AUTO INCREMENT can actually be 0 when the
SET insert_id= 0; was done.
2021-10-26 00:29:27 +04:00
Alexey Botchkov
d627d00b13 MDEV-26556 An improper locking bug(s) due to unreleased lock.
Get rid of the global big_buffer.
2021-10-25 19:53:25 +04:00
Sergei Golubchik
d22c8cae00 compilation fixes for sys-devel/gcc-11.2.0:11 2021-10-25 17:30:03 +02:00
Marko Mäkelä
481aa0af46 MDEV-23267 Assertion on --bootstrap --innodb-force-recovery
SysTablespace::file_not_found(): If the system tablespace cannot be
found and innodb_force_recovery has been specified, refuse to start up.
The system tablespace is necessary for accessing any InnoDB tables,
because it contains the TRX_SYS page (the state of transactions)
and the InnoDB data dictionary.

This is similar to our handling of innodb_read_only except that
we will happily create the InnoDB temporary tablespace even if
innodb_force_recovry is set.
2021-10-25 15:14:43 +03:00
Anel Husakovic
395a033237 MDEV-22552: mytop packaging
`mytop` and `my_print_defaults` for RPM

- Add `mytop` to client package
- Add man page of `my_print_defaults` to client package
- Add dependencies for RPMs
- Remove old comment
- Remove dead link

Reviewed by: serg@mariadb.com
2021-10-25 12:15:49 +02:00
Bernard Spil
2011fcf87d MDEV-22552: mytop packaging
Make `my_print_defaults` a client app (required by `mytop`).

Closes #1566

Reviewed by: serg@mariadb.com, anel@mariadb.org
2021-10-25 12:15:49 +02:00
Marko Mäkelä
1193a793c4 MDEV-26674: Set innodb_use_native_aio=OFF when using io_uring on a potentially affected kernel
We have observed hangs of the io_uring subsystem when using a
Linux kernel newer than 5.10. Also 5.15-rc6 is affected by this.

The exact cause of the hangs has not been diagnosed yet.
As a safety measure, we will disable innodb_use_native_aio by default
when the server has been configured with io_uring and the kernel
version is between 5.11 and 5.15.

If the start-up parameter innodb_use_native_aio=ON is set, then
we will issue a warning to the server error log.
2021-10-25 10:55:04 +03:00