Commit graph

190848 commits

Author SHA1 Message Date
Marko Mäkelä
47ab793d71 Merge 10.3 into 10.4 2021-11-09 08:40:14 +02:00
Marko Mäkelä
524b4a89da Merge 10.2 into 10.3 2021-11-09 08:26:59 +02:00
Marko Mäkelä
d6d1a1fc21 Remove a warning for clang 11 or earlier
This fixes up commit d22c8cae00
2021-11-09 08:23:25 +02:00
Marko Mäkelä
8c7e551da1 Remove restarts from encrypt_and_grep test 2021-11-09 08:08:29 +02:00
Marko Mäkelä
75f0c595d9 MariaDB 10.2.41 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEF39AEP5WyjM2MAMF8WVvJMdM0dgFAmGJTwkACgkQ8WVvJMdM
 0dgWYA/9EzjoEEB+b0iwhhnw8VRrsIEsLo9Cf0yhAg2dfFqKpzHGP9DNxsnb/pdS
 gbNKODrzSqRFdkO79ThfRa2FwIEOlAGJIbV5njRPqAsRoZ3qQd20RqJ+gbr8/PYn
 Xf0KZz82FcBePZjzcrbCUkqhrfnCtFsbg41YzYFZT6ETDtXOvusd4/eTZr+lhptk
 dfxItFsaJMhi1RxNFQlj+u7rFGpLbXtgfGEoQXj0CjtVvV2tyBPLP4siuaUXWQ88
 XfH7ZFHL/0LxEVNO4QGzp2yc6N4ePXYVLGqDjn8HxquG0YrZ37Z+G++nNudrC+K8
 +THgKihP162lMS770TL+4WLLBDWpIE01Fsf9GhxJclK2oIpxxdiw4rCPNwL4LoGw
 0N6yQdYN50CxFFzBOFOid0fp401G4w0FxkwDRhRcN895vSFMpZ60QLBv6MOcLH9D
 OqFKYT29bG4zr7mzV6uNXXrdQ5q9VFeU8coUV4MLQymatxlVOpOLYnEY2qQ8AcN3
 EwnVaacoo1ZrmBnG56H3TNrUQSpFXtXRmDgR3wwWB9CcqeWU/ImWYbETgObhvSSG
 O0QzLtAgSMsdfRWDxjPgi8di3t7k9Yi2kZUAs8nQFQNwFbGJ5O1LlJrnpfbJcngi
 GR2t8Rbvm1hk0AJIIAWg2T48Dc/OOWUtXjbL+HszdfqsuwFvWT4=
 =MSGM
 -----END PGP SIGNATURE-----

Merge mariadb-10.2.41 into 10.2
2021-11-09 08:07:58 +02:00
Daniel Black
7c30bc38a5 MDEV-26561 mariabackup release locks
The previous threads locked need to be released too.

This occurs if the initialization of any of the non-first
mutex/conditition variables errors occurs.
2021-11-09 17:05:55 +11:00
ryancaicse
e1eb39a446 MDEV-26561 Fix a bug due to unreleased lock
Fix a bug of unreleased lock ctrl_mutex in the method create_worker_threads
2021-11-09 17:05:55 +11:00
Marko Mäkelä
f7054ff5df MariaDB 10.3.32 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEF39AEP5WyjM2MAMF8WVvJMdM0dgFAmGJZJ0ACgkQ8WVvJMdM
 0dj8Jw//QD4uSbC4EHVdCDXWPQ9K/+Wv2A1DG4kCtngtQAVd/MgOpWK+9gdDCbKE
 Ce6m7627YLLzgBDzkEX/VkciHPd9GqvquqmgVKY1MdQ6efmmwgbzaGcaWcuJF8Z/
 C1pa7j0Duxn6nEuRbvM8OTgN4KfFlAc0OxpraJ7Fr8NvduLZQYMokBBW9DrJT1f1
 zGp4k05wUImBsmBt6teS073FS89frDL4J2aYGTXAxMjiqtno2MCopUIF2rpk5B29
 sJFaDpHCitNYDXuXZvWEWmuuss4vHz/NUYXM/GygfIteJqXKRLEOLAFBfvETyt4q
 6pYZDVfEGdKquHQu1a2XDI3W9+W1inmZ11dtebGnRexJTp9xeSxPhxiUvOQJj84A
 w6cQICCtlDCql3VlOIbt0vvAuXu+rOqhlqHorz0l62o6YjGE92z+NUL7B6gODip9
 RGd0gwCloPo+jGHnfpC6rvfcjA32vEx6L8giYTAYybxqjN1bMNIrix+7zwgfpZPZ
 0QRZtWtio/Iozj41q6x7dmP2Pxjll58+fPUEKevQn2iPm5WoPe+zrq3/lUdXFbZY
 3cz9fZch4YMTlhhu9BwuEmc2T9aIIm/YwYaB0Kmg55J/KT9xyerpMFZmRaF0VWcQ
 70ODJSMEDBhBW3n19LuYK/p3uJr551V/dFbZ/6lCXzbyp5i5MO8=
 =yIEG
 -----END PGP SIGNATURE-----

Merge mariadb-10.3.32 into 10.3
2021-11-09 07:59:36 +02:00
Oleksandr Byelkin
de2fa9eced Merge branch '10.4' into bb-10.4-release 2021-11-08 19:39:13 +01:00
Daniel Bartholomew
e026eddc7d
bump the VERSION 2021-11-08 12:56:18 -05:00
Daniel Bartholomew
227b782ad6
bump the VERSION 2021-11-08 12:39:31 -05:00
Daniel Bartholomew
8b6dfc3c02
bump the VERSION 2021-11-08 11:26:47 -05:00
Alexey Bychko
2a3dae115b MDEV-22522 RPM packages have meaningless summary/description
added summary/description per package.
2021-11-08 16:45:23 +07:00
Alexander Barkov
9dc05f1f11 MDEV-25610 Assertion `escape != -1' failed in Item_func_like::val_int 2021-11-08 08:16:48 +04:00
Oleksandr Byelkin
a19ab67318 Merge branch '10.3' into 10.4 2021-11-05 19:59:58 +01:00
Oleksandr Byelkin
a2f147af35 Merge branch '10.2' into 10.3 2021-11-05 19:58:32 +01:00
Andrei Elkin
561b6c7e51 MDEV-26833 Missed statement rollback in case transaction drops or create temporary table
When transaction creates or drops temporary tables and afterward its statement
faces an error even the transactional table statement's cached ROW
format events get involved into binlog and are visible after the transaction's commit.

Fixed with proper analysis of whether the errored-out statement needs
to be rolled back in binlog.
For instance a fact of already cached CREATE or DROP for temporary
tables by previous statements alone
does not cause to retain the being errored-out statement events in the
cache.
Conversely, if the statement creates or drops a temporary table
itself it can't be rolled back - this rule remains.
2021-11-05 19:33:28 +02:00
Vladislav Vaintroub
322fc4f4b3 appveyor - do not use buggy cygwin bison. 2021-11-04 07:34:55 +01:00
Sergei Krivonos
2df99f2193 Revert "MDEV-19129: Xcode compatibility update: deprecated vfork -> fork"
This reverts commit 5d6f3cebca.
2021-11-03 18:14:02 +02:00
Oleksandr Byelkin
3021b929c8 Merge branch '10.3' into 10.4 2021-11-03 14:01:20 +01:00
Oleksandr Byelkin
69c70c18af Merge branch '10.2' into 10.3 2021-11-03 13:52:52 +01:00
Oleksandr Byelkin
eb2c3d38e4 post merge result fix 2021-11-02 15:34:47 +01:00
Oleksandr Byelkin
ef968c9e63 Merge branch '10.3' into 10.4 2021-11-02 13:11:23 +01:00
Oleksandr Byelkin
bb46b79c8c Fix mutex order according to a new sequence. 2021-11-02 13:09:35 +01:00
Oleksandr Byelkin
f0b9194d89 Merge branch '10.2' into 10.3 2021-11-02 10:01:31 +01:00
Oleksandr Byelkin
d7c179e65c move "bad" test in seperate file with valgrind prohibited (different size of allocated memory) 2021-11-02 09:50:49 +01:00
Aleksey Midenkov
c8cece9144 MDEV-26928 Column-inclusive WITH SYSTEM VERSIONING doesn't work with explicit system fields
versioning_fields flag indicates that any columns were specified WITH
SYSTEM VERSIONING. In that case we imply WITH SYSTEM VERSIONING for
the whole table and WITHOUT SYSTEM VERSIONING for the other columns.
2021-11-02 11:49:47 +03:00
Aleksey Midenkov
8ce5635a3e MDEV-22284 Aria table key read crash because wrong index used
When restoring lastinx last_key.keyinfo must be updated as well. The
good example is in _ma_check_index().

The point of failure is extra(HA_EXTRA_NO_KEYREAD) in
ha_maria::get_auto_increment():

  1. extra(HA_EXTRA_KEYREAD) saves lastinx;
  2. maria_rkey() changes index, so the lastinx and last_key.keyinfo;
  3. extra(HA_EXTRA_NO_KEYREAD) restores lastinx but not
     last_key.keyinfo.

So we have discrepancy between lastinx and last_key.keyinfo after 3.
2021-11-02 11:26:35 +03:00
Jan Lindström
eab7f5d8bc MDEV-23328 Server hang due to Galera lock conflict resolution
* Fix error handling NULL-pointer reference
* Add mtr-suppression on galera_ssl_upgrade
2021-11-02 10:08:54 +02:00
Jan Lindström
db64924454 MDEV-23328 Server hang due to Galera lock conflict resolution
* Fix error handling NULL-pointer reference
* Add mtr-suppression on galera_ssl_upgrade
2021-11-02 07:23:40 +02:00
Jan Lindström
e571eaae9f MDEV-23328 Server hang due to Galera lock conflict resolution
Use better error message when KILL fails even in case TOI
fails.
2021-11-02 07:20:30 +02:00
Alexander Barkov
d0b611a76d MDEV-24335 Unexpected question mark in the end of a TINYTEXT column
my_copy_fix_mb() passed MIN(src_length,dst_length) to
my_append_fix_badly_formed_tail(). It could break a multi-byte
character in the middle, which put the question mark to the
destination.

Fixing the code to pass the true src_length to
my_append_fix_badly_formed_tail().
2021-11-02 09:00:49 +04:00
Aleksey Midenkov
1be39f86cc MDEV-25552 system versioned partitioned by LIMIT tables break CHECK TABLE
Replaced HA_ADMIN_NOT_IMPLEMENTED error code by HA_ADMIN_OK. Now CHECK
TABLE does not fail by unsupported check_misplaced_rows(). Admin
message is not needed as well.

Test case is the same as for MDEV-21011 (a7cf0db3d8), the result have
been changed.
2021-11-02 04:52:03 +03:00
Aleksey Midenkov
c6207ecba4 MDEV-25803 innodb.alter_candidate_key fix
There is a case when implicit primary key may be changed when removing
NOT NULL from the part of unique key. In that case we update
modified_primary_key which is then used to not skip key sorting.

According to is_candidate_key() there is no other cases when primary
kay may be changed implicitly.
2021-11-02 04:52:03 +03:00
Aleksey Midenkov
63c922ae0c MDEV-25803 Inplace ALTER breaks MyISAM/Aria table when order of keys is changed
mysql_prepare_create_table() does my_qsort(sort_keys) on key
info. This sorting is indeterministic: a table is created with one
order and inplace alter may overwrite frm with another order. Since
inplace alter does nothing about key info for MyISAM/Aria storage
engines this results in discrepancy between frm and storage engine key
definitions.

The fix avoids the sorting of keys when no new keys added by ALTER
(and this is ok for MyISAM/Aria since it cannot add new keys inplace).

Notes:

mi_keydef_write()/mi_keyseg_write() are used only in mi_create(). They
should be used in ha_inplace_alter_table() as well.

Aria corruption detection is unimplemented: maria_check_definition()
is never used!

MySQL 8.0 has this bug as well as of 8.0.26.

This breaks main.long_unique in 10.4. The new result is correct and
should be applied as it just different (original) order of keys.
2021-11-02 04:52:03 +03:00
Jan Lindström
ea239034de MDEV-23328 Server hang due to Galera lock conflict resolution
* Fix error handling NULL-pointer reference
* Add mtr-suppression on galera_ssl_upgrade
2021-11-01 13:07:55 +02:00
Marko Mäkelä
026984c360 MDEV-26949 --debug-gdb installs redundant signal handlers
There is a server startup option --gdb a.k.a. --debug-gdb that requests
signals to be set for more convenient debugging. Most notably, SIGINT
(ctrl-c) will not be ignored, and you will be able to interrupt the
execution of the server while GDB is attached to it.

When we are debugging, the signal handlers that would normally display
a terse stack trace are useless.

When we are debugging with rr, the signal handlers may interfere with
a SIGKILL that could be sent to the process by the environment, and ruin
the rr replay trace, due to a Linux kernel bug
https://lkml.org/lkml/2021/10/31/311

To be able to diagnose bugs in kill+restart tests, we may really need
both a trace before the SIGKILL and a trace of the failure after a
subsequent server startup. So, we had better avoid hitting the problem
by simply not installing those signal handlers.
2021-11-01 10:29:58 +02:00
Sergei Krivonos
0c77c5f6e7 MDEV-19129: Xcode compatibility update: #include <editline/readline.h> path 2021-10-31 00:24:26 +03:00
Sergei Krivonos
5d6f3cebca MDEV-19129: Xcode compatibility update: deprecated vfork -> fork 2021-10-31 00:23:53 +03:00
Oleksandr Byelkin
5900f3a782 Merge branch '10.3' into 10.4 2021-10-29 19:18:24 +02:00
Oleksandr Byelkin
6953af3650 Merge branch '10.2' into 10.3 2021-10-29 19:09:32 +02:00
Sergei Petrunia
d352bc5b67 MDEV-26929: Make the main testsuite runnable with optimizer trace enabled
Part#2: check that compile-time default @@optimizer_trace is correct.
2021-10-29 11:44:09 +03:00
Alexander Barkov
059797ed44 MDEV-24901 SIGSEGV in fts_get_table_name, SIGSEGV in ib_vector_size, SIGSEGV in row_merge_fts_doc_tokenize, stack smashing
strmake() puts one extra 0x00 byte at the end of the string.
The code in my_strnxfrm_tis620[_nopad] did not take this into
account, so in the reported scenario the 0x00 byte was put outside
of a stack variable, which made ASAN crash.

This problem is already fixed in in MySQL:

  commit 19bd66fe43c41f0bde5f36bc6b455a46693069fb
  Author: bin.x.su@oracle.com <>
  Date:   Fri Apr 4 11:35:27 2014 +0800

But the fix does not seem to be correct, as it breaks when finds a zero byte
in the source string.

Using memcpy() instead of strmake().

- Unlike strmake(), memcpy() it does not write beyond the destination
  size passed.
- Unlike the MySQL fix, memcpy() does not break on the first 0x00 byte found
  in the source string.
2021-10-29 12:37:29 +04:00
sjaakola
157b3a637f MDEV-23328 Server hang due to Galera lock conflict resolution
Mutex order violation when wsrep bf thread kills a conflicting trx,
the stack is

          wsrep_thd_LOCK()
          wsrep_kill_victim()
          lock_rec_other_has_conflicting()
          lock_clust_rec_read_check_and_lock()
          row_search_mvcc()
          ha_innobase::index_read()
          ha_innobase::rnd_pos()
          handler::ha_rnd_pos()
          handler::rnd_pos_by_record()
          handler::ha_rnd_pos_by_record()
          Rows_log_event::find_row()
          Update_rows_log_event::do_exec_row()
          Rows_log_event::do_apply_event()
          Log_event::apply_event()
          wsrep_apply_events()

and mutexes are taken in the order

          lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data

When a normal KILL statement is executed, the stack is

          innobase_kill_query()
          kill_handlerton()
          plugin_foreach_with_mask()
          ha_kill_query()
          THD::awake()
          kill_one_thread()

        and mutexes are

          victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach,  purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This also fixed unprotected calls to wsrep_thd_abort
that will use wsrep_abort_transaction. This is fixed
by holding THD::LOCK_thd_data while we abort transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-10-29 10:00:17 +03:00
Jan Lindström
30337addfc MDEV-25114: Crash: WSREP: invalid state ROLLED_BACK (FATAL)
Revert "MDEV-23328 Server hang due to Galera lock conflict resolution"

This reverts commit 29bbcac0ee.
2021-10-29 10:00:05 +03:00
sjaakola
5c230b21bf MDEV-23328 Server hang due to Galera lock conflict resolution
Mutex order violation when wsrep bf thread kills a conflicting trx,
the stack is

          wsrep_thd_LOCK()
          wsrep_kill_victim()
          lock_rec_other_has_conflicting()
          lock_clust_rec_read_check_and_lock()
          row_search_mvcc()
          ha_innobase::index_read()
          ha_innobase::rnd_pos()
          handler::ha_rnd_pos()
          handler::rnd_pos_by_record()
          handler::ha_rnd_pos_by_record()
          Rows_log_event::find_row()
          Update_rows_log_event::do_exec_row()
          Rows_log_event::do_apply_event()
          Log_event::apply_event()
          wsrep_apply_events()

and mutexes are taken in the order

          lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data

When a normal KILL statement is executed, the stack is

          innobase_kill_query()
          kill_handlerton()
          plugin_foreach_with_mask()
          ha_kill_query()
          THD::awake()
          kill_one_thread()

        and mutexes are

          victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach,  purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This also fixed unprotected calls to wsrep_thd_abort
that will use wsrep_abort_transaction. This is fixed
by holding THD::LOCK_thd_data while we abort transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-10-29 09:52:52 +03:00
Jan Lindström
aa7ca987db MDEV-25114: Crash: WSREP: invalid state ROLLED_BACK (FATAL)
Revert "MDEV-23328 Server hang due to Galera lock conflict resolution"

This reverts commit eac8341df4.
2021-10-29 09:52:40 +03:00
sjaakola
db50ea3ad3 MDEV-23328 Server hang due to Galera lock conflict resolution
Mutex order violation when wsrep bf thread kills a conflicting trx,
the stack is

          wsrep_thd_LOCK()
          wsrep_kill_victim()
          lock_rec_other_has_conflicting()
          lock_clust_rec_read_check_and_lock()
          row_search_mvcc()
          ha_innobase::index_read()
          ha_innobase::rnd_pos()
          handler::ha_rnd_pos()
          handler::rnd_pos_by_record()
          handler::ha_rnd_pos_by_record()
          Rows_log_event::find_row()
          Update_rows_log_event::do_exec_row()
          Rows_log_event::do_apply_event()
          Log_event::apply_event()
          wsrep_apply_events()

and mutexes are taken in the order

          lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data

When a normal KILL statement is executed, the stack is

          innobase_kill_query()
          kill_handlerton()
          plugin_foreach_with_mask()
          ha_kill_query()
          THD::awake()
          kill_one_thread()

        and mutexes are

          victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach,  purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This also fixed unprotected calls to wsrep_thd_abort
that will use wsrep_abort_transaction. This is fixed
by holding THD::LOCK_thd_data while we abort transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-10-29 07:57:18 +03:00
Jan Lindström
c8b39f7ee2 MDEV-25114: Crash: WSREP: invalid state ROLLED_BACK (FATAL)
Revert "MDEV-23328 Server hang due to Galera lock conflict resolution"

This reverts commit 29bbcac0ee.
2021-10-29 07:57:03 +03:00
Nikita Malyavin
1fdac57447 MDEV-26453 Assertion `0' failed in row_upd_sec_index_entry & corruption
Long UNIQUE HASH index silently creates virtual column index, which should
be impossible for base columns featuring AUTO_INCREMENT.

Fix: add a relevant check; add new vcol type for a prettier error message.
2021-10-29 05:05:21 +03:00