Commit graph

194367 commits

Author SHA1 Message Date
sjaakola
66b492f3e4 Fix for the test galera.galera_UK_conflict
mtr test galera.galera_UK_conflict, has rather distorted logic in test scenario 2.

The test would fail always with:

CURRENT_TEST: galera.galera_UK_conflict
mysqltest: In included file "./include/galera_wait_sync_point.inc":
included from /home/seppo/work/wsrep/mariadb-server/mysql-test/suite/galera/t/galera_UK_conflict.test at line 216:
At line 3: query 'SET SESSION wsrep_on = 0' failed: 1179: You are not allowed to execute this command in a transaction

This happens because wait_condition is called in wrong connection (node_1, which is excuting MST transaction)

This test is fixed by using contl connection for wait contdition check, and new result is recorded as well
2021-12-14 08:25:31 +02:00
Julius Goryavsky
7bc629a5ce MDEV-27181: Galera SST scripts should use ssl_capath for CA directory
1. Galera SST scripts should use ssl_capath (not ssl_ca) for CA
   directory. The current implementation tries to automatically
   detect the path using the trailing slash in the ssl_ca variable
   value, but this approach is not compatible with the server
   configuration. Now, by analogy with the server, SST scripts
   also use a separate ssl_capath variable. In addition, a similar
   tcapath variable has been added for the old-style configuration
   (in the "sst" section).
2. Openssl utility detection made more reliable.
3. Removed extra spaces in automatically generated command lines -
   to simplify debugging of the SST scripts.
4. In general, the code for detecting the presence or absence of
   auxiliary utilities has been improved - it is made more reliable
   in some configurations (and for shells other than bash).
2021-12-14 03:32:35 +01:00
Julius Goryavsky
8bb5563369 MDEV-27181: Galera SST scripts should use ssl_capath for CA directory
1. Galera SST scripts should use ssl_capath (not ssl_ca) for CA
   directory. The current implementation tries to automatically
   detect the path using the trailing slash in the ssl_ca variable
   value, but this approach is not compatible with the server
   configuration. Now, by analogy with the server, SST scripts
   also use a separate ssl_capath variable. In addition, a similar
   tcapath variable has been added for the old-style configuration
   (in the "sst" section).
2. Openssl utility detection made more reliable.
3. Removed extra spaces in automatically generated command lines -
   to simplify debugging of the SST scripts.
4. In general, the code for detecting the presence or absence of
   auxiliary utilities has been improved - it is made more reliable
   in some configurations (and for shells other than bash).
2021-12-14 03:25:19 +01:00
Marko Mäkelä
6b066ec332 MDEV-27235: Crash on SET GLOBAL innodb_encrypt_tables
fil_crypt_set_encrypt_tables(): If no encryption threads have been
initialized, do nothing.
2021-12-13 08:04:45 +02:00
Thirunarayanan Balathandayuthapani
660cfe4782 MDEV-27014 InnoDB fails to restore page 0 from the doublewrite buffer
- Addressing the format issue in deferred_dblwr() and changed the
function comment.
2021-12-12 15:09:59 +05:30
Thirunarayanan Balathandayuthapani
18c335a39e MDEV-27111 atomic.rename_table test case fails
InnoDB fails to identify the deferred tablespace after recovery.
Problem is that InnoDB fails to rename the tablespace present
in recovered tablespace. Fix is that InnoDB should try to rename the
recovered tablespace when tablespace is being deferred
2021-12-12 09:58:54 +05:30
Thirunarayanan Balathandayuthapani
be5990d0c8 MDEV-27014 InnoDB fails to restore page 0 from the doublewrite buffer
This patch reverts the commit cab8f4b552.
InnoDB fails to restore page0 from doublewrite buffer when the
tablespace is being deferred. In that case, InnoDB doesn't find
INIT_PAGE redo log record for page0 and it leads to failure.
InnoDB should recovery page0 from doublewrite buffer for the
deferred tablespace before applying the redo log records.

Added deferred_dblwr() to restore page0 of deferred tablespace
from doublewrite buffer
2021-12-12 09:58:54 +05:30
Marko Mäkelä
88b339805d Fix a test for cmake -DPLUGIN_PERFSCHEMA=NO 2021-12-11 15:27:14 +02:00
sjaakola
74b3d4252a MDEV-27218 mtr test galera.MDEV-20793 has sporadic false positive failures
Fixes to make the galera.MDEV-20793 test deterministic.
Specifically, after --send COMMIT, there is now a sync point to catch a known state of the COMMIT execution
2021-12-10 12:10:42 +02:00
Marko Mäkelä
50ed0bd891 MDEV-27219 Some error messages might report table names incorrectly on LLP64
In commit 49e2c8f0a6 (MDEV-25743)
some more use of the printf-style format "%.*s" was added.
The length parameter is of type int, not size_t.
On 64-bit platforms that follow the LLP64 convention (such as
64-bit Microsoft Windows), sizeof(int)==4 and sizeof(size_t)==8.

Let us explicitly cast the lengths to the correct type in order
to avoid any trouble.
2021-12-10 11:29:06 +02:00
Sergei Golubchik
0745db7179 don't use buffered_option_error_reporter without perfschema
it's not printed, not cleaned up without perfschema,
so isn't supposed to be written into either

this fixes "Memory not freed" warnings when early command line
options produce warnings in non-perfschema builds
2021-12-10 09:40:50 +01:00
forkfun
eafa2a1411 enable partition_open_files_limit test 2021-12-09 16:29:22 +01:00
forkfun
a9b5f59989 enable rpl_parallel2 test, as MDEV-23089 is fixed 2021-12-08 15:16:57 +01:00
forkfun
5e8148819e minor cleanup of backup_lock_binlog.test, so that test passes also with statement binlog format 2021-12-08 14:28:42 +01:00
Alexey Yurchenko
5c8e628dda wsrep-lib update: bugfixes, cleanups, event API, state transition cleanups
Don't switch state to DONOR in `wsrep_sst_donate()` - wsrep-lib does it now
2021-12-08 13:16:23 +02:00
Sergei Golubchik
186c1fa250 Merge branch '10.5' into 10.6 2021-12-07 22:11:30 +01:00
Sergei Golubchik
62ea1b4407 BUG#31761802 STATISTICS ANY QUERIES USING VIEWS ARE SUMMARIZED TOGETHER WITH THE VIEW DEFINITION SELECT
test case only
2021-12-07 21:31:27 +01:00
Sergei Golubchik
e27b1c3137 require system pcre2 in rpms
now when SLES12.3 is gone, we can enforce it
2021-12-07 21:31:13 +01:00
Sergei Golubchik
88ac91c7cc ColumnStore and S3 SUMMARY/DESCRIPTION for RPM 2021-12-07 21:31:00 +01:00
Sergei Golubchik
de70f921ce Merge branch '10.4' into 10.5 2021-12-07 21:30:27 +01:00
Sergei Krivonos
f974062c51 MDEV-19129: Fixed configure for Xcode, CMake generate
CMake Error in wsrep-lib/CMakeLists.txt:
  The custom command generating

    /Users/name/build/mariadb-server/sql/lex_token.h

  is attached to multiple targets:

    GenServerSource
    sql

  but none of these is a common dependency of the other(s).  This is not
  allowed by the Xcode "new build system".
2021-12-07 20:29:25 +01:00
Vladislav Vaintroub
f13c2107b3 Don't beep in mysql_upgrade_service.exe
This beep looks especially strange, as mysqladmin output is redirected to
the log file
2021-12-07 17:46:29 +01:00
Vladislav Vaintroub
8dd1f01d09 MDEV-27191 MariaDB client - "system" command does not work on Windows
- define USE_POPEN, like it is done elsewhere.
- use Notepad as default editor on Windows for the "edit" command.
2021-12-07 17:42:47 +01:00
Marko Mäkelä
cfcfdc65df MDEV-27190 InnoDB upgrade from 10.2, 10.3, 10.4 is not crash-safe
During startup, InnoDB must write a FILE_CHECKPOINT record.
However, before MDEV-12353 (in MariaDB Server 10.2, 10.3, 10.4)
the corresponding record MLOG_CHECKPOINT was encoded in a different way.

When we are upgrading from a logically empty 10.2, 10.3, or 10.4 redo log,
we must not write anything to the old log file, because if the server were
killed during the upgrade, we would end up with a corrupted log file, and
both the old and the new server would refuse to start up.

On upgrade, we must simply create a new logically empty log file
and replace the old ib_logfile0 with that.
2021-12-07 17:00:46 +02:00
forkfun
375ae890c7 enable rpl_semi_sync_after_sync and rpl_semi_sync_slave_compressed_protocol tests 2021-12-07 15:25:43 +01:00
Eugene Kosov
890c55177d MDEV-27183 optimize std::map lookup in during crash recovery
This is a low hanging fruit. Before this patch std::map::emplace() was
a ~50% of the whole recv_sys_t::parse() operation in by test.
After the fix it's only ~20%.

recv_sys_t::parse() recv_sys_t::pages is a collection of all pages
to recovery. Often, there are multiple changes for a single page.
Often, they go in a row and for such cases let's avoid
lookup in a std::map. cached_pages_it serves as a cache
of size 1.

recv_sys_t::add(): replace page_id argument with a std::map::iterator
2021-12-07 15:50:00 +06:00
Sergei Golubchik
e8a91c18ea Merge branch '10.3' into 10.4 2021-12-07 09:47:42 +01:00
Vladislav Vaintroub
d5ceddb391 Appveyor - cache chocolatey packages 2021-12-07 01:28:51 +01:00
Sergei Golubchik
153b75b576 Merge branch '10.2' into 10.3 2021-12-06 22:23:07 +01:00
Sergei Golubchik
71027eceac fix srpm builds after fe065f8d90 2021-12-06 18:50:19 +01:00
Sergei Golubchik
214cad8c3b fix ./mtr --manual warning after f5441ef4da 2021-12-06 18:50:19 +01:00
Eugene Kosov
0064316f19 cleanup: reduce code bloat 2021-12-06 14:06:17 +06:00
Marko Mäkelä
6deaff58a9 MDEV-27058 fixup: GCC 11 -march=i686 -Warray-bounds
page_zip_des_t::clear(): Avoid a bogus GCC warning with
some pointer arithmetics. Yes, storing the unrelated member "fix"
in this object is ugly, but it avoids memory alignment overhead
on 64-bit architectures.
2021-12-04 12:55:26 +02:00
Marko Mäkelä
3f7040fa97 Merge 10.5 into 10.6 2021-12-04 12:42:29 +02:00
Marko Mäkelä
fa1325512b Correct some comments
In commit aae3f921ad
recv_apply_hashed_log_recs() was replaced by recv_sys_t::apply().
2021-12-03 17:01:48 +02:00
Eugene Kosov
5d7da02793 MDEV-27139 32-bit systems fail to use big innodb-log-file-size
log_write_buf(): do not cast to size_t which prevents to write to files
which a bigger that 4G and remove useless assertion
2021-12-03 14:57:23 +06:00
Marko Mäkelä
d2a7710635 Merge 10.4 into 10.5 2021-12-03 10:27:35 +02:00
Marko Mäkelä
f458acc81e MDEV-27160 Out of memory in main.long_unique
A part of the test main.long_unique attempts to insert records
with two 60,000,001-byte columns. Let us move that test into
a separate file main.long_unique_big, declared as big test,
so that it can be skipped in environments with limited memory.
2021-12-03 09:56:30 +02:00
Christopher Odenbach
658a1e1fed Use mysqladmin ping instead of pid files
How 'bout know?
2021-12-03 15:34:08 +11:00
Christopher Odenbach
1b0fb2faa9 Fix logrotate problem with twice configured pid-file option
Hi,

if the pid-file option is configured more than once (e.g. multiple times in different files), my_print_defaults prints it twice, resulting in the logrotate postrotate script failing because of a syntax error. Debian fixed this already (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830976#42).

Perhaps you could implement this small change in the other branches as well?

Thanks,

Christopher
2021-12-03 15:34:08 +11:00
Lukas Javorsky
045f5f7b10 MDEV-21108 Add option for setting install paths of groonga
Include gronnga and groonga-normalizer-mysql install path
2021-12-02 16:40:29 +02:00
Marko Mäkelä
f9726ced25 MDEV-12353 fixup: Correct a comment
Several EXTENDED type records have already been implemented.
2021-12-01 13:43:43 +02:00
Thirunarayanan Balathandayuthapani
e0e24b180d MDEV-27014 InnoDB fails to restore page 0 from the doublewrite buffer
- Replaced the pointer parameter of validate_for_recovery() with uint32_t
2021-12-01 13:37:06 +05:30
Jan Lindström
d7b37de936 Fix bad galera tests
* galera_kill_applier : we should make sure that node has
  correct number of wsrep appliers
* galera_bad_wsrep_new_cluster: This test restarts both nodes,
so it is bad on mtr. Make sure it is run alone
* galera_update_limit : Make sure we have PK when needed
galera_as_slave_replay : bf abort was not consistent
* galera_unicode_pk : Add wait_conditions so that all nodes
are part of cluster and DDL and INSERT has replicated before
any further operations are done.
* galera_bf_abort_at_after_statement : Add wait_conditions
to make sure all nodes are part of cluster and that DDL
and INSERT has replicated. Make sure we reset DEBUG_SYNC.
2021-12-01 08:27:08 +02:00
Martin Beck
4ebac0fc86 MDEV-27088: Server crash on ARM (WMM architecture) due to missing barriers in lf-hash (10.5)
MariaDB server crashes on ARM (weak memory model architecture) while
concurrently executing l_find to load node->key and add_to_purgatory
to store node->key = NULL. l_find then uses key (which is NULL), to
pass it to a comparison function.

The specific problem is the out-of-order execution that happens on a
weak memory model architecture. Two essential reorderings are possible,
which need to be prevented.

a) As l_find has no barriers in place between the optimistic read of
the key field lf_hash.cc#L117 and the verification of link lf_hash.cc#L124,
the processor can reorder the load to happen after the while-loop.

In that case, a concurrent thread executing add_to_purgatory on the same
node can be scheduled to store NULL at the key field lf_alloc-pin.c#L253
before key is loaded in l_find.

b) A node is marked as deleted by a CAS in l_delete lf_hash.cc#L247 and
taken off the list with an upfollowing CAS lf_hash.cc#L252. Only if both
CAS succeed, the key field is written to by add_to_purgatory. However,
due to a missing barrier, the relaxed store of key lf_alloc-pin.c#L253
can be moved ahead of the two CAS operations, which makes the value of
the local purgatory list stored by add_to_purgatory visible to all threads
operating on the list. As the node is not marked as deleted yet, the
same error occurs in l_find.

This change three accesses to be atomic.

* optimistic read of key in l_find lf_hash.cc#L117
* read of link for verification lf_hash.cc#L124
* write of key in add_to_purgatory lf_alloc-pin.c#L253

Reviewers: Sergei Vojtovich, Sergei Golubchik

Fixes: MDEV-23510 / d30c1331a18d875e553f3fcf544997e4f33fb943
2021-11-30 15:16:16 +11:00
Martin Beck
17802165a6 MDEV-27088: lf unit tests - cycles insufficient
Per bug report, cycles was woefully insufficient to
detect any implementation error.
2021-11-30 11:30:52 +11:00
Martin Beck
4e0dcf1083 MDEV-27088: Server crash on ARM (WMM architecture) due to missing barriers in lf-hash
MariaDB server crashes on ARM (weak memory model architecture) while
concurrently executing l_find to load node->key and add_to_purgatory
to store node->key = NULL. l_find then uses key (which is NULL), to
pass it to a comparison function.

The specific problem is the out-of-order execution that happens on a
weak memory model architecture. Two essential reorderings are possible,
which need to be prevented.

a) As l_find has no barriers in place between the optimistic read of
the key field lf_hash.cc#L117 and the verification of link lf_hash.cc#L124,
the processor can reorder the load to happen after the while-loop.

In that case, a concurrent thread executing add_to_purgatory on the same
node can be scheduled to store NULL at the key field lf_alloc-pin.c#L253
before key is loaded in l_find.

b) A node is marked as deleted by a CAS in l_delete lf_hash.cc#L247 and
taken off the list with an upfollowing CAS lf_hash.cc#L252. Only if both
CAS succeed, the key field is written to by add_to_purgatory. However,
due to a missing barrier, the relaxed store of key lf_alloc-pin.c#L253
can be moved ahead of the two CAS operations, which makes the value of
the local purgatory list stored by add_to_purgatory visible to all threads
operating on the list. As the node is not marked as deleted yet, the
same error occurs in l_find.

This change three accesses to be atomic.

* optimistic read of key in l_find lf_hash.cc#L117
* read of link for verification lf_hash.cc#L124
* write of key in add_to_purgatory lf_alloc-pin.c#L253

Reviewers: Sergei Vojtovich, Sergei Golubchik

Fixes: MDEV-23510 / d30c1331a18d875e553f3fcf544997e4f33fb943
2021-11-30 11:30:52 +11:00
Thirunarayanan Balathandayuthapani
cab8f4b552 MDEV-27014 InnoDB fails to restore page 0 from the doublewrite buffer
InnoDB fails to restore page0 from doublewrite buffer when the
tablespace is being deferred. In that case, InnoDB doesn't find
INIT_PAGE redo log record for page0 and it leads to failure.
InnoDB should recovery page0 from doublewrite buffer.
2021-11-29 21:31:58 +05:30
Sergei Petrunia
5f22e83a29 Make the Optimizer Trace of reqular query and PS EXECUTE be identical
Print this piece when we've just made the choice to convert to semi-join.
Also, print it when we've already made that choice before:

  transformation": {
     "select_id": 2,
     "from": "IN (SELECT)",
     "to": "semijoin",
     "chosen": true
   }
2021-11-29 16:25:27 +03:00
Sergei Petrunia
cca4e14f9b Make the Optimizer Trace of reqular query and PS EXECUTE be identical
Print this piece when we've just made the choice to convert to semi-join.
Also, print it when we've already made that choice before:

  transformation": {
     "select_id": 2,
     "from": "IN (SELECT)",
     "to": "semijoin",
     "chosen": true
   }
2021-11-29 16:19:10 +03:00