Commit graph

193275 commits

Author SHA1 Message Date
Daniel Black
2776635cb9 MDEV-24788: mariadbd --help Can't lock aria control file
... '/var/lib/mysql/aria_log_control' for exclusive use, error: 11

As such don't lock under aria_readonly (which is set by opt_help in the handler
initialization.

Fixes: 8eba777c2b
2021-12-16 10:53:32 +11:00
Julius Goryavsky
03678bcf55 MDEV-27181: Galera SST scripts should use ssl_capath for CA directory
1. Galera SST scripts should use ssl_capath (not ssl_ca) for CA
   directory. The current implementation tries to automatically
   detect the path using the trailing slash in the ssl_ca variable
   value, but this approach is not compatible with the server
   configuration. Now, by analogy with the server, SST scripts
   also use a separate ssl_capath variable. In addition, a similar
   tcapath variable has been added for the old-style configuration
   (in the "sst" section).
2. Openssl utility detection made more reliable.
3. Removed extra spaces in automatically generated command lines -
   to simplify debugging of the SST scripts.
4. In general, the code for detecting the presence or absence of
   auxiliary utilities has been improved - it is made more reliable
   in some configurations (and for shells other than bash).
2021-12-14 13:36:13 +01:00
Marko Mäkelä
88b339805d Fix a test for cmake -DPLUGIN_PERFSCHEMA=NO 2021-12-11 15:27:14 +02:00
forkfun
a9b5f59989 enable rpl_parallel2 test, as MDEV-23089 is fixed 2021-12-08 15:16:57 +01:00
Sergei Golubchik
62ea1b4407 BUG#31761802 STATISTICS ANY QUERIES USING VIEWS ARE SUMMARIZED TOGETHER WITH THE VIEW DEFINITION SELECT
test case only
2021-12-07 21:31:27 +01:00
Sergei Golubchik
e27b1c3137 require system pcre2 in rpms
now when SLES12.3 is gone, we can enforce it
2021-12-07 21:31:13 +01:00
Sergei Golubchik
88ac91c7cc ColumnStore and S3 SUMMARY/DESCRIPTION for RPM 2021-12-07 21:31:00 +01:00
Sergei Golubchik
de70f921ce Merge branch '10.4' into 10.5 2021-12-07 21:30:27 +01:00
Marko Mäkelä
cfcfdc65df MDEV-27190 InnoDB upgrade from 10.2, 10.3, 10.4 is not crash-safe
During startup, InnoDB must write a FILE_CHECKPOINT record.
However, before MDEV-12353 (in MariaDB Server 10.2, 10.3, 10.4)
the corresponding record MLOG_CHECKPOINT was encoded in a different way.

When we are upgrading from a logically empty 10.2, 10.3, or 10.4 redo log,
we must not write anything to the old log file, because if the server were
killed during the upgrade, we would end up with a corrupted log file, and
both the old and the new server would refuse to start up.

On upgrade, we must simply create a new logically empty log file
and replace the old ib_logfile0 with that.
2021-12-07 17:00:46 +02:00
Eugene Kosov
890c55177d MDEV-27183 optimize std::map lookup in during crash recovery
This is a low hanging fruit. Before this patch std::map::emplace() was
a ~50% of the whole recv_sys_t::parse() operation in by test.
After the fix it's only ~20%.

recv_sys_t::parse() recv_sys_t::pages is a collection of all pages
to recovery. Often, there are multiple changes for a single page.
Often, they go in a row and for such cases let's avoid
lookup in a std::map. cached_pages_it serves as a cache
of size 1.

recv_sys_t::add(): replace page_id argument with a std::map::iterator
2021-12-07 15:50:00 +06:00
Sergei Golubchik
e8a91c18ea Merge branch '10.3' into 10.4 2021-12-07 09:47:42 +01:00
Sergei Golubchik
153b75b576 Merge branch '10.2' into 10.3 2021-12-06 22:23:07 +01:00
Sergei Golubchik
71027eceac fix srpm builds after fe065f8d90 2021-12-06 18:50:19 +01:00
Sergei Golubchik
214cad8c3b fix ./mtr --manual warning after f5441ef4da 2021-12-06 18:50:19 +01:00
Eugene Kosov
0064316f19 cleanup: reduce code bloat 2021-12-06 14:06:17 +06:00
Marko Mäkelä
fa1325512b Correct some comments
In commit aae3f921ad
recv_apply_hashed_log_recs() was replaced by recv_sys_t::apply().
2021-12-03 17:01:48 +02:00
Eugene Kosov
5d7da02793 MDEV-27139 32-bit systems fail to use big innodb-log-file-size
log_write_buf(): do not cast to size_t which prevents to write to files
which a bigger that 4G and remove useless assertion
2021-12-03 14:57:23 +06:00
Marko Mäkelä
d2a7710635 Merge 10.4 into 10.5 2021-12-03 10:27:35 +02:00
Marko Mäkelä
f458acc81e MDEV-27160 Out of memory in main.long_unique
A part of the test main.long_unique attempts to insert records
with two 60,000,001-byte columns. Let us move that test into
a separate file main.long_unique_big, declared as big test,
so that it can be skipped in environments with limited memory.
2021-12-03 09:56:30 +02:00
Christopher Odenbach
658a1e1fed Use mysqladmin ping instead of pid files
How 'bout know?
2021-12-03 15:34:08 +11:00
Christopher Odenbach
1b0fb2faa9 Fix logrotate problem with twice configured pid-file option
Hi,

if the pid-file option is configured more than once (e.g. multiple times in different files), my_print_defaults prints it twice, resulting in the logrotate postrotate script failing because of a syntax error. Debian fixed this already (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830976#42).

Perhaps you could implement this small change in the other branches as well?

Thanks,

Christopher
2021-12-03 15:34:08 +11:00
Lukas Javorsky
045f5f7b10 MDEV-21108 Add option for setting install paths of groonga
Include gronnga and groonga-normalizer-mysql install path
2021-12-02 16:40:29 +02:00
Marko Mäkelä
f9726ced25 MDEV-12353 fixup: Correct a comment
Several EXTENDED type records have already been implemented.
2021-12-01 13:43:43 +02:00
Jan Lindström
d7b37de936 Fix bad galera tests
* galera_kill_applier : we should make sure that node has
  correct number of wsrep appliers
* galera_bad_wsrep_new_cluster: This test restarts both nodes,
so it is bad on mtr. Make sure it is run alone
* galera_update_limit : Make sure we have PK when needed
galera_as_slave_replay : bf abort was not consistent
* galera_unicode_pk : Add wait_conditions so that all nodes
are part of cluster and DDL and INSERT has replicated before
any further operations are done.
* galera_bf_abort_at_after_statement : Add wait_conditions
to make sure all nodes are part of cluster and that DDL
and INSERT has replicated. Make sure we reset DEBUG_SYNC.
2021-12-01 08:27:08 +02:00
Martin Beck
4ebac0fc86 MDEV-27088: Server crash on ARM (WMM architecture) due to missing barriers in lf-hash (10.5)
MariaDB server crashes on ARM (weak memory model architecture) while
concurrently executing l_find to load node->key and add_to_purgatory
to store node->key = NULL. l_find then uses key (which is NULL), to
pass it to a comparison function.

The specific problem is the out-of-order execution that happens on a
weak memory model architecture. Two essential reorderings are possible,
which need to be prevented.

a) As l_find has no barriers in place between the optimistic read of
the key field lf_hash.cc#L117 and the verification of link lf_hash.cc#L124,
the processor can reorder the load to happen after the while-loop.

In that case, a concurrent thread executing add_to_purgatory on the same
node can be scheduled to store NULL at the key field lf_alloc-pin.c#L253
before key is loaded in l_find.

b) A node is marked as deleted by a CAS in l_delete lf_hash.cc#L247 and
taken off the list with an upfollowing CAS lf_hash.cc#L252. Only if both
CAS succeed, the key field is written to by add_to_purgatory. However,
due to a missing barrier, the relaxed store of key lf_alloc-pin.c#L253
can be moved ahead of the two CAS operations, which makes the value of
the local purgatory list stored by add_to_purgatory visible to all threads
operating on the list. As the node is not marked as deleted yet, the
same error occurs in l_find.

This change three accesses to be atomic.

* optimistic read of key in l_find lf_hash.cc#L117
* read of link for verification lf_hash.cc#L124
* write of key in add_to_purgatory lf_alloc-pin.c#L253

Reviewers: Sergei Vojtovich, Sergei Golubchik

Fixes: MDEV-23510 / d30c1331a18d875e553f3fcf544997e4f33fb943
2021-11-30 15:16:16 +11:00
Martin Beck
17802165a6 MDEV-27088: lf unit tests - cycles insufficient
Per bug report, cycles was woefully insufficient to
detect any implementation error.
2021-11-30 11:30:52 +11:00
Martin Beck
4e0dcf1083 MDEV-27088: Server crash on ARM (WMM architecture) due to missing barriers in lf-hash
MariaDB server crashes on ARM (weak memory model architecture) while
concurrently executing l_find to load node->key and add_to_purgatory
to store node->key = NULL. l_find then uses key (which is NULL), to
pass it to a comparison function.

The specific problem is the out-of-order execution that happens on a
weak memory model architecture. Two essential reorderings are possible,
which need to be prevented.

a) As l_find has no barriers in place between the optimistic read of
the key field lf_hash.cc#L117 and the verification of link lf_hash.cc#L124,
the processor can reorder the load to happen after the while-loop.

In that case, a concurrent thread executing add_to_purgatory on the same
node can be scheduled to store NULL at the key field lf_alloc-pin.c#L253
before key is loaded in l_find.

b) A node is marked as deleted by a CAS in l_delete lf_hash.cc#L247 and
taken off the list with an upfollowing CAS lf_hash.cc#L252. Only if both
CAS succeed, the key field is written to by add_to_purgatory. However,
due to a missing barrier, the relaxed store of key lf_alloc-pin.c#L253
can be moved ahead of the two CAS operations, which makes the value of
the local purgatory list stored by add_to_purgatory visible to all threads
operating on the list. As the node is not marked as deleted yet, the
same error occurs in l_find.

This change three accesses to be atomic.

* optimistic read of key in l_find lf_hash.cc#L117
* read of link for verification lf_hash.cc#L124
* write of key in add_to_purgatory lf_alloc-pin.c#L253

Reviewers: Sergei Vojtovich, Sergei Golubchik

Fixes: MDEV-23510 / d30c1331a18d875e553f3fcf544997e4f33fb943
2021-11-30 11:30:52 +11:00
Sergei Petrunia
cca4e14f9b Make the Optimizer Trace of reqular query and PS EXECUTE be identical
Print this piece when we've just made the choice to convert to semi-join.
Also, print it when we've already made that choice before:

  transformation": {
     "select_id": 2,
     "from": "IN (SELECT)",
     "to": "semijoin",
     "chosen": true
   }
2021-11-29 16:19:10 +03:00
Marko Mäkelä
d4cb177603 Merge 10.4 into 10.5 2021-11-29 11:16:20 +02:00
Marko Mäkelä
4da2273876 Merge 10.3 into 10.4 2021-11-29 10:59:22 +02:00
Marko Mäkelä
fafe60e7e2 MDEV-27134: Sporadic failure of DROP DATABASE test
Let us create and drop a separate database for getting rid of the
default database in the MDEV-22781 test.
2021-11-29 10:34:33 +02:00
Marko Mäkelä
289721de9a Merge 10.2 into 10.3 2021-11-29 10:33:06 +02:00
Igor Babaev
ac963142ee MDEV-26553 NOT IN subquery construct crashing 10.1 and up
This bug was introduced by commit be00e279c6
The commit was applied for the task MDEV-6480 that allowed to remove top
level disjuncts from WHERE conditions if the range optimizer evaluated them
as always equal to FALSE/NULL.
If such disjuncts are removed the WHERE condition may become an AND formula
and if this formula contains multiple equalities the field JOIN::item_equal
must be updated to refer to these equalities. The above mentioned commit
forgot to do this and it could cause crashes for some queries.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2021-11-26 13:02:39 -08:00
Sergei Golubchik
f5441ef4da MDEV-26972 MTR worker aborts after server restart failure
restore the old behavior where without a debugger mtr does not
wait for mysqld to start. It was broken in feacc0aaf2
2021-11-26 17:15:41 +01:00
Sergei Golubchik
a96b428269 MDEV-26755 innodb.undo_truncate: ilink::assert_linked(): Assertion `prev != 0 && next != 0' failed
close_connections() in mysqld.cc sends a signal to all threads.
But InnoDB is too busy purging, doesn't react immediately.
close_connections() waits 20 seconds, which isn't enough in this
particular case, and then unlinks all threads from
the list and forcibly closes their vio connection.

InnoDB background  threads have no vio connection to close, but
they're unlinked all the same. So when later they finally notice
the shutdown request and try to unlink themselves, they fail to
assert that they're still linked.

Fix: don't assert_linked, as another thread can unlink this THD anytime
2021-11-26 11:06:09 +01:00
Sergei Golubchik
4ba7478553 add a test case
MDEV-20330 Combination of "," (comma), cross join and left join fails to parse
2021-11-26 09:33:26 +02:00
Hugo Wen
e9572e53e6 MDEV-27124: Update definer of Add/DropGeometryColumn procedures from 'root' to 'mariadb.sys'
From 10.4.13, the `mariadb.sys` user was created to replace `root` definers.
 - In commit 0253ea7f22, definer of
   Add/DropGeometryColumn procedures was changed to `mariadb.sys`, in
   `scripts/maria_add_gis_sp.sql.in`.
   However, maria_add_gis_sp.sql only applies to new databases created by
   installation script. Databases upgraded from old versions will miss this
   change.
 - In addition, according to commit
   0d6d801e5886208b2632247d88da106a543e1032(MDEV-23102), in some scenarios
   when root user is replaced it will skip creating `mariadb.sys` user.

This commit is to update the definer from `root` to `mariadb.sys` during
upgrade. It only makes the change if the original definers are root.

Doesn't choose to execute `maria_add_gis_sp.sql` in upgrade script to
recreate the procedures is because of considering the scenarios of
MDEV-23102 that `root` user is replaced and `mariadb.sys` is not created.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2021-11-26 13:12:51 +11:00
Marko Mäkelä
ca26953924 Merge 10.4 into 10.5 2021-11-25 07:49:58 +02:00
Sergei Krivonos
cba065f4fe Json_writer_object add integers 2021-11-24 13:01:49 +02:00
Sergei Krivonos
691f7e165a MDEV-27036: allow Json_writer_[array|object] from Json_writer 2021-11-24 13:01:18 +02:00
ryancaicse
f809a4fbd0 MDEV-26558 Fix a deadlock due to cyclic dependence
Fix a potential deadlock bug between locks ctrl_mutex and entry->mutex
2021-11-24 12:57:44 +02:00
Marko Mäkelä
b53ee760ff Cleanup: offsetof instead of my_offsetof
Sys_lc_messages, Sys_lc_time_names: Use offsetof(MY_LOCALE, ...)
because the type has standard layout.
2021-11-24 12:42:31 +02:00
Marko Mäkelä
917b421012 MDEV-26682 fixup: GCC -Wunused-variable 2021-11-24 12:05:44 +02:00
Marko Mäkelä
6b2b510839 PFS_events_statements cleanup: Use offsetof
The macro my_offsetof() performs pointer arithmetics that may be
undefined behavior. As reported in MDEV-26272, it may cause
clang -fsanitize=undefined to generate invalid memory references.

struct PFS_events_statements: Convert to std::is_standard_layout
by encapsulating the standard-layout struct PFS_events instead of
deriving from it, so that the standard macro offsetof() can be used.

PFS_events_statements::copy(): Renamed from copy_events_statements().
A cast to void* is now needed in memcpy() to avoid GCC -Wclass-memaccess
"writing to an object ... leaves 64 bytes unchanged".
2021-11-24 12:04:51 +02:00
Marko Mäkelä
7e7235faa9 Cleanup: offsetof instead of my_offsetof
mdl_hash_function(): Use plain offsetof; MDL_key has standard layout.
2021-11-24 09:55:29 +02:00
Daniel Black
8d398710b3 Debian: comment default config
There's no need for Debian to set config items to their
default. Left commented user, datadir and tmpdir as
these may want to be changed. lc-messages and skip-external-locks
are so infrequently set even listing them looks overly verbose.

socket left uncommented in [client-server] as various client
implementations may have different defaults compiled in.
2021-11-24 18:13:57 +11:00
Daniel Black
ef179dadf4 mysql_install_db: remove MySQL references
MySQL documentation isn't going to help our
users and we shouldn't refer to it.
2021-11-24 17:19:55 +11:00
Marc Olivier Bergeron
749d8dedc3 MDEV-27066: Fixed scientific notation parsing bug
The bug occurs where the float token containing a dot with an 'e'
notation was dropped from the request completely.

This causes a manner of invalid SQL statements like:

select id 1.e, char 10.e(id 2.e), concat 3.e('a'12356.e,'b'1.e,'c'1.1234e)1.e, 12 1.e*2 1.e, 12 1.e/2 1.e, 12 1.e|2 1.e, 12 1.e^2 1.e, 12 1.e%2 1.e, 12 1.e&2 from test;

To be parsed correctly as if it was:

select id, char(id), concat('a','b','c'), 12*2, 12/2, 12|2, 12^2, 12%2, 12&2 from test.test;

This correct parsing occurs when e is followed by any of:

( ) . , | & % * ^ /
2021-11-24 16:46:01 +11:00
Alexey Bychko
fe065f8d90 MDEV-22522 RPM packages have meaningless summary/description
this patch moves cpack summury and description for optional packages
to the appropriate CMakeLists.txt files
2021-11-23 11:29:24 +07:00
Julius Goryavsky
1c7dd233a7 MDEV-26915 post-fix: deleted unnecessary file 2021-11-23 04:41:14 +01:00