Commit graph

195004 commits

Author SHA1 Message Date
Daniel Bartholomew
0d8b0493ee
bump the VERSION 2023-05-10 08:43:49 -04:00
Oleksandr Byelkin
d821fd7fab Merge branch 'merge-perfschema-5.7' into 10.5 2023-04-28 08:22:17 +02:00
Oleksandr Byelkin
512dbc4527 5.7.42 (only copyright year in all files changed) 2023-04-28 08:09:26 +02:00
Marko Mäkelä
902c622215 Merge 10.4 into 10.5 2023-04-27 09:39:53 +03:00
Yuchen Pei
6fccf8ba05
MDEV-29644 post-merge fixup
Applying b98375f instead of 9b32e4b for version 10.5-10.8.
Applying 5075f4e instead of 9b32e4b for version 10.9+.
2023-04-27 16:14:05 +10:00
Sergei Golubchik
b942f41438 MDEV-30218 update test result
followup for d1a46c68cd
2023-04-26 09:07:10 +02:00
Sergei Golubchik
b3817425d9 MDEV-11356 Option skip-core-file does not work
remove ancient hard-coded treatment of --core-file. This enables
normal my_getopt behavior for the already existing sysvar
2023-04-26 09:07:10 +02:00
Andrei
e22a57da82 MDEV-30620 Trying to lock uninitialized LOCK_parallel_entry
The error was seen by a number of mtr tests being caused
by overdue initialization of rpl_parallel::LOCK_parallel_entry.
Specifically, SHOW-SLAVE-STATUS might find in
rpl_parallel::workers_idle() a gtid domain hash entry
already inserted whose mutex had not done
mysql_mutex_init().

Fixed with swapping the mutex init and the its entry's stack insertion.

Tested with a generous number of `mtr --repeat` of a few of the reported
to fail tests, incl rpl.parallel_backup.
2023-04-25 19:43:04 +03:00
Sergei Petrunia
a72b2c3ffb MDEV-31121: ANALYZE statement produces 0 for all timings in embedded server
Timers require my_timer_init() call.
It was made only in mysqld_main(). Call it also from init_embedded_server().
2023-04-25 13:47:46 +03:00
Marko Mäkelä
50f3b7d164 MDEV-31124 Innodb_data_written miscounts doublewrites
When commit a5a2ef079c
implemented asynchronous doublewrite, the writes via
the doublewrite buffer started to be counted incorrectly,
without multiplying them by innodb_page_size.

srv_export_innodb_status(): Correctly count the
Innodb_data_written.

buf_dblwr_t: Remove submitted(), because it is close to written()
and only Innodb_data_written was interested in it. According to
its name, it should count completed and not submitted writes.

Tested by: Axel Schwenke
2023-04-25 12:17:06 +03:00
Brandon Nesterenko
31f09e36c1 MDEV-31038: Parallel Replication Breaks if XA PREPARE Fails Updating Slave GTID State
If a replica failed to update the GTID slave state when committing
an XA PREPARE, the replica would retry the transaction and get an
out-of-order GTID error. This is because the commit phase of an XA
PREPARE is bifurcated. That is, first, the prepare is handled by the
relevant storage engines. Then second, the GTID slave state is
updated as a separate autocommit transaction. If the second phase
fails, and the transaction is retried, then the same transaction is
attempted to be committed again, resulting in a GTID out-of-order
error.

This patch fixes this error by immediately stopping the slave and
reporting the appropriate error. That is, there was logic to bypass
the error when updating the GTID slave state table if the underlying
error is allowed for retry on a parallel slave. This patch adds a
parameter to disallow the error bypass, thereby forcing the error
state to still happen.

Reviewed By
============
Andrei Elkin <andrei.elkin@mariadb.com>
2023-04-24 16:20:36 -06:00
Brandon Nesterenko
29fb041007 MDEV-30430: Enabling system versioning on tables without primary key breaks replication
When replicating MDL events for a table that uses system versioning
without primary keys, ensure that for data sets with duplicate
records, the updates to these records with duplicates are enacted on
the correct row. That is, there was a bug (reported in MDEV-30430)
such that the function to find the row to update would stop after
finding the first matching record. However, in the absence of
primary keys, the version of the record is needed to compare the row
to ensure we are updating the correct one.

The fix, therefore, updates the record comparison functionality to
use system version columns when there are no primary keys on the
table.

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>
2023-04-24 15:09:40 -06:00
Brandon Nesterenko
4ec3dca34b MDEV-28798: Cosmetic Changes Only
Removed trailing whitespaces
2023-04-24 15:07:15 -06:00
Brandon Nesterenko
d3e7dba329 MDEV-28798: Previously Binlog Encrypted Master Segfaults on Binlog Dump with Using_Gtid=Slave_Pos
Problem:
========
A master can segfault if it can't set up decryption for its binary
log during a binlog dump with Using_Gtid=Slave_Pos. If slave
connects using GTID mode, the master will call into
log.cc::get_gtid_list_event(), which iterate through binlog events
looking for a Gtid_list_log_event. On an encrypted binlog that the
master cannot decrypt, the first event will be a
START_ENCRYPTION_EVENT which will call into the following decryption branch

if (fdle->start_decryption((Start_encryption_log_event*) ev))
  errormsg= ‘Could not set up decryption for binlog.’;

The event iteration however, does not stop in spite of this error.
The master will try to read the next event, but segfault while
trying to decrypt it because decryption failed to initialize.

Solution:
========
Break the event iteration if decryption cannot be set up.

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>
2023-04-24 15:07:15 -06:00
Igor Babaev
5dc9a6b455 MDEV-31102 Crash when pushing condition into view defined as union
This bug could manifest itself at the first execution of prepared statement
created for queries using a materialized view defined as union. A crash
could happen for sure if the query contained a condition pushable into
the view and this condition was over the column defined via a complex string
expression requiring implicit conversion from one charset to another for
some of its sub-expressions. The bug could cause crashes when executing
PS for some other queries whose optimization needed building clones for
such expressions.

This bug was introduced in the patch for MDEV-29988 where the class
Item_direct_ref_to_item was added. The implementations of the virtual
methods get_copy() and build_clone() were invalid for the class and this
could cause crashes after the method build_clone() was called for
expressions containing objects of the Item_direct_ref_to_item type.

Approved by Sergei Golubchik <serg@mariadb.com>
2023-04-24 07:39:19 -07:00
Oleksandr Byelkin
1d74927c58 Merge branch '10.4' into 10.5 2023-04-24 12:43:47 +02:00
Alexander Barkov
d3e394b3b1 A cleanup for MDEV-30968 mariadb-backup does not copy Aria logs if aria_log_dir_path is used
Fixing buildbot failures on mariabackup.aria_log_dir_path_rel.

The problem was that directory_exists() was called with the
relative aria_log_dir_path value, while the current directory
in mariadb-backup is not necessarily equal to datadir when MTR is running.

Fix:

- Moving building the absolute path un level upper:
  from the function copy_back_aria_logs() to the function copy_back().
- Passing the built absolute path to both directory_exists() and
  copy_back_aria_logs() as a parameter.
2023-04-24 10:36:13 +04:00
Igor Babaev
6dc6c22c14 MDEV-31085 Crash when processing multi-update using view with optimizer_trace on
This bug caused server crash when processing a multi-update statement that
used views if optimizer tracing was enabled.
The bug was introduced in the patch for MDEV-30539 that could incorrectly
detect the most top level selects of queries if views were used in them.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2023-04-22 12:32:38 -07:00
Alexander Barkov
9f98a2acd7 MDEV-30968 mariadb-backup does not copy Aria logs if aria_log_dir_path is used
- `mariadb-backup --backup` was fixed to fetch the value of the
   @@aria_log_dir_path server variable and copy aria_log* files
   from @@aria_log_dir_path directory to the backup directory.
   Absolute and relative (to --datadir) paths are supported.

   Before this change aria_log* files were copied to the backup
   only if they were in the default location in @@datadir.

- `mariadb-backup --copy-back` now understands a new my.cnf and command line
   parameter --aria-log-dir-path.

  `mariadb-backup --copy-back` in the main loop in copy_back()
   (when copying back from the backup directory to --datadir)
   was fixed to ignore all aria_log* files.

   A new function copy_back_aria_logs() was added.
   It consists of a separate loop copying back aria_log* files from
   the backup directory to the directory specified in --aria-log-dir-path.
   Absolute and relative (to --datadir) paths are supported.
   If --aria-log-dir-path is not specified,
   aria_log* files are copied to --datadir by default.

- The function is_absolute_path() was fixed to understand MTR style
  paths on Windows with forward slashes, e.g.
   --aria-log-dir-path=D:/Buildbot/amd64-windows/build/mysql-test/var/...
2023-04-21 19:08:35 +04:00
Daniel Black
da1c91fb92 MDEV-30713 field length handling for CONNECT engine
fp->field_length was unsigned and therefore the negative
condition around it.

Backport of cc182aca93 fixes it, however to correct the
consistent use of types pcf->Length needs to be unsigned
too.

At one point pcf->Precision is assigned from pcf->Length so
that's also unsigned.

GetTypeSize is assigned to length and has a length argument.
A -1 default value seemed dangerious to case, so at least 0
should assert if every hit.
2023-04-21 15:47:53 +01:00
Marko Mäkelä
c6e58a8d17 MDEV-30753 fixup: Unsafe buffer page restoration
trx_purge_free_segment(): The buffer-fix only prevents a block from
being freed completely from the buffer pool, but it will not prevent
the block from being evicted. Recheck the page identifier after
acquiring an exclusive page latch. If it has changed, backtrack and
invoke buf_page_get_gen() to look up the page normally.
2023-04-21 16:19:39 +03:00
Oleksandr Byelkin
3d27f6d7f4 Merge branch '10.3' into 10.4 2023-04-21 09:10:58 +02:00
Mikhail Chalov
fc6e8a3d32 Minimize unsafe C functions usage - replace strcat() and strcpy()
Similar to 567b6812 continue to replace use of strcat() and
strcpy() with safer options strncat() and strncpy().

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services
2023-04-20 15:27:20 +01:00
Marko Mäkelä
854e8b189e MDEV-28976 fixup: A better fix
do_shutdown_server(): After sending SIGKILL, invoke wait_until_dead().
Thanks to Sergei Golubchik for pointing out that the previous fix
does not actually work.
2023-04-19 15:53:26 +03:00
Denis Protivensky
210db2935c MDEV-30804 Rollback multi-engine transaction requiring 2PC but committing in one phase
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-04-19 14:08:57 +02:00
Thirunarayanan Balathandayuthapani
660afb1e9c MDEV-30076 ibuf_insert tries to insert the entry for uncommitted index
- Change buffer should not buffer the changes for uncommitted index
2023-04-19 17:11:14 +05:30
Thirunarayanan Balathandayuthapani
2bfd04e314 MDEV-31025 Redundant table alter fails when fixed column
stored externally

row_merge_buf_add(): Has strict assert that fixed length mismatch
shouldn't happen while rebuilding the redundant row format table

btr_index_rec_validate(): Fixed size column can be stored externally.
So sum of inline stored length and external stored length of the
column should be equal to total column length
2023-04-19 17:11:14 +05:30
Thirunarayanan Balathandayuthapani
b2bbc66a41 MDEV-24011 InnoDB: Failing assertion: index_cache->words == NULL in fts0fts.cc line 551
This issue happens when race condition happens when DDL
and fts optimize thread. DDL adds the new index to fts cache.
At the same time, fts optimize thread clears the cache
and reinitialize it. Take cache init lock before reinitializing
the cache. fts_sync_commit() should take dict_sys mutex
to avoid the deadlock with create index.
2023-04-19 17:11:14 +05:30
Marko Mäkelä
d665186477 MDEV-28976: mtr must wait for server to actually die
do_shutdown_server(): Call wait_until_dead() also when we are forcibly
killing the process (timeout=0). We have evidence that killing
the process may take some time and cause mystery failures in
crash recovery tests. For InnoDB, several failures were observed between
commit da094188f6 and
commit 0ee1082bd2
when no advisory file locking was being used by default.
2023-04-19 14:08:53 +03:00
Yuchen Pei
75063d1288
MDEV-30542 Add multilength spider self-reference detection test 2023-04-19 11:18:17 +10:00
Daniele Sciascia
feeeacc4d7 MDEV-30955 Explicit locks released too early in rollback path
Assertion `thd->mdl_context.is_lock_owner()` fires when a client is
disconnected, while transaction and and a table is opened through
`HANDLER` interface.
Reason for the assertion is that when a connection closes, its ongoing
transaction is eventually rolled back in
`Wsrep_client_state::bf_rollback()`. This method also releases explicit
which are expected to survive beyond the transaction lifetime.
This patch also removes calls to `mysql_ull_cleanup()`. User level
locks are not supported in combination with Galera, making these calls
unnecessary.
2023-04-18 13:57:59 +02:00
Sergei Petrunia
be7ef6566f MDEV-30605: Wrong result while using index for group-by
A GROUP BY query which uses "MIN(pk)" and has "pk<>const" in the
WHERE clause would produce wrong result when handled with "Using index
for group-by".  Here "pk" column is the table's primary key.

The problem was introduced by fix for MDEV-23634. It made the range
optimizer to not produce ranges for conditions in form "pk != const".

However, LooseScan code requires that the optimizer is able to
convert the condition on the MIN/MAX column into an equivalent range.
The range is used to locate the row that has the MIN/MAX value.

LooseScan checks this in check_group_min_max_predicates(). This fix
makes the code in that function to take into account that "pk != const"
does not produce a range.
2023-04-18 14:42:47 +03:00
Daniele Sciascia
bc3bfcf943 MDEV-30862 Assertion `mode_ == m_high_priority' failed
CREATE TABLE AS SELECT is not supported in combination with streaming
replication.
2023-04-18 10:02:22 +02:00
Florian Weimer
f575de39af rocksdb: Define _GNU_SOURCE during fallocate CMake probe
The glibc headers declare fallocate only if _GNU_SOURCE is defined.
Without this change, the probe fails with C compilers which do not
support implicit function declarations even if the system does in
fact support the fallocate function.

Upstream rocksdb does not need this because the probe is run with the
C++ compiler, and current g++ versions define _GNU_SOURCE
automatically.
2023-04-17 17:21:26 +10:00
Sergei Petrunia
6c196090c8 Fix compilation on gcc 11.2.0
It is used in the out-of date Ubuntu 21.10 Impish.
2023-04-13 20:13:13 +03:00
Daniel Black
2e1c532bd2 alloca() fix
Corrections from 1e58b8afc0.
* Re-add #pragma alloca for AIX - now in my_alloca.h
2023-04-13 21:47:56 +08:00
Julius Goryavsky
d1a4315f4c MDEV-30402: Encrypted mariabackup SST breaks on distributions with newer socat
This commit adds a new 'no-sni' option to socat which is required to
properly authenticate with newer socat versions (after version 1.7.4+).
This option is needed to disable the automatic use of the SNI feature
(Server Name Indication) since the SST script directly specifies the
commonname if necessary and automatic activation of the SNI feature
is unnecessary in such scenarios.
2023-04-13 07:49:35 +02:00
Igor Babaev
ef4d09948d MDEV-20773 Error from UPDATE when estimating selectivity of a range
This bug could affect multi-update statements as well as single-table
update statements processed as multi-updates when the where condition
contained a range condition over a non-indexed varchar column. The
optimizer calculates selectivity of such range conditions using histograms.
For each range the buckets containing endpoints of the the range are
determined with a procedure that stores the values of the endpoints in the
space of the record buffer where values of the columns are usually stored.
For a range over a varchar column the value of a endpoint may exceed the
size of the buffer and in such case the value is stored with truncation.
This truncations cannot affect the result of the calculation of the range
selectivity as the calculation employes only the beginning of the value
string. However it can trigger generation of an unexpected error on this
truncation if an update statement is processed.
This patch prohibits truncation messages when selectivity of a range
condition is calculated for a non-indexed column.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2023-04-12 08:14:56 -07:00
Alexander Barkov
7bcfa00a6a MDEV-31039 mariadb-backup: remove global variables ds_data and ds_meta
This is a non-functional change.

simplifying the code logic:
- removing global variables ds_data and ds_meta
- passing these variables as parameters to functions instead
- adding helper classes: Datasink_free_list and Backup_datasinks
- moving some function accepting a ds_ctxt parameter
  as methods to ds_ctxt.
2023-04-12 15:14:03 +04:00
Jan Lindström
f83b7ae13d MDEV-26175 : Assertion `! thd->in_sub_stmt' failed in bool trans_rollback_stmt(THD*)
If we are inside stored function or trigger we should not commit
or rollback current statement transaction.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-04-11 09:07:33 +02:00
lilinjie
4daea2f8b6 fix typo
Signed-off-by: lilinjie <1136268146@qq.com>
2023-04-10 12:18:16 +02:00
Alexander Barkov
ed2adc8c6f MDEV-28190 sql_mode makes MDEV-371 virtual column expressions nondeterministic
This problem was fixed earlier by MDEV-27653.
Adding MTR tests only.
2023-04-06 16:17:50 +04:00
Alexander Barkov
54715a1074 MDEV-30072 Wrong ORDER BY for a partitioned prefix key + NOPAD
This problem was earlier fixed by MDEV-30034.
Adding MTR tests only.
2023-04-06 09:57:58 +04:00
Alexander Barkov
fb72dfbf7f MDEV-30415 MDEV-30415 PERIOD false positive overlap wtih utf8mb4_unicode_nopad_ci
The problem was earlier fixed by the patch for MDEV-30034.
Adding MTR tests only.
2023-04-06 09:45:05 +04:00
Sergei Golubchik
79e27a6bf9 MDEV-25887 "Got notification message from PID xxxx, but reception only permitted for main PID yyyy" in systemd during SST
server has systemd support and calls sd_notify() to communicate
the status to systemd.

mariabackup links the whole server in, but it should not notify
systemd, because it's not started or managed by systemd.
2023-04-05 23:41:30 +02:00
Alexander Barkov
62e137d4d7 Merge remote-tracking branch 'origin/10.4' into 10.5 2023-04-05 16:16:19 +04:00
Jan Lindström
06393cd8f8 MDEV-29602 : Galera debug build crashes when the spider plugin is enabled
Spider system tables should be created so that wsrep_on=OFF.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-04-04 20:12:36 +02:00
Alexander Barkov
8f9bb82640 MDEV-30971 Add a new system variable aria_data_home_dir 2023-04-04 16:05:55 +04:00
Jan Lindström
afdf19cf33 MDEV-28641 : Query cache entries not invalidated on slave of a Galera cluster
Query cache should be invalidated if we are not in applier. For some
reason this condition was incorrect starting from 10.5 but it is
correct in 10.4.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-04-04 12:12:16 +02:00
Alexander Barkov
8020b1bd73 MDEV-30034 UNIQUE USING HASH accepts duplicate entries for tricky collations
- Adding a new argument "flag" to MY_COLLATION_HANDLER::strnncollsp_nchars()
  and a flag MY_STRNNCOLLSP_NCHARS_EMULATE_TRIMMED_TRAILING_SPACES.
  The flag defines if strnncollsp_nchars() should emulate trailing spaces
  which were possibly trimmed earlier (e.g. in InnoDB CHAR compression).
  This is important for NOPAD collations.

  For example, with this input:
   - str1= 'a '    (Latin letter a followed by one space)
   - str2= 'a  '   (Latin letter a followed by two spaces)
   - nchars= 3
  if the flag is given, strnncollsp_nchars() will virtually restore
  one trailing space to str1 up to nchars (3) characters and compare two
  strings as equal:
  - str1= 'a  '  (one extra trailing space emulated)
  - str2= 'a  '  (as is)

  If the flag is not given, strnncollsp_nchars() does not add trailing
  virtual spaces, so in case of a NOPAD collation, str1 will be compared
  as less than str2 because it is shorter.

- Field_string::cmp_prefix() now passes the new flag.
  Field_varstring::cmp_prefix() and Field_blob::cmp_prefix() do
  not pass the new flag.

- The branch in cmp_whole_field() in storage/innobase/rem/rem0cmp.cc
  (which handles the CHAR data type) now also passed the new flag.

- Fixing UCA collations to respect the new flag.
  Other collations are possibly also affected, however
  I had no success in making an SQL script demonstrating the problem.
  Other collations will be extended to respect this flags in a separate
  patch later.

- Changing the meaning of the last parameter of Field::cmp_prefix()
  from "number of bytes" (internal length)
  to "number of characters" (user visible length).

  The code calling cmp_prefix() from handler.cc was wrong.
  After this change, the call in handler.cc became correct.

  The code calling cmp_prefix() from key_rec_cmp() in key.cc
  was adjusted according to this change.

- Old strnncollsp_nchar() related tests in unittest/strings/strings-t.c
  now pass the new flag.
  A few new tests also were added, without the flag.
2023-04-04 12:30:50 +04:00