Commit graph

192016 commits

Author SHA1 Message Date
Alexander Barkov
284ac6f2b7 MDEV-27653 long uniques don't work with unicode collations 2023-01-19 20:33:03 +04:00
Oleksandr Byelkin
9924466b3b v5.5.4-stable 2023-01-17 22:46:25 +01:00
Marko Mäkelä
2b3423c462 Merge 10.3 into 10.4 2023-01-17 18:03:58 +02:00
Marko Mäkelä
489b556947 MDEV-30422 Merge new release of InnoDB 5.7.41 to 10.3
MySQL 5.7.41 includes one InnoDB change
mysql/mysql-server@d2d6b2dd00
that seems to be applicable to MariaDB Server 10.3 and 10.4.
Even though commit 5b9ee8d819
seems to have fixed sporadic failures on our CI systems, it is
theoretically possible that another race condition remained.

buf_flush_page_cleaner_coordinator(): In the final loop,
wait also for buf_get_n_pending_read_ios() to reach 0.
In this way, if a secondary index leaf page was read into the
buffer pool and ibuf_merge_or_delete_for_page() modified that
page or some change buffer pages, the flush loop would execute
until the buffer pool really is in a clean state.

This potential data corruption bug does not affect MariaDB Server 10.5
or later, thanks to commit b42294bc64
which removed change buffer merges that are not explicitly requested.
2023-01-17 17:52:16 +02:00
Oleksandr Byelkin
834650c7cf New CC 3.1 2023-01-16 14:59:59 +01:00
Daniel Black
3e8b6a79b7 Update sponsors 2023-01-16 10:23:22 +11:00
sjaakola
a44d896f98 10.4-MDEV-29684 Fixes for cluster wide write conflict resolving
If two high priority threads have lock conflict, we look at the
order of these transactions and honor the earlier transaction.
for_locking parameter in lock_rec_has_to_wait() has become
obsolete and it is now removed from the code .

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2023-01-14 07:50:04 +02:00
sjaakola
0ff7f33c7b 10.4-MDEV-29684 Fixes for cluster wide write conflict resolving
The rather recent thd_need_ordering_with() function does not take
high priority transactions' order in consideration. Chaged this
funtion to compare also transaction seqnos and favor earlier transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2023-01-13 13:11:03 +02:00
sjaakola
68cfcf9cb6 MDEV-29512 deadlock between commit monitor and THD::LOCK_thd_data mutex
This commit contains only a mtr test for reproducing the issue in MDEV-29512
The actual fix will be pushed in wsrep-lib repository

The hanging in MDEV-29512 happens when binlog purging is attempted, and there is
one local BF aborted transaction waiting for commit monitor.

The test will launch two node cluster and enable binlogging with expire log days,
to force binlog purging to happen.
A local transaction is executed so that will become BF abort victim, and has advanced
to replication stage waiting for commit monitor for final cleanup (to mark position in innodb)
after that, applier is released to complete the BF abort and due to binlog configuration,
starting the binlog purging. This is where the hanging would occur, if code is buggy

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2023-01-13 13:11:03 +02:00
sjaakola
cd97523dcf MDEV-30317 Transaction savepoint may cause failure in galera replaying
Created mtr test for reproducing the crash

Developed actual fix for the issue.
Setting THD::system_thread_info.rpl_sql_info for replayer thread,
same way as it is handled for appliers.

Recorded test result, with the fix

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2023-01-13 13:11:03 +02:00
sjaakola
66c05326d2 MDEV-29684 Fixes for cluster wide write conflict resolving
Cluster conflict victim's THD is marked with wsrep_aborter.
THD::wsrep_aorter holds the thread ID of the hight priority tread,
which is currently carrying out BF aborting for this victim.

However, the BF abort operation is not always successful,
and in such case the wsrep_aborter mark should be removed.
In the old code, this wsrep_aborter resetting did not happen,
and this could lead to a situation where the sticky wsrep_aborter
mark prevents any further attempt to BF abort this transaction.

This commit fixes this issue, and resets wsrep_aborter after
unsuccesful BF abort attempt.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2023-01-13 13:11:03 +02:00
Marko Mäkelä
71e8e4934d Merge 10.3 into 10.4 2023-01-13 09:28:25 +02:00
Nikita Malyavin
7a98d232e4 MDEV-30378 Versioned REPLACE succeeds with ON DELETE RESTRICT constraint
node->is_delete was incorrectly set to NO_DELETE for a set of operations.

In general we shouldn't rely on sql_command and look for more abstract ways
to control the behavior.

trg_event_map seems to be a suitable way. To mind replica nodes, it is ORed
with slave_fk_event_map, which stores trg_event_map when replica has
triggers disabled.
2023-01-12 21:51:48 +03:00
Weijun-H
12618cfb28 MDEV-19160 json_pretty() alias for json_detailed() 2023-01-12 17:51:42 +00:00
Rucha Deodhar
17920291a6 fixup for MDEV-19160 2023-01-12 20:11:19 +05:30
lilinjie
eb145e5ad7 fix typos
Signed-off-by: lilinjie <lilinjie@uniontech.com>
2023-01-12 14:02:20 +11:00
Monty
c08dba7b28 Fixed failing test main.func_json 2023-01-12 02:48:33 +02:00
Monty
f3d8a546e7 MDEV-30345 DML does not find rows it is supposed to
This only happens with 'timestamp_column IN (select ...)

The reason was a missing assignment in Item_cache_timestamp::cache_value()
2023-01-11 20:15:26 +02:00
Brandon Nesterenko
b194c83b7b MDEV-25277: mysqlbinlog --verbose cannot read row events with compressed columns: Don't know how to handle column type: 140
Problem:
=======
Mysqlbinlog cannot show the type of a compressed
column when two levels of verbosity is provided.

Solution:
========
Extend the log event printing logic to handle and
tag compressed types.

Behavioral Changes:
==================
  Old: When mysqlbinlog is called in verbose mode and
the database uses compressed columns, an error is
returned to the user.

  New: The output will append “ COMPRESSED” on the
type of compressed columns

Reviewed By
===========
Andrei Elkin <andrei.elkin@mariadb.com>
2023-01-11 10:37:49 -07:00
Julius Goryavsky
53c4be7bc0 MDEV-30220: rsync SST completely ignores aria-log-dir-path
This commit adds support for the --aria-log-dir-path
option on the command line and for the aria-log-dir-path
option in the configuration file to the SST scripts, since
before this change these parameters were completely ignored
during SST - SST scripts assumed that aria logs files are
always located in the same directory as logs for innodb.

Tests for this change will be added as a separate commit,
along with tests for MDEV-30157 and MDEV-28669.
2023-01-11 15:01:00 +01:00
Julius Goryavsky
b84f3fa769 MDEV-30157: Galera SST doesn't properly handle undo* files from innodb
This fix adds separate handling for "undo*" files that contain undo
logs as part of innodb files and adds a filter for undo* to the main
filter used when initially transferring files with rsync.
2023-01-11 15:01:00 +01:00
Julius Goryavsky
e4a4aad7cf pre-MDEV-30157 & pre-MDEV-28669: fixes before the main corrections
This commit adds even more correct handling of parameters
with paths when they contain leading or trailing spaces and/or
slashes. Also it fixes problems that occur when the user specified
explicit paths to additional directories, but these paths match
the specified path of the data directory - in this case, additional
subdirectories should be treated (in relation to the data directory)
in the same way as if these paths were not specified or as if they
are implicitly specified as "." or "./". But prior to this fix,
existing code treated any values as if they were completely
separate directories, whether or not they actually point to the
same location to which datadir points to - and this sometimes
resulted in incorrect file transfers.

This fix does not contain separate tests, as tests will be
part of the main commit(s). This fix has been made as a separate
commit to facilitate review for major substantive fixes related
to MDEV-30157 and MDEV-28669.
2023-01-11 15:01:00 +01:00
Sergei Petrunia
b928c849d2 MDEV-28602 Wrong result with outer join, merged derived table and view
(Variant 3, initial variant was by Rex Jonston)

A LEFT JOIN with a constant as a column of the inner table produced wrong
query result if the optimizer had to write the inner table column into a
temp table. Query pattern:

  SELECT ...
  FROM (SELECT /*non-mergeable select*/
        FROM t1 LEFT JOIN (SELECT 'Y' as Val) t2 ON ...) as tbl

Fixed this by adding Item_direct_view_ref::save_in_field() which follows
the pattern of Item_direct_view_ref's save_org_in_field(),
save_in_result_field() and val_XXX() functions:
* call check_null_ref() and handle NULL value
* if we didn't get a NULL-complemented row, call Item_direct_ref's function.
2023-01-11 16:19:16 +03:00
Marko Mäkelä
b218dfead2 Remove an unused parameter
lock_rec_has_to_wait(): Remove the unused parameter for_locking
that had been originally added
in commit df4dd593f2
2023-01-11 08:37:27 +02:00
Sergei Golubchik
fdcfc25127 Merge branch '10.3' into 10.4 2023-01-10 21:04:17 +01:00
Daniel Black
56948ee54c clang15 warnings - unused vars and old prototypes
clang15 finally errors on old prototype definations.

Its also a lot fussier about variables that aren't used
as is the case a number of time with loop counters that
aren't examined.

RocksDB was complaining that its get_range function was
declared without the array length in ha_rocksdb.h. While
a constant is used rather than trying to import the
Rdb_key_def::INDEX_NUMBER_SIZE header (was causing a lot of
errors on the defination of other orders). If the constant
does change can be assured that the same compile warnings will
tell us of the error.

The ha_rocksdb::index_read_map_impl DBUG_EXECUTE_IF was similar
to the existing endless functions used in replication tests.
Its rather moot point as the rocksdb.force_shutdown test that
uses myrocks_busy_loop_on_row_read is currently disabled.
2023-01-10 17:10:43 +00:00
Sergei Golubchik
6cb84346e1 MDEV-17869 AddressSanitizer: use-after-poison in Item_change_list::rollback_item_tree_changes
it's incorrect to use change_item_tree() to replace arguments
of top-level AND/OR, because they (arguments) are stored in a List,
so a pointer to an argument is in the list_node, and individual
list_node's of top-level AND/OR can be deleted in Item_cond::build_equal_items().
In that case rollback_item_tree_changes() will modify the deleted object.

Luckily, it's not needed to use change_item_tree() for top-level
AND/OR, because the whole top-level item is copied and preserved
in prep_where and prep_on, and restored from there.

So, just don't.

Additionally to the test case in the commit it fixes
* ASAN failure of main.opt_tvc --ps
* ASAN failure of main.having_cond_pushdown --ps
2023-01-09 18:06:06 +01:00
Sergei Golubchik
df82d68421 cleanup
remove casts. uchar can store 1/0 as good as bool.
2023-01-09 18:06:06 +01:00
Sergei Golubchik
1e6ad0ce13 don't set default value in temp table if NO_DEFAULT_VALUE_FLAG
when an internal temporary table field is created from a real field,
a new temp field should only copy a default from the source field
when the latter has it
2023-01-09 18:06:06 +01:00
Sergei Golubchik
32f09df2b8 MDEV-29890 Update with inner join false row count result
when creating a temp table field from an actual table field,
these two fields are supposed to be mostly identical
(except for BIT field storage), in particular, temp field should
have the same default as the orig field, even if the sql_mode has
been changed meanwhile (e.g. to include NO_ZERO_DATE)
2023-01-09 18:06:06 +01:00
Sergei Golubchik
610cea3dda cleanup
Helper class to swicth to relaxed checks during field copy.
Temporarily.
2023-01-09 18:06:06 +01:00
Sergei Golubchik
ad27e95d54 disable hanging galera test 2023-01-09 18:06:05 +01:00
Sergei Golubchik
3d95737e55 galera.galera_wsrep_new_cluster: fix bad merge 2023-01-09 18:06:05 +01:00
Daniel Black
d7f447915c MDEV-30342 Wrong "Truncated incorrect DECIMAL value" warning/error
regression from MDEV-29540 / 8c38939369.

INSERT SELECT errors needed to be unconditionally ignored.

As this touches the CREATE .. SELECT functionality, show
the equalivent test there.
2023-01-09 19:20:31 +11:00
Daniel Black
e64e6768e0 MDEV-17093: SOURCE_REVISION in log (postfix - not in help)
Don't display the source revision in the mysqld --help output.
2023-01-08 09:53:58 +11:00
Igor Babaev
b21832ef15 MDEV-27624 Wrong result for nested left join using not_exists optimization
This bug affected queries with nested left joins having the same last inner
table such that not_exists optimization could be applied to the most inner
outer join when optimizer chose to use join buffers. The bug could lead to
producing wrong a result set.
If the WHERE condition a query contains a conjunctive IS NULL predicate
over a non-nullable column of an inner table of a not nested outer join
then not_exists optimization can be applied to tho the outer join. With
this optimization when looking for matches for a certain record from the
outer table of the join the records of the inner table can be ignored
right after the first match satisfying the ON condition is found.
In the case of nested outer joins having the same last inner table this
optimization still can be applied but only if all ON conditions of the
embedding outer joins are satisfied. Such check was missing in the code
that tried to apply not_exists optimization when join buffers were used
for outer join operations.
This problem has been already fixed in the patch for bug MDEV-7992. Yet
there it was resolved only for the cases when join buffers were not used
for outer joins.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2023-01-06 14:20:42 -08:00
Haidong Ji
af0ff8b455 MDEV-17093: SOURCE_REVISION in log and handle_fatal_signal
MariaDB MDEV-12583 added `SOURCE_REVISION` variable that exposes the
SHA1 of source code commit that the current running engine was built
from. This info is useful for troubleshooting and debugging.

This commit does the following:
- addes the `SOURCE_REVISION` value into engine error log.
- when a crash triggers handle_fatal_signal, the `SOURCE_REVISION` will
  be included in crash report.
- resolves MDEV-20344: startup messages belong in stderr/error-log not
  stdout

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license.  I am contributing on behalf of my employer Amazon Web
Services, Inc.
2023-01-06 19:37:41 +11:00
Weijun-H
111a752b96 MDEV-19160 JSON_DETAILED output unnecessarily verbose 2023-01-05 13:12:24 +00:00
lilinjie
758c24dae2 fix typos
Signed-off-by: lilinjie <lilinjie@uniontech.com>
2023-01-05 07:57:51 +11:00
Marko Mäkelä
fb0808c450 Merge 10.3 into 10.4 2023-01-03 16:10:02 +02:00
Andrew Hutchings
e51a1d6fc0 MDEV-30329: mariadb-service-convert resets systemd service to default User=root
If mariadb-service-convert is run and the user variable is unset then
this sets `User=` in `[Service]`, which then tries to run mariadb as
root, which in-turn fails. This only happens when mysqld_safe is missing
which is all the time now. So don't set `User=` if there is no user variable.

Reviewer: Sergei Golubchik <serg@mariadb.org> (in PR )
2023-01-03 11:45:24 +11:00
Sergei Golubchik
8760f6907c MDEV-30102 file missing in development libraries
move mariadb_capi_rename.h out of private server headers,
because it's included by mysql.h which is not private.
2023-01-02 00:04:03 +01:00
Sergei Golubchik
21223c0461 MDEV-29988 group by fix 2023-01-02 00:04:03 +01:00
Sergei Golubchik
ca23558a05 --skip-name-resolve=0 didn't work
custom code in `case OPT_SKIP_RESOLVE`
was overriding the correct value from handle_options().
2023-01-02 00:04:03 +01:00
Sergei Golubchik
908c48a34d fixes for json.json_table and main.func_json in --ps 2023-01-02 00:04:03 +01:00
Sergei Golubchik
d78ac04ee6 ignore changes in submodules when committing everything
covers `git commit -a` and `git citool` (`git add`, `git status`, etc).
they can still be added explicitly
2023-01-02 00:04:03 +01:00
Oleksandr Byelkin
60f646e2f3 MDEV-29988: (spider fix) Major performance regression with 10.6.11
Make Item_direct_ref_to_item transparent for Spider
2023-01-02 00:04:03 +01:00
Sergei Golubchik
eba099184e MDEV-30151 parse error 1=2 not between/in
the parser couldn't parse `1=2 not between 3 and 5`
after `2` it expected only NOT2_SYM, but not NOT_SYM
(visible from the sql_yacc.output file), which resulted in
Syntax error ... near 'not between 3 and 4'

The parser was confused by a rather low NOT_SYM precedence and
%prec BETWEEN_SYM didn't resolve this confusion.

As a fix, let's remove any %precedence from NOT_SYM and
specify %prec explicitly in the only place where it matters for NOT_SYM.

In other places, such as for NOT BETWEEN, NOT_SYM won't have a
precedence, so bison won't be confused about it.
2023-01-02 00:04:03 +01:00
Dmitry Shulga
37a316c01d MDEV-29988: Major performance regression with 10.6.11
The idea is to put Item_direct_ref_to_item as a transparent and
permanent wrapper before a string which require conversion.
So that Item_direct_ref_to_item would be the only place where
the pointer to the string item is stored, this pointer can be changed
and restored during PS execution as needed. And if any permanent
(subquery) optimization would need a pointer to the item,
it'll use a pointer to the Item_direct_ref_to_item - which is
a permanent item and won't go away.
2023-01-02 00:04:03 +01:00
Sergei Golubchik
f8adc47b69 MDEV-19071 Wrong results when using STDDEV_SAMP() and view 2023-01-02 00:04:03 +01:00