Commit graph

192362 commits

Author SHA1 Message Date
Marko Mäkelä
12a5fb4b36 MDEV-31641 innochecksum dies with Floating point exception
print_summary(): Skip index_ids for which index.pages is 0.
Tablespaces may contain some freed pages that used to refer to indexes
or tables that were dropped.
2023-07-10 13:46:34 +03:00
Lawrin Novitsky
02cd3675c4 MDEV-31064 Changes in a SP are not immediately seen in I_S.parameters
If procedure is changed in one connection, and other procedure has
already called the initial version of the procedure, the query to
INFORMATION_SCHEMA.PARAMETERS would use obsolete information from sp
cache for that connection. That happens because cache invalidating
method only increments cache version, and does not flush (all) the
cache(s), and changing of a procedure only invalidates cache, and
removes the procedure's cache entry from local thread cache only.

The fix adds the check if sp info obtained from the cache for forming of
results for the query to I_S, is not obsoleted, and does not use it, if
it is.

The test has been added to main.information_schema. It changes the SP in
one connection, and ensures, that the change is seen in the query to the
I_S.PARAMETERS in other connection, that already has called the
procedure before the change.
2023-07-07 15:36:46 +02:00
Yury Chaikou
8fb863e6a4 MDEV-24712 get_partition_set is never executed in ha_partition::multi_range_key_create_key due to bitwise & with 0 constant
use == to compare enum (despite the name it is not a bit flag)
2023-07-07 14:27:17 +02:00
Oleg Smirnov
94a8921e9d MDEV-29284 ANALYZE doesn't work with pushed derived tables
There was no actual execution of the SQL of a pushed derived table,
which caused "r_rows" to be always displayed as 0 and "r_total_time_ms"
to show inaccurate numbers.
This commit makes a derived table SQL to be executed by the storage
engine, so the server is able to calculate the number of rows returned
and measure the execution time more accurately
2023-07-07 15:15:24 +07:00
Vlad Lesin
1bfd3cc457 MDEV-10962 Deadlock with 3 concurrent DELETEs by unique key
PROBLEM:
A deadlock was possible when a transaction tried to "upgrade" an already
held Record Lock to Next Key Lock.

SOLUTION:
This patch is based on observations that:
(1) a Next Key Lock is equivalent to Record Lock combined with Gap Lock
(2) a GAP Lock never has to wait for any other lock
In case we request a Next Key Lock, we check if we already own a Record
Lock of equal or stronger mode, and if so, then we change the requested
lock type to GAP Lock, which we either already have, or can be granted
immediately, as GAP locks don't conflict with any other lock types.
(We don't consider Insert Intention Locks a Gap Lock in above statements).

The reason of why we don't upgrage Record Lock to Next Key Lock is the
following.

Imagine a transaction which does something like this:

for each row {
    request lock in LOCK_X|LOCK_REC_NOT_GAP mode
    request lock in LOCK_S mode
}

If we upgraded lock from Record Lock to Next Key lock, there would be
created only two lock_t structs for each page, one for
LOCK_X|LOCK_REC_NOT_GAP mode and one for LOCK_S mode, and then used
their bitmaps to mark all records from the same page.

The situation would look like this:

request lock in LOCK_X|LOCK_REC_NOT_GAP mode on row 1:
// -> creates new lock_t for LOCK_X|LOCK_REC_NOT_GAP mode and sets bit for
// 1
request lock in LOCK_S mode on row 1:
// -> notices that we already have LOCK_X|LOCK_REC_NOT_GAP on the row 1,
// so it upgrades it to X
request lock in LOCK_X|LOCK_REC_NOT_GAP mode on row 2:
// -> creates a new lock_t for LOCK_X|LOCK_REC_NOT_GAP mode (because we
// don't have any after we've upgraded!) and sets bit for 2
request lock in LOCK_S mode on row 2:
// -> notices that we already have LOCK_X|LOCK_REC_NOT_GAP on the row 2,
// so it upgrades it to X
    ...etc...etc..

Each iteration of the loop creates a new lock_t struct, and in the end we
have a lot (one for each record!) of LOCK_X locks, each with single bit
set in the bitmap. Soon we run out of space for lock_t structs.

If we create LOCK_GAP instead of lock upgrading, the above scenario works
like the following:

// -> creates new lock_t for LOCK_X|LOCK_REC_NOT_GAP mode and sets bit for
// 1
request lock in LOCK_S mode on row 1:
// -> notices that we already have LOCK_X|LOCK_REC_NOT_GAP on the row 1,
// so it creates LOCK_S|LOCK_GAP only and sets bit for 1
request lock in LOCK_X|LOCK_REC_NOT_GAP mode on row 2:
// -> reuses the lock_t for LOCK_X|LOCK_REC_NOT_GAP by setting bit for 2
request lock in LOCK_S mode on row 2:
// -> notices that we already have LOCK_X|LOCK_REC_NOT_GAP on the row 2,
// so it reuses LOCK_S|LOCK_GAP setting bit for 2

In the end we have just two locks per page, one for each mode:
LOCK_X|LOCK_REC_NOT_GAP and LOCK_S|LOCK_GAP.
Another benefit of this solution is that it avoids not-entirely
const-correct, (and otherwise looking risky) "upgrading".

The fix was ported from
mysql/mysql-server@bfba840dfa
mysql/mysql-server@75cefdb1f7

Reviewed by: Marko Mäkelä
2023-07-06 15:06:10 +03:00
Alexander Barkov
19cdddf17d A cleanup for MDEV-30932 UBSAN: negation of -X cannot be represented in type ..
"mtr --view-protocol func_math" failed because of a too long
column names imlicitly generated for the underlying expressions.

With --view-protocol they were replaced to "Name_exp_1".

Adding column aliases for these expressions.
2023-07-06 13:49:06 +04:00
Rucha Deodhar
23e252aef2 MDEV-23187 misses resetting collation connection
MDEV-23187 misses resetting collation connection causing test failures for
10.5+ bugs.
2023-07-05 16:35:01 +05:30
Kristian Nielsen
9856bb4245 MDEV-31602: Race on rpl_global_gtid_slave_state when starting IO thread
Fix that rpl_slave_state::load() was calling rpl_slave_state::update() without
holding LOCK_slave_state.

Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2023-07-04 22:18:31 +02:00
Yuchen Pei
922db0642b
MDEV-31421 Fix spider test cleanup
This fixes mdev_26541.test, and the new clean_up_spider.inc will be
useful for other tests where part of deinit_spider does not apply,
e.g. those testing spider initialisation only.
2023-07-04 10:39:25 +10:00
Vicentiu Ciorbaru
ea386c9d06 Fix use of uninitialized variable
The original code generated a warning in gcc 13.1
2023-07-03 15:46:58 +02:00
Sergei Golubchik
5c81c50f10 MDEV-31214 Recursive CTE execution is interrupted without errors or warnings 2023-07-03 15:46:24 +02:00
Sergei Golubchik
22e5a5ff6e generalize ER_QUERY_EXCEEDED_ROWS_EXAMINED_LIMIT
make it "query reached <some limit> result may be incomplete"
2023-07-03 15:46:24 +02:00
Sergei Golubchik
d458136e7d cleanup: ER_QUERY_TIMEOUT -> ER_UNUSED_1
also make sure all unused error messages are "You should never see it"
and have no translations
2023-07-03 15:46:24 +02:00
Marko Mäkelä
b8088487e4 MDEV-19216 Assertion ...SYS_FOREIGN failed in btr_node_ptr_max_size
btr_node_ptr_max_size(): Handle BINARY(0) and VARBINARY(0)
as special cases, similar to CHAR(0) and VARCHAR(0).
2023-07-03 16:09:18 +03:00
Marko Mäkelä
0105220e3b Remove tests that duplicate innodb.max_record_size 2023-07-03 16:06:10 +03:00
Anel Husakovic
77a229cd2d MDEV-31358: Update description for MariaDB debian/rpm packages
Reviewer: <daniel@mariadb.org>
2023-07-03 17:45:58 +10:00
Sergei Golubchik
e146940ab3 MDEV-31480 RPM packages fail to install because they require /bin/sh for %pretrans
workaround cmake bug #25044

https://gitlab.kitware.com/cmake/cmake/-/issues/25044
2023-06-30 01:28:29 +02:00
Alexander Barkov
67657a01bf MDEV-30932 UBSAN: negation of -X cannot be represented in type ..
'long long int'; cast to an unsigned type to negate this value ..
  to itself in Item_func_mul::int_op and Item_func_round::int_op

Problems:

  The code in multiple places in the following methods:
    - Item_func_mul::int_op()
    - longlong Item_func_int_div::val_int()
    - Item_func_mod::int_op()
    - Item_func_round::int_op()

  did not properly check for corner values LONGLONG_MIN
  and (LONGLONG_MAX+1) before doing negation.
  This cuased UBSAN to complain about undefined behaviour.

Fix summary:

  - Adding helper classes ULonglong, ULonglong_null, ULonglong_hybrid
    (in addition to their signed couterparts in sql/sql_type_int.h).

  - Moving the code performing multiplication of ulonglong numbers
    from Item_func_mul::int_op() to ULonglong_hybrid::ullmul().

  - Moving the code responsible for extracting absolute values
    from negative numbers to Longlong::abs().
    It makes sure to perform negation without undefinite behavior:
    LONGLONG_MIN is handled in a special way.

  - Moving negation related code to ULonglong::operator-().
    It makes sure to perform negation without undefinite behavior:
    (LONGLONG_MAX + 1) is handled in a special way.

  - Moving signed<=>unsigned conversion code to
    Longlong_hybrid::val_int() and ULonglong_hybrid::val_int().

  - Reusing old and new sql_type_int.h classes in multiple
    places in Item_func_xxx::int_op().

Fix details (explain how sql_type_int.h classes are reused):

  - Instead of straight negation of negative "longlong" arguments
    *before* performing unsigned multiplication,
    Item_func_mul::int_op() now calls ULonglong_null::ullmul()
    using Longlong_hybrid_null::abs() to pass arguments.
    This fixes undefined behavior N1.

  - Instead of straight negation of "ulonglong" result
    *after* performing unsigned multiplication,
    Item_func_mul::int_op() now calls ULonglong_hybrid::val_int(),
    which recursively calls ULonglong::operator-().
    This fixes undefined behavior N2.

  - Removing duplicate negating code from Item_func_mod::int_op().
    Using ULonglong_hybrid::val_int() instead.
    This fixes undefinite behavior N3.

  - Removing literal "longlong" negation from Item_func_round::int_op().
    Using Longlong::abs() instead, which correctly handler LONGLONG_MIN.
    This fixes undefinite behavior N4.

  - Removing the duplicate (negation related) code from
    Item_func_int_div::val_int(). Reusing class ULonglong_hybrid.
    There were no undefinite behavior in here.
    However, this change allowed to reveal a bug in
    "-9223372036854775808 DIV 1".
    The removed negation code appeared to be incorrect when
    negating +9223372036854775808. It returned the "out of range" error.
    ULonglong_hybrid::operator-() now handles all values correctly
    and returns +9223372036854775808 as a negation for -9223372036854775808.

    Re-recording wrong results for
      SELECT -9223372036854775808 DIV  1;
    Now instead of "out of range", it returns -9223372036854775808,
    which is the smallest possible value for the expression data type
    (signed) BIGINT.

  - Removing "no UBSAN" branch from Item_func_splus::int_opt()
    and Item_func_minus::int_opt(), as it made UBSAN happy but
    in RelWithDebInfo some MTR tests started to fail.
2023-06-29 11:50:17 +04:00
Yuchen Pei
428c7964a2
MDEV-30370 [fixup] Spider: mdev_30370.test needs wsrep to run. 2023-06-29 11:22:13 +10:00
Yuchen Pei
ea4b8d4ce9
MDEV-31101 Spider: temporarily disable mdev_29904.test
Will re-enable once MDEV-31101 is no longer blocked by MDEV-22979,
as the patch for the latter might fix the former.
2023-06-28 14:58:24 +10:00
Sergei Golubchik
d214628af4 mtr: fix the help text for debuggers 2023-06-27 17:25:20 +02:00
Thirunarayanan Balathandayuthapani
5f09b53bdb MDEV-31086 MODIFY COLUMN can break FK constraints, and lead to unrestorable dumps
- When foreign_key_check is disabled, allowing to modify the
column which is part of foreign key constraint can lead to
refusal of TRUNCATE TABLE, OPTIMIZE TABLE later. So it make
sense to block the column modify operation when foreign key
is involved irrespective of foreign_key_check variable.

Correct way to modify the charset of the column when fk is involved:

SET foreign_key_checks=OFF;
ALTER TABLE child DROP FOREIGN KEY fk, MODIFY m VARCHAR(200) CHARSET utf8mb4;
ALTER TABLE parent MODIFY m VARCHAR(200) CHARSET utf8mb4;
ALTER TABLE child ADD CONSTRAINT FOREIGN KEY (m) REFERENCES PARENT(m);
SET foreign_key_checks=ON;

fk_check_column_changes(): Remove the FOREIGN_KEY_CHECKS while
checking the column change for foreign key constraint. This
is the partial revert of commit 5f1f2fc0e4
and it changes the behaviour of copy alter algorithm

ha_innobase::prepare_inplace_alter_table(): Find the modified
column and check whether it is part of existing and newly
added foreign key constraint.
2023-06-27 16:58:22 +05:30
Yuchen Pei
423c28f0aa
MDEV-29447 MDEV-26285 MDEV-31338 Refactor spider_db_mbase_util::open_item_func
spider_db_mbase_util::open_item_func() is a monster function.
It is difficult to maintain while it is expected that we need to
modify it when a new SQL function or a new func_type is added.

We split the function into two distinct functions: one handles the
case of str != NULL and the other handles the case of str == NULL.

This refactoring was done in a conservative way because we do not
have comprehensive tests on the function.

It also fixes MDEV-29447 and MDEV-31338 where field items that are
arguments of a func item may be used before created / initialised.

Note this commit is adapted from a patch by Nayuta for MDEV-26285.
2023-06-27 11:15:07 +10:00
Marko Mäkelä
b37357eb46 Fix GCC 13 -Wmaybe-uninitialized 2023-06-26 11:03:15 +03:00
Sergei Golubchik
9c0e91a27c Adjust OpenSSL context sizes for CiscoSSL
also, add static
2023-06-22 15:26:23 +02:00
Sergei Golubchik
1f72450260 Revert "MDEV-23925: Fixed warnings generated during compilation of mysys_ssl/openssl.c on MacOS"
This reverts commit a1b6691f93.

because #ifdef checks the symbol defined in ssl_compat.h,
so #include <ssl_compat.h> cannot be inside #ifdef
2023-06-22 15:25:34 +02:00
Sergei Golubchik
d32fc5b8e0 MDEV-31461 mariadb SIGSEGV when built with -DCLIENT_PLUGIN_DIALOG=STATIC 2023-06-19 12:12:21 +02:00
Sergei Petrunia
f5dceafd0b MDEV-30964: MAX_SEL_ARG memory exhaustion is not visible in the optimizer trace
Add printing
2023-06-08 14:02:34 +03:00
Sergei Petrunia
2165c30486 Fix testcase for MDEV-31240 to work with --view-protocol. 2023-06-08 11:35:21 +03:00
Oleksandr Byelkin
78b1831c9f Merge branch '10.4' into 10.4.30 2023-06-07 15:08:29 +02:00
Daniel Bartholomew
78a1f3ce81
bump the VERSION 2023-06-07 08:09:02 -04:00
Yuchen Pei
7e17a88e75
MDEV-30435 MDEV-30981 Fix ubsan errors w.r.t. memcpy in spd_trx.cc
Extract the indexed string memcopy pattern in spd_trx.cc to a static
inline function.

Also updated the ubsan check in mdev_26541.test (h/t roel).
2023-06-07 14:07:31 +10:00
Brandon Nesterenko
8ed88e3455 Revert "MDEV-13915: STOP SLAVE takes very long time on a busy system"
This reverts commit 0a99d457b3
because it should go into only 10.5+
2023-06-06 08:11:38 -06:00
Sergei Golubchik
677d6f0f23 MDEV-31183 binlog_encryption.encrypted_master_switch_to_unencrypted_gtid fails in BB with UBSAN runtime error: downcast of address
sql/log.cc:11101:56: runtime error: downcast of address 0x7f9dc801e9c8 which does not point to an object of type 'Gtid_list_log_event'
sql/sql_repl.cc:1429:12: runtime error: member call on address 0x7f1ca401ea48 which does not point to an object of type 'Gtid_list_log_event'
2023-06-06 13:22:39 +02:00
Brandon Nesterenko
0a99d457b3 MDEV-13915: STOP SLAVE takes very long time on a busy system
The problem is that a parallel replica would not immediately stop
running/queued transactions when issued STOP SLAVE. That is, it
allowed the current group of transactions to run, and sometimes the
transactions which belong to the next group could be started and run
through commit after STOP SLAVE was issued too, if the last group
had started committing. This would lead to long periods to wait for
all waiting transactions to finish.

This patch updates a parallel replica to try and abort immediately
and roll-back any ongoing transactions. The exception to this is any
transactions which are non-transactional (e.g. those modifying
sequences or non-transactional tables), and any prior transactions,
will be run to completion.

The specifics are as follows:

 1. A new stage was added to SHOW PROCESSLIST output for the SQL
Thread when it is waiting for a replica thread to either rollback or
finish its transaction before stopping. This stage presents as
“Waiting for worker thread to stop”

 2. Worker threads which error or are killed no longer perform GCO
cleanup if there is a concurrently running prior transaction. This
is because a worker thread scheduled to run in a future GCO could be
killed and incorrectly perform cleanup of the active GCO.

 3. Refined cases when the FL_TRANSACTIONAL flag is added to GTID
binlog events to disallow adding it to transactions which modify
both transactional and non-transactional engines when the binlogging
configuration allow the modifications to exist in the same event,
i.e. when using binlog_direct_non_trans_update == 0 and
binlog_format == statement.

 4. A few existing MTR tests relied on the completion of certain
transactions after issuing STOP SLAVE, and were re-recorded
(potentially with added synchronizations) under the new rollback
behavior.

Reviewed By
===========
Andrei Elkin <andrei.elkin@mariadb.com>
2023-06-05 10:03:06 -06:00
Sergei Petrunia
928012a27a MDEV-31403: Server crashes in st_join_table::choose_best_splitting
The code in choose_best_splitting() assumed that the join prefix is
in join->positions[].

This is not necessarily the case. This function might be called when
the join prefix is in join->best_positions[], too.
Follow the approach from best_access_path(), which calls this function:
pass the current join prefix as an argument,
"const POSITION *join_positions" and use that.
2023-06-05 18:24:39 +03:00
heyingquan0030
8de6740a2f MDEV-31205 Typo: complatible > compatible 2023-06-05 09:16:44 +10:00
Sergei Golubchik
eb472f77e3 Revert "MDEV-30473 : Do not allow GET_LOCK() / RELEASE_LOCK() in cluster"
This reverts commit b05218e08f.
2023-06-03 10:39:34 +02:00
Sergei Golubchik
0fd54c9892 Revert "MDEV-30473 : Do not allow GET_LOCK() / RELEASE_LOCK() in cluster"
This reverts commit 844ddb1109.

This fixes MDEV-30967, MDEV-31325, MDEV-31388
2023-06-03 10:39:34 +02:00
Igor Babaev
8f3bf593d2 MDEV-31240 Crash with condition pushable into derived and containing outer reference
This bug could affect queries containing a subquery over splittable derived
tables and having an outer references in its WHERE clause. If such subquery
contained an equality condition whose left part was a reference to a column
of the derived table and the right part referred only to outer columns
then the server crashed in the function st_join_table::choose_best_splitting()
The crashing code was added in the commit ce7ffe61d8
that made the code of the function sensitive to presence of the flag
OUTER_REF_TABLE_BIT in the KEYUSE_EXT::needed_in_prefix fields.

The field needed_in_prefix of the KEYUSE_EXT structure should not contain
table maps with OUTER_REF_TABLE_BIT or RAND_TABLE_BIT.

Note that this fix is quite conservative: for affected queries it just
returns the query plans that were used before the above mentioned commit.
In fact the equalities causing crashes should be pushed into derived tables
without any usage of split optimization.

Approved by Sergei Petrunia <sergey@mariadb.com>
2023-06-03 10:39:34 +02:00
Igor Babaev
aa713f5ae2 MDEV-31224 Crash with EXPLAIN EXTENDED for multi-table update of system table
EXPLAIN EXTENDED should always print the field item used in the left part
of an equality expression from the SET clause of an update statement as a
reference to table column.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2023-06-03 10:39:34 +02:00
Daniel Bartholomew
54324e542f bump the VERSION 2023-06-03 10:39:34 +02:00
Sergei Golubchik
c05ecda61f fix string literal escaping in views
process multibyte characters correctly, don't escape half of the character
2023-06-02 17:51:40 +02:00
Sergei Golubchik
69684f689c use correct collation_connection in --view
mysqltest should use the same collation_connection in the service
connection (that creates views) as in the main connection

this makes weight_string("aaa") to return the expected value in --view
and fixes main.func_str failure in --view
2023-06-02 17:51:40 +02:00
Sergei Golubchik
c0463704c2 fix the test for --view 2023-06-02 17:51:40 +02:00
Sergei Golubchik
aca641da28 mtr: handle the case of existing but unreadable /proc/cpuinfo 2023-06-02 17:51:40 +02:00
Sergei Golubchik
d14c485e1c test fixes for 32bit
* disable main.join_cache_notasan on 32bit
  as it uses join_buffer_size=5250229460064350213;

* update sysvars_server_embedded,32bit.rdiff
2023-06-02 17:51:02 +02:00
Sergei Golubchik
d785fa8d0b cmake warnings 2023-06-02 12:53:25 +02:00
Sergei Golubchik
270c233847 clarify why cmake is looking for Java and JNI 2023-06-02 10:46:13 +02:00
Sergei Golubchik
dc9498beb6 Revert "MDEV-31230: Fix CONNECT_JDBC in CMake"
This reverts commit 1d0e3d80d8.
2023-06-02 09:02:41 +02:00