Commit graph

76023 commits

Author SHA1 Message Date
ParadoxV5
cf2d49ddcf Extract some of #3360 fixes to 10.5.x
That PR uncovered countless issues on `my_snprintf` uses.
This commit backports a squashed subset of their fixes.
2024-11-21 22:43:56 +11:00
Monty
93fb364cd9 Removed not used ha_drop_table()
This was done after changing call in sql_select.cc from
ha_drop_table() to drop_table(), like in 11.5
2024-11-20 09:59:43 +02:00
Monty
0de9e40f4b Added status variable "stack_usable" to be able to check stack usage 2024-11-19 19:02:45 +02:00
Alexander Barkov
ae0cbfe934 MDEV-28001 greatest/least with bigint unsigned maxium has unexpected results compared to 0
LEAST() and GREATEST() erroneously calcucalted the result as signed
for BIGINT UNSIGNED arguments.

Adding a new method for unsigned arguments:
  Item_func_min_max::val_uint_native()
2024-11-19 14:26:39 +04:00
Alexander Barkov
74184074a0 MDEV-28652 SUBSTRING(str,pos,len) returns incorrect result in view (returns an empty string)
Item_func_substr::fix_length_and_dec() incorrecltly calculated its max_length
to 0 when a huge number was passed as the third argument:
  substring('hello', 1, 4294967295)
Fixing this.
2024-11-19 12:35:00 +04:00
Alexander Barkov
09fe74c7fd MDEV-25174 DOUBLE columns do not accept large hex hybrids
Limit only signed integer fields fields to LONGLONG_MAX.
Double and decimal fields do not need this limit, as they
can store integers up to ULONGLONG_MAX without problems.
2024-11-19 11:50:12 +04:00
Alexander Barkov
70dbd63e02 MDEV-24337 Server crash in DTCollation::set_repertoire_from_charset
The loop in Item_func_in::get_func_mm_tree incorrectly used array->count
in the loop. Fixing it to array->used_count.
2024-11-19 10:57:14 +04:00
Alexander Barkov
b65504b8db MDEV-23138 Odd behavior of character_set variables set to utf16 (when allowed)
Sys_var_typelib did not work when assigned to an expression
with character sets with mbminlen>1.
Using val_str_ascii() instead of val_str() to fix this.
2024-11-18 11:34:13 +04:00
ParadoxV5
79cc0f9f78 MDEV-11675 fixup: Fill in the GTID TODO
Fill in the `todo:gtid` in `check_and_remove_stale_alter`
(Note that `Master_info::master_id` is a `ulong`,
 unlike `rpl_gtid::server_id`)

> We could have caught this before MDEV-11675 was
> published if we'd had this validation earlier 😇 .
> ⸺ Brandon, reply in #3360 (MDEV-21978)

Co-authored-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
2024-11-18 14:04:56 +11:00
ParadoxV5
687377633d Extract some of #3360 fixes to 10.11.x
That PR uncovered countless issues on `my_snprintf` uses.
This commit backports a squashed subset of their fixes.
(Excludes previous parts #3485 and #3493)
2024-11-18 14:04:56 +11:00
ParadoxV5
9da7b41151 Fix DBUG_PRINT format of group_trp->records 2024-11-18 14:02:32 +11:00
ParadoxV5
d5f16d6305 Extract some of #3360 fixes to 10.6.x
That PR uncovered countless issues on `my_snprintf` uses.
This commit backports a squashed subset of their fixes (excludes #3485).
2024-11-18 13:29:04 +11:00
Dave Gosselin
3f114a0930 MDEV-35046 SIGSEGV in list_delete in optimized builds when using pseudo_slave_mode
slave_applier_reset_xa_trans() should clear the THD::pseudo_thread_id when called
to reset XA transaction state completely.  Clearing when pseudo_thread_id models
the binlog applier that handles BASE64-encoded events which possibly contain the
pseudo_thread_id, allowing us to restore the pre-event's state of the
connection's respective session var.
2024-11-15 09:48:28 -05:00
Daniel Black
7f55c61060 Update Windows TZ data from unicode source (2024b) 2024-11-15 12:18:06 +11:00
Alexander Barkov
c4843c10a3 MDEV-35416 CONV(1<<63, 10, -2) fails with --view-protocol
Item_func_conv::fix_length_and_dec() incorrectly set maximum
length as 64 character. But for negative numbers it can
return up to 65 charcters (including the sign).
2024-11-14 17:05:31 +04:00
Alexander Barkov
3b20045071 MDEV-35393 ASAN unknown-crash in Field_varstring::reset when inserting NULL value to a table with filename charset
Also fixes
MDEV-35392 Assertion `!__asan_region_is_poisoned((vo id*) dest,templ->mysql_col_len)' failed in void row_sel_field_store_in_mysql_format_func(byte *, const mysql_row_templ_t *, const byte *, ulint)

Conversion from CHAR to VARCHAR must be done before the call
for create_length_to_internal_length_string().
Moving the conversion code from Column_definition::prepare_blob_field()
to Column_definition::prepare_stage1_string().
2024-11-14 16:06:04 +04:00
Alexander Barkov
13a14c0d78 MDEV-33987 Server crashes at Item_func_as_wkt::val_str_ascii
Item_func_boundary::val_str() did not set null_value when
it could not construct a geomery object from the input.
2024-11-14 13:27:22 +04:00
Alexander Barkov
25be7da202 MDEV-32755 Stack-Buffer-Overflow at /mariadb-11.3.0/strings/int2str.c:122
The buffer ans[65] in Item_func_conv::val_str() was too small.
Fixing it to ans[66].
Thanks to Kristian Nielsen for the analysis.
2024-11-13 14:32:03 +04:00
Thirunarayanan Balathandayuthapani
074831ec61 Merge branch 10.5 into 10.6 2024-11-08 18:17:15 +05:30
Oleg Smirnov
41f54da46f MDEV-35342 Server crashes when creating index on a rocksdb table
Debug assertion, added with MDEV-34880, did not take into account
that MyRocks sometimes uses hidden indexes that SQL layer isn't
aware of. This commit adds a proper handling for such cases
2024-11-08 19:35:20 +07:00
Alexander Barkov
b9f9d804f2 MDEV-28686 Assertion `0' in Type_handler_string_result::make_sort_key or unexpected result
The code in the can_eval_in_optimize() branch in
Item_func_pad::fix_length_and_dec() did not take into account
that the constant can be negative. So the function will return NULL.

This later crashed on DBUG_ASSERT() because a NOT NULL function returned NULL.

Adding set_maybe_null() into this branch if the constant is negative.
2024-11-06 15:45:59 +04:00
Alexander Barkov
4ded2cbe13 MDEV-31910 ASAN memcpy-param-overlap upon CONCAT in ORACLE mode
Fixing Item_func_concat_operator_oracle::val_str() to use
String::copy_or_move(), like Item_func_oracle::val_str() does.
2024-11-06 11:39:50 +04:00
Jan Lindström
4b38af06a4 MDEV-35157 : wrong binlog timestamps on secondary nodes of a galera cluster
Problem was missing thd->set_time() before binlog event
execution in wsrep_apply_events.

Removed part of earlier commit 1363580 because it had
nothing to do with VERSIONED tables.

Note that this commit does not contain mtr-testcase
because actual timestamps on binlog file depends the
actual time when events are executed and how long
their execution takes.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-11-06 04:59:10 +01:00
Jan Lindström
e4a3a11dcc MDEV-35344 : Galera test failure on galera_sync_wait_upto
Ignoring configured server_id should not be a warning because
correct configuration is documented. Changed message to info
level with more detailed message what was configured and
what will be actually used.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-11-06 04:59:10 +01:00
Denis Protivensky
6d5fe9ed0d MDEV-28378: Don't hang trying to peek log event past the end of log
While applying CTAS log event, we peek the relay log to see if CTAS
contains inserted rows or if it's empty.
The peek function didn't check for end-of-file condition when tried to
get the next event from the log, and thus it hanged.

The fix includes checking for end-of-file while peeking for log events
and considering returned XID_EVENT value as a sign of an empty CTAS.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-11-06 04:59:09 +01:00
Vladislav Vaintroub
a2a0adbfc3 MDEV-34533 post-fix
Cache results of expensive my_get_stack_bounds(), instead of calling
it for every query in thread_attach()

See MDEV-27943 for effort to reduce thread_attach() overhead
2024-11-05 21:32:48 +01:00
Vladislav Vaintroub
37b7986467 Merge branch '10.5' into 10.6 2024-11-05 21:02:22 +01:00
Alexander Barkov
7741065936 MDEV-23895 Server crash, ASAN heap-buffer-overflow or Valgrind Invalid write in Item_func_rpad::val_str
Item_cache_int::val_str() and Item_cache_real::val_str() erroneously
used default_charset(). Fixing to return my_charset_numeric instead.
2024-11-05 12:36:08 +04:00
Alexander Barkov
eb41c1171e MDEV-33942 View cuts off the end of string with the utf8 character set in INSERT function
Item_func_insert::fix_length_and_dec() incorrectly calculated max_length
when its collation.collation evaluated to my_charset_bin.

Fixing the code to calculate max_length in terms of octets rather
than in terms of characters when collation.collation is my_charset_bin.
2024-11-05 11:16:10 +04:00
Alexander Barkov
c2bf1d4781 MDEV-29552 LEFT and RIGHT with big value for parameter 'len' >0 return empty value in view
The code in max_length_for_string() erroneously returned 0
for huge numbers like 4294967295.

Rewriting the code in a more straightforward way.
2024-11-05 09:19:05 +04:00
Brandon Nesterenko
b07258a0d5 MDEV-35109: Semi-sync Replication stalling Primary using wait point=AFTER_SYNC
For a primary configured with wait_point=AFTER_SYNC, if two threads
T1 (binlogging through MYSQL_BIN_LOG::write()) and T2 were
binlogging at the same time, T1 could accidentally wait for its
semi-sync ACK using the binlog coordinates of T2. Prior to
MDEV-33551, this only resulted in delayed transactions, because all
transactions shared the same condition variable for ACK signaling.
However, with the MDEV-33551 changes, each thread has its own
condition variable to signal. So T1 could wait indefinitely when
either:
  1) T1's ACK is received but not T2's when T1 goes into
wait_after_sync(), because the ACK receiver thread has already
notified about the T1 ACK, but T1 was _actually_ waiting on T2's
ACK, and therefore tries to wait (in vain).

  2) T1 goes to wait_after_sync() before any ACKs have arrived. When
T1's ACK comes in, T1 is woken up; however, sees it needs to wait
more (because it was actually waiting on T2's ACK), and goes to wait
again (this time, in vain).

Note that the actual cause of T1 waiting on T2's binlog coordinates
is when MYSQL_BIN_LOG::write() would call
Repl_semisync_master::wait_after_sync(), the binlog offset parameter
was read as the end of MYSQL_BIN_LOG::log_file, which is shared
among transactions. So if T2 had updated the binary log _after_ T1
had released LOCK_log, but not yet invoked wait_after_sync(), it
would use the end of the binary log file as the binlog offset, which
was that of T2 (or any future transaction).

The fix in this patch ensures consistency between the binary log
coordinates a transaction uses between report_binlog_update() and
wait_after_sync().

Reviewed By
============
Kristian Nielsen <knielsen@knielsen-hq.org>
Andrei Elkin <andrei.elkin@mariadb.com>
2024-11-04 10:45:58 -07:00
Brandon Nesterenko
5290fa043b MDEV-35109 PREP: simulate_delay_semisync_slave_reply use debug_sync
This is a preparatory commit for MDEV-35109 to make its
testing code cleaner (and harden other tests too).

The DEBUG_DBUG point simulate_delay_semisync_slave_reply
up to this patch used my_sleep() to delay an ACK response,
but sleeps are prone to test failures on machines that
run tests when already having a heavy load (e.g. on
buildbot).

This patch changes this DEBUG_DBUG sleep to use DEBUG_SYNC
to coordinate exactly when a slave should send its reply,
which is safer and faster.

As DEBUG_SYNC can't be used while a server is shutting
down, to synchronize threads with SHUTDOWN WAIT FOR SLAVES
logic, we use and extend wait_for_pattern_in_file.inc to
wait for an informational error message in the logic to
indicate that the shutdown process has reached the
intended state (i.e. indicating that the shutdown has
been delayed to await semi-sync ACKs). Specifically, the
extensions are as follows:

 1. wait_for_pattern_in_file.inc is extended with parameter
    wait_for_pattern_count as a number that indicates the
    number of times a pattern should occur in the file before
    return control back to the calling script.

 2. search_for_pattern_in_file.inc is extended with parameter
    SEARCH_ABORT_IS_SUCCESS to inverse the error/success
    logic, so the SEARCH_ABORT condition can be used to
    indicate success, rather than error.
2024-11-04 10:45:58 -07:00
Oleksandr Byelkin
a37f71bd10 Merge branch '10.11' into mariadb-10.11.10 2024-11-04 07:42:26 +01:00
Oleksandr Byelkin
f2bb2ab58c Merge branch '10.6' into mariadb-10.6.20 2024-11-04 07:40:45 +01:00
Oleksandr Byelkin
ecdccddaae Merge branch '10.5' into mariadb-10.5.27 2024-11-04 07:35:28 +01:00
Alexander Barkov
e60fd6c204 MDEV-28767 Collation "binary" is not accepted for databases, tables, columns
MariaDB in a COLLATE clause supported 'binary' only as an identifier:
  COLLATE `binary`

Fixing the parser to understand 'binary' as a keyword:
  COLLATE binary

This is for MySQL compatibility.
2024-11-02 12:47:28 +04:00
Alexander Barkov
d661bc1552 MDEV-20944 Wrong result of LEAST() and ASAN heap-use-after-free in my_strnncollsp_simple / Item::temporal_precision on TIME()
The code tried to avoid String::copy() but did it in a wrong way,
so asan detected heap-use-after-free errors.
Removing the wrong optimization, using copy() instead.
2024-11-01 15:55:09 +04:00
Alexander Barkov
dd41be2a51 MDEV-29184 Assertion `0' in Item_row::illegal_method_call, Type_handler_row::Item_update_null_value, Item::update_null_value
- Moving the check_cols(1) test from fix_fields() to fix_length_and_dec().
  So the test is now done before the code calling val_decimal()
  in fix_length_and_dec().

- Removing Item_func_interval::fix_fields(), as it become equal
  to the inherited one.
2024-11-01 12:40:43 +04:00
Oleksandr Byelkin
c770bce898 Merge branch '11.2' into 11.4 2024-10-30 15:11:17 +01:00
Oleg Smirnov
c3a7a3c7a2 MDEV-34665 Simplify IN predicate processing for NULL-aware materialization involving only one column
It was found that unnecessary work of building Ordered_key structures
is being done when processing NULL-aware materialization for IN predicates
having only one column. In fact, the logic for that simplified case can be
expressed as follows.
Say we have predicate left_expr IN (SELECT <subq1>), where left_expr is
scalar(not a tuple).
Then
    if (left_expr is NULL) {
      if (subq1 produced any rows) {
        // note that we don't care if subq1 has produced
        // NULLs or not.
        NULL IN (<some values>) -> UNKNOWN, i.e. NULL.
      } else {
        NULL IN ({empty-set}) -> FALSE.
      }
    } else {
      // left_expr is a non-NULL value
      if (subq1 output has a match for left_expr) {
        left_expr IN (..., left_expr ...) -> TRUE
      } else {
        // no "known" matches.
        if (subq1 output has a NULL) {
          left_expr IN ( ... NULL ...) ->
           (NULL could have been a match or not)
           -> NULL.
        } else {
          // subq1 didn't produce any "UNKNOWNs" so
          // we're positive there weren't any matches
          -> FALSE.
        }
      }
    }

This commit introduces subselect_single_column_partial_engine class
implementing the logic described.

Reviewer: Sergei Petrunia <sergey@mariadb.com>
2024-10-30 16:48:36 +07:00
Oleksandr Byelkin
69d033d165 Merge branch '10.11' into 11.2 2024-10-29 16:42:46 +01:00
Oleksandr Byelkin
3d0fb15028 Merge branch '10.6' into 10.11 2024-10-29 15:24:38 +01:00
Oleksandr Byelkin
f00711bba2 Merge branch '10.5' into 10.6 2024-10-29 14:20:03 +01:00
Oleksandr Byelkin
6aa47fae30 MDEV-35276 Assertion failure in find_producing_item upon a query from a view
Two problem solved:

1) Item_default_value makes a shallow copy so the copy
 should not delete field belong to the Item

2) Item_default_value should not inherit
 derived_field_transformer_for_having and
 derived_field_transformer_for_where (in this variant
 pushing DEFAULT(f) is prohibited (return NULL) but
 if return "this" it will be allowed (should go with
 a lot of tests))
2024-10-29 11:44:43 +01:00
Sergei Petrunia
9bb95de186 MDEV-35253: xa_prepare_unlock_unmodified fails ... : part 2
instead of our own prev_bits(), make use of my_set_bits().
2024-10-29 12:09:13 +02:00
Sergei Golubchik
953f847aed Post-fix for MDEV-35236
when the option_list is initially empty, its value doesn't need
to be restored, as it'll be shallow-copied every time.
Furthermore, the CREATE_INFO is allocated on the stack, so it's
even wrong to restore it after its frame was left.

followup for 3cd706b107
2024-10-28 15:29:10 +01:00
Oleg Smirnov
52723ec09a MDEV-34880 Incorrect result for query with derived table having TEXT field
Fixup: check key flags only in the case of successful index initialization
2024-10-28 18:49:01 +07:00
Monty
066f920484 MDEV-35110 Deadlock on Replica during BACKUP STAGE BLOCK_COMMIT on XA transactions
This is an extension of MDEV-30423 "Deadlock on Replica during BACKUP
STAGE BLOCK_COMMIT on XA transactions"

The original commit in MDEV-30423 was not complete as some usage in XA of
MDL_BACKUP_COMMIT locks did not set thd->backup_commit_lock.
This is required to be set when using parallel replication.

Fixed by ensuring that all usage of BACKUP_COMMIT lock i XA is uniform and
all sets thd->backup_commit_lock. I also changed all locks to be
MDL_EXPLICIT to keep also that part uniform.

A regression test is added.
2024-10-28 13:29:21 +02:00
Marko Mäkelä
63a7e4c96b MDEV-35257 Backup fails during an ALTER TABLE with FULLTEXT INDEX
In commit 1c55b845e0 (MDEV-32932) the
test mariabackup.innodb_ddl_on_intermediate_table was introduced but
disabled.

xb_load_single_table_tablespace(): Properly handle missing FTS_ tables.

backup_file_op_fail(): Properly handle FILE_DELETE records.
2024-10-28 07:44:18 +02:00
Sergei Petrunia
284593413f MDEV-35253: xa_prepare_unlock_unmodified fails: shift exponent 32 is too large
The code in best_access_path() uses PREV_BITS(uint, N) to
compute a bitmap of all keyparts: {keypart0, ... keypart{N-1}).

The problem is that PREV_BITS($type, N) macro code can't handle the case
when N=<number of bits in $type).
Also, why use PREV_BITS(uint, ...) for key part map computations when
we could have used PREV_BITS(key_part_map) ?

Fixed both:
- Change PREV_BITS(type, N) to handle any N in [0; n_bits(type)].
- Change PREV_BITS() to use key_part_map when computing key_part_map bitmaps.
2024-10-25 18:02:14 +03:00
Yuchen Pei
b8c2bd9f69
MDEV-35249 Fix regression caused by MDEV-34447
MDEV-34447 Removed setting first_cond_optimization to 0 in update and
delete when leaf_tables_saved. This can cause problems when two ps
executions of an update go through different paths, where the first ps
execution goes through single table update only and the second ps
execution also goes through multi table update. When this happens, the
first_cond_optimization of the outer query is not set to false during
the first ps execution because optimize() is not called for the outer
query. But then the second ps execution will call optimize() on the
outer query, which with first_cond_optimization==true trips the 2nd ps
mem leak detection.

This is not a problem in higher version as both executions go through
multi table updates, possibly due to MDEV-28883.

We fix this problem by restoring the FALSE assignments to
first_cond_optimization.
2024-10-25 18:03:40 +11:00
Sergei Golubchik
3cd706b107 MDEV-35236 Assertion `(mem_root->flags & 4) == 0' failed in safe_lexcstrdup_root
Post-fix for MDEV-35144.

Cannot allocate options values on the statement arena, because
HA_CREATE_INFO is shallow-copied for every execution, so if the
option_list was initially empty, it will be reset for every execution
and any values allocated on the  statement arena will be lost.

Cannot allocate option values on the execution arena, because
HA_CREATE_INFO is shallow-copied for every execution, so if the
option_list was  initially NOT empty, any values appended to the
end will be preserved and if they're on the execution arena their
content will be destroyed.

Let's use thd->change_item_tree() to save and restore necessary pointers
for every execution.

followup for 3da565c41d
2024-10-23 14:58:57 +02:00
Sergei Golubchik
eac33a23da MDEV-32022 ERROR 1054 (42S22): Unknown column 'X' in 'NEW' in trigger
add missing do_get_copy/do_build_clone
2024-10-23 14:58:57 +02:00
Oleg Smirnov
6bd1cb0ea0 MDEV-34880 Incorrect result for query with derived table having TEXT field
When a derived table which has distinct values and BLOB fields is
materialized, an index is created over all columns to ensure only
unique values are placed to the result.
This index is created in a special mode HA_UNIQUE_HASH to support BLOBs.
Later the optimizer may incorrectly choose this index to retrieve values
from the derived table, although such type of index cannot be used
for data retrieval.

This commit excludes HA_UNIQUE_HASH indexes from adding to
`JOIN::keyuse` array thus preventing their subsequent usage for
data retrieval
2024-10-23 17:55:00 +07:00
Vlad Lesin
8c7786e7d5 MDEV-34690 lock_rec_unlock_unmodified() causes deadlock
lock_rec_unlock_unmodified() is executed either under lock_sys.wr_lock()
or under a combination of lock_sys.rd_lock() + record locks hash table
cell latch. It also requests page latch to check if locked records were
changed by the current transaction or not.

Usually InnoDB requests page latch to find the certain record on the
page, and then requests lock_sys and/or record lock hash cell latch to
request record lock. lock_rec_unlock_unmodified() requests the latches
in the opposite order, what causes deadlocks. One of the possible
scenario for the deadlock is the following:

thread 1 - lock_rec_unlock_unmodified() is invoked under locks hash table
           cell latch, the latch is acquired;
thread 2 - purge thread acquires page latch and tries to remove
           delete-marked record, it invokes lock_update_delete(), which
           requests locks hash table cell latch, held by thread 1;
thread 1 - requests page latch, held by thread 2.

To fix it we need to release lock_sys.latch and/or lock hash cell latch,
acquire page latch and re-acquire lock_sys related latches.

When lock_sys.latch and/or lock hash cell latch are released in
lock_release_on_prepare() and lock_release_on_prepare_try(), the page on
which the current lock is held, can be merged. In this case the bitmap
of the current lock must be cleared, and the new lock must be added to
the end of trx->lock.trx_locks list, or bitmap of already existing lock
must be changed.

The new field trx_lock_t::set_nth_bit_calls indicates if new locks
(bits in existing lock bitmaps or new lock objects) were created during
the period when lock_sys was released in trx->lock.trx_locks list
iteration loop in lock_release_on_prepare() or
lock_release_on_prepare_try(). And, if so, we traverse the list again.

The block can be freed during pages merging, what causes assertion
failure in buf_page_get_gen(), as btr_block_get() passes BUF_GET as page
get mode to it. That's why page_get_mode parameter was added to
btr_block_get() to pass BUF_GET_POSSIBLY_FREED from
lock_release_on_prepare() and lock_release_on_prepare_try() to
buf_page_get_gen().

As searching for id of trx, which modified secondary index record, is
quite expensive operation, restrict its usage for master. System variable
was added to remove the restriction for testing simplifying. The
variable exists only either for debug build or for build with
-DINNODB_ENABLE_XAP_UNLOCK_UNMODIFIED_FOR_PRIMARY option to increase the
probability of catching bugs for release build with RQG.

Note that the code, which does primary index lookup to find out what
transaction modified secondary index record, is necessary only when
there is no primary key and no unique secondary key on replica with row
based replication, because only in this case extra X locks on unmodified
records can be set during scan phase.

Reviewed by Marko Mäkelä.
2024-10-23 12:36:17 +03:00
Jan Lindström
b3be3c2157 MDEV-30653 : With wsrep_mode=REPLICATE_ARIA only part of mixed-engine transactions is replicated
Replication of non-transactional engines is experimental and
uses TOI. This naturally means that if there is open transaction
with transactional engine it's changes will be rolled back.

Fixed by adding error message if non-transactional engine
is part of multi-engine transaction with warning.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-10-23 04:00:52 +02:00
Brandon Nesterenko
1ed30e08af MDEV-34122: Assertion `entry' failed in Active_tranx::assert_thd_is_waiter
If semi-sync is switched off then on while a transaction is
in-between binlogging and waiting for an ACK, the semi-sync state of
the transaction is removed, leading to a debug assertion that
indicates the transaction tried to wait, but cannot receive an ACK
signal. More specifically, when semi-sync is switched off, the
Active_tranx list is cleared (where a transaction adds an entry to
this list during binlogging), and each entry in this list saves the
thread which will wait for an ACK, and the thread has the COND
variable to signal to wake itself. So if the entry is lost, the
Ack_receiver thread won’t be able to find the thread to wake up when
an ACK comes in

The fix is to ensure that the entry exists before awaiting the ACK,
and if there is no entry, skip the wait. In debug builds, an
informative message is written explaining that the transaction is
skipping its wait. Additional debug-build only logic is added to
ensure that the cause of the missing entry is due to semi-sync being
turned off and on

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>
2024-10-21 15:35:54 -06:00
Sergei Petrunia
a68e74b5a4 MDEV-35164: optimizer_join_limit_pref_ratio: assertion when the ORDER BY table becomes constant
Assertion failure has happened due to this scenario:

A query was ran with optimizer_join_limit_pref_ratio=1.
The query had "ORDER BY t1.col LIMIT N".
The optimizer set join->limit_shortcut_applicable=1.
Then, table t1 was marked as constant.
The code in choose_query_plan() still set join->limit_optimization_mode=1
which caused the optimizer to only consider t1 as the first non-const table.
But t1 was already put into the join prefix as the constant table.
The optimizer couldn't produce any join order at all and crashed.

Fixed by not searching for shortcut plan if ORDER BY table is a constant.
We will not try to do sorting anyway in this case (and LIMIT short-cutting
will be done for any join order).
2024-10-18 15:42:05 +03:00
Rucha Deodhar
e14d2b7974 MDEV-8578: Wrong error code/message with enforce_storage_engine and
NO_ENGINE_SUBSTITUTION

Analysis:
When the error is hit, wrong error code is passed in my_error
Fix:
Pass a better error code.
2024-10-18 16:42:52 +05:30
Sergei Petrunia
0540eac05c MDEV-35180: ref_to_range rewrite causes poor query plan
(Variant 2: only allow rewrite for ref(const))

make_join_select() has a "ref_to_range" rewrite: it would rewrite
any ref access to a range access on the same index if the latter uses
more keyparts.
It seems, he initial intent of this was to fix poor query plan choice
in cases like

  t.keypart1=const AND t.keypart2 < 'foo'

Due to deficiency in cost model, ref access could be picked while range
would enumerate fewer rows and be cheaper.
However, the condition also forces a rewrite in cases like:

  t.keypart1=prev_table.col AND t.keypart1<='foo' AND t.keypart2<'bar'

Here, it can be that
* keypart1=prev_table.col is highly selective
* (keypart1, keypart2) <= ('foo', 'bar') is not at all selective.

Still, the rewrite would be made and poor query plan chosen.
Fixed this by only doing the rewrite if ref access was ref(const)
so we can be certain that quick select also used these restrictions
and will scan a subset of rows that ref access would scan.
2024-10-18 13:37:04 +03:00
Marko Mäkelä
ebefef658e Merge 10.11 into 11.2 2024-10-18 11:32:22 +03:00
Sergei Golubchik
3a1cf2c85b MDEV-34679 ER_BAD_FIELD uses non-localizable substrings 2024-10-17 21:37:37 +02:00
Sergei Golubchik
3da565c41d MDEV-35144 CREATE TABLE ... LIKE uses current innodb_compression_default instead of the create value
When adding a column or index that uses plugin-defined
sysvar-based options with CREATE ... LIKE the server
was using the current value of the sysvar, not the default one.

Because parse_option_list() function was used both in create
and open and it tried to guess when it's create (need to use
current sysvar value and add a new name=value pair to the list)
or open (need to use default, without extending the list).

Let's move the list extending functionality into a separate
function and call it explicitly when needed. Operations that
add new objects (CREATE, ALTER ... ADD) will extend the list,
other operations (ALTER, CREATE ... LIKE, open) will not.
2024-10-17 16:28:39 +02:00
Oleksandr Byelkin
600c42ea86 MDEV-34883 LOAD DATA INFILE with geometry data fails
We write field using field data charset, so we should read it
using the field charset.
2024-10-17 10:33:36 +02:00
Sergei Golubchik
3b58c6b93f MDEV-35079 Migrate MySQL5.7 to MariaDB 10.4, then to MariaDB 10.11 Failed
correctly detect when partitioning is disabled
2024-10-17 10:08:24 +02:00
Monty
4955f6018a MDEV-29351 SIGSEGV when doing forward reference of item in select list
The reason for the crash was the code assumed that
SELECT_LEX.ref_pointer_array would be initialized with zero, which was
not the case. This cause the test of
if (!select->ref_pointer_array[counter]) in item.cc to be unpredictable
and causes crashes.

Fixed by zero-filling ref_pointer_array on allocation.
2024-10-16 17:24:46 +03:00
Monty
7e5ad5dd9e Replace some usage of safe_strcat() with strxnmov() 2024-10-16 17:24:46 +03:00
Monty
0de2613e7a Fixed that SHOW CREATE TABLE for sequences shows used table options 2024-10-16 17:24:46 +03:00
Monty
2c52fdd28a MDEV-32350 Can't selectively restore sequences using innodb tables from backup
Added support for sequences to do  discard and import tablespace
2024-10-16 17:24:46 +03:00
Monty
bddbef3573 MDEV-34533 asan error about stack overflow when writing record in Aria
The problem was that when using clang + asan, we do not get a correct value
for the thread stack as some local variables are not allocated at the
normal stack.

It looks like that for example clang 18.1.3, when compiling with
-O2 -fsanitize=addressan it puts local variables and things allocated by
alloca() in other areas than on the stack.

The following code shows the issue

Thread 6 "mariadbd" hit Breakpoint 3, do_handle_one_connection
    (connect=0x5080000027b8,
    put_in_cache=<optimized out>) at sql/sql_connect.cc:1399

THD *thd;
1399      thd->thread_stack= (char*) &thd;
(gdb) p &thd
(THD **) 0x7fffedee7060
(gdb) p $sp
(void *) 0x7fffef4e7bc0

The address of thd is 24M away from the stack pointer

(gdb) info reg
...
rsp            0x7fffef4e7bc0      0x7fffef4e7bc0
...
r13            0x7fffedee7060      140737185214560

r13 is pointing to the address of the thd. Probably some kind of
"local stack" used by the sanitizer

I have verified this with gdb on a recursive call that calls alloca()
in a loop. In this case all objects was stored in a local heap,
not on the stack.

To solve this issue in a portable way, I have added two functions:

my_get_stack_pointer() returns the address of the current stack pointer.
The code is using asm instructions for intel 32/64 bit, powerpc,
arm 32/64 bit and sparc 32/64 bit.
Supported compilers are gcc, clang and MSVC.
For MSVC 64 bit we are using _AddressOfReturnAddress()

As a fallback for other compilers/arch we use the address of a local
variable.

my_get_stack_bounds() that will return the address of the base stack
and stack size using pthread_attr_getstack() or NtCurrentTed() with
fallback to using the address of a local variable and user provided
stack size.

Server changes are:

- Moving setting of thread_stack to THD::store_globals() using
  my_get_stack_bounds().
- Removing setting of thd->thread_stack, except in functions that
  allocates a lot on the stack before calling store_globals().  When
  using estimates for stack start, we reduce stack_size with
  MY_STACK_SAFE_MARGIN (8192) to take into account the stack used
  before calling store_globals().

I also added a unittest, stack_allocation-t, to verify the new code.

Reviewed-by: Sergei Golubchik <serg@mariadb.org>
2024-10-16 17:24:46 +03:00
Sergei Petrunia
89493a9980 MDEV-34993: fix merge into 10.6: OPTIMIZER_ADJ_FIX_CARD_MULT should be ON by default 2024-10-16 16:46:38 +03:00
Sergei Golubchik
5ebda30ccc Revert "MDEV-35019 Provide a way to enable "rollback XA on disconnect" behavior we had before 10.5.2"
This reverts commit 8ae462a220.
2024-10-16 13:23:47 +02:00
Kristian Nielsen
8ae462a220 MDEV-35019 Provide a way to enable "rollback XA on disconnect" behavior we had before 10.5.2
Implement variable legacy_xa_rollback_at_disconnect to support
backwards compatibility for applications that rely on the pre-10.5
behavior for connection disconnect, which is to rollback the
transaction (in violation of the XA specification).

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-10-16 10:18:36 +02:00
Daniel Black
e927e28ebe Remove HAVE_BROKEN_REALPATH
Was only applicable to FreeBSD < ~5.1.9
per: 33173fbe82
2024-10-16 11:46:19 +11:00
Dave Gosselin
e9c71f3a25 MDEV-34814 mysqld hangs on startup when --init-file target does not exist
Set select_thread_in_use only when we're about to enter into the polling
loop, not sooner, allowing early proces aborts to exist cleanly: the
process won't be waiting for a polling loop that isn't yet polling.
2024-10-15 09:22:31 -04:00
Sergei Petrunia
9849e3f948 MDEV-35072: Assertion with optimizer_join_limit_pref_ratio and 1-table select
Pre-11.0 variant:

1. In recompute_join_cost_with_limit(), add an assertion that that
partial_join_cost >= 0.0.

2. best_extension_by_limited_search() subtracts COST_EPS from
   join->best_read. But it is not subtracted from
   join->positions[0].read_time, add it back.

2. We could get very small negative partial_join_cost due to rounding
  errors. For fraction=1.0, we were computing essentially this (denote
  as EXPR-1):

  $row_read_cost + $where_cost - ($row_read_cost + $where_cost)

which should compute to 0.
But the computation was done in the following order (left-to-right):
EXPR-2:

  ($row_read_cost + $where_cost) - $row_read_cost - $where_cost

this produced a value of -1.1102230246251565e-16 due to a rounding
error. Change the computation use EXPR-1 instead of EXPR-2.
2024-10-15 15:56:41 +03:00
Sergei Petrunia
5619f29384 Replace 0.001 with symbolic name COST_EPS
optimize_straight_join and best_extension_by_limited_search()
use 0.001 to make choice between plans with identical cost deterministic.

Use COST_EPS instead of 0.001, like it's done in newer versions.
2024-10-15 15:56:41 +03:00
Sergei Petrunia
66b8d32b75 MDEV-35072: Assertion with optimizer_join_limit_pref_ratio and 1-table select
Variant for 11.2+:

In recompute_join_cost_with_limit(), do not subtract the cost of checking
the WHERE:
   pos->records_read* WHERE_COST_THD(join->thd)

It is already included in pos->read_time.

Also added comments about difference between this fix and the pre-11.2
variant.
2024-10-15 15:01:29 +03:00
Yuchen Pei
d797c9879e
[fixup] Restore the Chinese error translation of ER_SET_PASSWORD_AUTH_PLUGIN
It was removed in MDEV-18151
2024-10-15 20:35:55 +11:00
Yuchen Pei
cd5577ba4a
Merge branch '10.5' into 10.6 2024-10-15 16:00:44 +11:00
Yuchen Pei
77ed235d50
MDEV-26345 Spider GBH should execute original queries on the data node
Stop skipping const items when selecting but skip them when storing
their results to spider row to avoid storing in mismatching temporary
table fields.

Skip auxiliary fields in SELECTing, and do not store
the (non-existing) results to the corresponding temporary table
accordingly.

When there are BOTH auxiliary fields AND const items in the auxiliary
field items, do not use the spider GBH. This is a rare occasion if it
happens at all and not worth the added complexity to cover it.

Use the original item (item_ptr) in constructing GROUP BY and ORDER
BY, which also means using item->name instead of field->field_name as
aliases in constructing SELECT items. This fixes spurious regressions
caused by the above changes in some tests using ORDER BY, such as
mdev_24517.test. As a by-product, this also fixes MDEV-29546.
Therefore we update mdev_29008.test to include the MDEV-29546 case.
2024-10-15 15:36:12 +11:00
Yuchen Pei
d3b84ff10d
MDEV-30067 Remove some overly enthusiastic asserts when deleting from a partitioned table
When an DDL statement results in a local partition table with
partitions not covering all values in the table, a failure is emitted.
However, when the table in question is a spider table, the issue does
not surface until some future statements (DELETE in the test examples
in this commit) are executed. This is consistent with the design of
spider which aims to minimise connections with the data node. The
resulting error is legitimate and should not result in an assertion
failure. Similarly, a partitioned spider table could have misplaced
rows, so we remove the other assertion as well.
2024-10-15 14:18:10 +11:00
Oleksandr Byelkin
a79c9b3812 MDEV-35135 Assertion `!is_cond()' failed in Item_bool_func::val_int / do_select
Change val_int with val_bool when it is a condition.
2024-10-14 09:36:17 +02:00
Oleksandr Byelkin
b138f428ea MDEV-18151 Skipped error returning for GRANT/SET PASSWORD
Make message of error not warning.
2024-10-09 15:48:14 +02:00
Oleksandr Byelkin
cc59fbfffa MDEV-18151 Skipped error returning for GRANT/SET PASSWORD
Make error issueing for GRANT and SET PASSWORD the same.
Report errors wich were skipped before.
2024-10-09 13:29:59 +02:00
Oleksandr Byelkin
1d0e94c55f Merge branch '10.5' into 10.6 2024-10-09 08:38:48 +02:00
Thirunarayanan Balathandayuthapani
23820f1d79 MDEV-34392 Inplace algorithm violates the foreign key constraint
- Fixing the compilation issue for the compiler lesser than gcc-6

Reviewed-by : Marko Mäkelä <marko.makela@mariadb.com>
2024-10-09 10:14:29 +05:30
Sergei Golubchik
4281e0068b MDEV-35082 HANDLER with FULLTEXT keys is not always rejected
need to check for index capabilities also for HANDLER READ NEXT
2024-10-08 18:20:13 +02:00
Sergei Golubchik
23d7dc1ea1 MDEV-35086 Trying to lock mutex when the mutex was already locked (session_tracker.cc), server hangs
followup for 9021f40b8e
2024-10-08 15:31:02 +02:00
Sergei Golubchik
3ea71a2c8e MDEV-16699 heap-use-after-free in group_concat with compressed or GIS columns
Field_blob::store() has special code for GROUP_CONCAT temporary table
(to store blob values in Blob_mem_storage - this prevents them
from being freed/overwritten when a next row is read).

Field_geom and Field_blob_compressed inherit from Field_blob but they
have their own ::store() method without this special Blob_mem_storage
support.

Considering that non-grouping CONCAT() of such fields converts
them to plain BLOB, let's do the same for GROUP_CONCAT. To do it,
Item_func_group_concat::setup will signal that it's creating
a temporary table for GROUP_CONCAT, and Field_blog::make_new_field()
override will create base Field_blob when under group concat.
2024-10-08 15:31:02 +02:00
Thirunarayanan Balathandayuthapani
65418ca9ad MDEV-34392 Inplace algorithm violates the foreign key constraint
- Fix the compilation error in gcc-5
2024-10-08 16:43:57 +05:30
Aleksey Midenkov
706a8bcb5b MDEV-33470 Unique hash index is broken on DML for system-versioned table
Hash index is vcol-based wrapper (MDEV-371). row_end is added to
unique index. So when row_end is updated unique hash index must be
recalculated via vcol_update_fields(). DELETE did not update virtual
fields, so DELETE HISTORY was getting wrong hash value.

The fix does update_virtual_fields() on vers_update_end() so in every
case row_end is updated virtual fields are updated as well.
2024-10-08 13:08:10 +03:00
Aleksey Midenkov
d37bb140b1 MDEV-31297 Create table as select on system versioned tables do not
work consistently on replication

Row-based replication does not execute CREATE .. SELECT but instead
CREATE TABLE. CREATE .. SELECT creates implict system fields on
unusual place: in-between declared fields and select fields. That was
done because select_field_pos logic requires select fields go last in
create_list.

So, CREATE .. SELECT on master and CREATE TABLE on slave create system
fields on different positions and replication gets field mismatch.

To fix this we've changed CREATE .. SELECT to create implicit system
fields on usual place in the end and updated select_field_pos for
handling this case.
2024-10-08 13:08:10 +03:00
Alexander Barkov
a931da82fa MDEV-34123 CONCAT Function Returns Unexpected Empty Set in Query
Search conditions were evaluated using val_int(), which was wrong.
Fixing the code to use val_bool() instead.

Details:
- Adding a new item_base_t::IS_COND flag which marks Items used
  as <search condition> in WHERE, HAVING, JOIN ON, CASE WHEN clauses.
  The flag is at the parse time.
  These expressions must be evaluated using val_bool() rather than val_int().

  Note, the optimizer creates more Items which are used as search conditions.
  Most of these items are not marked with IS_COND yet. This is OK for now,
  but eventually these Items can also be fixed to have the flag.

- Adding a method Item::is_cond() which tests if the Item has the IS_COND flag.

- Implementing Item_cache_bool. It evaluates the cached expression using
  val_bool() rather than val_int().
  Overriding Type_handler_bool::Item_get_cache() to create Item_cache_bool.

- Implementing Item::save_bool_in_field(). It uses val_bool() rather than
  val_int() to evaluate the expression.

- Implementing Type_handler_bool::Item_save_in_field()
  using Item::save_bool_in_field().

- Fixing all Item_bool_func descendants to implement a virtual val_bool()
  rather than a virtual val_int().

- To find places where val_int() should be fixed to val_bool(), a few
  DBUG_ASSERT(!is_cond()) where added into val_int() implementations
  of selected (most frequent) classes:

  Item_field
  Item_str_func
  Item_datefunc
  Item_timefunc
  Item_datetimefunc
  Item_cache_bool
  Item_bool_func
  Item_func_hybrid_field_type
  Item_basic_constant descendants

- Fixing all places where DBUG_ASSERT() happened during an "mtr" run
  to use val_bool() instead of val_int().
2024-10-08 11:58:46 +02:00
Marko Mäkelä
b53b81e937 Merge 11.2 into 11.4 2024-10-03 14:32:14 +03:00
Monty
6f6c1911dc MDEV-34251 Conditional jump or move depends on uninitialised value in ha_handler_stats::has_stats
Fixed by checking handler_stats if it's active instead of
thd->variables.log_slow_verbosity & LOG_SLOW_VERBOSITY_ENGINE.

Reviewed-by: Sergei Petrunia <sergey@mariadb.com>
2024-10-03 13:45:26 +03:00
Marko Mäkelä
12a91b57e2 Merge 10.11 into 11.2 2024-10-03 13:24:43 +03:00
Marko Mäkelä
63913ce5af Merge 10.6 into 10.11 2024-10-03 10:55:08 +03:00
Marko Mäkelä
7e0afb1c73 Merge 10.5 into 10.6 2024-10-03 09:31:39 +03:00
Sergei Petrunia
1cda4726ca MDEV-34993, part2: backport optimizer_adjust_secondary_key_costs
...and make the fix for MDEV-34993 switchable. It is enabled by default
and controlled with @optimizer_adjust_secondary_key_costs=fix_card_multiplier
2024-10-02 10:52:09 +03:00