Commit graph

200139 commits

Author SHA1 Message Date
Vladislav Vaintroub
40abd973ab MDEV-34236 Mroonga build with ASAN/UBSAN with GCC 12+ extremely slow.
Workaround by disabling sanitizer for single source file.
2024-06-05 11:58:53 +02:00
Tuukka Pasanen
b204817986 MDEV-34261: Detect if build is running under 32-bit container
When building on 64-bit kernel machine in 32-bit docker container
CMake falsely (but it works as expected) detects that container
runtime in also 64-bits. Use linux32 command to change runtime
enviroment to 32-bit and then CMake will correctly disable for
example ColumnStore and not try to build it

This commit only works with debian/autobake-debs.sh
2024-06-05 09:01:32 +01:00
Monty
38cbef8b3f MDEV-22935 Erroneous Aria Index / Optimizer behaviour
The problem was in the Aria part of the range optimizer,
maria_records_in_range(), which wrong concluded that there was no rows
in the range.

This error would happen in the unlikely case when searching for a range
on a partial key and there was a match for the first key part in the
upper part of the b-tree (node) and also a match in the underlying
node page.

In other words, for this bug to happen one have to use Aria, have a multi
part key with a lot of identical values for the first key part and do a
range search on the second part of the key.

Fixed by ensuring that we do not stop searching for partial keys found
on node.

Other things:
- Added some comments
- Changed a variable name to more clearly explain it's purpose.
- Fixed wrong cast in _ma_record_pos() that could cause problems on 32 bit
  systems.
2024-06-05 10:29:49 +03:00
Marko Mäkelä
c6d36c3e7c MDEV-34297 get_rnd_value() of ib_counter_t is unnecessarily complex
The shared counter template ib_counter_t uses the function
my_timer_cycles() as a source of pseudo-random numbers to pick a shard.
On some platforms, my_timer_cycles() could return the constant value 0.

get_rnd_value(): Remove.

my_pseudo_random(): Implement as an alias of my_timer_cycles() or
a wrapper for pthread_self().

Reviewed by: Vladislav Vaintroub
2024-06-05 09:54:14 +03:00
ilyasa1211
ecf4a26107 Fix Indonesian month name.
Noticed on MySQL: https://github.com/mysql/mysql-server/pull/531

Matches https://icu4c-demos.unicode.org/icu-bin/locexp?d_=en&_=in_IN.
2024-06-05 14:06:16 +10:00
Igor Babaev
4d38267fc7 MDEV-29307 Wrong result when joining two derived tables over the same view
This bug could affect queries containing a join of derived tables over
grouping views such that one of the derived tables contains a window
function while another uses view V with dependent subquery DSQ containing
a set function aggregated outside of the subquery in the view V. The
subquery also refers to the fields from the group clause of the view.Due to
this bug execution of such queries could produce wrong result sets.

When the fix_fields() method performs context analysis of a set function AF
first, at the very beginning the function Item_sum::init_sum_func_check()
is called. The function copies the pointer to the embedding set function,
if any, stored in THD::LEX::in_sum_func into the corresponding field of the
set function AF simultaneously changing the value of THD::LEX::in_sum_func
to point to AF. When at the very end of the fix_fields() method the function
Item_sum::check_sum_func() is called it is supposed to restore the value
of THD::LEX::in_sum_func to point to the embedding set function. And in
fact Item_sum::check_sum_func() did it, but only for regular set functions,
not for those used in window functions. As a result after the context
analysis of AF had finished THD::LEX::in_sum_func still pointed to AF.
It confused the further context analysis. In particular it led to wrong
resolution of Item_outer_ref objects in the fix_inner_refs() function.
This wrong resolution forced reading the values of grouping fields referred
in DSQ not from the temporary table used for aggregation from which they
were supposed to be read, but from the table used as the source table for
aggregation.

This patch guarantees that the value of THD::LEX::in_sum_func is properly
restored after the call of fix_fields() for any set function.
2024-06-04 17:54:01 -07:00
Yuchen Pei
042a0d85ad
MDEV-27186 spider/partition: Report error on info() failure
Like MDEV-28105, spider may attempt to connect to remote server in
info(), and it may emit an error upon failure to connect. In this
case, the downstream caller ha_partition::open() should return the
error to avoid inconsistency.

This fixes MDEV-27186, MDEV-27237, MDEV-27334, MDEV-28241, MDEV-34101.
2024-06-05 10:13:30 +10:00
Tuukka Pasanen
1bf0950b19 MDEV-34146: Remove duplicate #DEBHELPER# from MariaDB server postrm
Moving to use Debian systemd install and uninstall scripts
caused duplicated DEBHELP to server postrm script. Commit
removes unneeded and makes postrm work better and pass lintian
tests
2024-06-05 09:54:41 +10:00
Tuukka Pasanen
e9f4b87e53 MDEV-33919: Remove less standard format directive an-trap
Few man pages have less standard format directive:
.it 1 an-trap which specifying a formatting instruction
related to indentation (adds tab in man page in this)

There is no traces what an-trap should do and removing
it does not affect rendering of man page
2024-06-05 09:53:52 +10:00
Alexander Barkov
5e12d49205 MDEV-34295 CAST(char_col AS DOUBLE) prints redundant spaces in a warning
Field_string::val_int(), Field_string::val_real(), Field_string::val_decimal()
passed the whole buffer of field_length bytes to data type conversion routines.
This made conversion routines to print redundant trailing spaces in case of warnings.

Adding a method Field_string::to_lex_cstring() and using it inside
val_int(), val_real(), val_decimal(), val_str().

After this change conversion routines get the same value with what val_str() returns,
and no redundant trailing spaces are displayed.
2024-06-04 15:34:14 +04:00
Alexander Barkov
76e0dc18b6 MDEV-34288 SET NAMES DEFAULT crashes mariadbd --collation-server=utf8mb4_unicode_ci
The @@global.character_set_client variable could erroneously be set
to a non-default collation of its character set, which further made
the `SET NAMES DEFAULT` statement crash the server.

Fixing the code to make sure that the global value these variables:
  @@character_set_client
  @@character_set_connection
  @@character_set_server
  @@character_set_database
  @@character_set_connection
point to the default compiled collations of the character set.
2024-06-04 12:38:43 +04:00
Yuchen Pei
581712b989
MDEV-33490 MENT-1504 Fix some english strings in spider. 2024-06-04 12:25:08 +10:00
Thirunarayanan Balathandayuthapani
58a0e1e3dd MDEV-34223 Innodb - add status variable for number of bulk inserts
- Added a counter innodb_num_bulk_insert_operation in
INFORMATION_SCHEMA.GLOBAL_STATUS. This counter is incremented
whenever a InnoDB undergoes bulk insert operation.

- Change the innodb_instant_alter_column to atomic variable.
2024-06-03 16:27:22 +05:30
Julius Goryavsky
c21aa486a8 MDEV-32633: additional post-merge changes for 10.5+ 2024-06-03 09:48:13 +02:00
Denis Protivensky
a4838721a2 MDEV-32633: Fix Galera cluster <-> native replication interaction
GTID events are applied without a running server transaction,
we need to set next transaction ID for Wsrep transaction.

The whole Galera cluster now has a single GTID value (including
the server ID throughout the cluster), fix the config accordingly.

Add force restart so that repeated MTR test execution prints
consistent GTID values, otherwise they would have been recovered
from the previous run.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-03 09:48:13 +02:00
Denis Protivensky
0cc9b49751 MDEV-32633: Fix Galera cluster <-> native replication interaction
It's possible to establish Galera multi-cluster setups connected
through the native replication when every Galera cluster is configured
to have a separate domain ID.
For this setup to work, we need to replace domain ID values in generated
GTID events when they are written at transaction commit to the values
configured by Wsrep replication.

At the same time, it's possible that the GTID event already contains
a correct domain ID if it comes through the native replication from
another Galera cluster.
In this case, when such an event is applied either through a native
replication slave thread or through Wsrep applier, we write GTID event
on transaction start and avoid writing it during transaction commit.

The code contained multiple problems that were fixed:
- applying GTID events didn't work because it's applied without a
running server transaction and Wsrep transaction was not started
- GTID event generation on transaction start didn't contain proper
"standalone" and "is_transactional" flags that the original applied
GTID event contained
- condition determining that GTID event is written on transaction start
to avoid writing it on commit relied on the fact that the GTID event
is the first found in transaction/statement caches, which wasn't the
case and resulted in duplicate GTID events written
- instead of relying on the caches to find a GTID event, a simple check
is introduced that follows the exact rules for checking if event is
written at transaction start as described above
- the test case is improved to check that exact GTID events are
applied after two Galera clusters have synced.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-03 09:48:13 +02:00
Denis Protivensky
a6b7203d65 MDEV-33952: Fix flaky galera_create_table_as_select test with debug sync
The test that triggers multi-master conflict between two CTAS commands
uses LOCK/UNLOCK TABLES to block local CTAS from progress. It could
result in a race when UNLOCK TABLES command is issued a bit earlier
then needed, causing local CTAS to run further and change wsrep
transaction state, so that a different code path is taken later and
the original error gets overridden, causing the test to fail.
The solution is to replace LOCK/UNLOCK TABLES with debug sync points.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2024-06-03 09:48:13 +02:00
Sergei Golubchik
d61947550a fix a typo 2024-05-31 19:37:51 +02:00
Yuchen Pei
22774ddb53
Merge branch '10.6' into 10.11 2024-05-31 09:13:40 +10:00
Yuchen Pei
f2302a62e3
Merge branch '10.5' into 10.6 2024-05-31 09:10:17 +10:00
Yuchen Pei
25476ba1ae
MDEV-29027 ASAN errors in spider_db_free_result after partition DDL
Spider calls ha_spider::close() at least twice on ALTER TABLE ... ADD
PARTITION. The first call frees wide_handler and the second call
accesses wide_handler->trx->thd (heap-use-after-free).

In general, there seems to be no problem with using THD obtained by
the macro current_thd() except in background threads. Thus, we simply
replace wide_handler->trx->thd with current_thd().

Original author: Nayuta Yanagasawa
2024-05-31 09:06:55 +10:00
Nayuta Yanagisawa
6d0c9872d9
MDEV-28522 Delete constant SPIDER_SQL_TYPE_*_HS
The HandlerSocket support of Spider has been deleted by MDEV-26858.
Thus, the constants, SPIDER_SQL_TYPE_*_HS, are no longer necessary.
2024-05-31 09:06:55 +10:00
Yuchen Pei
6c30220780
MDEV-26858 Spider: Remove dead code related to HandlerSocket
Remove the dead-code, in Spider, which is related to the Spider's
HandlerSocket support. The code has been disabled for a long time
and it is unlikely that the code will be enabled.

- rm all files under storage/spider/hs_client/ except hs_compat.h
- rm storage/spider/spd_db_handlersocket.*
- unifdef -UHS_HAS_SQLCOM -UHAVE_HANDLERSOCKET \
  -m storage/spider/spd_* storage/spider/ha_spider.* storage/spider/hs_client/*
- remove relevant files from storage/spider/CMakeLists.txt
2024-05-31 09:06:55 +10:00
Marko Mäkelä
22ba7e4ff8 Merge 10.6 into 10.11 2024-05-30 16:04:00 +03:00
Marko Mäkelä
5ba542e9ee Merge 10.5 into 10.6 2024-05-30 14:27:07 +03:00
Marko Mäkelä
0c440abd5e MDEV-31340 fixup: Add end-of-test marker 2024-05-30 14:23:45 +03:00
Marko Mäkelä
c71275b69e Fix ./mtr --repeat=2 main.func_str 2024-05-30 14:22:00 +03:00
Dave Gosselin
b0b463a894 MDEV-33616 Fix memleak in pfs_noop
Invoke cleanup routine at the end of pfs_noop.
2024-05-29 16:49:51 -04:00
Andrew Hutchings
1929a698a3 Update README for branch choice
This commit updates the README to indicate that the "Get the code, build
it, test it" link will help decide the correct branch to work in.

Also fixes a grammar issue and cleans-up the Markdown a little bit.
2024-05-29 13:49:32 +01:00
Brian White
a960e95feb don't require resolveip if it won't be used 2024-05-29 13:12:39 +02:00
Souradeep Saha
83a04be84a Fix Various Typos
Fix various typos, in comments and DEBUG statements, and code changes
are non-functional.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2024-05-28 11:31:49 +10:00
Sergei Petrunia
36ab6cc80c MDEV-34125: ANALYZE FORMAT=JSON: r_engine_stats.pages_read_time_ms has wrong scale
- Change the comments in class ha_handler_stats to say the members
  are in ticks, not milliseconds.
- In sql_explain.cc, adjust the scale to print milliseconds.
2024-05-27 15:28:57 +03:00
Alexander Barkov
4a158ec167 MDEV-34226 On startup: UBSAN: applying zero offset to null pointer in my_copy_fix_mb from strings/ctype-mb.c and other locations
nullptr+0 is an UB (undefined behavior).

- Fixing my_string_metadata_get_mb() to handle {nullptr,0} without UB.
- Fixing THD::copy_with_error() to disallow {nullptr,0} by DBUG_ASSERT().
- Fixing parse_client_handshake_packet() to call THD::copy_with_error()
  with an empty string {"",0} instead of NULL string {nullptr,0}.
2024-05-27 13:19:13 +04:00
Alexander Barkov
7925326183 MDEV-30931 UBSAN: negation of -X cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in get_interval_value on SELECT
- Fixing the code in get_interval_value() to use Longlong_hybrid_null.
  This allows to handle correctly:

  - Signed and unsigned arguments
    (the old code assumed the argument to be signed)
  - Avoid undefined negation behavior the corner case with LONGLONG_MIN

  This fixes the UBSAN warning:
    negation of -9223372036854775808 cannot be represented
    in type 'long long int';

- Fixing the code in get_interval_value() to avoid overflow in
  the INTERVAL_QUARTER and INTERVAL_WEEK branches.
  This fixes the UBSAN warning:
    signed integer overflow: -9223372036854775808 * 7 cannot be represented
    in type 'long long int'

- Fixing the INTERVAL_WEEK branch in date_add_interval() to handle
  huge numbers correctly. Before the change, huge positive numeber
  were treated as their negative complements.
  Note, some other branches still can be affected by this problem
  and should also be fixed eventually.
2024-05-27 13:19:13 +04:00
Thirunarayanan Balathandayuthapani
44b23bb184 MDEV-34222 Alter operation on redundant table aborts the server
- InnoDB page compression works only on COMPACT or DYNAMIC row
format tables. So InnoDB should throw error when alter table
tries to enable PAGE_COMPRESSED for redundant table.
2024-05-24 15:48:19 +05:30
Thirunarayanan Balathandayuthapani
0ffa340a49 MDEV-34221 Errors about checksum mismatch on crash recovery are confusing
- InnoDB should avoid printing the error message before
restoring the first page from doublewrite buffer.
2024-05-24 12:57:42 +05:30
Marko Mäkelä
82ba486e54 MDEV-4742 fixup: g++-14 -Wmaybe-uninitialized
to_natsort_key(): Zero-initialize also num_start. This silences a
compiler warning. There is no impact on correctness, because
before the first read of num_start, !n_digits would always hold
and hence num_start would have been initialized.
2024-05-24 10:06:11 +03:00
Vladislav Vaintroub
736449d30f MDEV-34205: ASAN stack buffer overflow in strxnmov() in frm_file_exists
Correct the second parameter for strxnmov to prevent potential buffer
overflows. The second parameter must be one less than the size of the
input buffer to avoid writing past the end of the buffer.

While the second parameter is usually correct, there are exceptions
that need fixing.

This commit addresses the issue within frm_file_exists() and other
affected places.
2024-05-23 22:08:27 +02:00
Alexander Barkov
7c4c082349 MDEV-28387 UBSAN: runtime error: negation of -9223372036854775808 cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in my_strtoll10 on SELECT
Fixing the condition to raise an overflow in the ulonglong
representation of the number is greater or equal to 0x8000000000000000ULL.
Before this change the condition did not catch -9223372036854775808
(the smallest possible signed negative longlong number).
2024-05-23 14:18:34 +04:00
Daniel Black
f146ba82c4 MDEV-34206 compile failure: fmt use incompatible with libfmt-10.2.[2]+ (7.1.3 compat)
Preserve compatibility with 7.1.3 by including the previous non-const
function.

The error was:

fmt/format.h:3466:8: note: candidate function template not
viable: no known conversion from 'const formatter<String, [2 * ...]>' to
'formatter<fmt::basic_string_view<char>, [2 * ...]>' for object argument
 3466 |   auto format(const T& val, FormatContext& ctx) ->
decltype(ctx.out()) {
2024-05-23 12:54:46 +10:00
Daniel Black
4375245d5d MDEV-34206 compile failure: fmt use incompatible with libfmt-10.2.[2]+
Upstream libfmt commit d70729215f
now requires the format function to be const.

Adjust the function prototype so it is const and can compile.
2024-05-23 12:54:46 +10:00
Thirunarayanan Balathandayuthapani
be0dfcdb99 MDEV-34200 InnoDB tries to write to read-only system tablespace
in buf_dblwr_t::init_or_load_pages()

- InnoDB fails to set the TRX_SYS_DOUBLEWRITE_SPACE_ID_STORED
flag in transaction system header page while recreating
the undo log tablespaces

buf_dblwr_t::init_or_load_pages(): Tries to reset the
space id and try to write into doublewrite buffer even
when read_only mode is enabled.

In srv_all_undo_tablespaces_open(), InnoDB should try to
open the extra unused undo tablespaces instead of trying to
creating it.
2024-05-22 13:16:10 +05:30
Yuchen Pei
c4020b541c
MDEV-24610 MEMORY SE: check overflow in info calls with HA_STATUS_AUTO 2024-05-22 09:18:09 +10:00
Marko Mäkelä
266495b93e MDEV-33817 fixup: Disable for macOS
According to https://discussions.apple.com/thread/8256853
an attempt to use AVX512 registers on macOS will result in #UD
(crash at runtime).

Also, starting with clang-18 and GCC 14, we must add "evex512" to the
target flags so that AVX and SSE instructions can use AVX512 specific
encodings. This flag was introduced together with the avx10.1-512 target.
Older compiler versions do not recognize "evex512". We do not want to
write "avx10.1-512" because it could enable some AVX512 subfeatures
that we do not have any CPUID check for.

Reviewed by: Vladislav Vaintroub
Tested on macOS by: Valerii Kravchuk
2024-05-21 16:45:07 +03:00
Alexander Barkov
310fd6ff69 Backporting bugs fixes fixed by MDEV-31340 from 11.5
The patch for MDEV-31340 fixed the following bugs:

MDEV-33084 LASTVAL(t1) and LASTVAL(T1) do not work well with lower-case-table-names=0
MDEV-33085 Tables T1 and t1 do not work well with ENGINE=CSV and lower-case-table-names=0
MDEV-33086 SHOW OPEN TABLES IN DB1 -- is case insensitive with lower-case-table-names=0
MDEV-33088 Cannot create triggers in the database `MYSQL`
MDEV-33103 LOCK TABLE t1 AS t2 -- alias is not case sensitive with lower-case-table-names=0
MDEV-33108 TABLE_STATISTICS and INDEX_STATISTICS are case insensitive with lower-case-table-names=0
MDEV-33109 DROP DATABASE MYSQL -- does not drop SP with lower-case-table-names=0
MDEV-33110 HANDLER commands are case insensitive with lower-case-table-names=0
MDEV-33119 User is case insensitive in INFORMATION_SCHEMA.VIEWS
MDEV-33120 System log table names are case insensitive with lower-cast-table-names=0

Backporting the fixes from 11.5 to 10.5
2024-05-21 14:58:01 +04:00
mariadb-DebarunBanerjee
b2944adb76 MDEV-34166 Server could hang with BP < 80M under stress
BUF_LRU_MIN_LEN (256) is too high value for low buffer pool(BP) size.
For example, for BP size lower than 80M and 16 K page size, the limit is
more than 5% of total BP and for lowest BP 5M, it is 80% of the BP.
Non-data objects like explicit locks could occupy part of the BP pool
reducing the pages available for LRU. If LRU reaches minimum limit and
if no free pages are available, server would hang with page cleaner not
able to free any more pages.

Fix: To avoid such hang, we adjust the LRU limit lower than the limit
for data objects as checked in buf_LRU_check_size_of_non_data_objects()
i.e. one page less than 5% of BP.
2024-05-21 14:13:29 +05:30
Marko Mäkelä
0907df3d89 MDEV-34204 Assertion `!*detailed_error' failed on shutdown after XA PREPARE
trx_free_at_shutdown(): Similar to trx_t::commit_in_memory(),
clear the detailed_error (FOREIGN KEY constraint error) before
invoking trx_t::free(). We only do this on debug instrumented
builds in order to avoid a debug assertion failure on shutdown.
2024-05-21 09:52:35 +03:00
mariadb-DebarunBanerjee
2d5cba22a9 MDEV-34167 We fail to terminate transaction early with ER_LOCK_TABLE_FULL when lock memory is growing
This regression is introduced in 10.6 by following commit.
commit b6a2472489
MDEV-27891: SIGSEGV in InnoDB buffer pool resize

During DML, we check if buffer pool is running out of data pages in
buf_pool_t::running_out. Here is 75% of the buffer pool is occupied by
non-data pages we rollback the current transaction and exit with
ER_LOCK_TABLE_FULL.

The integer division (n_chunks_new / 4) becomes zero whenever the total
number of chunks are < 4 making the check completely ineffective for
such cases. Also the check is inaccurate for larger chunks.

Fix-1: Correct the check in buf_pool_t::running_out.

Fix-2: While waiting for free page, check for
buf_LRU_check_size_of_non_data_objects.
2024-05-21 08:33:44 +05:30
Yuchen Pei
698dae54ef
MDEV-31475 Spider: only reset wide_handler when owning it
A wide_handler is shared among ha_spider of partitions of the same
spider table, where the last partition is designated the owner of the
wide_handler, and is responsible for its deallocation. Therefore in
case of failure, we only reset wide_handler in error handling if the
current ha_spider is the owner of the wide_handler, otherwise it will
result in segv in the destructor of ha_spider, or during
ha_spider::close().
2024-05-21 09:17:12 +10:00
Yuchen Pei
86adee3806
MDEV-31475 remove unnecessary assignment to spider share init_error
The init, init_error, and init_error_time fields of a SPIDER_SHARE
should only be assigned when actually doing the initialisation of a
SPIDER_SHARE, otherwise they could result in spurious failures from
spider_get_share() in a subsequent statement.
2024-05-21 09:17:12 +10:00