Commit graph

1,112 commits

Author SHA1 Message Date
Marko Mäkelä
9aca89fa9f MDEV-37619 The forced use of THD accessors hurts InnoDB performance
Let us access some data members of THD directly, instead of invoking
non-inline accessor functions. Note: my_thread_id will be used instead
of the potentially narrower ulong data type.

Also, let us remove some functions from sql_class.cc that were only
being used by InnoDB or RocksDB, for no reason. RocksDB always had
access to the internals of THD.

Reviewed by: Sergei Golubchik
Tested by: Saahil Alam
2025-09-16 15:18:32 +03:00
Daniel Black
4f1170d9db MDEV-37502: clang/debug/rocksdb - stack frame size
Follow up to MDEV-34388 82d7419e06

Relax limit on specific files only.

clang-20 + CMAKE_BUILD_TYPE=Debug:

options/cf_options.cc:0:0: stack frame size (17624) exceeds limit (16384) in function '__cxx_global_var_init.33'
options/db_options.cc:0:0: stack frame size (34328) exceeds limit (32768) in function '__cxx_global_var_init.45'

Reviewer: Jimmy Hu <jimmy.hu@mariadb.com>
2025-09-11 10:02:43 +10:00
Daniel Black
f4d203ca2c rocksdb: tests timeout under MSAN+Debug
But rocksdb.bulk_load_unsorted_rev and rocksdb.bulk_load_unsorted
succeed under non-debug builds, and because it was slow at 87 seconds)
there is a --big-test criteria for these tests.
2025-08-29 20:40:26 +10:00
Daniel Black
725874941d MDEV_37504 Rocks replace __unused__ attribute with nameless parameters
Per c++98.
2025-08-29 20:40:26 +10:00
Daniel Black
a0384c2f88 MDEV-37504 MemorySanitizer: use-of-uninitialized-value myrocks::Rdb_key_def::pack_field
m_charset_codec is uninitalized when calling m_make_unpack_info_func.

In the cases where m_make_unpack_info_func is one of:
* Rdb_key_def::make_unpack_unknown_varchar
* Rdb_key_def::make_unpack_unknown
* Rdb_key_def::dummy_make_unpack_info

the m_charset_coded that forms the first argument to this function
is unused.

In these limited cases we initialize the m_charset_codec member
as the only use is to pass though to the m_make_unpack_info_func

Ultimately MemorySanitizer shouldn't error on this as all
of these 3 functions clearly have the attribute
__unused__ on their first argument where the m_charset_coded is
passed.
2025-08-29 20:40:26 +10:00
Aleksey Midenkov
9a51709dba MDEV-29001 DROP DEFAULT makes SHOW CREATE non-idempotent
DROP DEFAULT adds DEFAULT NULL in case of nullable column. In case of
NOT NULL column it drops default expression if any exists.
2025-07-17 09:18:18 +02:00
Brad Smith
fadfd9ea28 MDEV-37001 Enable building RocksDB on non-Linux aarch64 OSes
Now that RocksDB has been synced up to 6.29 which includes the changes
mentioned in the CMake comment support for building on non-Linux aarch64
OSes can be enabled.
2025-06-13 15:39:30 +10:00
Brad Smith
6d684b64d5 Rocksdb: Add missing getauxval() test for Linux 2025-06-13 14:08:22 +10:00
Marko Mäkelä
1c7209e828 Merge 10.6 into 10.11 2025-05-21 07:36:35 +03:00
Marko Mäkelä
82d7419e06 MDEV-34388: Stack overflow on Alpine Linux
page_is_corrupted(): Do not allocate the buffers from stack,
but from the heap, in xb_fil_cur_open().

row_quiesce_write_cfg(): Issue one type of message when we
fail to create the .cfg file.

update_statistics_for_table(), read_statistics_for_table(),
delete_statistics_for_table(), rename_table_in_stat_tables():
Use a common stack buffer for Index_stat, Column_stat, Table_stat.

ha_connect::FileExists(): Invoke push_warning_printf() so that
we can avoid allocating a buffer for snprintf().

translog_init_with_table(): Do not duplicate TRANSLOG_PAGE_SIZE_BUFF.

Let us also globally enable the GCC 4.4 and clang 3.0 option
-Wframe-larger-than=16384 to reduce the possibility of introducing
such stack overflow in the future.  For RocksDB and Mroonga we relax
these limits.

Reviewed by: Vladislav Lesin
2025-05-20 17:27:05 +03:00
Sergei Petrunia
5033da6ed6 Fix rocksdb_sys_vars.rocksdb_stats_level_basic test
The minimum statistics level now is rocksdb::StatsLevel::kDisableAll.
The default remains rocksdb::StatsLevel::kExceptHistogramOrTimers
which is now 1 (it used to be 0).
2025-04-28 15:12:44 +03:00
Sergei Petrunia
1a044437a3 MDEV-16523 update RocksDB to 6.29fb 2025-04-23 11:12:56 +02:00
Sergei Golubchik
221aa5e08f Merge branch '10.6' into 10.11 2025-01-10 13:14:42 +01:00
Sergei Golubchik
311e88c67a fix rocksdb tests for buildbot 2025-01-10 13:13:53 +01:00
Sergei Golubchik
a0e5dd5433 mysqltest: fix --sorted_results
only sort actual results not warnings or metadata
also work for vertical results
warnings are sorted separately
2025-01-09 10:00:36 +01:00
Marko Mäkelä
420d9eb27f Merge 10.6 into 10.11 2025-01-08 12:51:26 +02:00
Monty
88d9348dfc Remove dates from all rdiff files 2025-01-05 16:40:11 +02:00
Marko Mäkelä
3d23adb766 Merge 10.6 into 10.11 2024-11-29 13:43:17 +02:00
Marko Mäkelä
7d4077cc11 Merge 10.5 into 10.6 2024-11-29 12:37:46 +02:00
Daniel Black
971a0ba23c MDEV-34408: Facilitate the addition of warnings into the build system
Create a MY_WARNING_FLAGS_NON_FATAL for testing warnings

Add -Weffc++ as no-error espect, disabling from rocksdb due
to excessive errors that will be corrected later.
2024-11-23 08:14:23 -07:00
Marko Mäkelä
63913ce5af Merge 10.6 into 10.11 2024-10-03 10:55:08 +03:00
Marko Mäkelä
7e0afb1c73 Merge 10.5 into 10.6 2024-10-03 09:31:39 +03:00
Lena Startseva
0a5e4a0191 MDEV-31005: Make working cursor-protocol
Updated tests: cases with bugs or which cannot be run
with the cursor-protocol were excluded with
"--disable_cursor_protocol"/"--enable_cursor_protocol"

Fix for v.10.5
2024-09-18 18:39:26 +07:00
Marko Mäkelä
27a3366663 Merge 10.6 into 10.11 2024-06-27 10:26:09 +03:00
Marko Mäkelä
0076eb3d4e Merge 10.5 into 10.6 2024-06-24 13:09:47 +03:00
Dave Gosselin
db0c28eff8 MDEV-33746 Supply missing override markings
Find and fix missing virtual override markings.  Updates cmake
maintainer flags to include -Wsuggest-override and
-Winconsistent-missing-override.
2024-06-20 11:32:13 -04:00
Sergei Golubchik
a6b2f820e0 Merge branch '10.6' into 10.11 2024-05-10 20:02:18 +02:00
Sergei Golubchik
7b53672c63 Merge branch '10.5' into 10.6 2024-05-08 20:06:00 +02:00
Sergei Petrunia
55754be20c MDEV-33781: rocksdb.locking_issues_case5_rc fails windows ... : Disable it 2024-05-06 13:34:39 +03:00
Sergei Petrunia
f90fcefdb2 MDEV-33866: rocksdb.write_sync fails on windows ... : Disable it 2024-05-06 13:31:31 +03:00
Sergei Petrunia
be60782103 MDEV-33789: rocksdb.bloomfilter2 failed on ... : Disable it. 2024-05-06 13:06:14 +03:00
Vladislav Vaintroub
029e2a5fd9 MDEV-33876 CMake, zlib - use names compatible with official FindZLIB.cmake
- ZLIB_LIBRARIES, not ZLIB_LIBRARY
- ZLIB_INCLUDE_DIRS, not ZLIB_INCLUDE_DIR

For building libmariadb, ZLIB_LIBRARY/ZLIB_INCLUDE_DIR are still defined
This workaround will be removed later.
2024-05-03 21:48:47 +02:00
Marko Mäkelä
788953463d Merge 10.6 into 10.11
Some fixes related to commit f838b2d799 and
Rows_log_event::do_apply_event() and Update_rows_log_event::do_exec_row()
for system-versioned tables were provided by Nikita Malyavin.
This was required by test versioning.rpl,trx_id,row.
2024-03-28 09:16:57 +02:00
Marko Mäkelä
fa8a46eb68 MDEV-33613 InnoDB may still hang when temporarily running out of buffer pool
By design, InnoDB has always hung when permanently running out of
buffer pool, for example when several threads are waiting to allocate
a block, and all of the buffer pool is buffer-fixed by the active threads.

The hang that we are fixing here occurs when the buffer pool is only
temporarily running out and the situation could be rescued by writing out
some dirty pages or evicting some clean pages.

buf_LRU_get_free_block(): Simplify the way how we wait for
the buf_flush_page_cleaner thread. This fixes occasional hangs
of the test encryption.innochecksum that were introduced by
commit a55b951e60 (MDEV-26827).
To play it safe, we use a timed wait when waiting for the
buf_flush_page_cleaner() thread to perform its job. Should that
thread get stuck, we will invoke buf_pool.LRU_warn() in order to
display a message that pages could not be freed, and keep trying
to wake up the buf_flush_page_cleaner() thread.

The INFORMATION_SCHEMA.INNODB_METRICS counters
buffer_LRU_single_flush_failure_count and
buffer_LRU_get_free_waits will be removed.
The latter is represented by buffer_pool_wait_free.

Also removed will be the message
"InnoDB: Difficult to find free blocks in the buffer pool"
because in d34479dc66 we
introduced a more precise message
"InnoDB: Could not free any blocks in the buffer pool"
in the buf_flush_page_cleaner thread.

buf_pool_t::LRU_warn(): Issue the warning message that we could
not free any blocks in the buffer pool. This may also be invoked
by buf_LRU_get_free_block() if buf_flush_page_cleaner() appears
to be stuck.

buf_pool_t::n_flush_dec(): Remove.

buf_pool_t::n_flush_dec_holding_mutex(): Rename to n_flush_dec().

buf_flush_LRU_list_batch(): Increment the eviction counter for blocks
of temporary, discarded or dropped tablespaces.

buf_flush_LRU(): Make static, and remove the constant parameter
evict=false. The only caller will be the buf_flush_page_cleaner()
thread.

IORequest::is_LRU(): Remove. The only case of evicting pages on
write completion will be when we are writing out pages of the
temporary tablespace. Those pages are not in buf_pool.flush_list,
only in buf_pool.LRU.

buf_page_t::flush(): Remove the parameter evict.

buf_page_t::write_complete(): Change the parameter "bool temporary"
to "bool persistent" and add a parameter for an already read state().

Reviewed by: Debarun Banerjee
2024-03-22 14:17:39 +02:00
Monty
b5d65fc105 Optimize performance of my_bitmap
MDEV-33502 Slowdown when running nested statement with many partitions

This change was triggered to help some MariaDB users with close to
10000 bits in their bitmaps.

- Change underlaying storage to be 64 bit instead of 32bit.
  - This reduses number of loops to scan bitmaps.
  - This can cause some bitmaps to be 4 byte large.
- Ensure that all not used top-bits are always 0 (simplifes code as
  the last 64 bit storage is not a special case anymore).
- Use my_find_first_bit() to find the first set bit which is much faster
  than scanning trough things byte by byte and then bit by bit.

Other things:
- Added a bool to remember if my_bitmap_init() did allocate the bitmap
  array. my_bitmap_free() will only free arrays it did allocate.
  This allowed me to remove setting 'bitmap=0' before calling
  my_bitmap_free() for cases where the bitmap's where allocated externally.
- my_bitmap_init() sets bitmap to 0 in case of failure.
- Added 'universal' asserts to most bitmap functions.
- Change all remaining calls to bitmap_init() to my_bitmap_init().
  - To finish the change from 2014.
- Changed all usage of uint32 in my_bitmap.h to my_bitmap_map.
- Updated bitmap_copy() to handle bitmaps of different size.
- Removed const from bitmap_exists_intersection() as this caused casts
  on all usage.
- Removed not used function bitmap_set_above().
- Renamed create_last_word_mask() to create_last_bit_mask() (to match
  name changes in my_bitmap.cc)
- Extended bitmap-t with test for more bitmap functions.
2024-02-27 14:51:33 +02:00
Oleg Smirnov
5b8493ba34 MDEV-15656 follow-up: fix rocksdb.group_min_max test 2024-02-23 21:29:21 +07:00
Marko Mäkelä
64cce8d5bf Merge 10.6 into 10.11 2024-02-14 16:12:53 +02:00
Marko Mäkelä
691f923906 Merge 10.5 into 10.6 2024-02-13 20:42:59 +02:00
Marko Mäkelä
8ec12e0d6d Merge 10.4 into 10.5 2024-02-12 11:38:13 +02:00
Oleksandr Byelkin
d816a5ca32 fix test 2024-02-09 10:26:46 +01:00
Oleksandr Byelkin
fecd78b837 Merge branch '10.10' into 10.11 2023-11-08 16:46:47 +01:00
Oleksandr Byelkin
04d9a46c41 Merge branch '10.6' into 10.10 2023-11-08 16:23:30 +01:00
Oleksandr Byelkin
b83c379420 Merge branch '10.5' into 10.6 2023-11-08 15:57:05 +01:00
Oleksandr Byelkin
6cfd2ba397 Merge branch '10.4' into 10.5 2023-11-08 12:59:00 +01:00
Marko Mäkelä
2ecc0443ec Merge 10.10 into 10.11 2023-10-17 16:04:21 +03:00
Marko Mäkelä
d5e15424d8 Merge 10.6 into 10.10
The MDEV-29693 conflict resolution is from Monty, as well as is
a bug fix where ANALYZE TABLE wrongly built histograms for
single-column PRIMARY KEY.
Also includes a fix for safe_malloc error reporting.

Other things:
- Copied main.log_slow from 10.4 to avoid mtr issue

Disabled test:
- spider/bugfix.mdev_27239 because we started to get
  +Error	1429 Unable to connect to foreign data source: localhost
  -Error	1158 Got an error reading communication packets
- main.delayed
  - Bug#54332 Deadlock with two connections doing LOCK TABLE+INSERT DELAYED
    This part is disabled for now as it fails randomly with different
    warnings/errors (no corruption).
2023-10-14 13:36:11 +03:00
Sergei Golubchik
3e2b1295bd MDEV-30658 fix failing test
followup for 96ae37abc5
2023-10-11 12:01:54 +02:00
Marko Mäkelä
625a150a86 Merge 10.5 into 10.6 2023-10-06 14:34:01 +03:00
Yuchen Pei
6b343de8ef
Merge branch '10.4' into 10.5 2023-09-25 13:06:57 +10:00
Daniel Black
cba102408c MDEV-29993: myrocks_hotbackup.1 and test suite files installed when engine is disabled 2023-09-20 08:18:57 +10:00