Commit graph

444 commits

Author SHA1 Message Date
Daniel Black
f278c7a3e7 MDEV-37502 mroonga + clang + Debug with exceeds stack frame size
Stack limits exceeded for clang-20 +Debug + MSAN.

Added disables for the following functions.

storage/mroonga/vendor/groonga/lib/ii.c:303:1: error: stack frame size (16696) exceeds limit (16384) in 'buffer_segment_reserve' [-Werror,-Wframe-larger-than]
 303 | buffer_segment_reserve(grn_ctx *ctx, grn_ii *ii,
     | ^
storage/mroonga/vendor/groonga/lib/ii.c:4803:1: error: stack frame size (20936) exceeds limit (16384) in 'grn_ii_delete_one' [-Werror,-Wframe-larger-than]
4803 | grn_ii_delete_one(grn_ctx *ctx, grn_ii *ii, grn_id tid, grn_ii_updspec *u, grn_hash *h)
     | ^
storage/mroonga/vendor/groonga/lib/ii.c:6313:1: error: stack frame size (25736) exceeds limit (16384) in 'grn_ii_column_update' [-Werror,-Wframe-larger-than]
6313 | grn_ii_column_update(grn_ctx *ctx, grn_ii *ii, grn_id rid, unsigned int section,
     | ^

For non-Debug the following stack frame sizes wher exceeded:

storage/mroonga/vendor/groonga/lib/proc/proc_select.c:3575:1: warning: stack frame size (94072) exceeds limit (49152) in 'command_select' [-Wframe-larger-than]
 3575 | command_select(grn_ctx *ctx, int nargs, grn_obj **args, grn_user_data *user_data)
      | ^
storage/mroonga/vendor/groonga/lib/proc/proc_schema.c:1134:1: warning: stack frame size (98360) exceeds limit (49152) in 'command_schema_output_tables' [-Wframe-larger-than]
 1134 | command_schema_output_tables(grn_ctx *ctx, grn_schema_data *data)

Reviewer: Jimmy Hu <jimmy.hu@mariadb.com>
2025-09-11 10:02:43 +10:00
Daniel Black
8ce0eee0e0 MDEV-37502: clang+debug mroonga remove -Wno-unused-but-set-variable
This is frequently violated within the mroonga implementatation and
therefore should not error.

Reviewer: Jimmy Hu <jimmy.hu@mariadb.com>
2025-09-11 10:02:43 +10:00
Monty
f65dda628d Fixed that one can compile MariaDB with ASAN with -Wframe-larger-than=16384
Added PRAGMA_DISABLE_CHECK_STACK_FRAME around some functions
2025-09-04 18:08:38 +03:00
Daniel Black
4f9221ae88 MDEV-36542: remove _lint macro which is unused
Attribute noreturn functions don't need
a return afterwards.

aria_pack was missing the noreturn attribute
on its my_end function.
2025-07-29 13:15:02 +10:00
Oleksandr Byelkin
28d6530571 Merge branch '10.6' into 10.11 2025-06-04 14:09:23 +02:00
Brad Smith
6409e43177 Replace deprecated CMAKE_COMPILER_IS_GNU(CC|CXX) with CMAKE_(C|CXX)_COMPILER_ID
As of CMake 3.24 CMAKE_COMPILER_IS_GNU(CC|CXX) are deprecated and should
be replaced with CMAKE_(C|CXX)_COMPILER_ID which were introduced with
CMake 2.6.
2025-06-03 18:16:35 +10:00
Daniel Black
88d35c5c51 MDEV-34388: Stack overflow on Alpine Linux (postfix) mroonga+asan
The following mroonga functions had approaching 64k stack frames, so
exclude these:

* chunk_merge - ~60k
* buffer_merge - ~78k
* grn_ii_update_one - ~60k
2025-05-28 17:07:53 +10:00
Marko Mäkelä
3da36fa130 Merge 10.6 into 10.11 2025-05-26 08:10:47 +03:00
Daniel Black
2c4fe3557a MDEV-36337: mroonga_* udf correct ptr types for is_null/error
Shows up in mroonga UDF tests under clang with UBSAN:

UndefinedBehaviorSanitizer: function-type-mismatch

Accepted upstream: https://github.com/mroonga/mroonga/pull/902
2025-05-21 09:47:55 +02:00
Marko Mäkelä
1c7209e828 Merge 10.6 into 10.11 2025-05-21 07:36:35 +03:00
Marko Mäkelä
82d7419e06 MDEV-34388: Stack overflow on Alpine Linux
page_is_corrupted(): Do not allocate the buffers from stack,
but from the heap, in xb_fil_cur_open().

row_quiesce_write_cfg(): Issue one type of message when we
fail to create the .cfg file.

update_statistics_for_table(), read_statistics_for_table(),
delete_statistics_for_table(), rename_table_in_stat_tables():
Use a common stack buffer for Index_stat, Column_stat, Table_stat.

ha_connect::FileExists(): Invoke push_warning_printf() so that
we can avoid allocating a buffer for snprintf().

translog_init_with_table(): Do not duplicate TRANSLOG_PAGE_SIZE_BUFF.

Let us also globally enable the GCC 4.4 and clang 3.0 option
-Wframe-larger-than=16384 to reduce the possibility of introducing
such stack overflow in the future.  For RocksDB and Mroonga we relax
these limits.

Reviewed by: Vladislav Lesin
2025-05-20 17:27:05 +03:00
Thirunarayanan Balathandayuthapani
f388222d49 MDEV-36504 Memory leak after CREATE TABLE..SELECT
Problem:
========
- After commit cc8eefb0dc (MDEV-33087),
InnoDB does use bulk insert operation for ALTER TABLE.. ALGORITHM=COPY
and CREATE TABLE..SELECT as well. InnoDB fails to clear the bulk
buffer when it encounters error during CREATE..SELECT. Problem
is that while transaction cleanup, InnoDB fails to identify
the bulk insert for DDL operation.

Fix:
====
- Represent bulk_insert in trx by 2 bits. By doing that, InnoDB
can distinguish between TRX_DML_BULK, TRX_DDL_BULK. During DDL,
set bulk insert value for transaction to TRX_DDL_BULK.

- Introduce a parameter HA_EXTRA_ABORT_ALTER_COPY which rollbacks
only TRX_DDL_BULK transaction.

- bulk_insert_apply() happens for TRX_DDL_BULK transaction happens
only during HA_EXTRA_END_ALTER_COPY extra() call.
2025-04-17 12:04:09 +05:30
Julius Goryavsky
1a013cea95 Merge branch '10.6' into '10.11' 2025-04-16 03:34:40 +02:00
Julius Goryavsky
88dfa6bcee Merge branch '10.5' into '10.6' 2025-04-15 01:49:48 +02:00
Nikita Malyavin
2a5a12b227 MDEV-36506 Build fails with cmake 4.0
Update cmake_minimum_required to 2.8...3.12 in root cmake and mroonga.

This will update "Policy Version" to 3.12, which will not prevent the
build by even higher cmake versions. There is also a reason to stay on
the compatible with windows "policy version", so 3.12 is conservatively
chosen.

On the other hand, it will require at least version 2.8.
2025-04-10 13:05:19 +02:00
Daniel Black
b316a7135b mroonga: undefined behaviour fix
The value of dv[0].data being null showed up
in the mtr tests:

mroonga/storage.alter_table_fulltext_add_no_primary_key

as:

/source/storage/mroonga/vendor/groonga/lib/ii.c:2052:37: runtime error: applying non-zero offset 28 to null pointer

Correct this by entrying the if condition on null pointer value.

The free is valid, and the data of size is allocated.
2025-04-07 11:04:53 +02:00
Daniel Black
2ba49bd804 mroonga: correct offsetof calculation
Using a null pointer as a basic for calculation
was undefined behaviour.
2025-04-07 11:04:53 +02:00
Marko Mäkelä
a54d151fc1 Merge 10.6 into 10.11 2024-12-19 15:38:53 +02:00
Marko Mäkelä
69e20cab28 Merge 10.5 into 10.6 2024-12-11 14:46:43 +02:00
Daniel Black
bf7cfa2535 MDEV-35574 remove obsolete pthread_exit calls
Threads can normally exit without a explicit pthread_exit call.

There seem to date to old glibc bugs, many around 2.2.5.

The semi related bug was https://bugs.mysql.com/bug.php?id=82886.

To improve safety in the signal handlers DBUG_* code was removed.

These where also needed to avoid some MSAN unresolved stack issues.

This is effectively a backport of 2719cc4925.
2024-12-10 12:12:20 +11:00
Marko Mäkelä
3d23adb766 Merge 10.6 into 10.11 2024-11-29 13:43:17 +02:00
Marko Mäkelä
7d4077cc11 Merge 10.5 into 10.6 2024-11-29 12:37:46 +02:00
Brandon Nesterenko
840fe316d4 MDEV-34348: my_hash_get_key fixes
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict

Change the type of my_hash_get_key to:
 1) Return const
 2) Change the context parameter to be const void*

Also fix casting in hash adjacent areas.

Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
2024-11-23 08:14:22 -07:00
ParadoxV5
cf2d49ddcf Extract some of #3360 fixes to 10.5.x
That PR uncovered countless issues on `my_snprintf` uses.
This commit backports a squashed subset of their fixes.
2024-11-21 22:43:56 +11:00
Marko Mäkelä
63913ce5af Merge 10.6 into 10.11 2024-10-03 10:55:08 +03:00
Marko Mäkelä
7e0afb1c73 Merge 10.5 into 10.6 2024-10-03 09:31:39 +03:00
Daniel Black
1f7898f686 mroonga: remove -Wunused-but-set-variable warnings
There where unused variable. They were not conditional
on defines, so removed them.

Added an error handing in proc_object if there was no db
as subsequent operations would have failed.
2024-10-03 15:05:09 +10:00
Daniel Black
3723fd1573 MDEV-35007 mroonga should modify source files during build
CMake rewriting the tests causes Mroonga to be un-buildable
on build environments where there source directory is read
only.

In the test results, the version wasn't particularly important.

Remove the version dependence of tests.
2024-10-03 15:05:09 +10:00
Max Kellermann
45298b730b sql/handler: referenced_by_foreign_key() returns bool
The method was declared to return an unsigned integer, but it is
really a boolean (and used as such by all callers).

A secondary change is the addition of "const" and "noexcept" to this
method.

In ha_mroonga.cpp, I also added "inline" to the two helper methods of
referenced_by_foreign_key().  This allows the compiler to flatten the
method.
2024-09-30 16:33:25 +03:00
Lena Startseva
0a5e4a0191 MDEV-31005: Make working cursor-protocol
Updated tests: cases with bugs or which cannot be run
with the cursor-protocol were excluded with
"--disable_cursor_protocol"/"--enable_cursor_protocol"

Fix for v.10.5
2024-09-18 18:39:26 +07:00
Oleksandr Byelkin
0fe39d368a Merge branch '10.6' into 10.11 2024-07-22 15:14:50 +02:00
Oleksandr Byelkin
9af2caca33 Merge branch '10.5' into 10.6 2024-07-18 16:25:33 +02:00
Sutou Kouhei
383d53edbc MDEV-21166 Add Mroonga initialized check to Mroonga UDFs
Mroonga UDFs can't be used without loading Mroonga.
2024-07-17 15:52:58 +10:00
Marko Mäkelä
27a3366663 Merge 10.6 into 10.11 2024-06-27 10:26:09 +03:00
Yuchen Pei
c9212b7a43
MDEV-33746 [fixup] Add suggested overrides in mroonga 2024-06-26 11:08:56 +08:00
Marko Mäkelä
0076eb3d4e Merge 10.5 into 10.6 2024-06-24 13:09:47 +03:00
Dave Gosselin
db0c28eff8 MDEV-33746 Supply missing override markings
Find and fix missing virtual override markings.  Updates cmake
maintainer flags to include -Wsuggest-override and
-Winconsistent-missing-override.
2024-06-20 11:32:13 -04:00
Marko Mäkelä
b81d717387 Merge 10.6 into 10.11 2024-06-11 12:50:10 +03:00
Marko Mäkelä
a687cf8661 Merge 10.5 into 10.6 2024-06-07 10:03:51 +03:00
Vladislav Vaintroub
40abd973ab MDEV-34236 Mroonga build with ASAN/UBSAN with GCC 12+ extremely slow.
Workaround by disabling sanitizer for single source file.
2024-06-05 11:58:53 +02:00
Sergei Golubchik
a6b2f820e0 Merge branch '10.6' into 10.11 2024-05-10 20:02:18 +02:00
Sergei Golubchik
7b53672c63 Merge branch '10.5' into 10.6 2024-05-08 20:06:00 +02:00
Sergei Golubchik
22b3ba9312 MDEV-25102 UNIQUE USING HASH error after ALTER ... DISABLE KEYS
on disable_indexes(HA_KEY_SWITCH_NONUNIQ_SAVE) the engine does
not know that the long unique is logically unique, because on the
engine level it is not. And the engine disables it,

Change the disable_indexes/enable_indexes API. Instead of the enum
mode, send a key_map of indexes that should be enabled. This way the
server will decide what is unique, not the engine.
2024-05-06 17:16:10 +02:00
Oleksandr Byelkin
c9b1ebee2f Merge branch '10.6' into 10.11 2024-04-26 08:02:49 +02:00
Monty
0ccdf54b64 Check and remove high stack usage
I checked all stack overflow potential problems found with
gcc -Wstack-usage=16384
and
clang -Wframe-larger-than=16384 -no-inline

Fixes:
Added '#pragma clang diagnostic ignored "-Wframe-larger-than="'
  to a lot of function to where stack usage large but resonable.
- Added stack check warnings to BUILD scrips when using clang and debug.

Function changed to use malloc instead allocating things on stack:
- read_bootstrap_query() now allocates line_buffer (20000 bytes) with
  malloc() instead of using stack. This has a small performance impact
  but this is not releant for bootstrap.
- mroonga grn_select() used 65856 bytes on stack. Changed it to use
  malloc().
- Wsrep_schema::replay_transaction() and
  Wsrep_schema::recover_sr_transactions().
- Connect zipOpen3()

Not fixed:
- mroonga/vendor/groonga/lib/expr.c grn_proc_call() uses
  43712 byte on stack.  However this is not easy to fix as the stack
  used is caused by a lot of code generated by defines.
- Most changes in mroonga/groonga where only adding of pragmas to disable
  stack warnings.
- rocksdb/options/options_helper.cc uses 20288 of stack space.
  (no reason to fix except to get rid of the compiler warning)
- Causes using alloca() where the allocation size is resonable.
- An issue in libmariadb (reported to connectors).
2024-04-23 14:12:31 +03:00
Marko Mäkelä
788953463d Merge 10.6 into 10.11
Some fixes related to commit f838b2d799 and
Rows_log_event::do_apply_event() and Update_rows_log_event::do_exec_row()
for system-versioned tables were provided by Nikita Malyavin.
This was required by test versioning.rpl,trx_id,row.
2024-03-28 09:16:57 +02:00
Marko Mäkelä
ccb7a1e9a1 Merge 10.5 into 10.6 2024-03-27 15:00:56 +02:00
Vladislav Vaintroub
7d36919f4b MDEV-33723 Mroonga ignored WITHOUT_DYNAMIC_PLUGINS
Make WITHOUT_DYNAMIC_PLUGINS ignore mrooonga also in its own DIY version
of MYSQL_ADD_PLUGIN
2024-03-19 13:12:56 +01:00
Monty
b5d65fc105 Optimize performance of my_bitmap
MDEV-33502 Slowdown when running nested statement with many partitions

This change was triggered to help some MariaDB users with close to
10000 bits in their bitmaps.

- Change underlaying storage to be 64 bit instead of 32bit.
  - This reduses number of loops to scan bitmaps.
  - This can cause some bitmaps to be 4 byte large.
- Ensure that all not used top-bits are always 0 (simplifes code as
  the last 64 bit storage is not a special case anymore).
- Use my_find_first_bit() to find the first set bit which is much faster
  than scanning trough things byte by byte and then bit by bit.

Other things:
- Added a bool to remember if my_bitmap_init() did allocate the bitmap
  array. my_bitmap_free() will only free arrays it did allocate.
  This allowed me to remove setting 'bitmap=0' before calling
  my_bitmap_free() for cases where the bitmap's where allocated externally.
- my_bitmap_init() sets bitmap to 0 in case of failure.
- Added 'universal' asserts to most bitmap functions.
- Change all remaining calls to bitmap_init() to my_bitmap_init().
  - To finish the change from 2014.
- Changed all usage of uint32 in my_bitmap.h to my_bitmap_map.
- Updated bitmap_copy() to handle bitmaps of different size.
- Removed const from bitmap_exists_intersection() as this caused casts
  on all usage.
- Removed not used function bitmap_set_above().
- Renamed create_last_word_mask() to create_last_bit_mask() (to match
  name changes in my_bitmap.cc)
- Extended bitmap-t with test for more bitmap functions.
2024-02-27 14:51:33 +02:00
Oleksandr Byelkin
04d9a46c41 Merge branch '10.6' into 10.10 2023-11-08 16:23:30 +01:00