MDEV-21978 introduces more preferrable alternatives to those extensions.
This commit removes the above old and deprecated syntax.
With the code for the old extension formats gone,
this commit also improves the code around the new extension suffixes:
* Remove code for formatting ``%`T``
* Those code are now dead,
for the new suffix-based syntax does not recognize `%sQT`/`%sTQ`.
* `suffix_q= TRUE` now additionally replaces `…|= ESCAPED_ARG`.
* Flatten `flag= true if /%iE/` and `do_iE if flag` code together
[Breaking] This commit removes obsolete features. Although my earlier
work (should have) migrated every usages direct or indirect in the
entire MariaDB/server, other codebases might still be using them. This
final change will break *everything* in those outdated foreign lands.
… and delete `%uU` (just use `%d` for that)
The follow-up #3360 discovered `%M` usages that suggest that it was
designed for `errno` and similar **signed** `int` (“errno_t”) variables.
Besides convenience, if the old `%M` read a `signed int`,
so should the new version to maintain compatibility.
I only added `%iE` (no `%dE`) to keep the new suffix mechanics
away from the popular `%d`. Beïng synonyms (originally),
this decision on preserving `%d` also saves the need for `%iI`/`%dD`.
This is the first part of
MDEV-21978 make my_vsnprintf to use gcc-compatible format extensions,
which adds these alternatives to the MySQL extensions.
There’s also the escapes `%sS` & `%uU` for any hippies needing them.
These suffixes are compatible with the C standard and
therefore as well as `printf` tools such as GCC checks.
The old extension formats (e.g., `%M`) are now effectively deprecated,
although they’re left intact for now. For a more sequential
MDEV-21978 process, a separate commit will delete them after we migrate
all `my_vsnprintf` usages to the new preferred syntax. The service’s
major version bumped nonetheless for the new significance of suffixes.
[Breaking] This commit may fail
* on places needing the aforementioned escapes
* because of the major version bump
Reviewed-by: Andrew Hutchings <andrew@mariadb.org>
Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
For InnoDB tables, parse the CREATE TABLE statement to defer index and
constraint creation until after data loading. For other storage engines,
the DISABLE/ENABLE KEYS commands achieve similar optimization.
This behavior is controlled by a new option, innodb-optimize-keys
(default: ON), compatible with mydumper.
Additionally, this commit separates the table creation phase from data
loading. Running DDL statements (such as DROP IF EXISTS) in a single
thread avoids the "table not locked" issue from MDEV-34741. As a bonus,
view creation no longer requires a separate step.
This demonstrates and tests the basic usage of DYNAMIC_ARRAY.
Additional tests could be added to demonstrate and test the
remaining functions and capabilities.
Signed-off-by: Eric Herman <eric@freesa.org>
The ok() function outputs the TAP context string in both the success
and failure cases. The strings were worded to make sense in the
failure case, yet were confusing in the success case. This changes
the strings to be appropriate, and even more informative in either
the success or failure cases. In the cases where input, expected,
and result values are included in the context string, they have been
adjusted such that expected and actual result values are aligned for
easy visual comparison.
Signed-off-by: Eric Herman <eric@freesa.org>
Threads can normally exit without a explicit pthread_exit call.
There seem to date to old glibc bugs, many around 2.2.5.
The semi related bug was https://bugs.mysql.com/bug.php?id=82886.
To improve safety in the signal handlers DBUG_* code was removed.
These where also needed to avoid some MSAN unresolved stack issues.
This is effectively a backport of 2719cc4925.
These tests rely on THR_KEY_mysys but it is not initialized. On
Linux, the corresponding thread variable is null, but on macOS it has a
nonzero value. In all cases, initialize the variable explicitly by
calling MY_INIT and my_end appropriately.
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
The functions queue_compare, qsort2_cmp, and qsort_cmp2
all had similar interfaces, and were used interchangable
and unsafely cast to one another.
This patch consolidates the functions all into the
qsort_cmp2 interface.
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
The problem was that when using clang + asan, we do not get a correct value
for the thread stack as some local variables are not allocated at the
normal stack.
It looks like that for example clang 18.1.3, when compiling with
-O2 -fsanitize=addressan it puts local variables and things allocated by
alloca() in other areas than on the stack.
The following code shows the issue
Thread 6 "mariadbd" hit Breakpoint 3, do_handle_one_connection
(connect=0x5080000027b8,
put_in_cache=<optimized out>) at sql/sql_connect.cc:1399
THD *thd;
1399 thd->thread_stack= (char*) &thd;
(gdb) p &thd
(THD **) 0x7fffedee7060
(gdb) p $sp
(void *) 0x7fffef4e7bc0
The address of thd is 24M away from the stack pointer
(gdb) info reg
...
rsp 0x7fffef4e7bc0 0x7fffef4e7bc0
...
r13 0x7fffedee7060 140737185214560
r13 is pointing to the address of the thd. Probably some kind of
"local stack" used by the sanitizer
I have verified this with gdb on a recursive call that calls alloca()
in a loop. In this case all objects was stored in a local heap,
not on the stack.
To solve this issue in a portable way, I have added two functions:
my_get_stack_pointer() returns the address of the current stack pointer.
The code is using asm instructions for intel 32/64 bit, powerpc,
arm 32/64 bit and sparc 32/64 bit.
Supported compilers are gcc, clang and MSVC.
For MSVC 64 bit we are using _AddressOfReturnAddress()
As a fallback for other compilers/arch we use the address of a local
variable.
my_get_stack_bounds() that will return the address of the base stack
and stack size using pthread_attr_getstack() or NtCurrentTed() with
fallback to using the address of a local variable and user provided
stack size.
Server changes are:
- Moving setting of thread_stack to THD::store_globals() using
my_get_stack_bounds().
- Removing setting of thd->thread_stack, except in functions that
allocates a lot on the stack before calling store_globals(). When
using estimates for stack start, we reduce stack_size with
MY_STACK_SAFE_MARGIN (8192) to take into account the stack used
before calling store_globals().
I also added a unittest, stack_allocation-t, to verify the new code.
Reviewed-by: Sergei Golubchik <serg@mariadb.org>
my_b_encr_write(): Initialize also block_length, and at the same time
last_block_length, so that all 128 bits can be initialized with fewer
writes. This fixes an error that was caught in the test
encryption.tempfiles_encrypted.
test_my_safe_print_str(): Skip a test that would attempt to
display uninitialized data in the test unit.stacktrace.
Previously, our CI did not build unit tests with MemorySanitizer.
handle_delayed_insert(): Remove a redundant call to pthread_exit(0),
which would for some reason cause MemorySanitizer in clang-19 to
report a stack overflow in a RelWithDebInfo build. This fixes a
failure of several tests.
Reviewed by: Vladislav Vaintroub
Without the call to my_mutex_init, the mutex attributes
my_fast_mutexattr and my_errorcheck_mutexattr are uninitialized.
Linux tolerates this but FreeBSD doesn't (and segfaults).
We fix for all since the unit text should be testing the
standard mutexes of the system.
Cleanup unnecessary whitespace at the end of lines and end of files
in the unittest/ directory. Note that all code changes are
non-functional.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
- ZLIB_LIBRARIES, not ZLIB_LIBRARY
- ZLIB_INCLUDE_DIRS, not ZLIB_INCLUDE_DIR
For building libmariadb, ZLIB_LIBRARY/ZLIB_INCLUDE_DIR are still defined
This workaround will be removed later.
In our unit test, let us rely on our own reference
implementation using the reflected
CRC-32 ISO 3309 and CRC-32C polynomials. Let us also
test with various lengths.
Let us refactor the CRC-32 and CRC-32C implementations
so that no special compilation flags will be needed and
that some function call indirection will be avoided.
pmull_supported: Remove. We will have pointers to two separate
functions crc32c_aarch64_pmull() and crc32c_aarch64().
I checked all stack overflow potential problems found with
gcc -Wstack-usage=16384
and
clang -Wframe-larger-than=16384 -no-inline
Fixes:
Added '#pragma clang diagnostic ignored "-Wframe-larger-than="'
to a lot of function to where stack usage large but resonable.
- Added stack check warnings to BUILD scrips when using clang and debug.
Function changed to use malloc instead allocating things on stack:
- read_bootstrap_query() now allocates line_buffer (20000 bytes) with
malloc() instead of using stack. This has a small performance impact
but this is not releant for bootstrap.
- mroonga grn_select() used 65856 bytes on stack. Changed it to use
malloc().
- Wsrep_schema::replay_transaction() and
Wsrep_schema::recover_sr_transactions().
- Connect zipOpen3()
Not fixed:
- mroonga/vendor/groonga/lib/expr.c grn_proc_call() uses
43712 byte on stack. However this is not easy to fix as the stack
used is caused by a lot of code generated by defines.
- Most changes in mroonga/groonga where only adding of pragmas to disable
stack warnings.
- rocksdb/options/options_helper.cc uses 20288 of stack space.
(no reason to fix except to get rid of the compiler warning)
- Causes using alloca() where the allocation size is resonable.
- An issue in libmariadb (reported to connectors).
