Commit graph

196245 commits

Author SHA1 Message Date
Dave Gosselin
89084c2ea4 MDEV-33078 SysInfo.pm reports incorrect CPU count on macOS
When running on macOS, MTR will ask the operating system for the core count when --parallel=auto
2024-05-07 09:08:57 +10:00
Sergei Golubchik
7ed9d2ac00 MDEV-9179 When binlog_annotate_row_events on , event of binlog file is truncated
cnt counter was incremented one extra time per line
2024-05-06 20:14:37 +02:00
Sergei Golubchik
7eeba92520 Merge branch '10.4' into 10.5 2024-05-06 17:17:18 +02:00
Sergei Golubchik
13663cb5c4 MDEV-33727 mariadb-dump trusts the server and does not validate the data
safety first - tell mariadb client not to execute dangerous
cli commands, they cannot be present in the dump anyway.

wrapping the command in /*!999999 ..... */ guarantees that
if a non-mariadb-cli client loads the dump and sends it to the
server - the server will ignore the command it doesn't understand
2024-05-06 17:16:10 +02:00
Sergei Golubchik
2025597c0b MDEV-21778 Disable system commands in mysql/mariadb client
mysql --sandbox

disables system (\!), tee (\T), pager with an argument(\P foo), source (\.)

does *not* disable edit (\e). Use EDITOR=/bin/false to disable
or, for example, EDITOR=rnano for something more useful

does *not* disable pager (\P) without an argument. Use
PAGER=cat or, for example PAGER=less LESSSECURE=1 for something
more useful

using a disabled command is an error, which can be ignored with --force

Also, a "sandbox" command (\-) - enables the sandbox mode until EOF
(current file or the session, if interactive)
2024-05-06 17:16:10 +02:00
Sergei Golubchik
83aedeacc4 cleanup: remove __attribute__((unused)) from mysql.cc
the client is C++, use a much more concise C++ syntax

as a bonus, arguments that are used, are no longer marked "unused"
2024-05-06 17:16:10 +02:00
Sergei Golubchik
994a0de96b cleanup: reorder mysql help output 2024-05-06 17:16:10 +02:00
Sergei Golubchik
98b157fdee cleanup: unused OPT_xxx and client_priv.h 2024-05-06 17:16:10 +02:00
Sergei Golubchik
22b3ba9312 MDEV-25102 UNIQUE USING HASH error after ALTER ... DISABLE KEYS
on disable_indexes(HA_KEY_SWITCH_NONUNIQ_SAVE) the engine does
not know that the long unique is logically unique, because on the
engine level it is not. And the engine disables it,

Change the disable_indexes/enable_indexes API. Instead of the enum
mode, send a key_map of indexes that should be enabled. This way the
server will decide what is unique, not the engine.
2024-05-06 17:16:10 +02:00
Thirunarayanan Balathandayuthapani
9b2bf09b95 MDEV-33980 mariadb-backup --backup is missing retry logic for undo tablespaces
- This is a merge of commit f378e76434
from 10.4 to 10.5.
2024-05-06 19:50:20 +05:30
Sergei Golubchik
16394f1aa1 update C/C 3.1 2024-05-06 16:14:11 +02:00
Julius Goryavsky
b88c20ce1b Merge branch 10.4 into 10.5 2024-05-06 13:55:42 +02:00
Sergei Petrunia
55754be20c MDEV-33781: rocksdb.locking_issues_case5_rc fails windows ... : Disable it 2024-05-06 13:34:39 +03:00
Sergei Petrunia
f90fcefdb2 MDEV-33866: rocksdb.write_sync fails on windows ... : Disable it 2024-05-06 13:31:31 +03:00
Sergei Petrunia
be60782103 MDEV-33789: rocksdb.bloomfilter2 failed on ... : Disable it. 2024-05-06 13:06:14 +03:00
Julius Goryavsky
52c45332a8 MDEV-34071: Failure during the galera_3nodes_sr.GCF-336 test
This commit fixes sporadic failures in galera_3nodes_sr.GCF-336
test. The following changes have been made here:

1) A small addition to the test itself which should make
   it more deterministic by waiting for non-primary state
   before COMMIT;
2) More careful handling of the wsrep_ready variable in
   the server code (it should always be protected with mutex).

No additional tests are required.
2024-05-06 03:16:59 +02:00
Sergei Golubchik
4045a87bcb test for Bug #16051817 GOT ERROR 124 FROM STORAGE ENGINE ON DELETE FROM A PARTITIONED TABLE
followup for c5896384bd
2024-05-05 21:37:08 +02:00
Sergei Golubchik
3d75cffa91 bugfix: INFORMATION_SCHEMA.STATISTICS doesn't show whether the index is disabled 2024-05-05 21:37:08 +02:00
Sergei Golubchik
179515a68c bugfix: dgcov.pl shows wrong line numbers
dgcov.pl was putting gcov execution counters on wrong lines
in the report (*.dgcov files were correct), because it was
incrementing the new line number for diff lines starting from '-'
(lines from the old file, not present in the new)
2024-05-05 21:37:08 +02:00
Sergei Golubchik
4f5dea43df cleanup
* remove dead code
* simplify the check for table->s->next_number_index
* misc
2024-05-05 21:37:08 +02:00
Sergei Golubchik
947eeaa6dc MDEV-29345 update case insensitive (large) unique key with insensitive change of value - duplicate key
use collation-sensitive comparison when comparing fields
2024-05-05 21:37:08 +02:00
Sergei Golubchik
91fb8b7fd3 MDEV-19949 mariabackup option of '--password' or '-p' without specifying password in commandline
password handling as in other command-line tools
2024-05-05 21:37:08 +02:00
Monty
cd0356a764 MDEV-34077 scripts/mariadb-install-db: Error in my_thread_global_end(): 1 threads didn't exit
followup for 952ab9a596

chery-pick of 814dc46748 into 10.5
2024-05-05 21:37:08 +02:00
Sergei Golubchik
983e6ca097 bugfix: buffer overwrite in mariadb-backup
this fixes galera.galera_sst_mariabackup_table_options

Note that `man snprintf` says

  The functions snprintf() and vsnprintf() do not write more
  than size bytes (including the terminating null byte
  ('\0')). If the output was truncated due to this limit, then
  the return value is the number of characters (excluding the
  terminating null byte) which would have been written to the
  final string if enough space had been available.
2024-05-05 21:37:08 +02:00
Sergei Golubchik
349ca2be74 mtr: remove innodb combinations
dead code for about 10 years
2024-05-05 21:37:08 +02:00
Sergei Golubchik
df6899b30b bugfix: mysqld --safe-mode crashes 2024-05-05 21:37:08 +02:00
Sergei Golubchik
d74fee9e8c MDEV-19487 fix for --view
followup for d7df63e1c9
2024-05-05 21:37:08 +02:00
Sergei Golubchik
03295f0c20 MDEV-30727 Check spider_hton_ptr in spider udfs
UDF isn't supposed to use my_error(), it should return
the error message in the provided error message buffer

Fixes valgrind:

==93993== Conditional jump or move depends on uninitialised value(s)
==93993==    at 0x484ECCD: strnlen (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==93993==    by 0x1AD2B2C: process_str_arg (my_vsnprintf.c:259)
==93993==    by 0x1AD47E0: my_vsnprintf_ex (my_vsnprintf.c:696)
==93993==    by 0x1A3B91E: my_error (my_error.c:120)
==93993==    by 0xF87BE8: udf_handler::fix_fields(THD*, Item_func_or_sum*, unsigned int, Item**) (item_func.cc:3638)

followup for 267dd5a993
2024-05-05 21:37:07 +02:00
Sergei Golubchik
7a789e2027 sporadic failures of rpl.rpl_parallel_sbm
the test waits for the event to get stuck on MASTER_DELAY,
but on a slow/overloaded slave the event might pass MASTER_DELAY
before the test starts waiting.

Wait for the event to get stuck on the LOCK TABLES (after MASTER_DELAY),
the event cannot avoid that,
2024-05-05 21:37:07 +02:00
Sergei Golubchik
cea083af9f cleanup: use THD_STAGE_INFO, not thd_proc_info
and put master-slave.inc *last* in the series of includes
2024-05-05 21:37:07 +02:00
Sergei Golubchik
cb7c99674e sporadic failure of perfschema.func_file_io
--- func_file_io.result
+++ func_file_io.reject
@@ -134,7 +134,7 @@
 Variable_name	Value
 Performance_schema_accounts_lost	0
 Performance_schema_cond_classes_lost	0
-Performance_schema_cond_instances_lost	0
+Performance_schema_cond_instances_lost	5
 Performance_schema_digest_lost	0
 Performance_schema_file_classes_lost	0
 Performance_schema_file_handles_lost	0
2024-05-05 21:37:07 +02:00
Nikita Malyavin
72429cad7f MDEV-30046 wrong row targeted with "insert ... on duplicate" and "replace"
When HA_DUPLICATE_POS is not supported, the row to replace was navigated by
ha_index_read_idx_map, which uses only hash to navigate.

Suchwise, given a hash collision it may choose an incorrect row.

handler::position would be correct and very convenient to use here.

dup_ref is already set by handler independently of the engine
capabilities, when an extra lookup is made (for long unique or something else,
for example WITHOUT OVERLAPS) such error will be indicated by
file->lookup_errkey != -1.
2024-05-05 18:38:34 +02:00
Alexander Barkov
7f161a5c58 MDEV-34088 The TIMESTAMP value of '1970-01-01 00:00:00' can be indirectly inserted in strict mode
In strict mode a timestamp(0) column could be directly assigned from
another timestamp(N>0) column with the value '1970-01-01 00:00:00.1'
(at time zone '+00:00'), or with any other value '1970-01-01 00:00:00.XXXXXX'
with non-zero microsecond value XXXXXX.
This assignment happened silently without warnings or errors.

It worked as follows:

- The value {tv_sec=0, tv_usec=100000}, which is '1970-01-01 00:00:00.1'
  was rounded to {tv_sec=0, tv_usec=0}, which is '1970-01-01 00:00:00.0'

- Then {tv_sec=0, tv_usec=0} was silently re-interpreted as zero datetime.

After the fix this assignment always raises a warning,
which in case of the strict mode is escalated to an error.

The problem in this scenario is that '1970-01-01 00:00:00' cannot be stored,
because its timeval value {tv_sec=0, tv_usec=0} is reserved for zero datetimes.
Thus the warning should be raised no matter if sql_mode allows or disallows
zero dates.
2024-05-05 16:31:18 +04:00
Alexander Barkov
2c19877015 MDEV-34061 unix_timestamp(coalesce(timestamp_column)) returns NULL on '1970-01-01 00:00:00.000001'
Field_timestampf::val_native() checked only the
first four bytes to detect zero dates.
That was not enough. Fixing the code to check all packed_length()
bytes to detect zero dates.
2024-05-04 23:41:55 +04:00
Alexander Barkov
1cdf22374b MDEV-34069 Zero datetime reinterprets as '1970-01-01 00:00:00' on field_datetime=field_timestamp
The code in Field_timestamp::save_in_field() did not catch
zero datetime and stored it to the other field like a usual value
using store_timestamp_dec(), which knows nothing about zero date and
treats {tv_sec=0, tv_usec=0} as a normal timeval value corresponding to
'1970-01-01 00:00:00 +00:00'.

Fixing the code to catch the special combination (ts==0 && sec_pat==0) and
store it using store_time_dec() with a zero datetime passed as an argument.
2024-05-04 22:39:58 +04:00
Vladislav Vaintroub
88f49da8e0 MDEV-34063 tpool - integer overflow in multiplication.
When calculating next wakeup timepoint for the timer thread, with large
thread_pool_stall_limit values, 32bit int overflow can happen.
Fixed by making one operand 8 byte large.

Also fixed the type of tick_interval to be unsigned, so it does not
go negative for very thread_pool_stall_limit.
2024-05-03 22:01:09 +02:00
Vladislav Vaintroub
b18259ecf5 Compiling - Fix MSVC compile warnings, on x86 2024-05-03 22:00:56 +02:00
Vladislav Vaintroub
029e2a5fd9 MDEV-33876 CMake, zlib - use names compatible with official FindZLIB.cmake
- ZLIB_LIBRARIES, not ZLIB_LIBRARY
- ZLIB_INCLUDE_DIRS, not ZLIB_INCLUDE_DIR

For building libmariadb, ZLIB_LIBRARY/ZLIB_INCLUDE_DIR are still defined
This workaround will be removed later.
2024-05-03 21:48:47 +02:00
Marko Mäkelä
9ec7819c58 MDEV-33817: AVX512BW and VPCLMULQDQ based CRC-32
This is based on https://github.com/intel/intel-ipsec-mb/
and has been tested both on x86 and x86-64, with code that
was generated by several versions of GCC and clang.
GCC 11 or clang 8 or later should be able to compile this,
and so should recent versions of MSVC.

Thanks to Intel Corporation for providing access to hardware,
for answering my questions regarding the code, and for
providing the coefficients for the CRC-32C computation.

crc32_avx512(): Compute a reverse polynomial CRC-32 using
precomputed tables and carry-less product, for up to 256 bytes
of unaligned input per loop iteration.

Reviewed by: Vladislav Vaintroub
2024-05-03 15:55:20 +03:00
Marko Mäkelä
611cd6b981 MDEV-33817 preparation: Restructuring and unit tests
In our unit test, let us rely on our own reference
implementation using the reflected
CRC-32 ISO 3309 and CRC-32C polynomials. Let us also
test with various lengths.

Let us refactor the CRC-32 and CRC-32C implementations
so that no special compilation flags will be needed and
that some function call indirection will be avoided.

pmull_supported: Remove. We will have pointers to two separate
functions crc32c_aarch64_pmull() and crc32c_aarch64().
2024-05-03 13:06:13 +03:00
Yuchen Pei
b84d335d9d
MDEV-33538 make auxiliary spider plugins init depend on actual spider
The two I_S plugins SPIDER_ALLOC_MEM and SPIDER_WRAPPER_PROTOCOL
only makes sense if the main SPIDER plugin is installed. Further,
SPIDER_ALLOC_MEM requires a mutex that requires SPIDER init to fill
the table.

We also update the spider init query to override
--transaction_read_only=on so that it does not affect the spider init.

Also fixed error handling in spider_db_init() so that failure in
spider table init does not result in memory leak
2024-05-03 14:47:54 +10:00
Vladislav Vaintroub
20f60fe70f postfix a09ebe5567 (PCRE-10.43)
- no need to switch off -fsanitize-address for MSVC anymore
- option /WX is MSVC only
- CMAKE_C_STANDARD C99 is already in pcre's own CMakeLists.txt
2024-05-02 22:13:48 +02:00
Daniel Black
e63ed4e004 MDEV-33631 Ubuntu/Debian MYSQL_SERVER_SUFFIX is version+suffix on MariaDB packaged versions
The debian/rules removed sufficient version information from the
downstream version, however the mariadb upstream has a +
separator from the suffix.

While we are at it, remove the Debian/Ubuntu revision -1.

Other revisions used:
* Debian unstable (inherited to both Debian and Ubuntu if package unchanged): 1:10.11.6-2
* Ubuntu stable updates example: 1:10.11.6-0ubuntu0.23.10.2
* Debian stable updates example: 1:10.11.4-1~deb12u1

The result is compulation with:
    -DMYSQL_SERVER_SUFFIX="-1:10.4.34+maria~deb10" \

Compared to Debian:
    -DMYSQL_SERVER_SUFFIX="-1~deb12u1" \

The impact comes into play with SELECT VERSION():
    10.4.34-MariaDB-1:10.4.34+maria~deb10

A more abbreviated form of the following is much more human/machine parsable:
    10.6.16-MariaDB~ubu2004

Reviewer: Otto Kekäläinen
2024-05-01 00:57:15 +10:00
Dimitri John Ledkov
bf77f9793d openssl: add a more specific DES support detection
Improve detection for DES support in OpenSSL, to allow compilation
against system OpenSSL without DES.

Note that MariaDB needs to be compiled against OpenSSL-like library
that itself has DES support which cmake detected. Positive detection
is indicated with CMake variable HAVE_des 1.

Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@surgut.co.uk>
2024-04-30 23:09:02 +10:00
Thirunarayanan Balathandayuthapani
f378e76434 MDEV-33980 mariadb-backup --backup is missing retry logic for undo tablespaces
Problem:
========
- Currently mariabackup have to reread the pages in case they are
modified by server concurrently. But while reading the undo
tablespace, mariabackup failed to do reread the page in case of
error.

Fix:
===
Mariabackup --backup functionality should have retry logic
while reading the undo tablespaces.
2024-04-30 16:15:26 +05:30
Bernard Spil
f9575495ce Fix typo
There is no component `SuportFiles` in MariaDB server, yet a `SupportFiles` component exists.
2024-04-30 20:30:43 +10:00
Rucha Deodhar
9e6858a426 MDEV-22141: JSON_REMOVE returns NULL on valid arguments
Analysis:
When we scan json to get to a beginning according to the path, we end up
scanning json even if we have exhausted it. When eventually returns error.

Fix:
Continue scanning json only if we have not exhausted it and return result
accordingly.
2024-04-29 22:32:17 +05:30
Rucha Deodhar
5ca64e65d0 MDEV-32287: JSON_EXTRACT not returning multiple values for same path
Analysis:
When scanning json and getting the exact path at each step, if a path
is reached, we end up adding the item in the result and immediately get the
next item which results in current path changing.
Fix:
Instead of immediately returning the item, count the occurences of the path
in argument and append in the result as needed.
2024-04-29 22:32:17 +05:30
Rucha Deodhar
d7df63e1c9 MDEV-19487: JSON_TYPE doesnt detect the type of String Values
(returns NULL) and for Date/DateTime returns "INTEGER"

Analysis:
When the first character of json is scanned it is number. Based on that
integer is returned.
Fix:
Scan rest of the json before returning the final result to ensure json is
valid in the first place in order to have a valid type.
2024-04-29 22:32:17 +05:30
Thirunarayanan Balathandayuthapani
a586b6dbc8 MDEV-22855 Assertion `!field->prefix_len || field->fixed_len == field->prefix_len' failed in btr_node_ptr_max_size
Problem:
========
- InnoDB wrongly calulates the record size in
btr_node_ptr_max_size() when prefix index of
the column has to be stored externally.

Fix:
====
- InnoDB should add the maximum field size to
record size when the field is a fixed length one.
2024-04-29 16:42:26 +05:30