Commit graph

3563 commits

Author SHA1 Message Date
Oleksandr Byelkin
fccfdc28b8 MDEV-34771 Types mismatch when cloning items causes debug assertion
Missing methods added to Item_bin_string
2024-08-19 17:15:46 +02:00
Monty
db8ab4aca2 Sort result from table_statistics and index_statistics
This is needed as the order of rows are not deterministic,
especially in future versions of table statistics.
2024-08-19 11:14:11 +03:00
Monty
e51d55a63f Revert "mtr: remove not_valgrind_build"
The original code is correct.

valgrind and asan binaries should be built with a specialiced version of
mem_root that makes it easier to find memory overwrites.
This is what the BUILD scripts is doing.

The specialiced mem_root code allocates a new block for every allocation
which is visiable for any test that depenmds on the default original malloc
size and usage.
2024-08-19 10:59:57 +03:00
Marko Mäkelä
62bfcfd8b2 Merge 10.6 into 10.11 2024-08-14 11:36:52 +03:00
Marko Mäkelä
757c368139 Merge 10.5 into 10.6 2024-08-14 10:56:11 +03:00
Alexander Barkov
0e27351028 MDEV-34376 Wrong data types when mixing an utf8 *TEXT column and a short binary
A mixture of a multi-byte *TEXT column and a short binary column
produced a too large column.
For example, COALESCE(tinytext_utf8mb4, short_varbinary)
produced a BLOB column instead of an expected TINYBLOB.

- Adding a virtual method Type_all_attributes::character_octet_length(),
  returning max_length by default.
- Overriding Item_field::character_octet_length() to extract
  the octet length from the underlying Field.
- Overriding Item_ref::character_octet_length() to extract
  the octet length from the references Item (e.g. as VIEW fields).
- Fixing Type_numeric_attributes::find_max_octet_length() to
  take the octet length using the new method character_octet_length()
  instead of accessing max_length directly.
2024-08-12 17:13:31 +04:00
Ian Gilfillan
c83ba513da Update sponsors 2024-08-12 09:32:30 +01:00
Oleksandr Byelkin
662bb50784 Merge branch '10.5' into mariadb-10.5.26 2024-08-09 08:47:24 +02:00
Oleg Smirnov
cf202decde MDEV-34683 Types mismatch when cloning items causes debug assertion
New runtime type diagnostic (MDEV-34490) has detected that classes
Item_func_eq, Item_default_value and Item_date_literal_for_invalid_dates
incorrectly return an instance of its ancestor classes when being cloned.
This commit fixes that.

Additionally, it fixes a bug at Item_func_case_simple::do_build_clone()
which led to an endless loop of cloning functions calls.

Reviewer: Oleksandr Byelkin <sanja@mariadb.com>
2024-08-03 16:08:29 +07:00
Oleksandr Byelkin
0e8fb977b0 Merge branch '10.6' into 10.11 2024-08-03 09:15:40 +02:00
Oleksandr Byelkin
8f020508c8 Merge branch '10.5' into 10.6 2024-08-03 09:04:24 +02:00
Oleksandr Byelkin
7a5b8bf0f5 lost in editinig line added 2024-08-03 08:53:19 +02:00
Galina Shalygina
d072a29601 MDEV-23983: Crash caused by query containing constant having clause
Before this patch the crash occured when a single row dataset is used and
Item::remove_eq_conds() is called for HAVING. This function is not supposed
to be called after the elimination of multiple equalities.

To fix this problem instead of Item::remove_eq_conds() Item::val_int() is
used. In this case the optimizer tries to evaluate the condition for the
single row dataset and discovers impossible HAVING immediately. So, the
execution phase is skipped.

Approved by Igor Babaev <igor@maridb.com>
2024-08-01 12:18:29 +02:00
Sergei Petrunia
fdda8171b2 MDEV-34580: Assertion `(key_part->key_part_flag & 4) == 0' failed key_hashnr
Remove an assert added by fix for MDEV-34417. BNL-H join can be used with
prefix keys. This happens when there are real prefix indexes on the
equi-join columns (although it probably doesn't make a lot of sense).

Anyway, remove the assert. The code receives properly truncated key values
for hashing/comparison so it can handle them just fine.
2024-07-30 17:49:09 +03:00
Thirunarayanan Balathandayuthapani
cc8eefb0dc MDEV-33087 ALTER TABLE...ALGORITHM=COPY should build indexes more efficiently
- During copy algorithm, InnoDB should use bulk insert operation
for row by row insert operation. By doing this, copy algorithm
can effectively build indexes. This optimization is disabled
for temporary table, versioning table and table which has
foreign key relation.

Introduced the variable innodb_alter_copy_bulk to allow
the bulk insert operation for copy alter operation
inside InnoDB. This is enabled by default

ha_innobase::extra(): HA_EXTRA_END_ALTER_COPY mode tries to apply
the buffered bulk insert operation, updates the non-persistent
table stats.

row_merge_bulk_t::write_to_index(): Update stat_n_rows after
applying the bulk insert operation

row_ins_clust_index_entry_low(): In case of copy algorithm,
switch to bulk insert operation.

copy_data_error_ignore(): Handles the error while copying
the data from source to target file.
2024-07-30 11:59:01 +05:30
Rex
48b256a7e2 MDEV-34506 2nd execution name resolution problem with pushdown into unions
Statements affected by this bug need all the following to be true
1) a derived table table or view whose specification contains a set
     operation at the top level.
2) a grouping operator (group by/having) operating on a column alias
     other than in the first select of the union/intersect
3) an outer condition that will be pushed into all selects in this
     union/intersect, either into the where or having clause

When pushing a condition into all selects of a unit with more than one
select, pushdown_cond_for_derived() renames items so we can re-use the
condition being pushed.
These names need to be saved and reset for correct name resolution on
second execution of prepared statements.

Reviewed by Igor Babaev (igor@mariadb.com)
2024-07-30 08:21:58 +11:00
Monty
4bf7c966b3 MDEV-34664: Add an option to fix InnoDB's doubling of secondary index cardinalities
(With trivial fixes by sergey@mariadb.com)
Added option fix_innodb_cardinality to optimizer_adjust_secondary_key_costs

Using fix_innodb_cardinality disables the 'divide by 2' of rec_per_key_int
in InnoDB that in effect doubles the Cardinality for secondary keys.
This has the biggest effect for indexes where a few rows has the same key
value. Using this may also cause table scans for very small tables (which
in some cases may be better than an index scan).

The user visible effect is that 'SHOW INDEX FROM table_name' will for
InnoDB show the true Cardinality (and not 2x the real value). It will
also allow the optimizer to chose a better index in some cases as the
division by 2 could have a bad effect for tables with 2-5 identical values
per key.

A few notes about using fix_innodb_cardinality:
- It has direct affect for SHOW INDEX FROM table_name. SHOW INDEX
  will also update the statistics in table share.
- The effect of fix_innodb_cardinality for query plans or EXPLAIN
  is only visible after first open of the table. This is why one must
  do a flush tables or use SHOW INDEX for the option to take effect.
- Using fix_innodb_cardinality can thus affect all user in their query
  plans if they are using the same tables.

Because of this, it is strongly recommended that one uses
optimizer_adjust_secondary_key_costs=fix_innodb_cardinality mainly
in configuration files to not cause issues for other users.
2024-07-29 16:40:53 +03:00
Daniel Black
0939bfc093 MDEV-19052 main.win postfix --view-protocol compat
Correct compatibility with view-protocol.

Thanks Lena Startseva
2024-07-27 14:11:03 +10:00
Daniel Black
7788593547 MDEV-19052 Range-type window frame supports only numeric datatype
When there is no bounds on the upper or lower part of the window,
it doesn't matter if the type is numeric.

It also doesn't matter how many ORDER BY items there are in the
query.

Reviewers: Sergei Petrunia and Oleg Smirnov
2024-07-25 19:16:37 +10:00
Oleksandr Byelkin
26f31bdd52 The test should be not for AddressSanitizer used becouse stack check tests
and this check switched off
2024-07-24 16:41:29 +02:00
Oleksandr Byelkin
2844895766 disabling view protcol untill fix 2024-07-24 11:27:05 +02:00
Oleg Smirnov
c91aeb3771 MDEV-34634 Types mismatch when cloning items causes debug assertion
New runtime diagnostic introduced with MDEV-34490 has detected
that `Item_int_with_ref` incorrectly returns an instance of its ancestor
class `Item_int`. This commit fixes that.

In addition, this commit reverts a part of the diagnostic related
to `clone_item()` checks. As it turned out, `clone_item()` is not required
to return an object of the same class as the cloned one. For example,
look at `Item_param::clone_item()`: it can return objects of `Item_null`,
`Item_int`, `Item_string`, etc, depending on the object state.
So the runtime type diagnostic is not applicable to `clone_item()` and
is disabled with this commit.

As the similar diagnostic failures are expected to appear again
in the future, this commit introduces a new test file in the main suite:
item_types.test, and new test cases may be added to this file

Reviewer: Oleksandr Byelkin <sanja@mariadb.com>
2024-07-23 20:11:28 +07:00
Oleksandr Byelkin
0fe39d368a Merge branch '10.6' into 10.11 2024-07-22 15:14:50 +02:00
Oleksandr Byelkin
a938503cfb Merge branch '10.5' into 10.6 2024-07-20 08:12:42 +02:00
Andrei
b8f92ade57 MDEV-15393 gtid_slave_pos duplicate key errors after mysqldump restore
When mysqldump is run to dump the `mysql` system database, it generates
INSERT statements into the table `mysql.gtid_slave_pos`.
After running the backup script
those inserts did not produce the expected gtid state on slave. In
particular the maximum of mysql.gtid_slave_pos.sub_id did not make
into
   rpl_global_gtid_slave_state.last_sub_id

an in-memory object that is supposed to match the current state of the
table. And that was regardless of whether --gtid option was specified
or not. Later when the backup recipient server starts as slave
in *non-gtid* mode this desychronization may lead to a duplicate key
error.

This effect is corrected for --gtid mode mysqldump/mariadb-dump only
as the following.  The fixes ensure the insert block of the dump
script is followed with a "summing-up" SET @global.gtid_slave_pos
assignment.

For the implemenation part, note a deferred print-out of
SET-gtid_slave_pos and associated comments is prefered over relocating
of the entire blocks if (opt_master,slave_data &&
do_show_master,slave_status) ...  because of compatiblity
concern. Namely an error inside do_show_*() is handled in the new code
the same way, as early as, as before.

A regression test can be run in how-to-reproduce mode as well.
One affected mtr test observed.
rpl_mysqldump_slave.result "mismatch" shows now the new deferring print
of SET-gtid_slave_pos policy in action.
2024-07-19 21:44:12 +03:00
Oleksandr Byelkin
b8b6cab2d7 Fix view protocol 2024-07-19 13:07:17 +02:00
Oleksandr Byelkin
9af2caca33 Merge branch '10.5' into 10.6 2024-07-18 16:25:33 +02:00
Alexander Barkov
9dafde575f Additional tests for MDEV-28345 ASAN: use-after-poison or unknown-crash in my_strtod_int from charset_info_st::strntod or test_if_number 2024-07-18 08:17:53 +04:00
Sergei Golubchik
8d813f080b MDEV-34539 Invalid "use" and "Schema" in slow query log file with multi-line schema
quote a database name in the slow log
2024-07-17 21:25:40 +02:00
Sergei Golubchik
d20518168a also protect the /*!999999 sandbox comment 2024-07-17 21:25:40 +02:00
Sergei Golubchik
d60f5c11ea MDEV-34318 mariadb-dump SQL syntax error with MAX_STATEMENT_TIME against Percona MySQL server
protect MariaDB conditional comments from a bug
in Percona MySQL comment parser
2024-07-17 21:25:40 +02:00
Sergei Golubchik
dea5746de2 MDEV-32155 MariaDB Server crashes with ill-formed partitions
for ALTER_PARTITION_ADMIN (CHECK/REPAIR/LOAD INDEX/CACHE INDEX/etc)
partitioning marks affected partitions with PART_ADMIN state.

The assumption is that the server will call a corresponding
method of ha_partition which will reset the state back to PART_NORMAL.

This assumption is invalid, the server is not required to do so,
indeed, in CHECK ... FOR UPGRADE the server might decide early that
the table is fine and won't call ha_partition::check(), leaving
partitions in the wrong state. It will thus leak into the next
statement confusing the engine about what it is doing (see
ha_partition::create_handler_file()), causing a crash later.

Let's force all partitions into PART_NORMAL state after the admin
operation succeeded, in case it did so without consulting the engine.
2024-07-17 21:25:40 +02:00
Alexander Barkov
b777b749ad MDEV-28345 ASAN: use-after-poison or unknown-crash in my_strtod_int from charset_info_st::strntod or test_if_number
This patch fixes two problems:

- The code inside my_strtod_int() in strings/dtoa.c could test the byte
  behind the end of the string when processing the mantissa.
  Rewriting the code to avoid this.

- The code in test_if_number() in sql/sql_analyse.cc called my_atof()
  which is unsafe and makes the called my_strtod_int() look behind
  the end of the string if the input string is not 0-terminated.
  Fixing test_if_number() to use my_strtod() instead, passing the correct
  end pointer.
2024-07-17 12:17:27 +04:00
Sergei Petrunia
e644e130b0 MDEV-30623: Fix the testcase
- Fix view-protocol: long expressions in SELECT
  list should have "expr AS column_name".

- Also, moved the test from subselect*test to
  suite/json/t/json_table.test.
2024-07-16 12:52:31 +03:00
Oleg Smirnov
972879f413 MDEV-33010 Crash when pushing condition with CHARSET()/COERCIBILITY() into derived table
Based on the current logic, objects of classes Item_func_charset and
Item_func_coercibility (responsible for CHARSET() and COERCIBILITY()
functions) are always considered constant.
However, SQL syntax allows their use in a non-constant manner, such as
CHARSET(t1.a), COERCIBILITY(t1.a).

In these cases, the `used_tables()` parameter corresponds to table names
in the function parameters, creating an inconsistency: the item is marked
as constant but accesses tables. This leads to crashes when
conditions with CHARSET()/COERCIBILITY() are pushed into derived tables.

This commit addresses the issue by setting `used_tables()` to 0 for
`Item_func_charset` and `Item_func_coercibility`. Additionally, the items
now store the return values during the preparation phase and return
them during the execution phase. This ensures that the items do not call
its arguments methods during the execution and are truly constant.

Reviewer: Alexander Barkov <bar@mariadb.com>
2024-07-16 16:20:17 +07:00
Yuchen Pei
f071b7620b
Merge branch '10.5' into 10.6 2024-07-16 15:54:22 +08:00
Julius Goryavsky
0802e5a7eb MDEV-34505: galera.mariadb_tzinfo_to_sql fails deterministically on Ubuntu 24.04
Fixed a sorting order condition that in its previous form could lead
to the formation of an incorrect pattern for comparing strings.
2024-07-13 04:38:10 +02:00
Oleg Smirnov
aae3233c4f MDEV-34041 Display additional information for materialized subqueries in EXPLAIN/ANALYZE FORMAT=JSON
This commits adds the "materialization" block to the output of
EXPLAIN/ANALYZE FORMAT=JSON when materialized subqueries are involved
into processing. In the case of ANALYZE additional runtime information
is displayed, such as:
  - chosen strategy of materialization
  - number of partial match/index lookup loops
  - sizes of partial match buffers
2024-07-11 17:40:39 +07:00
Galina Shalygina
a5e4c34991 MDEV-32608: Expression with constant subquery causes a crash in pushdown
from HAVING

The bug is caused by refixing of the constant subquery in pushdown from
HAVING into WHERE optimization.

Similarly to MDEV-29363 in the problematic query two references of the
constant subquery are used. After the pushdown one of the references of the
subquery is pushed into WHERE-clause and the second one remains as the part
of the HAVING-clause.
Before the represented fix, the constant subquery reference that was going to
be pushed into WHERE was cleaned up and fixed. That caused the changes of
the subquery itself and, therefore, changes for the second reference that
remained in HAVING. These changes caused a crash.

To fix this problem all constant objects that are going to be pushed into
WHERE should be marked with an IMMUTABLE_FL flag. Objects marked with this
flag are not cleaned up or fixed in the pushdown optimization.

Approved by Igor Babaev <igor@mariadb.com>
2024-07-11 11:05:32 +02:00
Daniel Black
eaf7c0cbea mtr: remove not_valgrind_build
The version test on not_valgrind_build.inc was
broken as in BB the sp-no-valgrind.test was
executed.

The implication that it wouldn't work on ASAN
was also incorrect as ASAN tests show it running
fine there.

Correct sp-no-valgrind.test for not_valgrind.inc.
2024-07-11 17:52:12 +10:00
Dave Gosselin
02e38e2ece MDEV-33971 NAME_CONST in WHERE clause replaced by inner item
Improve performance of queries like
  SELECT * FROM t1 WHERE field = NAME_CONST('a', 4);
by, in this example, replacing the WHERE clause with field = 4
in the case of ref access.

The rewrite is done during fix_fields and we disambiguate this
case from other cases of NAME_CONST by inspecting where we are
in parsing.  We rely on THD::where to accomplish this.  To
improve performance there, we change the type of THD::where to
be an enumeration, so we can avoid string comparisons during
Item_name_const::fix_fields.  Consequently, this patch also
changes all usages of THD::where to conform likewise.
2024-07-10 17:23:43 -04:00
Alexander Barkov
4d71a117a3 Merge remote-tracking branch 'origin/10.6' into 10.11 2024-07-08 21:52:08 +04:00
Rex
b418b60ebf MDEV-30623 JSON_TABLE in subquery not correctly marked as correlated
st_select_lex::update_correlated_cache() fails to take JSON_TABLE
functions in subqueries into account.

Reviewed by Sergei Petrunia (sergey@mariadb.com)
2024-07-09 04:45:29 +11:00
Alexander Barkov
e56040fee8 Merge remote-tracking branch 'origin/10.5' into 10.6 2024-07-08 18:59:04 +04:00
Alexander Barkov
d1e5fa8917 MDEV-34305 Redundant truncation errors/warnings with optimizer_trace enabled
my_like_range*() can create longer keys than Field::char_length().
This caused warnings during print_range().

Fix:

Suppressing warnings in print_range().
2024-07-08 18:01:01 +04:00
Sergei Petrunia
e40d232ad6 Stabilize analyze_engine_stats2.test 2024-07-04 15:24:49 +03:00
Sergei Petrunia
513c827041 MDEV-34190: r_engine_stats.pages_read_count is unrealistically low
The symptoms were: take a server with no activity and a table that's
not in the buffer pool. Run a query that reads the whole table and
observe that r_engine_stats.pages_read_count shows about 2% of the table
was read. Who reads the rest?

The cause was that page prefetching done inside InnoDB was not counted.

This counts page prefetch requests made in buf_read_ahead_random() and
buf_read_ahead_linear() and makes them visible in:

- ANALYZE: r_engine_stats.pages_prefetch_read_count
- Slow Query Log: Pages_prefetched:

This patch intentionally doesn't attempt to count the time to read the
prefetched pages:
* there's no obvious place where one can do it
* prefetch reads may be done in parallel (right?), it is not clear how
  to count the time in this case.
2024-07-04 15:24:49 +03:00
Galina Shalygina
6cb896a639 MDEV-29363: Constant subquery causing a crash in pushdown optimization
The crash is caused by the attempt to refix the constant subquery during
pushdown from HAVING into WHERE optimization.

Every condition that is going to be pushed into WHERE clause is first
cleaned up, then refixed. Constant subqueries are not cleaned or refixed
because they will remain the same after refixing, so this complicated
procedure can be omitted for them (introduced in MDEV-21184).
Constant subqueries are marked with flag IMMUTABLE_FL, that helps to miss
the cleanup stage for them. Also they are marked as fixed, so refixing is
also not done for them.
Because of the multiple equality propagation several references to the same
constant subquery can exist in the condition that is going to be pushed
into WHERE. Before this patch, the problem appeared in the following way.
After the first reference to the constant subquery is processed, the flag
IMMUTABLE_FL for the constant subquery is disabled.
So, when the second reference to this constant subquery is processed, the
flag is already disabled and the subquery goes through the procedure of
cleaning and refixing. That causes a crash.

To solve this problem, IMMUTABLE_FL should be disabled only after all
references to the constant subquery are processed, so after the whole
condition that is going to be pushed is cleaned up and refixed.

Approved by Igor Babaev <igor@maridb.com>
2024-07-04 13:46:19 +02:00
Oleksandr Byelkin
034a175982 Merge branch '10.6' into 10.11 2024-07-04 11:52:07 +02:00
Alexander Barkov
f6989d1767 MDEV-10865 COLLATE keyword doesn't work in PREPARE query
Fixing applying the COLLATE clause to a parameter caused an error error:
  COLLATION '...' is not valid for CHARACTER SET 'binary'

Fix:

- Changing the collation derivation for a non-prepared Item_param
  to DERIVATION_IGNORABLE.

- Allowing to apply any COLLATE clause to expressions with DERIVATION_IGNORABLE.
  This includes:
    1. A non-prepared Item_param
    2. An explicit NULL
    3. Expressions derived from #1 and #2

  For example:
    SELECT ? COLLATE utf8mb_unicode_ci;
    SELECT NULL COLLATE utf8mb_unicode_ci;
    SELECT CONCAT(?) COLLATE utf8mb_unicode_ci;
    SELECT CONCAT(NULL) COLLATE utf8mb_unicode_ci

- Additional change: preserving the collation of an expression when
  the expression gets assigned to a PS parameter and evaluates to SQL NULL.
  Before this change, the collation of the parameter was erroneously set
  to &my_charset_binary.

- Additional change: removing the multiplication to mbmaxlen from the
  fix_char_length_ulonglong() argument, because the multiplication already
  happens inside fix_char_length_ulonglong().
  This fixes a too large column size created for a COLLATE clause.
2024-07-04 11:08:47 +04:00
Oleksandr Byelkin
dcd8a64892 Merge branch '10.5' into 10.6 2024-07-03 13:27:23 +02:00
Monty
c91ec6a5c1 Added Lock_time_ms and Table_catalog columns to metadata_lock_info
If compiled for debugging, LOCK_DURATION is also filled in.
2024-07-03 13:20:33 +03:00
Monty
2739b5f5f8 MDEV-34494 Add server_uid global variable and add it to error log at startup
The feedback plugin server_uid variable and the calculate_server_uid()
function is moved from feedback/utils.cc to sql/mysqld.cc

server_uid is added as a global variable (shown in 'show variables') and
is written to the error log on server startup together with server version
and server commit id.
2024-07-02 11:26:13 +03:00
Monty
d8c9c5ead6 MDEV-34491 Setting log_slow_admin="" at startup should be converted to log_slow_admin=ALL
We have an issue if a user have the following in a configuration file:
log_slow_filter=""                  # Log everything to slow query log
log_queries_not_using_indexes=ON

This set log_slow_filter to 'not_using_index' which disables
slow_query_logging of most queries.
In effect, on should never use log_slow_filter="" in config files but
instead use log_slow_filter=ALL.

Fixed by changing log_slow_filter="" that comes either from a
configuration file or from the command line, when starting to the server,
to log_slow_filter=ALL.
A warning will be printed when this happens.

Other things:
- One can now use =ALL for any 'set' variable to set all options at once.
  (backported from 10.6)
2024-07-02 11:26:13 +03:00
Daniel Black
243dee7415 MDEV-34437: handle error on getaddrinfo
When getaddrinfo returns and error, the contents
of ai are invalid so we cannot continue based
on their data structures.

In the previous branch of the if statement, we
abort there if there is an error so for consistency
we abort here too.

The test case fixes the port number to UINTMAX32
for both an enumberated bind-address and the
default bind-address covering the two calls to
getaddrinfo.

Review thanks Sanja.
2024-07-02 17:11:32 +10:00
Lena Startseva
090cecd5e8 MDEV-31933: Make working view-protocol + ps-protocol (running two protocols together)
Fix for v. 10.5
2024-07-02 10:11:33 +07:00
Alexander Barkov
d046b13e7b MDEV-20548 Unexpected error on CREATE..SELECT HEX(num)
Item_func_hex::fix_length_and_dec() evaluated a too short data type
for signed numeric arguments, which resulted in a 'Data too long for column'
error on CREATE..SELECT.

Fixing the code to take into account that a short negative
numer can produce a long HEX value: -1  -> 'FFFFFFFFFFFFFFFF'

Also fixing Item_func_hex::val_str_ascii_from_val_real().
Without this change, MTR test with HEX with negative float point arguments
failed on some platforms (aarch64, ppc64le, s390-x).
2024-07-01 18:50:32 +04:00
Lena Startseva
9e74a7f4f3 Removing MDEV-27871 from tastcases because it is not a bug 2024-06-28 16:45:50 +07:00
Marko Mäkelä
27a3366663 Merge 10.6 into 10.11 2024-06-27 10:26:09 +03:00
Yuchen Pei
d7042ec4da
Merge branch '10.5' into 10.6 2024-06-26 09:16:54 +08:00
Rex
d513a4ce74 MDEV-19520 Extend condition normalization to include 'NOT a'
Having Item_func_not items in item trees breaks assumptions during the
optimization phase about transformation possibilities in fix_fields().
Remove Item_func_not by extending normalization during parsing.

Reviewed by Oleksandr Byelkin (sanja@mariadb.com)
2024-06-25 04:51:29 +11:00
Marko Mäkelä
0076eb3d4e Merge 10.5 into 10.6 2024-06-24 13:09:47 +03:00
Rex
9e800eda86 MDEV-32583 UUID() should be treated as stochastic for the purposes of forcing query materialization
RAND() and UUID() are treated differently with respect to subquery
materialization both should be marked as uncacheable, forcing materialization.
Altered Create_func_uuid(_short)::create_builder().
Added comment in header about UNCACHEABLE_RAND meaning also unmergeable.
2024-06-22 13:26:49 +11:00
Sergei Petrunia
49b4a6e26d Fix the testcase for MDEV-31558: log_slow_verbosity_innodb
log_slow_innodb.test sets "log_slow_verbosity_innodb_expected_matches"

in include/log_slow_grep.inc:
- Fix a typo: take value from log_slow_*VERBOSITY*_innodb_expected_matches
- Add a missing "--source include/log_grep.inc"
- two-line search patterns do not work, search for each line individually.
2024-06-21 10:53:40 +03:00
Alexander Barkov
6cecf61a59 MDEV-34417 Wrong result set with utf8mb4_danish_ci and BNLH join
There were erroneous calls for charpos() in key_hashnr() and key_buf_cmp().
These functions are never called with prefix segments.

The charpos() calls were wrong. Before the change BNHL joins
- could return wrong result sets, as reported in MDEV-34417
- were extremely slow for multi-byte character sets, because
  the hash was calculated on string prefixes, which increased
  the amount of collisions drastically.

This patch fixes the wrong result set as reported in MDEV-34417,
as well as (partially) the performance problem reported in MDEV-34352.
2024-06-20 11:30:02 +04:00
Vicențiu Ciorbaru
6382339144 MDEV-34311: Alter USER should reset all account limit counters
This commit introduces a reset of password errors counter on any alter user
command for the altered user. This is done so as to not require a
complete privilege system reload.
2024-06-19 23:08:35 +03:00
Marko Mäkelä
34813c1aa0 Merge 10.6 into 10.11 2024-06-19 15:04:07 +03:00
Souradeep Saha
10fbd1ce51 MDEV-34168: Extend perror utility to print link to KB page
As all MariaDB Server errors now have a dedicated web page, the
perror utility is extended to include a link to the KB page of
the corresponding error code.

All new code of the whole pull request, including one or several
files that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my
employer Amazon Web Services, Inc.
2024-06-18 13:25:39 +10:00
Alexander Barkov
83d3ed4908 MDEV-34014 mysql_upgrade failed
Adding a new statement into scripts/sys_schema/before_setup.sql:

  ALTER DATABASE sys CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci;

to fix db.opt in case:
- the database `sys` was altered to unexpected CHARACTER SET or COLLATE values
- or db.opt was erroneously removed

to make sure that sys objects are always recreated using utf8mb3_general_ci.
2024-06-17 16:38:48 +04:00
Sergei Petrunia
2eda310b15 Restore test coverage for MDEV-18956
(It was accidentally removed by fix for MDEV-28846)
2024-06-17 14:08:32 +03:00
Sergei Petrunia
a2066b2400 MDEV-30651: Assertion `sel->quick' in make_range_rowid_filters
The optimizer deals with Rowid Filters this way:

1. First, range optimizer is invoked. It saves information
   about all potential range accesses.
2. A query plan is chosen. Suppose, it uses a Rowid Filter on
   index $IDX.
3. JOIN::make_range_rowid_filters() calls the range optimizer
again to create a quick select on index $IDX which will be used
to populate the rowid filter.

The problem: KILL command catches the query in step #3. Quick
Select is not created which causes a crash.

Fixed by checking if query was killed. Note: the problem also
affects 10.6, even if error handling for
SQL_SELECT::test_quick_select is different there.
2024-06-17 14:08:32 +03:00
Sergei Petrunia
ef9e3e73ed MDEV-30651: Assertion `sel->quick' in make_range_rowid_filters
(Variant for 10.6: return error code from SQL_SELECT::test_quick_select)
The optimizer deals with Rowid Filters this way:

1. First, range optimizer is invoked. It saves information
   about all potential range accesses.
2. A query plan is chosen. Suppose, it uses a Rowid Filter on
   index $IDX.
3. JOIN::make_range_rowid_filters() calls the range optimizer
again to create a quick select on index $IDX which will be used
to populate the rowid filter.

The problem: KILL command catches the query in step #3. Quick
Select is not created which causes a crash.

Fixed by checking if query was killed.
2024-06-17 12:50:43 +03:00
Sergei Petrunia
b47bd3f8bf MDEV-33875: ORDER BY DESC causes ROWID Filter slowdown
Rowid Filter cannot be used with reverse-ordered scans, for the
same reason as IndexConditionPushdown cannot be.

test_if_skip_sort_order() already has logic to disable ICP when
setting up a reverse-ordered scan. Added logic to also disable
Rowid Filter in this case, factored out the code into
prepare_for_reverse_ordered_access(), and added a comment describing
the cause of this limitation.
2024-06-17 09:50:32 +03:00
Marko Mäkelä
346a0c1402 Merge 10.6 into 10.11 2024-06-17 09:08:07 +03:00
Marko Mäkelä
e60acae655 Merge 10.5 into 10.6 2024-06-17 08:40:07 +03:00
Monty
956bcf8f49 Change mysqldump to use DO instead of 'SELECT' for storing sequences.
This avoids a lot of SETVAL() results when applying a mysqldump with
sequences.
2024-06-16 10:51:33 +03:00
Monty
fef32fd9ad MDEV-34406 Enhance mariadb_upgrade to print failing query in case of error
To make this possible, it was also necessary to enhance the mariadb
client with the option --print-query-on-error.
This option can also be very useful when running a batch of queries
through the mariadb client and one wants to find out where things goes
wrong.

TODO: It would be good to enhance mariadb_upgrade to not call the mariadb
client for executing queries but instead do this internally.  This
would have made this patch much easier!

Reviewed by: Sergei Golubchik <serg@mariadb.com>
2024-06-16 10:51:33 +03:00
Marko Mäkelä
5b89cab44f Merge 10.6 into 10.11 2024-06-13 08:16:49 +03:00
Marko Mäkelä
fc9005adc4 Merge 10.5 into 10.6 2024-06-12 07:51:28 +03:00
Marko Mäkelä
b81d717387 Merge 10.6 into 10.11 2024-06-11 12:50:10 +03:00
Sergei Golubchik
40dd5b8676 fix the test for --view 2024-06-10 21:03:53 +02:00
Marko Mäkelä
27834ebc91 Merge 10.5 into 10.6 2024-06-10 15:22:15 +03:00
Alexander Barkov
21f56583bf MDEV-32376 SHOW CREATE DATABASE statement crashes the server when db name contains some unicode characters, ASAN stack-buffer-overflow
Adding the test for the length of lex->name into show_create_db().

Without this test writes beyond the end of db_name_buff were possible
upon a too long database name.
2024-06-10 09:31:14 +04:00
Oleksandr Byelkin
77c4c0f256 MDEV-34203 Sandbox mode \- is not compatible with --binary-mode
"Process" sandbox short command put by masqldump to avoid an error.
2024-06-07 14:07:54 +02:00
Marko Mäkelä
a687cf8661 Merge 10.5 into 10.6 2024-06-07 10:03:51 +03:00
Rucha Deodhar
0406b2a4ed MDEV-34143: Server crashes when executing JSON_EXTRACT after setting
non-default collation_connection

Analysis:
Due to different collation, the string has nothing to chop off.

Fix:
Got rid of chop(), only append " ," only when we have more elements to
add to the result.
2024-06-06 11:41:01 +05:30
Igor Babaev
4d38267fc7 MDEV-29307 Wrong result when joining two derived tables over the same view
This bug could affect queries containing a join of derived tables over
grouping views such that one of the derived tables contains a window
function while another uses view V with dependent subquery DSQ containing
a set function aggregated outside of the subquery in the view V. The
subquery also refers to the fields from the group clause of the view.Due to
this bug execution of such queries could produce wrong result sets.

When the fix_fields() method performs context analysis of a set function AF
first, at the very beginning the function Item_sum::init_sum_func_check()
is called. The function copies the pointer to the embedding set function,
if any, stored in THD::LEX::in_sum_func into the corresponding field of the
set function AF simultaneously changing the value of THD::LEX::in_sum_func
to point to AF. When at the very end of the fix_fields() method the function
Item_sum::check_sum_func() is called it is supposed to restore the value
of THD::LEX::in_sum_func to point to the embedding set function. And in
fact Item_sum::check_sum_func() did it, but only for regular set functions,
not for those used in window functions. As a result after the context
analysis of AF had finished THD::LEX::in_sum_func still pointed to AF.
It confused the further context analysis. In particular it led to wrong
resolution of Item_outer_ref objects in the fix_inner_refs() function.
This wrong resolution forced reading the values of grouping fields referred
in DSQ not from the temporary table used for aggregation from which they
were supposed to be read, but from the table used as the source table for
aggregation.

This patch guarantees that the value of THD::LEX::in_sum_func is properly
restored after the call of fix_fields() for any set function.
2024-06-04 17:54:01 -07:00
Alexander Barkov
5e12d49205 MDEV-34295 CAST(char_col AS DOUBLE) prints redundant spaces in a warning
Field_string::val_int(), Field_string::val_real(), Field_string::val_decimal()
passed the whole buffer of field_length bytes to data type conversion routines.
This made conversion routines to print redundant trailing spaces in case of warnings.

Adding a method Field_string::to_lex_cstring() and using it inside
val_int(), val_real(), val_decimal(), val_str().

After this change conversion routines get the same value with what val_str() returns,
and no redundant trailing spaces are displayed.
2024-06-04 15:34:14 +04:00
Alexander Barkov
76e0dc18b6 MDEV-34288 SET NAMES DEFAULT crashes mariadbd --collation-server=utf8mb4_unicode_ci
The @@global.character_set_client variable could erroneously be set
to a non-default collation of its character set, which further made
the `SET NAMES DEFAULT` statement crash the server.

Fixing the code to make sure that the global value these variables:
  @@character_set_client
  @@character_set_connection
  @@character_set_server
  @@character_set_database
  @@character_set_connection
point to the default compiled collations of the character set.
2024-06-04 12:38:43 +04:00
Marko Mäkelä
22ba7e4ff8 Merge 10.6 into 10.11 2024-05-30 16:04:00 +03:00
Marko Mäkelä
5ba542e9ee Merge 10.5 into 10.6 2024-05-30 14:27:07 +03:00
Marko Mäkelä
0c440abd5e MDEV-31340 fixup: Add end-of-test marker 2024-05-30 14:23:45 +03:00
Marko Mäkelä
c71275b69e Fix ./mtr --repeat=2 main.func_str 2024-05-30 14:22:00 +03:00
Sergei Petrunia
36ab6cc80c MDEV-34125: ANALYZE FORMAT=JSON: r_engine_stats.pages_read_time_ms has wrong scale
- Change the comments in class ha_handler_stats to say the members
  are in ticks, not milliseconds.
- In sql_explain.cc, adjust the scale to print milliseconds.
2024-05-27 15:28:57 +03:00
Alexander Barkov
4a158ec167 MDEV-34226 On startup: UBSAN: applying zero offset to null pointer in my_copy_fix_mb from strings/ctype-mb.c and other locations
nullptr+0 is an UB (undefined behavior).

- Fixing my_string_metadata_get_mb() to handle {nullptr,0} without UB.
- Fixing THD::copy_with_error() to disallow {nullptr,0} by DBUG_ASSERT().
- Fixing parse_client_handshake_packet() to call THD::copy_with_error()
  with an empty string {"",0} instead of NULL string {nullptr,0}.
2024-05-27 13:19:13 +04:00
Alexander Barkov
7925326183 MDEV-30931 UBSAN: negation of -X cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in get_interval_value on SELECT
- Fixing the code in get_interval_value() to use Longlong_hybrid_null.
  This allows to handle correctly:

  - Signed and unsigned arguments
    (the old code assumed the argument to be signed)
  - Avoid undefined negation behavior the corner case with LONGLONG_MIN

  This fixes the UBSAN warning:
    negation of -9223372036854775808 cannot be represented
    in type 'long long int';

- Fixing the code in get_interval_value() to avoid overflow in
  the INTERVAL_QUARTER and INTERVAL_WEEK branches.
  This fixes the UBSAN warning:
    signed integer overflow: -9223372036854775808 * 7 cannot be represented
    in type 'long long int'

- Fixing the INTERVAL_WEEK branch in date_add_interval() to handle
  huge numbers correctly. Before the change, huge positive numeber
  were treated as their negative complements.
  Note, some other branches still can be affected by this problem
  and should also be fixed eventually.
2024-05-27 13:19:13 +04:00
Vladislav Vaintroub
736449d30f MDEV-34205: ASAN stack buffer overflow in strxnmov() in frm_file_exists
Correct the second parameter for strxnmov to prevent potential buffer
overflows. The second parameter must be one less than the size of the
input buffer to avoid writing past the end of the buffer.

While the second parameter is usually correct, there are exceptions
that need fixing.

This commit addresses the issue within frm_file_exists() and other
affected places.
2024-05-23 22:08:27 +02:00
Alexander Barkov
7c4c082349 MDEV-28387 UBSAN: runtime error: negation of -9223372036854775808 cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in my_strtoll10 on SELECT
Fixing the condition to raise an overflow in the ulonglong
representation of the number is greater or equal to 0x8000000000000000ULL.
Before this change the condition did not catch -9223372036854775808
(the smallest possible signed negative longlong number).
2024-05-23 14:18:34 +04:00
Alexander Barkov
310fd6ff69 Backporting bugs fixes fixed by MDEV-31340 from 11.5
The patch for MDEV-31340 fixed the following bugs:

MDEV-33084 LASTVAL(t1) and LASTVAL(T1) do not work well with lower-case-table-names=0
MDEV-33085 Tables T1 and t1 do not work well with ENGINE=CSV and lower-case-table-names=0
MDEV-33086 SHOW OPEN TABLES IN DB1 -- is case insensitive with lower-case-table-names=0
MDEV-33088 Cannot create triggers in the database `MYSQL`
MDEV-33103 LOCK TABLE t1 AS t2 -- alias is not case sensitive with lower-case-table-names=0
MDEV-33108 TABLE_STATISTICS and INDEX_STATISTICS are case insensitive with lower-case-table-names=0
MDEV-33109 DROP DATABASE MYSQL -- does not drop SP with lower-case-table-names=0
MDEV-33110 HANDLER commands are case insensitive with lower-case-table-names=0
MDEV-33119 User is case insensitive in INFORMATION_SCHEMA.VIEWS
MDEV-33120 System log table names are case insensitive with lower-cast-table-names=0

Backporting the fixes from 11.5 to 10.5
2024-05-21 14:58:01 +04:00
Dmitry Shulga
5e6c122427 MDEV-33769: Memory leak found in the test main.rownum run with --ps-protocol against a server built with the option -DWITH_PROTECT_STATEMENT_MEMROOT
A memory leak happens on the second execution of a query that run in PS mode
and uses the function ROWNUM().

A memory leak took place on allocation of an instance of the class Item_int
for storing a limit value that is performed at the function set_limit_for_unit
indirectly called from JOIN::optimize_inner. Typical trace to the place where
the memory leak occurred is below:
 JOIN::optimize_inner
  optimize_rownum
   process_direct_rownum_comparison
    set_limit_for_unit
     new (thd->mem_root) Item_int(thd, lim, MAX_BIGINT_WIDTH);

To fix this memory leak, calling of the function optimize_rownum()
has to be performed only once on first execution and never called
after that. To control it, the new data member
  first_rownum_optimization
added into the structure st_select_lex.
2024-05-13 17:07:48 +07:00
Sergei Golubchik
a6b2f820e0 Merge branch '10.6' into 10.11 2024-05-10 20:02:18 +02:00
Daniel Black
d7d8c2c287 MDEV-31566: Fix buffer overrun of column_json function (postfix)
Test case failed --view protocol. Revert to using table for data
in the test.
2024-05-10 13:39:10 +10:00
He Guohua
867747204a MDEV-31566 Fix buffer overrun of column_json function
The accounting of the limit variable that represents the
amount of space left it the buffer was incorrect.

Also there was 1 or 2 bytes left to write that occured without
the buffer length being checked.

Review: Sanja Byelkin
2024-05-09 10:45:15 +10:00
Sergei Golubchik
7b53672c63 Merge branch '10.5' into 10.6 2024-05-08 20:06:00 +02:00
Sergei Golubchik
29c185bd77 test needs to cleanup after itself 2024-05-08 20:01:17 +02:00
Sergei Petrunia
40b3525fcc MDEV-28621: group by optimization incorrectly removing subquery where subject buried in a function
Workaround patch: Do not remove GROUP BY clause when it has
subquer(ies) in it.

remove_redundant_subquery_clauses() removes redundant GROUP BY clause
from queries in form:
  expr IN (SELECT no_aggregates GROUP BY ...)
  expr {CMP} {ALL|ANY|SOME} (SELECT no_aggregates GROUP BY ...)
This hits problems when the GROUP BY clause itself has subquer(y/ies).

This patch is just a workaround: it disables removal of GROUP BY clause
if the clause has one or more subqueries in it.

Tests:
- subselect_elimination.test has all known crashing cases.
- subselect4.result, insert_select.result are updated.
Note that in some cases results of SELECT are changed too (not just
EXPLAINs). These are caused by non-deterministic SQL: when running a
query like:

  x > ANY( SELECT col1 FROM t1 GROUP BY constant_expression)

without removing the GROUP BY, the executor is free to pick the value
of t1.col1 from any row in the GROUP BY group (denote it $COL1_VAL).
Then, it computes x > ANY(SELECT $COL1_VAL).

When running the same query and removing the GROUP BY:

   x > ANY( SELECT col1 FROM t1)

the executor will actually check all rows of t1.
2024-05-07 21:25:22 +02:00
Galina Shalygina
4bc1860eb4 MDEV-23878 Wrong result with semi-join and splittable derived table
Due to this bug a wrong result might be expected from queries with
an IN subquery predicate in the WHERE clause and a derived table in the
FROM clause to which split optimization could be applied.

The function JOIN::fix_all_splittings_in_plan() used the value of the
bitmap JOIN::sjm_lookup_tables() such as it had been left after the
search for the best plan for the select containing the splittable
derived table. That value could not be guaranteed to be correct. So the
recalculation of this bitmap is needed to exclude the plans with key
accesses from SJM lookup tables.

Approved by Igor Babaev <igor@maridb.com>
2024-05-07 12:21:35 +02:00
Sergei Golubchik
7ed9d2ac00 MDEV-9179 When binlog_annotate_row_events on , event of binlog file is truncated
cnt counter was incremented one extra time per line
2024-05-06 20:14:37 +02:00
Sergei Golubchik
13663cb5c4 MDEV-33727 mariadb-dump trusts the server and does not validate the data
safety first - tell mariadb client not to execute dangerous
cli commands, they cannot be present in the dump anyway.

wrapping the command in /*!999999 ..... */ guarantees that
if a non-mariadb-cli client loads the dump and sends it to the
server - the server will ignore the command it doesn't understand
2024-05-06 17:16:10 +02:00
Sergei Golubchik
2025597c0b MDEV-21778 Disable system commands in mysql/mariadb client
mysql --sandbox

disables system (\!), tee (\T), pager with an argument(\P foo), source (\.)

does *not* disable edit (\e). Use EDITOR=/bin/false to disable
or, for example, EDITOR=rnano for something more useful

does *not* disable pager (\P) without an argument. Use
PAGER=cat or, for example PAGER=less LESSSECURE=1 for something
more useful

using a disabled command is an error, which can be ignored with --force

Also, a "sandbox" command (\-) - enables the sandbox mode until EOF
(current file or the session, if interactive)
2024-05-06 17:16:10 +02:00
Sergei Golubchik
22b3ba9312 MDEV-25102 UNIQUE USING HASH error after ALTER ... DISABLE KEYS
on disable_indexes(HA_KEY_SWITCH_NONUNIQ_SAVE) the engine does
not know that the long unique is logically unique, because on the
engine level it is not. And the engine disables it,

Change the disable_indexes/enable_indexes API. Instead of the enum
mode, send a key_map of indexes that should be enabled. This way the
server will decide what is unique, not the engine.
2024-05-06 17:16:10 +02:00
Julius Goryavsky
b88c20ce1b Merge branch 10.4 into 10.5 2024-05-06 13:55:42 +02:00
Sergei Golubchik
3d75cffa91 bugfix: INFORMATION_SCHEMA.STATISTICS doesn't show whether the index is disabled 2024-05-05 21:37:08 +02:00
Sergei Golubchik
4f5dea43df cleanup
* remove dead code
* simplify the check for table->s->next_number_index
* misc
2024-05-05 21:37:08 +02:00
Sergei Golubchik
947eeaa6dc MDEV-29345 update case insensitive (large) unique key with insensitive change of value - duplicate key
use collation-sensitive comparison when comparing fields
2024-05-05 21:37:08 +02:00
Sergei Golubchik
349ca2be74 mtr: remove innodb combinations
dead code for about 10 years
2024-05-05 21:37:08 +02:00
Sergei Golubchik
d74fee9e8c MDEV-19487 fix for --view
followup for d7df63e1c9
2024-05-05 21:37:08 +02:00
Nikita Malyavin
72429cad7f MDEV-30046 wrong row targeted with "insert ... on duplicate" and "replace"
When HA_DUPLICATE_POS is not supported, the row to replace was navigated by
ha_index_read_idx_map, which uses only hash to navigate.

Suchwise, given a hash collision it may choose an incorrect row.

handler::position would be correct and very convenient to use here.

dup_ref is already set by handler independently of the engine
capabilities, when an extra lookup is made (for long unique or something else,
for example WITHOUT OVERLAPS) such error will be indicated by
file->lookup_errkey != -1.
2024-05-05 18:38:34 +02:00
Alexander Barkov
7f161a5c58 MDEV-34088 The TIMESTAMP value of '1970-01-01 00:00:00' can be indirectly inserted in strict mode
In strict mode a timestamp(0) column could be directly assigned from
another timestamp(N>0) column with the value '1970-01-01 00:00:00.1'
(at time zone '+00:00'), or with any other value '1970-01-01 00:00:00.XXXXXX'
with non-zero microsecond value XXXXXX.
This assignment happened silently without warnings or errors.

It worked as follows:

- The value {tv_sec=0, tv_usec=100000}, which is '1970-01-01 00:00:00.1'
  was rounded to {tv_sec=0, tv_usec=0}, which is '1970-01-01 00:00:00.0'

- Then {tv_sec=0, tv_usec=0} was silently re-interpreted as zero datetime.

After the fix this assignment always raises a warning,
which in case of the strict mode is escalated to an error.

The problem in this scenario is that '1970-01-01 00:00:00' cannot be stored,
because its timeval value {tv_sec=0, tv_usec=0} is reserved for zero datetimes.
Thus the warning should be raised no matter if sql_mode allows or disallows
zero dates.
2024-05-05 16:31:18 +04:00
Alexander Barkov
2c19877015 MDEV-34061 unix_timestamp(coalesce(timestamp_column)) returns NULL on '1970-01-01 00:00:00.000001'
Field_timestampf::val_native() checked only the
first four bytes to detect zero dates.
That was not enough. Fixing the code to check all packed_length()
bytes to detect zero dates.
2024-05-04 23:41:55 +04:00
Alexander Barkov
1cdf22374b MDEV-34069 Zero datetime reinterprets as '1970-01-01 00:00:00' on field_datetime=field_timestamp
The code in Field_timestamp::save_in_field() did not catch
zero datetime and stored it to the other field like a usual value
using store_timestamp_dec(), which knows nothing about zero date and
treats {tv_sec=0, tv_usec=0} as a normal timeval value corresponding to
'1970-01-01 00:00:00 +00:00'.

Fixing the code to catch the special combination (ts==0 && sec_pat==0) and
store it using store_time_dec() with a zero datetime passed as an argument.
2024-05-04 22:39:58 +04:00
Sergei Golubchik
9dfef3fb41 fix sporadic failures of main.lock_sync
wait for all connections to disconnect before the cleanup
2024-05-02 11:04:16 +02:00
Sergei Golubchik
0aae11ac28 Merge branch '10.6' into 10.11 2024-04-30 16:56:49 +02:00
Dimitri John Ledkov
bf77f9793d openssl: add a more specific DES support detection
Improve detection for DES support in OpenSSL, to allow compilation
against system OpenSSL without DES.

Note that MariaDB needs to be compiled against OpenSSL-like library
that itself has DES support which cmake detected. Positive detection
is indicated with CMake variable HAVE_des 1.

Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@surgut.co.uk>
2024-04-30 23:09:02 +10:00
Rucha Deodhar
9e6858a426 MDEV-22141: JSON_REMOVE returns NULL on valid arguments
Analysis:
When we scan json to get to a beginning according to the path, we end up
scanning json even if we have exhausted it. When eventually returns error.

Fix:
Continue scanning json only if we have not exhausted it and return result
accordingly.
2024-04-29 22:32:17 +05:30
Rucha Deodhar
5ca64e65d0 MDEV-32287: JSON_EXTRACT not returning multiple values for same path
Analysis:
When scanning json and getting the exact path at each step, if a path
is reached, we end up adding the item in the result and immediately get the
next item which results in current path changing.
Fix:
Instead of immediately returning the item, count the occurences of the path
in argument and append in the result as needed.
2024-04-29 22:32:17 +05:30
Rucha Deodhar
d7df63e1c9 MDEV-19487: JSON_TYPE doesnt detect the type of String Values
(returns NULL) and for Date/DateTime returns "INTEGER"

Analysis:
When the first character of json is scanned it is number. Based on that
integer is returned.
Fix:
Scan rest of the json before returning the final result to ensure json is
valid in the first place in order to have a valid type.
2024-04-29 22:32:17 +05:30
Alexander Barkov
c6e3fe29d4 MDEV-30646 View created via JSON_ARRAYAGG returns incorrect json object
Backporting add782a13e from 10.6, this fixes the problem.
2024-04-29 13:47:45 +04:00
Sergei Golubchik
c1f3eff53f Merge branch '10.5' into 10.6 2024-04-29 10:08:58 +02:00
Alexander Barkov
dc25d600ee MDEV-21058 CREATE TABLE with generated column and RLIKE results in sigabrt
Regexp_processor_pcre::fix_owner() called Regexp_processor_pcre::compile(),
which could fail on the regex syntax error in the pattern and put
an error into the diagnostics area. However, the callers:
  - Item_func_regex::fix_length_and_dec()
  - Item_func_regexp_instr::fix_length_and_dec()
still returned "false" in such cases, which made the code
crash later inside Diagnostics_area::set_ok_status().

Fix:

- Change the return type of fix_onwer() from "void" to "bool"
  and return "true" whenever an error is put to the DA
  (e.g. on the syntax error in the pattern).
- Fixing fix_length_and_dec() of the mentioned Item_func_xxx
  classes to return "true" if fix_onwer() returned "true".
2024-04-29 11:08:07 +04:00
Alexander Barkov
3141a68b7c MDEV-33534 UBSAN: Negation of -X cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in my_double_round from sql/item_func.cc|
The negation in this line:
  ulonglong abs_dec= dec_negative ? -dec : dec;
did not take into account that 'dec' can be the smallest possible
signed negative value -9223372036854775808. Its negation is
an operation with an undefined behavior.

Fixing the code to use Longlong_hybrid, which implements a safe
method to get an absolute value.
2024-04-27 23:16:35 +04:00
Oleksandr Byelkin
62287320d4 MDEV-33790 Incorrect DEFAULT expression evaluated in UPDATE
The problem was that Item_default_value::associate_with_target_field
assigned passed as argument field as an argument which changed argument
in case of default() call with certain field (i.e. deault(field)).

There is no way to get wrong field in constructor so we will not reassign
parameter.
2024-04-25 14:11:28 +02:00
Alexander Barkov
e02077aa03 MDEV-21076 NOT NULL and UNIQUE constraints cause SUM() to yield an incorrect result
This problem was earlier fixed by the patch for MDEV 33344.
Adding a test case only.
2024-04-23 10:45:39 +04:00
Alexander Barkov
24abbb9bdb MDEV-21034 GREATEST() and LEAST() malfunction for NULL
There is a convention that Item::val_int() and Item::val_real() return
SQL NULL doing effectively what this code does:
  null_value= true;
  return 0; // Always return 0 for SQL NULL

This is done to optimize boolean value evaluation:
if Item::val_int() or Item::val_real() returned 1 -
that always means TRUE and never can means SQL NULL.
This convention helps to avoid unnecessary testing
Item::null_value after getting a non-zero return value.

Item_func_min_max did not follow this convention.
It could return a non-zero value together with null_value==true.
This made evaluate_join_record() erroneously misinterpret
SQL NULL as TRUE in this call:

  select_cond_result= MY_TEST(select_cond->val_int());

Fixing Item_func_min_max to follow the convention.
2024-04-23 02:38:26 +04:00
Markus Staab
361b790392 Remove unnecessary whitespace in mysqldump 2024-04-22 21:41:29 +01:00
Sergei Golubchik
018d537ec1 Merge branch '10.6' into 10.11 2024-04-22 15:23:10 +02:00
Sergei Golubchik
75488a57f2 archive.archive and main.mysqlbinlog_row_compressed
fixes for zlib-ng
2024-04-22 00:14:03 +02:00
Sergei Golubchik
aa4bcdbbb8 main.func_sformat: fixes for fmt 10.2.1 2024-04-22 00:14:03 +02:00
Sergei Golubchik
f0f9dc8631 enable main.func_sformat in --view 2024-04-22 00:14:02 +02:00
Sergei Golubchik
63ac87c121 make main.mysqlbinlog_row_compressed independent from the environment
don't depend on the thread ids, table ids, and current binlog content
2024-04-22 00:14:02 +02:00
Sergei Golubchik
1437e734f7 adjust timeout value in main.ssl_timeout test
fixes sporadic failures under --valgrind
2024-04-21 10:47:20 +02:00
Sergei Golubchik
d8368ae289 Merge '10.5' into 10.6 2024-04-20 14:47:26 +02:00
Zhibo Zhang
7432a487b1 Update tests to be compatible with OpenSSL 3.2.0
As of version 3.2.0, OpenSSL updated the error message in new versions
("https://github.com/openssl/openssl/commit/81b741f68984"). Update the
tests and result files such that they are compatible with both original
and new error messages.

All new code of the whole pull request, including one or several files that are
either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services,
Inc.
2024-04-19 15:44:28 +01:00
Vladislav Vaintroub
2e84560dc4 MDEV-16944 postfix. Fix a typo 2024-04-18 09:45:41 +02:00
Marko Mäkelä
bb2e125d07 Merge 10.5 into 10.6
This excludes commit 040069f4ba
because it is specific to innodb_sync_debug, which had been removed
in commit ff5d306e29.
2024-04-18 07:14:56 +03:00
Vladislav Vaintroub
061adae9a2 MDEV-16944 Fix file sharing issues on Windows in mysqltest
On Windows systems, occurrences of ERROR_SHARING_VIOLATION due to
conflicting share modes between processes accessing the same file can
result in CreateFile failures.

mysys' my_open() already incorporates a workaround by implementing
wait/retry logic on Windows.

But this does not help if files are opened using shell redirection like
mysqltest traditionally did it, i.e via

--echo exec "some text" > output_file

In such cases, it is cmd.exe, that opens the output_file, and it
won't do any sharing-violation retries.

This commit addresses the issue by introducing a new built-in command,
'write_line', in mysqltest. This new command serves as a brief alternative
to 'write_file', with a single line output, that also resolves variables
like "exec" would.

Internally, this command will use my_open(), and therefore retry-on-error
logic.

Hopefully this will eliminate the very sporadic "can't open file because
it is used by another process" error on CI.
2024-04-17 16:52:37 +02:00
Marko Mäkelä
829cb1a49c Merge 10.5 into 10.6 2024-04-17 14:14:58 +03:00
Marko Mäkelä
9164c2b8bb Tests: remove a duplicated check
This fixes up the merge commit 9b18275623
2024-04-17 10:10:23 +03:00
Sergei Golubchik
41e7ceb0ac MDEV-33889 Read only server throws error when running a create temporary table as select statement
create_partitioning_metadata() should only mark transaction r/w
if it actually did anything (that is, the table is partitioned).

otherwise it's a no-op, called even for temporary tables and
it shouldn't do anything at all
2024-04-16 20:43:31 +02:00
Oleksandr Byelkin
9b18275623 Merge branch '10.4' into 10.5 2024-04-16 11:04:14 +02:00
Oleksandr Byelkin
50998a6c6f MDEV-33861 main.query_cache fails with embedded after enabling WITH_PROTECT_STATEMENT_MEMROOT
Synopsis: If SELECT returned answer from Query Cache it is not really executed.

The reason for firing of assertion
  DBUG_ASSERT((mem_root->flags & ROOT_FLAG_READ_ONLY) == 0);
is that in case the query_cache is on and the same query run by different
stored routines the following use case can take place:
First, lets say that bodies of routines used by the test case are the same
and contains the only query 'SELECT * FROM t1';
  call p1() -- a result set is stored in query cache for further use.
  call p2() -- the same query is run against the table t1, that result in
               not running the actual query but using its cached result.
               On finishing execution of this routine, its memory root is
               marked for read only since every SP instruction that this
               routine contains has been executed.
  INSERT INT t1 VALUE (1); -- force following invalidation of query cache
  call p2() -- query the table t1 will result in assertion failure since its
               execution would require allocation on the memory root that
               has been already marked as read only memory root

The root cause of firing the assertion is that memory root of the stored
routine 'p2' was marked as read only although actual execution of the query
contained inside hadn't been performed.

To fix the issue, mark a SP instruction as not yet run in case its execution
doesn't result in real query processing and a result set got from query cache
instead.

Note that, this issue relates server built in debug mode AND with the protect
statement memory root feature turned on. It doesn't affect server built
in release mode.
2024-04-16 08:52:51 +02:00
Kristian Nielsen
16aa4b5f59 Merge from 10.4 to 10.5
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-04-15 17:46:49 +02:00
Sergei Golubchik
6a4ac4c72d Fixed random failure in main.kill_processlist-6619 (take 3)
followup for 81f75ca83a

improve over take 2. It's technically possible, though unlikely,
to see THD after it already reset the info to NULL, but has not
changed the command to COM_SLEEP yet (see THD::mark_connection_idle()).

Let's wait for "Sleep", not for NULL.
2024-04-13 16:28:13 +02:00
Tony Chen
79706fd386 Minor improvements to options error handling
- Add additional MTRs for more coverage on invalid options
- Updating a few error messages to be more informative
- Use the exit code from handle_options() when there is an error processing
  user options

All new code of the whole pull request, including one or several files that are
either new files or modified ones, are contributed under the BSD-new license. I
am contributing on behalf of my employer Amazon Web Services, Inc.
2024-04-13 19:02:33 +07:00
Tony Chen
47d75cdd80 MDEV-33469 Fix behavior on invalid arguments
When passing in an invalid value (e.g. incorrect data type) for a variable, the
server startup will fail with misleading error messages.

The behavior **before** this change:

For server options:
- The error message will indicate that the argument is being adjusted to a valid value
- Server startup still fails

For plugin options:
- The error message will indicate that the argument is being adjusted to a valid value
- The plugin is still disabled
- Server startup fails with a message that it does not recognize the plugin option

The behavior **after** this change:

For server options:
- Output that an invalid argument was provided
- Exit server startup

For plugin options:
- Output that an invalid argument was provided
- Disable the plugin
- Attempt to continue server startup

All new code of the whole pull request, including one or several files that are
either new files or modified ones, are contributed under the BSD-new license. I
am contributing on behalf of my employer Amazon Web Services, Inc.
2024-04-13 19:02:33 +07:00
Tony Chen
dd639985c1 Simplify MTR for handling multiple invalid options
In 69a4d6ae, an MTR test was added to verify that we handled multiple invalid
options.  However, the logic to perform this test relied on a non-trivial regex
to filter out the noise in the logs.

Instead, we now just simply search for what we expect to be in the logs.

All new code of the whole pull request, including one or several files that are
either new files or modified ones, are contributed under the BSD-new license. I
am contributing on behalf of my employer Amazon Web Services, Inc.
2024-04-13 19:02:33 +07:00
Sergei Golubchik
41296a07c8 Merge branch '10.5' into 10.6 2024-04-11 13:58:22 +02:00
Alexander Barkov
b697dce8ca MDEV-29149 Assertion `!is_valid_datetime() || fraction_remainder(((item->decimals) < (6) ? (item->decimals) : (6))) == 0' failed in Datetime_truncation_not_needed::Datetime_truncation_not_needed
TIME-alike string and numeric arguments to TIMEDIFF()
can get additional fractional seconds during the supported
TIME range adjustment in get_time().

For example, during TIMEDIFF('839:00:00','00:00:00') evaluation
in Item_func_timediff::get_date(), the call for args[0]->get_time()
returns MYSQL_TIME '838:59:59.999999'.

Item_func_timediff::get_date() did not handle these extra digits
and returned a MYSQL_TIME result with fractional digits outside
of Item_func_timediff::decimals. This mismatch could further be
caught by a DBUG_ASSERT() in various other pieces of the code,
leading to a crash.

Fix:

In case if get_time() returned MYSQL_TIMESTAMP_TIME,
let's truncate all extra digits using my_time_trunc(&l_time,decimals).
This guarantees that the rest of the code returns a MYSQL_TIME
with second_part not conflicting with Item_func_timediff::decimals.
2024-04-10 17:02:24 +04:00
Alexander Barkov
d4936c8b26 MDEV-18898 SELECT using wrong index when using operator IN with mixed types
These patches:

  # commit 74891ed257
  #
  #  MDEV-11514, MDEV-11497, MDEV-11554, MDEV-11555 - IN and CASE type aggregation problems

  # commit 53499cd1ea
  #
  # MDEV-31303 Key not used when IN clause has both signed and usigned values

earlier fixed MDEV-18898.

Adding only an MTR case.

	modified:   mysql-test/main/func_in.result
	modified:   mysql-test/main/func_in.test
2024-04-09 16:05:56 +04:00
Alexander Barkov
6606abb6a4 MDEV-18319 BIGINT UNSIGNED Performance issue
The patch for MDEV-18319 BIGINT UNSIGNED Performance issue
fixed this problem in 10.5.23.

This patch adds only an MTR test to cover MDEV-18319.
2024-04-09 13:27:49 +04:00
Rucha Deodhar
fcd345de48 MDEV-32726: Fix failing test fir freebsd for json
Json test about max statement time fails with freebsd because on some
architectures the test might execute faster and the statement may not fail.

To simulate failure regardless of architecture, introduce a wait of seconds
longer than the max_statement_time.
2024-04-08 20:20:44 +05:30
Rucha Deodhar
3c40f8bafb MDEV-31402: SIGSEGV in json_get_path_next | Item_func_json_extract::read_json 2024-04-08 19:19:39 +05:30
Sergei Golubchik
a7bf0a42d0 sporadic failures of main.mdl_sync
main.mdl_sync 'innodb'                   w32 [ fail ]
        Test ended at 2024-04-06 14:11:15

CURRENT_TEST: main.mdl_sync
--- main/mdl_sync.result
+++ main/mdl_sync.reject
@@ -2458,6 +2458,7 @@
 SELECT LOCK_MODE, LOCK_TYPE, TABLE_SCHEMA, TABLE_NAME FROM information_schema.metadata_lock_info;
 LOCK_MODE	LOCK_TYPE	TABLE_SCHEMA	TABLE_NAME
 MDL_BACKUP_FTWRL2	Backup lock
+MDL_SHARED	Table metadata lock	test	t2
 unlock tables;
 connection default;
 # Reaping UPDATE
2024-04-06 23:16:21 +02:00
Sergei Petrunia
8cc36fb743 MDEV-21102: Server crashes in JOIN_CACHE::write_record_data upon EXPLAIN with subqueries
JOIN_CACHE has a light-weight initialization mode that's targeted at
EXPLAINs. In that mode, JOIN_CACHE objects are not able to execute.

Light-weight mode was used whenever the statement was an EXPLAIN. However
the EXPLAIN can execute subqueries, provided they enumerate less than
@@expensive_subquery_limit rows.

Make sure we use light-weight initialization mode only when the select is
more expensive @@expensive_subquery_limit.

Also add an assert into JOIN_CACHE::put_record() which prevents its use
if it was initialized for EXPLAIN only.
2024-04-04 10:30:09 +03:00
joshhn
4987b5e3b1 MDEV-33803 Error 4162 "Operator does not exists" is incorrectly-worded
"Operator does not exists" should rather read "Operator does not exist".
2024-04-03 10:03:02 +11:00
Vladislav Vaintroub
40973d855c MDEV-32926 mysql_install_db_win fails on buildbot
In mysql_install_db_win_admin test, dump bootstrap output in case of
bootstrap failure.
2024-04-02 20:59:01 +02:00
Alexander Barkov
29bb321f04 MDEV-33788 HEX(COLUMN_CREATE(.. AS CHAR ...)) fails with --view-protocol
Item_func_dyncol_create::print_arguments() printed only CHARSET clause
without COLLATE.

Therefore,

HEX(column_create(1,'1212' AS CHAR CHARACTER SET utf8mb3 COLLATE utf8mb3_bin))

inside a VIEW changed to just:

HEX(column_create(1,'1212' AS CHAR CHARACTER SET utf8mb3))

which changed the collation ID seen in the HEX output.

Note, the collation ID inside column_create() is not really much important.
(It's only important what the character set is).
And for COLLATE, the more important thing is what's later written
in the AS clause of COLUMN_GET:

SELECT
   COLUMN_GET(
    column_create(1,'1212' AS CHAR CHARACTER SET utf8mb3 COLLATE utf8mb3_bin)
    column_nr AS type  -- this type is more important
   );

Still, let's add the COLLATE clause into the COLUMN_CREATE() print output,
although it's not important for now for anything else than just the HEX output.
At least to make VIEW work in a more predictable way with HEX(COLUMN_CREATE()).

Also, in the future we can start using somehow the collation ID written inside
COLUMN_CREATE(), for example by making the `AS type` clause optional in
COLUMN_GET():
  COLUMN_GET(dyncol_blob, column_nr [AS type]);
instead of:
  COLUMN_GET(dyncol_blob, column_nr AS type);

SQL Server compatibility layer may need this for
the SQL_Variant data type support.
2024-03-29 05:45:06 +04:00
Dmitry Shulga
e1876e7f78 MDEV-33768: Memory leak found in the test main.constraints run with --ps-protocol against a server built with the option -DWITH_PROTECT_STATEMENT_MEMROOT
The discovered memory leak was introduced by the commit
  762bf7a03b
    (MDEV-22602 Disable UPDATE CASCADE for SQL constraints)

The reason why a memory leaked on running the test main.constraints
is that a statement arena was used for allocation a memory
for storing a constraint name. A constraint name is an entity having
temporary nature by its design so runtime arena should be used for its
allocation.
2024-03-28 14:53:58 +07:00
Marko Mäkelä
788953463d Merge 10.6 into 10.11
Some fixes related to commit f838b2d799 and
Rows_log_event::do_apply_event() and Update_rows_log_event::do_exec_row()
for system-versioned tables were provided by Nikita Malyavin.
This was required by test versioning.rpl,trx_id,row.
2024-03-28 09:16:57 +02:00
Daniel Black
9f1019ba3d MDEV-33044 Loading time zones does not work with alter_algorithm INPLACE (postfix)
Test case doesn't work on embedded builds.
2024-03-28 14:47:29 +11:00
Anson Chung
7890388d91 MDEV-33044 Loading time zones does not work with alter_algorithm INPLACE
$MYSQL_TZINFO_TO_SQL works by truncating tables. Truncation is an
operation that cannot be done in-place and therefore is fundamentally
incompatible with alter_algorithm='INPLACE'. As a result, we override
the default alter_algorithm setting in tztime.cc to
alter_algorithm='COPY' so that timezones can be loaded regardless
of the previously set alter_algorithm.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
2024-03-28 09:37:22 +11:00
Sergei Golubchik
81f75ca83a Fixed random failure in main.kill_processlist-6619
wait for all previous connections to disconnect and for all previous
queries to finish running
2024-03-27 16:14:56 +01:00
Sergei Golubchik
3226787bba Revert "Fixed random failure in main.kill_processlist-6619"
This reverts commit 8b3f470c0b.

because it doesn't work, the test still fails, and even more than before
2024-03-27 16:14:56 +01:00
Sergei Golubchik
c84d67a302 reenable main.mysqldump-system test 2024-03-27 16:14:55 +01:00
Sergei Golubchik
c77680768c MDEV-33460 use the correct sql_mode and fix for --view 2024-03-27 16:14:55 +01:00
Marko Mäkelä
ccb7a1e9a1 Merge 10.5 into 10.6 2024-03-27 15:00:56 +02:00
Alexander Barkov
0fc123c595 MDEV-33772 Bad SEPARATOR value in GROUP_CONCAT on character set conversion
Item_func_group_concat::print() did not take into account
that Item_func_group_concat::separator can be of a different character set
than the "String *str" (when the printing is being done to).
Therefore, printing did not work correctly for:
- non-ASCII separators when GROUP_CONCAT is done on 8bit data
  or multi-byte data with mbminlen==1.
- all separators (even including simple ones like comma)
  when GROUP_CONCAT is done on ucs2/utf16/utf32 data (mbminlen>1).

Because of this problem, VIEW definitions did not print correctly to
their FRM files. This later led to a wrong SELECT and SHOW CREATE output.

Fix:

- Adding new String methods:

  bool append_for_single_quote_using_mb_wc(const char *str, size_t length,
                                           CHARSET_INFO *cs);

  bool append_for_single_quote_opt_convert(const char *str,
                                           size_t length,
                                           CHARSET_INFO *cs)

  which perform both escaping and character set conversion at the same time.

- Adding a new String method escaped_wc_for_single_quote(),
  to reuse the code between the old and the new methods.

- Fixing Item_func_group_concat::print() to use the new
  method append_for_single_quote_opt_convert().
2024-03-27 15:22:58 +04:00
Dave Gosselin
58df20974b MDEV-33460 select '123' 'x'; unexpected result
Queries that select concatenated constant strings now have
colname and value that match.  For example,
  SELECT '123' 'x';
will return a result where the column name and value both
are '123x'.

Review: Daniel Black
2024-03-27 15:51:26 +11:00
Vladislav Vaintroub
d695e2de54 MDEV-33506 Show original IP in the "aborted" message.
Add "real ip:<ip_or_localhost>" part to the aborted message
Only for proxy-protocoled connection, so it does not  not to cause
confusion to normal users.
2024-03-26 13:10:36 +01:00
Vladislav Vaintroub
318000cffc MDEV-33506 Show original IP in the "aborted" message.
Add "real ip:<ip_or_localhost>" part to the aborted message
Only for proxy-protocoled connection, so it does not  not to cause
confusion to normal users.
2024-03-26 11:11:03 +01:00
Sergei Petrunia
ed027d65f1 MDEV-33747: Optimization of (SELECT) IN (SELECT ...) executes subquery at prepare stage
Make IN->EXISTS rewrite not to compute constant left expression if it
has a subquery in it.
2024-03-26 12:45:36 +03:00
Daniel Black
c1da568502 MDEV-33726 Moving from MariaDB 10.5 to 10.6 mysql_upgrade
.. is not updating some system tables

Some schema changes from MDEV-24312 master_host has 60 character limit, increase to 255 bytes
failed to happen in the upgrade for tables in the mysql schema:
* mysql.global_priv
* mysql.procs_priv
* mysql.proxies_priv
* mysql.roles_mapping
2024-03-26 10:28:27 +11:00
Marko Mäkelä
b8a6719889 MDEV-26642/MDEV-26643/MDEV-32898 Implement innodb_snapshot_isolation
https://jepsen.io/analyses/mysql-8.0.34 highlights that the
transaction isolation levels in the InnoDB storage engine do not
correspond to any widely accepted definitions, such as
"Generalized Isolation Level Definitions"
https://pmg.csail.mit.edu/papers/icde00.pdf
(PL-1 = READ UNCOMMITTED, PL-2 = READ COMMITTED, PL-2.99 = REPEATABLE READ,
PL-3 = SERIALIZABLE).
Only READ UNCOMMITTED in InnoDB seems to match the above definition.

The issue is that InnoDB does not detect write/write conflicts
(Section 4.4.3, Definition 6) in the above.

It appears that as soon as we implement write/write conflict detection
(SET SESSION innodb_snapshot_isolation=ON), the default isolation level
(SET TRANSACTION ISOLATION LEVEL REPEATABLE READ) will become
Snapshot Isolation (similar to Postgres), as defined in Section 4.2 of
"A Critique of ANSI SQL Isolation Levels", MSR-TR-95-51, June 1995
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf

Locking reads inside InnoDB used to read the latest committed version,
ignoring what should actually be visible to the transaction.
The added test innodb.lock_isolation illustrates this. The statement
	UPDATE t SET a=3 WHERE b=2;
is executed in a transaction that was started before a read view or
a snapshot of the current transaction was created, and committed before
the current transaction attempts to execute
	UPDATE t SET b=3;
If SET innodb_snapshot_isolation=ON is in effect when the second
transaction was started, the second transaction will be aborted with
the error ER_CHECKREAD. By default (innodb_snapshot_isolation=OFF),
the second transaction would execute inconsistently, displaying an
incorrect SELECT COUNT(*) FROM t in its read view.

If innodb_snapshot_isolation=ON, if an attempt to acquire a lock on a
record that does not exist in the current read view is made, an error
DB_RECORD_CHANGED (HA_ERR_RECORD_CHANGED, ER_CHECKREAD) will
be raised. This error will be treated in the same way as a deadlock:
the transaction will be rolled back.

lock_clust_rec_read_check_and_lock(): If the current transaction has
a read view where the record is not visible and
innodb_snapshot_isolation=ON, fail before trying to acquire the lock.

row_sel_build_committed_vers_for_mysql(): If innodb_snapshot_isolation=ON,
disable the "semi-consistent read" logic that had been implemented by
myself on the directions of Heikki Tuuri in order to address
https://bugs.mysql.com/bug.php?id=3300 that was motivated by a customer
wanting UPDATE to skip locked rows that do not match the WHERE condition.
It looks like my changes were included in the MySQL 5.1.5
commit ad126d90e019f223470e73e1b2b528f9007c4532; at that time, employees
of Innobase Oy (a recent acquisition of Oracle) had lost write access to
the repository.

The only reason why we set innodb_snapshot_isolation=OFF by default is
backward compatibility with applications, such as the one that motivated
the implementation of "semi-consistent read" back in 2005. In a later
major release, we can default to innodb_snapshot_isolation=ON.

Thanks to Peter Alvaro, Kyle Kingsbury and Alexey Gotsman for their work
on https://github.com/jepsen-io/ and to Kyle and Alexey for explanations
and some testing of this fix.

Thanks to Vladislav Lesin for the initial test for MDEV-26643,
as well as reviewing these changes.
2024-03-20 09:48:03 +02:00
Marko Mäkelä
50715bd2ed Merge 10.5 into 10.6 2024-03-18 17:07:32 +02:00
Marko Mäkelä
4592af2e84 Work around missing MSAN instrumentation
Let us skip the recently added test main.mysql-interactive if
an instrumented ncurses library is not available.

In InnoDB, let us work around an uninstrumented libnuma, by
declaring that the objects returned by numa_get_mems_allowed()
are initialized.
2024-03-18 16:01:58 +02:00
mariadb-DebarunBanerjee
d912a6369c MDEV-31154 Fatal InnoDB error or assertion `!is_v' failure upon multi-update with indexed virtual column
MDEV-33558 Fatal error InnoDB: Clustered record field for column x not found

This is issue is about row ID filtering used with index on virtual
column(s). We hit debug assert and crash while building the record
template in Innodb. The primary reason is that we try to force the code
path to use the ICP path. With ICP, we don't support index with virtual
column and we validate it while index condition is pushed.

Simplify the code for building template to handle both ICP and Row ID
filtering by skipping virtual columns.
2024-03-15 19:29:46 +05:30
Kristian Nielsen
ef7abc881c MDEV-10793: MDEV-33292: main.kill_processlist-6619 fails sporadically in buildbot
There were several races in the main.kill_processlist-6619 testcase:

 - Lingering connections from a previous test case could be visible in SHOW
   PROCESSLIST and cause .result diff.
 - A sync point "dispatch_command_end" was ineffective, as it was consumed at
   the end of the SET DEBUG command itself.
 - The signal from sync point "before_execute_sql_command" could override an
   earlier signal, causing DEBUG_SYNC timeout and test failure.
 - The final SHOW PROCESSLIST could occasionally see a connection in state
   "Busy" instead of the expected "Sleep".

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-03-14 22:48:12 +01:00
Monty
ae063e4ff5 Fixed random failure in main.kill_processlist-6619
The problem was that SHOW PROCESSLIST was done before the command of
the default connection was cleared.

Reviewer: Sergei Golubchik <serg@mariadb.org>
2024-03-14 21:02:29 +01:00
Dmitry Shulga
d7758debae MDEV-33218: Assertion `active_arena->is_stmt_prepare_or_first_stmt_execute() || active_arena->state == Query_arena::STMT_SP_QUERY_ARGUMENTS' failed in st_select_lex::fix_prepare_information
In case there is a view that queried from a stored routine or
a prepared statement and this temporary table is dropped between
executions of SP/PS, then it leads to hitting an assertion
at the SELECT_LEX::fix_prepare_information. The fired assertion
 was added by the commit 85f2e4f8e8
(MDEV-32466: Potential memory leak on executing of create view statement).
Firing of this assertion means memory leaking on execution of SP/PS.
Moreover, if the added assert be commented out, different result sets
can be produced by the statement SELECT * FROM the hidden table.

Both hitting the assertion and different result sets have the same root
cause. This cause is usage of temporary table's metadata after the table
itself has been dropped. To fix the issue, reload the cache of stored
routines. To do it  cache of stored routines is reset at the end of
execution of the function dispatch_command(). Next time any stored routine
be called it will be loaded from the table mysql.proc. This happens inside
the method Sp_handler::sp_cache_routine where loading of a stored routine
is performed in case it missed in cache. Loading is performed unconditionally
while previously it was controlled by the parameter lookup_only. By that
reason the signature of the method Sroutine_hash_entry::sp_cache_routine
was changed by removing unused parameter lookup_only.

Clearing of sp caches affects the test main.lock_sync since it forces
opening and locking the table mysql.proc but the test assumes that each
statement locks its tables once during its execution. To keep this invariant
the debug sync points with names "before_lock_tables_takes_lock" and
"after_lock_tables_takes_lock" are not activated on handling the table
mysql.proc
2024-03-14 15:43:03 +07:00
Sergei Golubchik
f71d7f2f0f Merge branch '10.5' into 10.6 2024-03-13 21:02:34 +01:00
Sergei Golubchik
0e8cda6130 MDEV-33313 Incorrect error message for "ALTER TABLE ... DROP CONSTRAINT ..., DROP col, DROP col" 2024-03-13 18:27:19 +01:00
Sergei Golubchik
4cda50afbd Merge branch '10.4' into 10.5 2024-03-13 16:18:37 +01:00
Sergei Golubchik
62a9a54a94 MDEV-33344 REGEXP empty string inconsistent 2024-03-13 15:01:32 +01:00
Sergei Golubchik
7828aadd3a MDEV-33318 ORDER BY COLLATE improperly applied to non-character columns
when changing charset from latin1 to utf8, adjust max_length accordingly
2024-03-13 15:01:32 +01:00
Dmitry Shulga
428a673152 MDEV-33549: Incorrect handling of UPDATE in PS mode in case a table's colum declared as NOT NULL
UPDATE statement that is run in PS mode and uses positional parameter
handles columns declared with the clause DEFAULT NULL incorrectly in
case the clause DEFAULT is passed as actual value for the positional
parameter of the prepared statement. Similar issue happens in case
an expression specified in the DEFAULT clause of table's column definition.

The reason for incorrect processing of columns declared as DEFAULT NULL
is that setting of null flag for a field being updated was missed
in implementation of the method Item_param::assign_default().
The reason for incorrect handling of an expression in DEFAULT clause is
also missed saving of a field inside implementation of the method
Item_param::assign_default().
2024-03-12 16:13:49 +07:00
Marko Mäkelä
c3a00dfa53 Merge 10.5 into 10.6 2024-03-12 09:19:57 +02:00
Marko Mäkelä
f703e72bd8 Merge 10.4 into 10.5 2024-03-11 10:08:20 +02:00
Kristian Nielsen
23c48474f7 MDEV-33212: mysqldump uses MASTER_LOG_POS with dump-slave
The patch for MDEV-15530 incorrectly added a column in the middle of SHOW
SLAVE STATUS output. This is wrong, as it breaks backwards compatibility
with existing applications and scripts. In this case, it even broke
mariadb-dump, which is included in the server source tree!

Revert the incorrect change, putting the new Replicate_Rewrite_DB at the end
of SHOW SLAVE STATUS output.

Add a testcase for the mariadb-dump --dump-slave wrong output problem. Also
add a testcase rpl.rpl_show_slave_status to hopefully prevent any future
incorrect additions to SHOW SLAVE STATUS.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-03-08 15:23:42 +01:00
Monty
11c75fc396 Fixed sporadically failing test show_explain_json.test
Code taken from MDEV-33423: show_analyze sporadically fails

Thanks to Sergei Petrunia for pointing this out.
2024-03-04 16:03:42 +02:00
Monty
611d442510 Fixed mtr random bug in lock_sync.test 2024-03-04 15:31:56 +02:00