Commit graph

77843 commits

Author SHA1 Message Date
Marko Mäkelä
be24e75229 Merge 10.11 into 11.0 2023-10-19 08:12:16 +03:00
Marko Mäkelä
2ecc0443ec Merge 10.10 into 10.11 2023-10-17 16:04:21 +03:00
Marko Mäkelä
0563106b1a Merge 10.6 into 10.10 2023-10-17 13:02:57 +03:00
Thirunarayanan Balathandayuthapani
ee5cadd5c8 MDEV-28122 Optimize table crash while applying online log
- InnoDB fails to check the overflow buffer while applying
the operation to the table that was rebuilt. This is caused
by commit 3cef4f8f0f (MDEV-515).
2023-10-16 20:17:09 +05:30
Monty
cca9547892 Post fix for MDEV-32449 2023-10-16 12:55:17 +03:00
Monty
1c554459b3 MDEV-32449 Server crashes in Alter_info::add_stat_drop_index upon CREATE TABLE
Fixed missing initialization of Alter_info()

This could cause crashes in some create table like scenarios
where some generated indexes where automatically dropped.

I also added a test that we do not try to drop from index_stats for
temporary tables.
2023-10-14 15:46:29 +03:00
Monty
ec277a70e8 Do not create histograms for single column unique key
The intentention was always to not create histograms for single value
unique keys (as histograms is not useful in this case), but because of
a bug in the code this was still done.

The changes in the test cases was mainly because hist_size is now NULL
for these kind of columns.
2023-10-14 13:43:26 +03:00
Marko Mäkelä
d5e15424d8 Merge 10.6 into 10.10
The MDEV-29693 conflict resolution is from Monty, as well as is
a bug fix where ANALYZE TABLE wrongly built histograms for
single-column PRIMARY KEY.
Also includes a fix for safe_malloc error reporting.

Other things:
- Copied main.log_slow from 10.4 to avoid mtr issue

Disabled test:
- spider/bugfix.mdev_27239 because we started to get
  +Error	1429 Unable to connect to foreign data source: localhost
  -Error	1158 Got an error reading communication packets
- main.delayed
  - Bug#54332 Deadlock with two connections doing LOCK TABLE+INSERT DELAYED
    This part is disabled for now as it fails randomly with different
    warnings/errors (no corruption).
2023-10-14 13:36:11 +03:00
Vlad Lesin
18fa00a54c MDEV-32272 lock_release_on_prepare_try() does not release lock if supremum bit is set along with other bits set in lock's bitmap
The error is caused by MDEV-30165 fix with the following commit:
d13a57ae81

There is logical error in lock_release_on_prepare_try():

        if (supremum_bit)
          lock_rec_unlock_supremum(*cell, lock);
        else
          lock_rec_dequeue_from_page(lock, false);

Because there can be other bits set in the lock's bitmap, and the lock
type can be suitable for releasing criteria, but the above logic
releases only supremum bit of the lock.

The fix is to release lock if it suits for releasing criteria and unlock
supremum if supremum is locked otherwise.

Tere is also the test for the case, which was reported by QA team. I
placed it in a separate files, because it requires debug build.

Reviewed by: Marko Mäkelä
2023-10-13 16:29:04 +03:00
Thirunarayanan Balathandayuthapani
cbad0bcd41 MDEV-31098 InnoDB Recovery doesn't display encryption message when no encryption configuration passed
- InnoDB fails to report the error when encryption configuration
wasn't passed. This patch addresses the issue by adding
the error while loading the tablespace and deferring the
tablespace creation.
2023-10-13 17:27:27 +05:30
Yuchen Pei
17810b7585
Merge branch '10.10' into 10.11 2023-10-13 18:05:20 +11:00
Daniel Black
3b38c2f358 MDEV-18200 MariaBackup full backup failed with InnoDB: Failing assertion: success
There are many filesystem related errors that can occur with
MariaBackup. These already outputed to stderr with a good description of
the error. Many of these are permission or resource (file descriptor)
limits where the assertion and resulting core crash doesn't offer
developers anything more than the log message. To the user, assertions
and core crashes come across as poor error handling.

As such we return an error and handle this all the way up the stack.
2023-10-13 10:01:01 +11:00
Monty
8bf17c579b MDEV-32388 MSAN / Valgrind errors in Item_func_like::get_mm_leaf upon query from partitioned table
The problem was that RANGE_OPT_PARAM was not completely initialized in
some cases.
Added bzero() to ensure that all elements are always initialized.
2023-10-10 14:05:02 +03:00
Monty
55534a2616 Removed warning from ssl_cipher.test 2023-10-10 14:05:02 +03:00
Monty
b159f05a63 MDEV-31957 Concurrent ALTER and ANALYZE collecting statistics can result in stale statistical data
Fixed hang when renaming index to original name
2023-10-10 11:12:26 +03:00
Otto Kekalainen
8941bdc474 Fix merge commit 5ea5291: No test file or result files should be executable
In commit 5ea5291 @sanja-byelkin for unknown reason switched the file mode
for 3 Galera tzinfo related test files from 644 -> 755. This exists only
from branch 10.6 onward:

    $ git checkout 10.5
    $ find mysql-test -executable -name *.test -or -executable -name *.result
    (no results)
    $ git checkout 10.6
    $ find mysql-test -executable -name *.test -or -executable -name *.result
    mysql-test/suite/galera/t/mysql_tzmysql-test/suite/galera/t/mysql_tzinfo_to_sql.test
    mysql-test/suite/galera/t/mariadb_tzinfo_to_sql.test
    mysql-test/suite/galera/r/mariadb_tzinfo_to_sql.resultinfo_to_sql.test

mysql-test/suite/galera/t/mariadb_tzinfo_to_sql.test
mysql-test/suite/galera/r/mariadb_tzinfo_to_sql.result

No test file nor test result file should be executable, so run chmod -x
on them.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2023-10-09 03:28:45 +11:00
Marko Mäkelä
625a150a86 Merge 10.5 into 10.6 2023-10-06 14:34:01 +03:00
Marko Mäkelä
6e9b421f77 MDEV-32364 Server crashes when starting server with high innodb_log_buffer_size
log_t::create(): Return whether the initialisation succeeded.
It may fail if too large an innodb_log_buffer_size is specified.
2023-10-06 14:16:01 +03:00
Aleksey Midenkov
05c99435a7 MDEV-31014 Database privileges are insufficient for CONVERT TABLE TO PARTITION
check_access() updates ACL of first TABLE_LIST (t1), but not second
(tp). After it is done we copy t1 ACL to tp.
2023-10-06 13:01:49 +03:00
Sergei Petrunia
5e2d08b5e8 MDEV-32335: main.partition_innodb2 fails in BB with different #rows
use --replace_column for rows, as the numbers are not important.
2023-10-03 15:40:42 +03:00
Michael Widenius
9ba8dc1413 MDEV-32164 Server crashes in JOIN::cleanup after erroneous query with view
The problem was that we did not handle errors properly in
JOIN::get_best_combination. In case an early error, JOIN->join_tab would
contain unintialized values, which would cause errors on cleanup().

The error in question was reported earlier, but not noticed until later.
One cause of this is that most of the sql_select.cc code just checks
thd->fatal_error and not thd->is_error().
Fixed by changing of checks of fatal_error to is_error().
2023-10-03 08:25:31 +03:00
Monty
d4347177c7 Change SEL_ARG::MAX_SEL_ARGS to a user defined variable optimizer_max_sel_args
This allows a user to to change the default value of MAX_SEL_ARGS (16000)
in the rare case where they neeed more generated SEL_ARGS (as part of
the range optimizer)
2023-10-03 08:25:31 +03:00
Monty
4e9322e2ff MDEV-32203 Raise notes when an index cannot be used on data type mismatch
Raise notes if indexes cannot be used:
- in case of data type or collation mismatch (diferent error messages).
- in case if a table field was replaced to something else
  (e.g. Item_func_conv_charset) during a condition rewrite.

Added option to write warnings and notes to the slow query log for
slow queries.

New variables added/changed:

- note_verbosity, with is a set of the following options:
  basic            - All old notes
  unusable_keys    - Print warnings about keys that cannot be used
                     for select, delete or update.
  explain          - Print unusable_keys warnings for EXPLAIN querys.

The default is 'basic,explain'. This means that for old installations
the only notable new behavior is that one will get notes about
unusable keys when one does an EXPLAIN for a query. One can turn all
of all notes by either setting note_verbosity to "" or setting sql_notes=0.

- log_slow_verbosity has a new option 'warnings'. If this is set
  then warnings and notes generated are printed in the slow query log
  (up to log_slow_max_warnings times per statement).

- log_slow_max_warnings   - Max number of warnings written to
                            slow query log.

Other things:
- One can now use =ALL for any 'set' variable to set all options at once.
  For example using "note_verbosity=ALL" in a config file or
  "SET @@note_verbosity=ALL' in SQL.
- mysqldump will in the future use @@note_verbosity=""' instead of
  @sql_notes=0 to disable notes.
- Added "enum class Data_type_compatibility" and changing the return type
  of all Field::can_optimize*() methods from "bool" to this new data type.

Reviewer & Co-author: Alexander Barkov <bar@mariadb.com>
- The code that prints out the notes comes mainly from Alexander
2023-10-03 08:25:31 +03:00
Monty
4c8d2410b6 Give warnings if open_stat_table_for_ddl() fails
The warning is given in case of table not found or if there is a lock
timeout. The warning is needed as in case of a lock timeout then the
persistent table stats are going to be wrong.
2023-10-03 08:25:31 +03:00
Monty
e3b36b8f1b MDEV-31957 Concurrent ALTER and ANALYZE collecting statistics can result in stale statistical data
Example of what causes the problem:
T1: ANALYZE TABLE starts to collect statistics
T2: ALTER TABLE starts by deleting statistics for all changed fields,
    then creates a temp table and copies data to it.
T1: ANALYZE ends and writes to the statistics tables.
T2: ALTER TABLE renames temp table in place of the old table.

Now the statistics from analyze matches the old deleted tables.

Fixed by waiting to delete old statistics until ALTER TABLE is
the only one using the old table and ensure that rename of columns
can handle swapping of column names.

rename_columns_in_stat_table() (former rename_column_in_stat_tables())
now takes a list of columns to rename. It uses the following algorithm
to update column_stats to be able to handle circular renames

- While there are columns to be renamed and it is the first loop or
  last rename loop did change something.
  - Loop over all columns to be renamed
    - Change column name in column_stat
      - If fail because of duplicate key
      - If this is first change attempt for this column
         - Change column name to a temporary column name
         - If there was a conflicting row, replace it with the current row.
    else
     - Remove entry from column list

- Loop over all remaining columns in the list
 - Remove the conflicting row
 - Change column from temporary name to final name in column_stat

Other things:
- Don't flush tables for every operation. Only flush when all updates
  are done.
- Rename of columns was not handled in case of ALGORITHM=copy (old bug).
  - Fixed that we do not collect statistics for hidden hash columns
    used by UNIQUE constraint on long values.
  - Fixed that we do not collect statistics for blob columns referred by
    generated virtual columns. This was achieved by storing the fields for
    which we want to have statistics in table->has_value_set instead of
    in table->read_set.
- Rename of indexes was not handled for persistent statistics.
  - This is now handled similar as rename of columns. Renamed columns
    are now stored in 'rename_stat_indexes' and handled in
    Alter_info::delete_statistics() together with drooped indexes.
- ALTER TABLE .. ADD INDEX may instead of creating a new index rename
  an existing generated foreign key index. This was not reflected in
  the index_stats table because this was handled in
  mysql_prepare_create_table instead instead of in the mysql_alter() code.
  Fixed by adding a call in mysql_prepare_create_table() to drop the
  changed index.
  I also had to change the code that 'marked the index' to be ignored
  with code that would not destroy the original index name.

Reviewer: Sergei Petrunia <sergey@mariadb.com>
2023-10-03 08:25:30 +03:00
Andrei
388296a1e6 MDEV-32257 dangling XA-rollback in binlog from emtpy XA in pseudo_slave_mode
In `pseudo_slave_mode=1` aka "psedo-slave" mode any prepared XA
transaction disconnects from the user session, as if the user
connection drops. The xid of such transaction remains in the server,
and should the prepared transaction be read-only, it is marked.
The marking makes sure that the following termination of the
read-only transaction  ends up with ER_XA_RBROLLBACK.
This did not take place actually for  `pseudo_slave_mode=1` read-only.

Fixed with checking the read-only status of a prepared transaction
at time it disconnects from the `pseudo_slave_mode=1` session, to mark
its xid when that's the case.
2023-09-29 15:54:09 +03:00
Sergei Golubchik
3f6bccb888 Merge branch '10.11' into 11.0 2023-09-29 12:24:54 +02:00
Vladislav Vaintroub
29e7f53bf5 MDEV-32232 mysql_install_db_win.test fails on second execution
Fix test
2023-09-27 15:02:54 +02:00
Yuchen Pei
6b343de8ef
Merge branch '10.4' into 10.5 2023-09-25 13:06:57 +10:00
Sergei Golubchik
034848c6c2 Merge branch '10.10' into 10.11 2023-09-24 19:41:43 +02:00
Sergei Golubchik
11c69177e9 fix galera.galera_as_slave_gtid_myisam for 10.10+ 2023-09-24 19:32:04 +02:00
Sergei Golubchik
9880006b5b update wsrep.wsrep_provider_plugin_defaults to 26.4.16 2023-09-23 23:02:20 +02:00
Vlad Lesin
d13a57ae81 Merge 10.5 into 10.6. 2023-09-22 15:21:15 +03:00
Oleksandr Byelkin
89a493d64c MDEV-31742 incorrect examined rows in case of stored function usage
The counter is global so we do not need add backup to it
if we do not zero it after taking the backup.
2023-09-22 13:58:35 +02:00
Oleksandr Byelkin
2bf291ba59 MDEV-30820 slow log Rows_examined out of range
Fix row counters to be able to get any possible value.
2023-09-22 12:10:38 +02:00
Vlad Lesin
95730372bd MDEV-30165 X-lock on supremum for prepared transaction for RR
trx_t::set_skip_lock_inheritance() must be invoked at the very beginning
of lock_release_on_prepare(). Currently  trx_t::set_skip_lock_inheritance()
is invoked at the end of lock_release_on_prepare() when lock_sys and trx
are released, and there can be a case when locks on prepare are released,
but "not inherit gap locks" bit has not yet been set, and page split
inherits lock to supremum.

Also reset supremum bit and rebuild waiting queue when XA is prepared.

Reviewed by: Marko Mäkelä
2023-09-21 20:07:53 +03:00
Marko Mäkelä
60b039a864 Merge 10.5 into 10.6 2023-09-20 08:32:04 +03:00
Marko Mäkelä
8513f8f1c9 Give a reason for disabling a test 2023-09-20 08:31:28 +03:00
Oleg Smirnov
73ef86d4ef MDEV-29731 Assertion failure when HAVING in a correlated subquery references columns in the outer query
When resolving a column from the HAVING clause, a new Item_field
object may be created inside Item_ref::fix_fields().
But the object is created with an empty name resolution context,
which then leads to debug assertion failure during
Item_field::fix_fields().

The solution is to pass the correct name resolution context
when creating the Item_field object.

Reviewer: Oleksandr Byelkin (sanja@mariadb.com)
2023-09-20 11:05:34 +07:00
Monty
2ed54611c7 Disabled gcol.innodb_virtual_fk
Constant failures with:
"InnoDB: tried to purge non-delete-marked record in index"
2023-09-19 22:23:59 +03:00
Monty
5c9d8315cc Fixed random failuring test main.innodb_ext_key
Fixed by copying the solution from 11.0
2023-09-19 18:41:56 +03:00
Marko Mäkelä
8096139b3a Merge 10.5 into 10.6 2023-09-19 10:47:26 +03:00
Marko Mäkelä
6c05edfdcd Merge 10.4 into 10.5 2023-09-19 10:20:09 +03:00
Marko Mäkelä
76b688f100 MDEV-30024 InnoDB: tried to purge non-delete-marked of a virtual column prefix
row_vers_vc_matches_cluster(): Invoke dtype_get_at_most_n_mbchars()
to extract the correct number of bytes corresponding to the number
of characters in a virtual column prefix index, just like we do in
row_sel_sec_rec_is_for_clust_rec().

The test case would occasionally reproduce the failure when this
fix is not present.
2023-09-19 09:31:34 +03:00
Dmitry Shulga
68353dc92a MDEV-23902: MariaDB crash on calling function
On creation of a VIEW that depends on a stored routine an instance of
the class Item_func_sp is allocated on a memory root of SP statement.
It happens since mysql_make_view() calls the method
  THD::activate_stmt_arena_if_needed()
before parsing definition of the view.
On the other hand, when sp_head's rcontext is created an instance of
the class Field referenced by the data member
  Item_func_sp::result_field
is allocated on the Item_func_sp's Query_arena (call arena) that set up
inside the method
  Item_sp::execute_impl
just before calling the method
  sp_head::execute_function()

On return from the method sp_head::execute_function() all items allocated
on the Item_func_sp's Query_arena are released and its memory root is freed
(see implementation of the method Item_sp::execute_impl). As a consequence,
the pointer
  Item_func_sp::result_field
references to the deallocated memory. Later, when the method
  sp_head::execute
cleans up items allocated for just executed SP instruction the method
  Item_func_sp::cleanup is invoked and tries to delete an object referenced
by data member Item_func_sp::result_field that points to already deallocated
memory, that results in a server abnormal termination.

To fix the issue the current active arena shouldn't be switched to
a statement arena inside the function mysql_make_view() that invoked indirectly
by the method sp_head::rcontext_create. It is implemented by introducing
the new Query_arena's state STMT_SP_QUERY_ARGUMENTS that is set when explicit
Query_arena is created for placing SP arguments and other caller's side items
used during SP execution. Then the method THD::activate_stmt_arena_if_needed()
checks Query_arena's state and returns immediately without switching to
statement's arena.
2023-09-19 08:57:36 +07:00
Daniel Black
8bbe3a3cd2 MDEV-21194: mariadb-install-db doesn't properly grant proxy privileges to all created user accounts
mariadb-install-db --auth-root-authentication-method=normal created 4
root accounts by default, but only two of these had PROXY privilege
granted.

mariadb-install-db (default option
--auth-root-authentication-method=socket) as non-root user also didn't
grant PROXY priv to the created nonroot@localhost user.

To fix this, in mysql_system_tables_data.sql, we re-use tmp_user_nopasswd
as this contains the list of all root users.

REPLACE INTO tmp_proxies_priv SELECT @current_hostname, IFNULL(@auth_root_socket, 'root')
creates the $user@$current_host but will not error if @auth_root_socket
is null. Note @current_hostname lines are filtered out with
--cross-bootstrap in mariadb-install-db so it was needed to include this
expression for consistency.

Like the existing mysql_system_tables.sql is used to create teh
$user@localhost proxies_priv.

Test cases roles.acl_statistics, perfschema,privilege_table_io depends on the number of proxy users.

After:

--auth-root-authentication-method=normal:

MariaDB [mysql]> select * from global_priv;
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
| Host      | User        | Priv                                                                                                                     |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
| localhost | mariadb.sys | {"access":0,"plugin":"mysql_native_password","authentication_string":"","account_locked":true,"password_last_changed":0} |
| localhost | root        | {"access":18446744073709551615}                                                                                          |
| bark      | root        | {"access":18446744073709551615}                                                                                          |
| 127.0.0.1 | root        | {"access":18446744073709551615}                                                                                          |
| ::1       | root        | {"access":18446744073709551615}                                                                                          |
| localhost |             | {}                                                                                                                       |
| bark      |             | {}                                                                                                                       |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
7 rows in set (0.001 sec)

MariaDB [mysql]> select * from proxies_priv;
+-----------+------+--------------+--------------+------------+---------+---------------------+
| Host      | User | Proxied_host | Proxied_user | With_grant | Grantor | Timestamp           |
+-----------+------+--------------+--------------+------------+---------+---------------------+
| localhost | root |              |              |          1 |         | 2023-07-10 12:12:24 |
| 127.0.0.1 | root |              |              |          1 |         | 2023-07-10 12:12:24 |
| ::1       | root |              |              |          1 |         | 2023-07-10 12:12:24 |
| bark      | root |              |              |          1 |         | 2023-07-10 12:12:24 |
+-----------+------+--------------+--------------+------------+---------+---------------------+

--auth-root-authentication-method=socket:

MariaDB [mysql]> select * from proxies_priv;
+-----------+------+--------------+--------------+------------+---------+---------------------+
| Host      | User | Proxied_host | Proxied_user | With_grant | Grantor | Timestamp           |
+-----------+------+--------------+--------------+------------+---------+---------------------+
| localhost | root |              |              |          1 |         | 2023-07-10 12:11:55 |
| localhost | dan  |              |              |          1 |         | 2023-07-10 12:11:55 |
| bark      | dan  |              |              |          1 |         | 2023-07-10 12:11:55 |
+-----------+------+--------------+--------------+------------+---------+---------------------+
3 rows in set (0.017 sec)

MariaDB [mysql]> select * from global_priv;
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Host      | User        | Priv                                                                                                                                       |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| localhost | mariadb.sys | {"access":0,"plugin":"mysql_native_password","authentication_string":"","account_locked":true,"password_last_changed":0}                   |
| localhost | root        | {"access":18446744073709551615,"plugin":"mysql_native_password","authentication_string":"invalid","auth_or":[{},{"plugin":"unix_socket"}]} |
| localhost | dan         | {"access":18446744073709551615,"plugin":"mysql_native_password","authentication_string":"invalid","auth_or":[{},{"plugin":"unix_socket"}]} |
| localhost |             | {}                                                                                                                                         |
| bark      |             | {}                                                                                                                                         |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.000 sec)

MariaDB [mysql]> show grants;
+----------------------------------------------------------------------------------------------------------------------------------------+
| Grants for dan@localhost                                                                                                               |
+----------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO `dan`@`localhost` IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket WITH GRANT OPTION |
| GRANT PROXY ON ''@'%' TO 'dan'@'localhost' WITH GRANT OPTION                                                                           |
+----------------------------------------------------------------------------------------------------------------------------------------+
2023-09-19 08:07:07 +10:00
Lena Startseva
d515fa72a0 MDEV-31795: main.flush_corruption times out with view-protocol
Fixed tests:
main.flush_corruption - disabled service connection for some queries
since it is necessary that the query SELECT pass in the same session
2023-09-15 18:04:22 +07:00
Lena Startseva
7bef86ea74 MDEV-31460: main.order_by_pack_big fails with view-protocol
Fixed tests:
main.order_by_pack_big - disabled view-protocol for some queries
because the view is created with wrong column name if column
name > 64 symbols
2023-09-15 17:58:28 +07:00
Yuchen Pei
b70d8fbf18
Merge branch '10.5' into 10.6 2023-09-15 12:12:46 +10:00
Yuchen Pei
e95e9a221f
Merge branch '10.4' into 10.5 2023-09-15 12:04:44 +10:00