Commit graph

191623 commits

Author SHA1 Message Date
Oleksandr Byelkin
e509065247 Merge branch '10.3' into 10.4 2022-08-03 19:51:44 +02:00
Sergei Petrunia
37a3d4467e MDEV-23809: Server crash in JOIN_CACHE::free ...: part
Part : make sure we allocate space for two JOIN_TABs that
use temporary tables.
The dbug_join_tab_array_size is still set to catch cases where
we try to access more JOIN_TAB object than we thought we would have.
2022-08-03 19:40:02 +03:00
Sergei Petrunia
2cd98c95de MDEV-23809: Server crash in JOIN_CACHE::free or ...
The problem was caused by use of COLLATION(AVG('x')). This is an
item whose value is a constant.
Name Resolution code called convert_const_to_int() which removed AVG('x').
However, the item representing COLLATION(...) still had with_sum_func=1.

This inconsistent state confused the code that handles grouping and
DISTINCT: JOIN::get_best_combination() decided to use one temporary
table and allocated one JOIN_TAB for it, but then
JOIN::make_aggr_tables_info() attempted to use two and made writes
beyond the end of the JOIN::join_tab array.

The fix:
- Do not replace constant expressions which contain aggregate functions.
- Add JOIN::dbug_join_tab_array_size to catch attempts to use more
  JOIN_TAB objects than we've allocated.
2022-08-03 19:40:02 +03:00
Thirunarayanan Balathandayuthapani
f9ec9b6abb MDEV-27282 InnoDB: Failing assertion: !query->intersection
- query->intersection fails to get freed if the query exceeds
innodb_ft_result_cache_limit

- errors from init_ftfuncs were not propogated by delete command

This is taken from percona/percona-server@ef2c0bcb9a
2022-08-03 20:35:12 +05:30
Oleksandr Byelkin
8fd8a81a99 Fix Oracle parser. 2022-08-03 12:49:49 +02:00
Igor Babaev
c2300d06f7 MDEV-28617 Crash with INSERT...SELECT using derived table in GROUP BY clause
This bug manifested itself for INSERT...SELECT and DELETE statements whose
WHERE condition used an IN/ANY/ALL predicand or a EXISTS predicate with
such grouping subquery that:
 - its GROUP BY clause could be eliminated,
 - the GROUP clause contained a subquery over a mergeable derived table
   referencing the updated table.

The bug ultimately caused a server crash when the prepare phase of the
statement processing was executed. This happened after removal redundant
subqueries used in the eliminated GROUP BY clause from the statement tree.
The function that excluded the subqueries from the did not do it properly.
As a result the specification of any derived table contained in a removed
subquery was not marked as excluded.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2022-08-02 12:29:24 -07:00
Sergei Golubchik
07a670b884 MDEV-23097 heap-use-after-free in mysqlimport
mysqlimport starts many worker threads. when one of the worker
encounters an error, it frees global memory and calls exit().

it suppresses memory leak detector, because, as the comment says
"dirty exit, some threads are still running", indeed, it cannot
free the memory from other threads.

but precisely because some threads are still running, they
might use this global memory, so it cannot be freed.

fix: if we know that some threads are still running and accept
that we cannot free all memory anyway, let's not free global
allocations either
2022-08-02 20:10:37 +02:00
Daniel Black
92b0a367aa MDEV-26447: mysqldump to use temporary view instead of tables.
This is particularly important for Azure where there is no
MyISAM support in their MariaDB cloud product.

Like mysqldumper does, a view can satisfy the requirement
like a table, without constraints. The views in frm files are
text form and don't have column limits.

Thanks Thomas Casteleyn for the suggestion.
2022-08-03 02:51:11 +10:00
Oleksandr Byelkin
48e35b8cf6 Merge branch '10.3' into 10.4 2022-08-02 14:15:39 +02:00
Mike Griffin
53c4e4d054 MDEV-18702 mysqldump: add variable 'max-statement-time'
With a global non-default max-statement-time of a time interval that exceed
the query time mysqldump queries when doing a backup.

To solve both, add a max-statement-time option, defaulting to 0 (unlimited time).

Also like mariabackup, set the session wait_timeout=DEFAULT (28800). The
time/processing between mysqldump times isn't expected to get that
close ever, but let's adopt the standard of mariabackup as no-one has
challenged it has having a detrimental effect.

Reviewer and test case author Daniel Black
2022-08-02 20:31:08 +10:00
Oleksandr Byelkin
5ac528a91f Merge remote-tracking branch 'connect/10.3' into 10.3 2022-08-02 10:23:53 +02:00
Daniel Black
182a6383cd MDEV-16605 Always include buf_madvise_do_dump in binaries
The "used" attribute seems to do this

ref: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#Common-Function-Attributes
2022-08-02 12:29:11 +10:00
Aleksey Midenkov
1ea5e402a8 MDEV-28727 ALTER TABLE ALGORITHM=NOCOPY does not work after upgrade
MDEV-20704 changed the rules of how (HA_BINARY_PACK_KEY |
HA_VAR_LENGTH_KEY) flags are added. Older FRMs before that fix had
these flags for DOUBLE index. After that fix when ALTER sees such old
FRM it thinks it cannot do instant alter because of failed
compare_keys_but_name(): it compares flags against tmp table created
by ALTER.

MDEV-20704 fix was actually not about DOUBLE type but about
FIELDFLAG_BLOB which affected DOUBLE. So there is no direct knowledge
that any other types were not affected.

The proposed fix under CHECK TABLE checks if FRM has
(HA_BINARY_PACK_KEY | HA_VAR_LENGTH_KEY) flags and was created prior
MDEV-20704 and if so issues "needs upgrade". When mysqlcheck and
mysql_upgrade see such status they issue ALTER TABLE FORCE and upgrade
the table to the version of server.
2022-08-01 19:14:46 +03:00
Aleksey Midenkov
231feabd2b MDEV-21540 Initialization of already inited long unique index on reorganize partition
Handler for existing partition was already index-inited at the
beginning of copy_partitions().

In the case of REORGANIZE PARTITION we fill new partition by calling
its ha_write_row() (handler is storage engine of new partition). From
that we go through the below conditions:

    if (this->inited == RND)
      table->clone_handler_for_update();
    handler *h= table->update_handler ? table->update_handler : table->file;

First, the above misses the meaning of this->inited check. Now it is
new partition and this handler is not inited. So, we assign
table->file which is ha_partition and is really not known to be inited
or not. It is supposed (this == table->file), otherwise we are
out of the logic for using update_handler. This patch adds DBUG_ASSERT
for that.

Second, we call check_duplicate_long_entries() for table->file and
that calls ha_partition::index_init() which calls index_init() for
each partition's handler. But the existing parititions' handlers was
already inited in copy_partitions() and we fail on assertion.

The fix implies that we don't need check_duplicate_long_entries()
per-partition as we've already done check_duplicate_long_entries() for
ha_partition. For REORGANIZE PARTITION that means existing row was
already checked at previous INSERT/UPDATE commands, so no need to
check it again (see NOTE in handler::ha_write_row()).

The fix also optimizes ha_update_row() so
check_duplicate_long_entries_update() is not called per-partition
considering it was already called for ha_partition. Besides,
per-partition duplicate check is not really usable.
2022-08-01 19:14:46 +03:00
Julius Goryavsky
7fb1f919d0 MDEV-28758: Mariabackup copies binary logs to backup directory
This commit restores defaults and functionality regarding binlogs
to the way it was prior to MDEV-27524. The mariabackup utility no
longer saves binlogs files as part of a backup without the --galera-info
option. However, since we use --galera-info during SST, the behavior
of mariabackup changes and, in combination with GTIDs support enabled,
mariabackup transfers one (most recent) binlog file obtained after
FLUSH BINARY LOGS. In other cases, binlogs are not transferred during
SST in mariabackup mode. As for SST in the rsync mode, it works the
same way as before MDEV-27524 - by default it transfers one last
binlog file.

The --sst-max-binlogs option for mariabackup and the sst_max_binlogs
parameter in the [sst] / server sections are no longer supported for
SST via mariabackup.
2022-08-01 15:53:14 +02:00
Sergei Golubchik
5b4154373a only copy buffer pool dump in SST galera mode
and then only into the default name, so that the joiner could find it
2022-08-01 15:53:14 +02:00
Sergei Golubchik
5197519f4f revert mariabackup part of MDEV-27524, fix the test 2022-08-01 15:53:13 +02:00
Marko Mäkelä
6a3fbfdb2d MDEV-14804 innodb.update_time occasionally fails
Let simplify the test.
The update_time is stored in the table metadata (dict_table_t);
it has nothing to do with buffer pool page eviction or replacement.
2022-08-01 15:15:06 +03:00
Sergei Golubchik
40c2460d8c in INFORMATION_SCHEMA.ALL_PLUGINS match installed plugins better
look for an installed plugin with the same name _and the same type_
(in case there are many plugins with the same name and different type,
which is, technically, possible for built-in plugins).
2022-08-01 11:11:45 +02:00
Sergei Golubchik
8ea529ecba MDEV-29131 Assertion `status == 0' failed when renaming user after deleting table roles_mapping
if mysql.roles_mapping table doesn't exist (it's optional, after all),
we still update in-memory structures to keep them consistent
2022-07-29 21:58:38 +02:00
Sergei Golubchik
6c7e3e5c13 bugfix: RAND is VCOL_SESSION_FUNC
it's not "non deterministic", it's completely defined
by @@rand_seed1 and @@rand_seed2. And as a session func it needs
to be re-fixed at the beginning of every statement.
2022-07-29 21:58:38 +02:00
Rucha Deodhar
25219920f5 MDEV-28762: recursive call of some json functions without stack control
Fixup to MDEV-28762. Fixes warnings about unused variable "stack_used_up"
during building with RelWithDebInfo
2022-07-29 22:09:45 +05:30
Georg Richter
f9315b3321 CC 3.1 update
Test fixes:

Since fix for CONC-603 (wrong error handling in TLS read/write) in case
of a read/write error client doesn't return always error 2013 (server
has gone away), so in addition we need to check for error 2026
(TLS/SSL error) and 5014 (write error).
2022-07-29 13:39:12 +02:00
Oleksandr Byelkin
cbcc0101ee MDEV-29188 Crash in JSON_EXTRACT
If we have null_value set then decimal/string value/result shoud be 0 pointer.
2022-07-29 09:03:54 +02:00
Marko Mäkelä
4b77d38c26 Fix GCC -Og -Wmaybe-uninitialized 2022-07-28 08:51:19 +03:00
Oleksandr Byelkin
f0107c90a0 wolfssl v5.4.0-stable 2022-07-27 16:21:28 +02:00
Vladislav Vaintroub
4329720b79 Fixes for WolfSSL 5.4.0 2022-07-27 16:18:18 +02:00
Oleksandr Byelkin
15a2ff1231 MDEV-26647 (simple_password_check) Include password validation plugin information in the error message if the SQL statement is not satisfied password policy
Make the plugin reporting cause of the error.
2022-07-27 16:04:20 +02:00
Oleksandr Byelkin
cc6bba008d MDEV-26647 (plugin name) Include password validation plugin information in the error message if the SQL statement is not satisfied password policy
Add plugin name to the error message.
2022-07-27 15:23:39 +02:00
Marko Mäkelä
e5c4f4e590 Merge 10.3 into 10.4 2022-07-27 14:25:36 +03:00
Marko Mäkelä
0ee1082bd2 MDEV-28495 InnoDB corruption due to lack of file locking
Starting with commit da094188f6 (MDEV-24393),
MariaDB will no longer acquire advisory file locks on InnoDB data
files by default, because it would create a large number of
entries in Linux /proc/locks.

The motivation for acquiring the file locks is to prevent accidental
concurrent startup of multiple server processes on the same data files.
Such mistake still turns out to be relatively common, based on
corruption bug reports from the community.

To prevent corruption due to concurrent startup attempts, the
Aria storage engine would unconditionally acquire an advisory lock
on one of its log files.

Solution: InnoDB will always lock its system tablespace files.
(Ever since commit 685d958e38
the InnoDB log file will not necessarily be open while the
server is running, because it can be accessed via memory-mapped I/O.)

If more protection is desired, then the option --external-locking
can be used.

The mandatory advisory lock also fixes intermittent failures of
some crash recovery tests. It turns out that when the mtr test harness
kills and restarts the server, it will not actually ensure that the
old process has terminated before starting the new one.
2022-07-27 14:15:14 +03:00
Oleksandr Byelkin
3bb36e9495 Merge branch '10.3' into 10.4 2022-07-27 11:02:57 +02:00
Igor Babaev
bd935a4106 MDEV-29139 Crash when using ANY predicand with redundant subquery in GROUP BY clause
This bug could cause a crash of the server when executing queries containing
ANY/ALL predicands with redundant subqueries in GROUP BY clauses.
These subqueries are eliminated by remove_redundant_subquery_clause()
together with elimination of GROUP BY list containing these subqueries.
However the references to the elements of the GROUP BY remained in the
JOIN::all_fields list of the right operand of of the ALL/ANY predicand.
Later these references confused make_aggr_tables_info() when forming
proper execution structures after ALL/ANY predicands had been replaced
with expressions containing MIN/MAX set functions.
The patch just removes these references from JOIN::all_fields list used
by the subquery of the ALL/ANY predicand when its GROUP BY clause is
eliminated.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2022-07-26 18:03:34 -07:00
Sergei Golubchik
e8eb6d9c93 zlib: remove redundant and generated files
followup for 4bc34ef36f
2022-07-26 16:46:27 +02:00
Marko Mäkelä
9a897335eb MDEV-26420 Buffer overflow on instant ADD/DROP of generated column
prepare_inplace_add_virtual(): Over-estimate the size of the arrays
by not subtracting table->s->virtual_fields (which may refer to
stored, not virtual generated columns). InnoDB only distinguishes
virtual columns.
2022-07-26 16:45:10 +03:00
Sergei Golubchik
bc4098582b MDEV-29074 GET_BIT variables crash in SET STATEMENT 2022-07-26 14:42:32 +02:00
Sergei Golubchik
0ee5cf837e disks plugin: check for build prerequisites properly 2022-07-26 14:42:32 +02:00
Thirunarayanan Balathandayuthapani
15a68fc255 MDEV-29058 Assertion `index->type == 32' failed in dict_index_build_internal_fts
- Import tablespace re-evicts and reload the table definition. During that
time, innodb has to load the table even though the secondary fts index
marked as corrupted
2022-07-26 15:01:38 +05:30
Thirunarayanan Balathandayuthapani
f076dc2f66 MDEV-20797 FULLTEXT search with apostrophe, and mandatory words
- InnoDB should ignore the single word followed by apostrophe while
tokenising the document. Example is that if the input string is O'brien
then right now, InnoDB seperates into two tokens as O, brien. But
after this patch, InnoDB can ignore the token 'O' and consider
only 'brien'.
2022-07-26 15:01:17 +05:30
Marko Mäkelä
a8a27f1edf Correct a bogus comment.
The logic on Windows was originally simplified in
commit 4bca1a786f
for storage/xtradb and later in
commit 6304c0bfc7
for storage/innobase.
2022-07-26 11:03:53 +03:00
Marko Mäkelä
3bf10012e0 MDEV-28762: Fixup for clang
Unlike GCC, clang could optimize away alloca() and thus the
ALLOCATE_MEM_ON_STACK() instrumentation. To make it harder, let us
invoke a non-inline function on the entire allocated buffer.
2022-07-26 11:00:11 +03:00
Mikhail Chalov
19af1890b5 Use memory safe snprintf() in Connect Engine
This commit replaces sprintf(buf, ...) with
snprintf(buf, sizeof(buf), ...),
specifically in the "easy" cases where buf is allocated with a size
known at compile time.

The changes make sure we are not write outside array/string bounds which
will lead to undefined behaviour. In case the code is trying to write
outside bounds - safe version of functions simply cut the string
messages so we process this gracefully.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license.  I am contributing on behalf of my employer Amazon Web Services,
Inc.

bsonudf.cpp warnings cleanup by Daniel Black

Reviewer: Daniel Black
2022-07-26 16:28:59 +10:00
Marko Mäkelä
95eb5e5a12 Fix clang -Wunused-but-set-variable in unit tests 2022-07-26 08:18:36 +03:00
Marko Mäkelä
5d0f75349f MDEV-28980: Disable the test for --embedded 2022-07-26 08:08:48 +03:00
Brandon Nesterenko
555c12a541 MDEV-21087/MDEV-21433: ER_SLAVE_INCIDENT arrives at slave without failure specifics
Problem:
=======

This patch addresses two issues:

 1. An incident event can be incorrectly reported for transactions
which are rolled back successfully. That is, an incident event
should only be generated for failed “non-transactional transactions”
(i.e., those which modify non-transactional tables) because they
cannot be rolled back.

 2. When the mariadb slave (error) stops at receiving the incident
event there's no description of what led to it. Neither in the event
nor in the master's error log.

Solution:
========

Before reporting an incident event for a transaction, first validate
that it is “non-transactional” (i.e. cannot be safely rolled back).
To determine if a transaction is non-transactional,
  lex->stmt_accessed_table(LEX::STMT_WRITES_NON_TRANS_TABLE)
is used because it is set previously in
THD::decide_logging_format().

Additionally, when an incident event is written, write an error
message to the server’s error log to indicate the underlying issue.

Reviewed by:
===========
Andrei Elkin <andrei.elkin@mariadb.com>
2022-07-25 16:26:53 -06:00
Rucha Deodhar
46ff660883 This commit is a fixup for MDEV-28762 2022-07-25 22:13:40 +05:30
Marko Mäkelä
f1c8749f46 Fix DBUG_ENTER/return mismatch
Spotted by Thirunarayanan Balathandayuthapani.
2022-07-25 15:58:16 +03:00
Marko Mäkelä
8aa37c264f MDEV-28980 InnoDB: Failing assertion: len <= MAX_TABLE_NAME_LEN
dict_load_foreigns(): Use a correctly sized buffer for the maximum-length
SYS_FOREIGN.ID. In case of overflow, do not crash the server but instead
return DB_CORRUPTION.
2022-07-25 10:28:45 +03:00
Brad Smith
e55c3dc33f Fix building Connect storage engine ioapi code on OpenBSD / NetBSD / DragonFly 2022-07-25 15:06:32 +10:00
Rucha Deodhar
95989e8211 MDEV-28762: recursive call of some json functions without stack control
This commit is a fixup for MDEV-28762

    Analysis: Some recursive json functions dont check for stack control
    Fix: Add check_stack_overrun(). The last argument is NULL because it is not
    used
2022-07-23 23:02:12 +05:30