Commit graph

197588 commits

Author SHA1 Message Date
Daniel Black
72f1384c3a Merge branch 10.4 into 10.5 2022-12-13 09:57:19 +11:00
Daniel Black
382e85fe70 MDEV-30065: mariadb-install-db allow for --enforce-storage-engine=InnoDB
Hide the errors related to missing innodb stats tables in bootstrap mode
on the assumption that because we are in bootstrap mode they are going
to be created.
2022-12-13 08:47:48 +11:00
Marko Mäkelä
1862273c43 MDEV-30209 Race condition between fil_node_open_file() and renaming files
mtr_t::commit_file(): Protect the rename operation with fil_system.mutex
like we used to do before commit 2e43af69e3
in order to prevent fil_node_open_file() from running concurrently.
In other words, fil_system.mutex will protect the consistency of
fil_node_t::name and the file name in the file system.

This race condition should be very hard to trigger. We would need
a low value of innodb_open_files or table_cache limit so that
fil_space_t::try_to_close() will be invoked frequently. Simultaneously
with a RENAME operation, something (such as a write of a data page)
would have to try to open the file.
2022-12-12 11:41:12 +02:00
Marko Mäkelä
12786f0e77 Merge 10.10 into 10.11 2022-12-12 08:10:25 +02:00
Marko Mäkelä
7ab42fdd23 Merge 10.9 into 10.10 2022-12-12 08:10:12 +02:00
Marko Mäkelä
21ef68d52e Merge 10.8 into 10.9 2022-12-12 08:09:55 +02:00
Marko Mäkelä
57737258be Merge 10.7 into 10.8 2022-12-12 08:09:27 +02:00
Marko Mäkelä
dbcbee1017 Merge 10.6 into 10.7 2022-12-12 08:08:44 +02:00
Marko Mäkelä
cf437f6be9 Fix GCC -Og -Wmaybe-uninitialized
This fixes up commit dd5f4b3625
2022-12-12 08:07:47 +02:00
Marko Mäkelä
c4d7939989 MDEV-30180 Server hang with innodb_undo_log_truncate=ON
trx_purge_truncate_history(): In case of an apparent conflict with
buf_pool_t::release_freed_page(), always momentarily acquire
buf_pool.mutex, to let the thread that is holding it and waiting for
buf_pool.flush_list_mutex to proceed.

The fix in commit c410f7aaea was
insufficient, because simple yielding would not necessarily let
the conflicting thread to acquire buf_pool.flush_list_mutex.

This hang was reported and the fix tested by Axel Schwenke.
2022-12-12 07:54:38 +02:00
Daniel Black
c1cc6e8496 Correct DBUG_ENTER for Pushdown_derived::execute 2022-12-12 11:24:07 +11:00
Julius Goryavsky
a491400833 MDEV-29814: galera_var_notify_ssl_ipv6 causes testing system to hang
This commit fixes the test system hanging due to
the galera_var_notify_ssl_ipv6 test and also brings
the wsrep_notify[_ssl].sh files in line with each other
between the user template and the mtr suite.

Quotes are also added here to avoid problems if the
user specifies the value of one of the variables at the
beginning of the file containing shell-specific characters,
for example, if the password or username specified in the
PSWD and USER variables will contain the "$" character.

Also fixed an issue with automatic --ssl-verify-server-cert
option substitution when the corresponding value is set
by the user to "1" or "on".

Also fixed some tests here to avoid joining one of the nodes
to another cluster when the nodes are restarted from the mtr
side, which can lead to random failures when testing with
buildbot.
2022-12-10 01:11:55 +01:00
Julius Goryavsky
8f30973234 MDEV-29814: galera_var_notify_ssl_ipv6 causes testing system to hang
This commit fixes the test system hanging due to
the galera_var_notify_ssl_ipv6 test and also brings
the wsrep_notify[_ssl].sh files in line with each other
between the user template and the mtr suite.

Quotes are also added here to avoid problems if the
user specifies the value of one of the variables at the
beginning of the file containing shell-specific characters,
for example, if the password or username specified in the
PSWD and USER variables will contain the "$" character.

Also fixed an issue with automatic --ssl-verify-server-cert
option substitution when the corresponding value is set
by the user to "1" or "on".

Also fixed some tests here to avoid joining one of the nodes
to another cluster when the nodes are restarted from the mtr
side, which can lead to random failures when testing with
buildbot.
2022-12-09 17:52:26 +01:00
Marko Mäkelä
782b2a7500 MDEV-29144 ER_TABLE_SCHEMA_MISMATCH or crash on DISCARD/IMPORT
mysql_discard_or_import_tablespace(): On successful
ALTER TABLE...DISCARD TABLESPACE, evict the table handle from the
table definition cache, so that ha_innobase::close() will be invoked,
like InnoDB expects to be the case. This will avoid an assertion failure
ut_a(table->get_ref_count() == 0) during IMPORT TABLESPACE.

ha_innobase::open(): Do not issue any ER_TABLESPACE_DISCARDED warning.
Member functions for DML will do that.

ha_innobase::truncate(), ha_innobase::check_if_supported_inplace_alter():
Issue ER_TABLESPACE_DISCARDED warnings, to compensate for the removal of
the warning in ha_innobase::open().

row_quiesce_write_indexes(): Only write information about committed
indexes. The ALTER TABLE t NOWAIT ADD INDEX(c) in the nondeterministic
test case will most of the time fail due to a metadata lock (MDL) timeout
and leave behind an uncommitted index.

Reviewed by: Sergei Golubchik
2022-12-09 10:42:19 +02:00
Daniel Black
8f3631d009 MDEV-30150 ST_GeomFromGeoJSON, 'geometry' before 'type: feature' error
The geometry type requires Type:"Feature" but the feature need
not be first in the JSON structure.

Adjust code to return an error if geometry isn't a JSON object,
but continue parsing searching for Type: "Feature" to trigger
the geometry parsing.

Thanks Derick Magnusen for the bug report.
2022-12-09 08:49:43 +11:00
Tuukka Pasanen
851816532b MDEV-28834: Add minimal support for Lintian version 2.115 and above
Convert minimal amount of Lintian overrides to make Lintian
test pass also with Debian Sid latest Lintian 2.115 version.

Old style overrides are kept so they can be used with
older versions of Lintian.

Introduce minimal Lintian overrides which are common
from MariaDB version 10.5 up-to to 10.8.

Overrides added files:
  * debian/mariadb-test-data.lintian-overrides
    - MariaDB installs some shared objects to test-suite directory and not in
      '/usr/lib' or similar. Share objects is pam_mariadb_mtr.so. Tags are
      arch-dependent-file-in-usr-share and
      arch-independent-package-contains-binary-or-object Lintia
 * debian/mariadb-test.lintian-overrides
   - MariaDB installs some some binaries to test-sute directory and
     in mariadb-test package they are my_safe_process and
     wsrep_check_version. Tags is
     arch-dependent-file-in-usr-share
 * debian/source/lintian-overrides
   - In source there is some source files missing which should be addressed
     sql/share/charsets/languages.html and
     and storage/rocksdb/rocksdb/docs/_includes/footer.html.
     Tags is source-is-missing
   - Add Lintian override for missing:
     storage/columnstore/columnstore/utils/jemalloc/libjemalloc.so.2
   - Add Lintian override for substvar external resources:
     ${source:Version} libmariadb-dev -> libmysqlclient-dev [debian/control:66]
     ${source:Version} libmariadb-dev -> libmysqld-dev [debian/control:66]
     ${source:Version} libmariadbd-dev -> libmariadbclient-dev [debian/control:216]
2022-12-08 17:01:39 +00:00
Monty
dd5f4b3625 Fixed bug in Aria when used with enterprise mariadb-backup
If the backup finished in the middle of a Aria bulk load insert,
which could happen with LOAD DATA INFILE, CREATE ... SELECT etc)
there was a chance that Aria recovery would fail on the backup.

Fixed by ensuring that bulk load operations for Aria are not allowed
under BACKUP LOCK.
I also changed so that the table TRN is updated just before truncate
which ensures that old redo's for the table are ignored.
I also enabled Aria redo for DDL's to be able to repeat REPAIR commands.
Without this change recovery would not work on repaired tables.

Notes:
- We take the backup lock protection at the end of bulk insert (as we
  don't want to keep the lock over a very long running insert).
  If mariadb-backup keeps the backup lock too long,  this may fail with
  a lock timeout. In this case the batch insert will fail and the table
  will be truncated (set to it's original state).
2022-12-08 12:11:33 +02:00
Vladislav Vaintroub
d360fa6fa8 MDEV-30162 Fix occasional "Permission denied" on Windows caused by buggy 3rd party
Add retry logic for CreateFile, DeleteFile, or MoveFile
when GetLastError() is ERROR_SHARING_VIOLATION.
2022-12-07 14:26:10 +01:00
Marko Mäkelä
64071d30bd Merge 10.10 into 10.11 2022-12-07 10:00:52 +02:00
Marko Mäkelä
3ff4eb07ed Merge 10.9 into 10.10 2022-12-07 09:49:38 +02:00
Marko Mäkelä
23f705f3a2 Merge 10.8 into 10.9 2022-12-07 09:43:38 +02:00
Marko Mäkelä
b3c254339b Merge 10.7 into 10.8 2022-12-07 09:43:13 +02:00
Marko Mäkelä
9e27e53dfa Merge 10.6 into 10.7 2022-12-07 09:39:46 +02:00
Nayuta Yanagisawa
2beede9ba4 MDEV-29636 Assertion `part_share->auto_inc_initialized || !can_use_for_auto_inc_init()' failed in ha_partition::set_auto_increment_if_higher upon REPLACE with partition pruning
The bug is caused by a similar mechanism as MDEV-21027.

The function, check_insert_or_replace_autoincrement, failed to open
all the partitions on REPLACE SELECT statements and it results in the
assertion error.
2022-12-07 16:34:56 +09:00
Jan Lindström
0174a9ff3d MDEV-30172: Galera test case cleanup
* Delete tests that are not supported and not going to be supported
  any time soon
* Fix result set on tests that are not run on bb
* Fix tests that fail because of auto increment offset
* Make sure that disabled tests have open bug report
2022-12-07 07:33:51 +02:00
Vladislav Vaintroub
9044e016c5 MDEV-29822 - disable a test that fails sporadically 2022-12-06 13:31:11 +01:00
Marko Mäkelä
e55397a46d Merge 10.5 into 10.6 2022-12-05 18:04:23 +02:00
Marko Mäkelä
0a7d85c97f MDEV-30148 Race condition between non-persistent statistics and purge
btr_cur_t::open_random_leaf(): Replaces btr_cur_open_at_rnd_pos().
Acquire a shared latch on each page, and finally release all
latches except the one on the leaf page.

This fixes a race condition between the purge of history and
btr_estimate_number_of_different_key_vals(), which turned out
to only hold a buffer-fix on the randomly chosen leaf page.
Typically, an assertion would fail in page_rec_is_supremum().

ibuf_contract(): Start from the beginning of the change buffer,
to simplify the logic. Starting with
commit b42294bc64
it does not matter much where the change buffer merge is being initiated.

The race condition may have been introduced as early as
mysql/mysql-server@ac74632293
from where it was copied to
commit 2e814d4702.

Reviewed by: Vladislav Lesin
Tested by: Matthias Leich
2022-12-05 18:00:22 +02:00
Anel Husakovic
a59dffb0e9 dgcov: parsing without dot to get specific version
- Regression introduced 7baf24a0f8 for multidigit gcc dump.
There is no dot in `dumpversion`.
```
$ gcc -dumpversion
10
```
Otherwise it will fail and not produce the output
```
Running dgcov
Name "IO::Uncompress::Gunzip::GunzipError" used only once: possible typo at ./dgcov.pl line 197.
Cannot parse gcc -dumpversion: 9
```

- The warning `once` is always generated:
```
Running dgcov
Name "IO::Uncompress::Gunzip::GunzipError" used only once: possible typo at ./dgcov.pl line 197.
<number>
```
Suppresing the line `Name "IO::Uncompress::Gunzip::GunzipError" used only once: possible typo at ./dgcov.pl line 197.`
with the patch.

- Reviewed by: <>
2022-12-05 15:24:36 +01:00
Monty
e748f5cc83 Fixed a crash during automatic zerofill of moved Aria table
This could happen if one did a DML with a moved table that one had done
an external zerofill on.
The crash happend because a message that was supposed to be sent to
a repair report was instead sent to the result, which caused an ASSERT
2022-12-05 14:53:23 +02:00
Thirunarayanan Balathandayuthapani
922f7ba75c MDEV-30158 InnoDB fails to start ther server 10.11 when innodb_undo_tablespaces mismatch
- InnoDB should throw the warning when user specified undo tablespaces
doesn't match with the number of opened undo tablespaces and should
reflect the value in innodb_undo_tablespaces variable
2022-12-05 16:26:05 +05:30
Otto Kekäläinen
95d71272ef Gitlab-CI: Upgrade Fedora build always use latest (now 37) version
The version was fixed to be Fedora 36 due to previous issues on
Gitlab-CI, but those seem to be solved now.

Use 'mariadb' name in scripts and server binary as Fedora switched name in
df76620f9e

Switch to using the `default:` section supported by newer Gitlab-CI,
see https://docs.gitlab.com/ee/ci/yaml/#default.

Also define an explicit timeout of 3 hours to ensure builds don't time
out if the default timeout is too short.

NOTE TO MERGERS: These changes are version independent and should be
merged up on all MariaDB branches 10.6 -> 10.11.
2022-12-05 07:23:22 +00:00
Sergei Golubchik
180b2bcd53 dgcov: also remove rpm dependency on IO::Uncompress::Gunzip
and enable option bundling
2022-12-04 00:31:29 +01:00
Sergei Petrunia
e0dbec1ce3 MDEV-29129: Performance regression starting in 10.6: select order by limit ...
The cause of regression was handling for ROWNUM() function.
For queries like

  SELECT ROWNUM() FROM ... ORDER BY ...

ROWNUM() should be computed before the ORDER BY.
The computation was moved to be before the ORDER BY for any entries in
the select list that had RAND_TABLE_BIT set.

This had a negative impact on queries in form:

  SELECT sp_func() FROM t1 ORDER BY ... LIMIT n

where sp_func() is NOT declared as DETERMINISTIC (and so has
RAND_TABLE_BIT set).

The fix is to require evaluation for sorting only for the ROWNUM()
function. Functions that just have RAND_TABLE_BIT() can be computed
after ORDER BY ... LIMIT is applied.

(think about a possible index that satisfies the ORDER BY clause. In
that case, the the rows would be read in the needed order and we would
stop after reading LIMIT rows, achieving the same effect).
2022-12-03 15:46:00 +03:00
Sergei Golubchik
7baf24a0f8 MDEV-26102 followup
* update README
* fix version regex to support versions with two digits
* die if the version cannot be parsed
* support gcc versions 11+
* require JSON::PP not use, to avoid introducing new rpm dependency
  into MariaDB-test
2022-12-02 16:19:13 +01:00
Anel Husakovic
43173ef261 MDEV-26102 dgcov: add support for *.gcda.gcov.json.gz files of gcov 9.1+
- Script covers situation for gcov (gcc) < 9 with non-json format of
generated files as well as for gcov (gcc) >=8 with json generated format

Reviewed by: serg@mariadb.com
2022-12-02 16:19:13 +01:00
Sergei Golubchik
1547e55489 fix more sporadic failures on main.kill
sometimes `KILL QUERY ID @id` was executed before the previous
`send SELECT SLEEP(1000)` has reached the parser. As the statement
resets the kill status before execution, the effect of the KILL
was ignored.
2022-12-02 16:19:13 +01:00
Sergei Golubchik
401ae95a60 MDEV-30082 View definition losing brackets changes semantics of the query and causes wrong result
Item_func_not_all::print() either uses Item_func::print() or
directly invokes args[0]->print(). Thus the precedence should be
either the one of Item_func or of args[0].

Item_allany_subselect::print() prints args[0], then a comparison op,
then a subquery. That is, the precedence should be the one of
a comparison.
2022-12-02 16:19:13 +01:00
Sergei Golubchik
37bfe32c6d try harder to reject not strictly deterministic vcols in indexes/stored
detect non-determinism in vcol of vcol, like:

create table t1 (a int, b real as (rand()), c real as (b) stored);
2022-12-02 16:19:13 +01:00
Sergei Golubchik
ae53f684d3 MDEV-30016 Virtual columns do not support autoincrement columns
change vcol_upgrade test to use stored gcols
2022-12-02 16:19:13 +01:00
Sergei Golubchik
a6b327e90a cleanup: VCOL_NOT_VIRTUAL->VCOL_NEXTVAL
rename to stress that is a specific hack for Item_func_nextval
and should not be used for other items.

If a vcol uses Item_func_nextval, a corresponding table for the sequence
should be added to the prelocking list (in that sense NEXTVAL is not
simply a function, but more like a subquery), see add_internal_tables()
in DML_prelocking_strategy::handle_table(). At the moment it is only
implemented for DEFAULT, not for GENERATED ALWAYS AS, thus the
VCOL_NEXTVAL hack.
2022-12-02 16:19:13 +01:00
Sergei Golubchik
53e57a8681 MDEV-30056 Impossible to export column grants 2022-12-02 16:19:13 +01:00
Sergei Golubchik
f915681d2f MDEV-30036 NULL pointer dereference in partition_info::set_partition_bitmaps_from_table
remove dead code
2022-12-02 16:19:13 +01:00
Sergei Golubchik
c7c1461b94 fix embedded startup with no command line arguments
&fake_argv cannot be cast to char***
this causes a crash when resolving
2022-12-02 16:19:13 +01:00
Sergei Golubchik
cfb47ddde2 MDEV-30066 (limit + offset) union all (...) limit = incorrect result
select_union_direct::send_data() only sends a record when
the LIMIT ... OFFSET clause of the individual select won't skip it.

Thus, select_union_direct::send_data() should not do any actions
related to a sending a record if the offset of a select isn't
reached yet
2022-12-02 16:19:13 +01:00
Sergei Golubchik
da3fc33e88 cleanup: union.test 2022-12-02 16:19:12 +01:00
Sergei Golubchik
d08f2ab6d6 MDEV-28855 SEGV around dict_free_vc_templ during DROP INDEX
protect shared dict_table_t::vc_templ with a mutex,
same as in ha_innobase::open
2022-12-02 16:19:12 +01:00
Sergei Golubchik
4fb8f7d07a cleanup: clarify innobase_init_vc_templ usage 2022-12-02 16:19:12 +01:00
Oleksandr Byelkin
126619047a MDEV-28643: view protocol fails due to different column name
Decent name given to the column.
2022-12-02 11:59:36 +01:00
Thirunarayanan Balathandayuthapani
dd20a43c6c MDEV-30114 Incremental prepare fails when innodb_undo_tablespaces > 0
- Mariabackup fails to open the undo tablespaces while applying delta
files to the corresponding data file. Mariabackup opens the
undo tablespaces first time in srv_undo_tablespaces_init() and does
tries to open the undo tablespaces in xtrabackup_apply_deltas() with
conflicting mode and leads to the failure.

- Mariabackup should close the undo tablespaces before applying
the incremental delta files.
2022-12-02 15:48:37 +05:30