Commit graph

23526 commits

Author SHA1 Message Date
Sergei Petrunia
eebe2090c8 Merge 10.3 -> 10.4 2021-06-30 18:41:46 +03:00
Sergei Petrunia
586870f9ef Merge 10.2->10.3 2021-06-30 15:06:54 +03:00
Marko Mäkelä
58252fff15 MDEV-26040 os_file_set_size() may not work on O_DIRECT files
os_file_set_size(): Trim the current size down to the file system
block size, to obey the constraints for unbuffered I/O.
2021-06-29 14:28:23 +03:00
Marko Mäkelä
09b03ff31b Merge 10.3 into 10.4 2021-06-23 08:05:27 +03:00
Marko Mäkelä
35a9aaebe2 MDEV-25981 InnoDB upgrade fails
trx_undo_mem_create_at_db_start(): Relax too strict upgrade checks
that were introduced in
commit e46f76c974 (MDEV-15912).
On commit, pages will typically be set to TRX_UNDO_CACHED state.
Having the type TRX_UNDO_INSERT in such pages is common and
unproblematic; the type would be reset in trx_undo_reuse_cached().

trx_rseg_array_init(): On failure, clean up the rollback segments
that were initialized so far, to avoid an assertion failure later
during shutdown.
2021-06-22 09:30:25 +03:00
Marko Mäkelä
e07f0a2d80 Merge 10.2 into 10.3 2021-06-22 09:15:38 +03:00
Marko Mäkelä
19716ad5a8 MDEV-25982 Upgrade of MariaDB 10.1 log crashes due to missing encryption key
init_crypt_key(): On failure, set info->key_version to
ENCRYPTION_KEY_VERSION_INVALID.

log_crypt_101_read_block(): Refuse to attempt decryption if
info->key_version is ENCRYPTION_KEY_VERSION_INVALID.
2021-06-22 09:11:41 +03:00
Marko Mäkelä
ce868cd89e Merge 10.3 into 10.4 2021-06-21 18:44:14 +03:00
Marko Mäkelä
9dc50ea229 MDEV-25979 Invalid page number written to DB_ROLL_PTR
trx_undo_report_row_operation(): Fix a race condition that was introduced
in commit f74023b955 (MDEV-15090).
We must not access undo_block after the page latch has been released
in mtr_t::commit(), because the block could be evicted or replaced.
2021-06-21 18:13:28 +03:00
Marko Mäkelä
baf0ef9a18 After-merge fix: Remove duplicated code
In the merge commit d3e4fae797
a message about innodb_force_recovery was accidentally duplicated.
2021-06-21 14:05:43 +03:00
Marko Mäkelä
d3e4fae797 Merge 10.3 into 10.4 2021-06-21 12:38:25 +03:00
Marko Mäkelä
e46f76c974 MDEV-15912: Remove traces of insert_undo
Let us simply refuse an upgrade from earlier versions if the
upgrade procedure was not followed. This simplifies the purge,
commit, and rollback of transactions.

Before upgrading to MariaDB 10.3 or later, a clean shutdown
of the server (with innodb_fast_shutdown=1 or 0) is necessary,
to ensure that any incomplete transactions are rolled back.
The undo log format was changed in MDEV-12288. There is only
one persistent undo log for each transaction.
2021-06-21 12:34:07 +03:00
Marko Mäkelä
241d30d3fa After-merge fixes for MDEV-14180 2021-06-21 12:09:00 +03:00
Marko Mäkelä
c9a85fb1b1 Merge 10.2 into 10.3 2021-06-21 09:07:40 +03:00
Sergei Golubchik
690ae1de45 fix spider tests for --ps in 10.4
see also 068246c006
2021-06-19 13:46:11 +02:00
Sergei Golubchik
a4f485917e spider tests aren't big in 10.4
see also a5f6eca50d
2021-06-19 13:46:11 +02:00
Sergei Golubchik
068246c006 fix spider tests for --ps
spider tests use

--let $var= many;sql;statements
--eval $var

and this doesn't work in ps
2021-06-19 00:45:49 +02:00
Sergei Golubchik
a5f6eca50d spider tests aren't big
and *never* disable tests in suite.pm based on $::opt_big_test,
this will make the test skipped both as too big (for ./mtr)
and as too small (for ./mtr --big --big).
2021-06-19 00:22:15 +02:00
Marko Mäkelä
c307dc6efd Remove a unused variable
In commit 1c35a3f6fd a useless
computation that used the variable was removed.
2021-06-16 07:50:04 +03:00
Eugene Kosov
1c35a3f6fd fix clang build
a new warning -Wunused-but-set-variable was introduced recently to clang
2021-06-15 13:10:16 +03:00
Thirunarayanan Balathandayuthapani
7d591cf850 MDEV-24713 Assertion `dict_table_is_comp(index->table)' failed in row_merge_buf_add()
- During online alter conversion from compact to redundant,
virtual column field length already set during
innobase_get_computed_value(). Skip the char(n) check for
virtual column in row_merge_buf_add()
2021-06-15 13:15:32 +05:30
Thirunarayanan Balathandayuthapani
7229107e3e MDEV-25872 InnoDB: Assertion failure in row_merge_read_clustered_index upon ALTER on table with indexed virtual columns
- InnoDB fails to check DB_COMPUTE_VALUE_FAILED error in
row_merge_read_clustered_index() and wrongly asserts that
the buffer shouldn't be ran out of memory. Alter table
should give warning when the column value is being
truncated.
2021-06-15 13:15:32 +05:30
Thirunarayanan Balathandayuthapani
8c7d8b716c MDEV-14180 Automatically disable key rotation checks for file_key_managment plugin
Problem:
=======
- InnoDB iterates the fil_system space list to encrypt the
tablespace in case of key rotation. But it is not
necessary for any encryption plugin which doesn't do
key version rotation.

Solution:
=========
- Introduce a new variable called srv_encrypt_rotate to
indicate whether encryption plugin does key rotation

fil_space_crypt_t::key_get_latest_version(): Enable the
srv_encrypt_rotate only once if current key version is
higher than innodb_encyrption_rotate_key_age

fil_crypt_must_default_encrypt(): Default encryption tables
should be added to default_encryp_tables list if
innodb_encyrption_rotate_key_age is zero and encryption
plugin doesn't do key version rotation

fil_space_create(): Add the newly created space to
default_encrypt_tables list if
fil_crypt_must_default_encrypt() returns true

Removed the nondeterministic select from
innodb-key-rotation-disable test. By default,
InnoDB adds the tablespace to the rotation list and
background crypt thread does encryption of tablespace.
So these select doesn't give reliable results.
2021-06-15 13:15:32 +05:30
Thirunarayanan Balathandayuthapani
7d2c338c48 MDEV-14180 preparation: Rename key_rotation_list
- Rename the rotation list to default_encrypt_tables in
fil_system_t. Because rotation list naming could be
misleading when it comes to key version rotation

- Rename is_in_rotation_list to is_in_default_encrypt
in fil_space_t

- Rename keyrotate_next function to default_encrypt_next

fil_system_t::default_encrypt_next(): Find the next
suitable default encrypt table if beginning of
default_encrypt_tables list has been scheduled
to be deleted
2021-06-15 13:15:32 +05:30
Marko Mäkelä
396864c6b3 Remove orphan type name trx_sig_t
This was missed in commit 1d0f70c2f8.
2021-06-10 11:17:40 +03:00
Marko Mäkelä
c7ee039d36 Merge 10.3 to 10.4 2021-06-09 14:28:57 +03:00
Marko Mäkelä
75a65d3201 MDEV-25886 CHECK TABLE crash with DB_MISSING_HISTORY if innodb_read_only
Occasionally, the test innodb.alter_copy would fail in MariaDB 10.6.1,
reporting DB_MISSING_HISTORY during CHECK TABLE. It started to occur during
the development of MDEV-25180, which introduced purge_sys.stop_SYS().
If we delay purge more during DDL operations, then the test would
almost always fail. The reason is that during startup we will restore
a purge view, and CHECK TABLE would still use REPEATABLE READ
even though innodb_read_only is set and other isolation levels
than READ UNCOMMITTED are not guaranteed to work.

ha_innobase::check(): Use READ UNCOMMITTED isolation level if
innodb_read_only is set or innodb_force_recovery exceeds 3.

dict_set_corrupted(): Do not update the persistent data dictionary
if innodb_force_recovery exceeds 3.
2021-06-09 14:23:20 +03:00
Marko Mäkelä
b8d38c5e39 dict_index_build_internal_clust(): Catch MDEV-20131 on CREATE TABLE 2021-06-08 15:03:50 +03:00
Marko Mäkelä
72b2489621 Merge 10.3 into 10.4 2021-06-08 15:02:40 +03:00
Marko Mäkelä
6e9642beb2 Merge 10.2 into 10.3 2021-06-08 14:33:07 +03:00
Marko Mäkelä
dfa2d0bc13 MDEV-25869 Change buffer entries are lost on InnoDB restart
buf_read_ibuf_merge_pages(): If space->size is 0, invoke
fil_space_get_size() to determine the size of the tablespace
by reading the header page. Only after that proceed to delete
any entries that are beyond the end of the tablespace.
Otherwise, we could be deleting valid entries that actually
need to be applied.

This fixes a regression that had been introduced in
commit b80df9eba2 (MDEV-21069),
which aimed to avoid crashes during DROP TABLE of corrupted tables.
2021-06-08 13:21:33 +03:00
Vladislav Vaintroub
b1b4d67bcd MDEV-21373 DBUG compilation - bad synchronization in ha_heap::external_lock()
ha_heap::external_lock contains some consistency checks for the table,#
in a debug compilation.

This code suffers from lack of synchronization, in a rare case
where mysql_lock_tables() fail, and unlock is forced, even if lock was
not previously taken.

To workaround, require EXTRA_DEBUG compile definition in order to activate
the consistency checks.The code still might be useful in some cases - but
the audience are developers looking for errors in single-threaded scenarios,
rather than multiuser stress-tests.
2021-06-04 15:02:48 +02:00
Anel Husakovic
7eed97ed9f MDEV-25777: JAVA_INCLUDE_PATH and JAVA_INCLUDE_PATH2 not found with out of source configuration and Ninja generator
- As solution `PLUGIN_CONNECT=NO` use early check to disable plugin:
  Solution suggested by wlad@mariadb.com
- `JNI_FOUND` is a internal result variable and should be set with
cached library and header variables (like `JAVA_INCLUDE_PATH`) defined.
  * Note: wrapper cmake/FindJNI.cmake runs first time and cmake native Find<module> returns only cached variable, like `JAVA_INCLUDE_PATH`, results variable are not cached).

Reviewed by: serg@mariadb.com
2021-06-04 10:52:18 +02:00
Monty
fa0bbff032 Fixed that compile-pentium64-valgrind-max works
- Removed Tokudb (no need to test this anymore with valgrind)
- Added __attribute__(unused)) to a few places to be able to compile even
  if valgrind/memcheck.h is not installed.

Reviewer: Marko Mäkelä <marko.makela@mariadb.com>
2021-06-02 18:54:49 +03:00
Marko Mäkelä
77d8da57d7 Merge 10.3 into 10.4 2021-06-01 09:14:59 +03:00
Marko Mäkelä
950a220060 Merge 10.2 into 10.3 2021-06-01 08:40:59 +03:00
Sergei Golubchik
d06205ba37 CONNECT: use my_snprintf 2021-05-27 17:36:32 +02:00
Thirunarayanan Balathandayuthapani
c11c5f36d8 MDEV-25758 InnoDB spatial indexes miss large geometry fields after MDEV-25459
InnoDB should calculate the MBR for the first field of
spatial index and do the comparison with the clustered
index field MBR. Due to MDEV-25459 refactoring, InnoDB
calculate the length of the first field and fails with
too long column error.
2021-05-27 15:01:19 +05:30
Marko Mäkelä
ab87fc6c7a Cleanup: Remove handler::update_table_comment()
The only call of the virtual member function
handler::update_table_comment() was removed in
commit 82d28fada7 (MySQL 5.5.53)
but the implementation was not removed.

The only non-trivial implementation was for InnoDB. The information
is now returned via handler::get_foreign_key_create_info() and
ha_statistics::delete_length.
2021-05-27 09:31:19 +03:00
Marko Mäkelä
1dea7f7977 Merge 10.3 into 10.4 2021-05-25 15:38:57 +03:00
Marko Mäkelä
1864a8ea93 Merge 10.2 into 10.3 2021-05-24 09:38:49 +03:00
Thirunarayanan Balathandayuthapani
5c75ba9cad MDEV-25663 Double free of transaction during truncate operation
- Patch addresses the problem to fix double free of
transaction if it is own transaction.
2021-05-24 11:33:01 +05:30
Thirunarayanan Balathandayuthapani
349d77ecdd MDEV-25721 Double free of table when inplace alter
FTS add index fails

Problem:
========
InnoDB double frees the table if auxiliary fts table
creation fails and fails to set the dict operation
for the transaction. It leads to failure while
dropping newly added index.

Solution:
=========
  InnoDB should avoid double freeing and set the
dictionary operation of transaction in
fts_create_common_tables()
2021-05-23 15:53:59 +05:30
Thirunarayanan Balathandayuthapani
98f7b2cb09 MDEV-25663 Double free of transaction during truncate operation
InnoDB truncate table fails to load the fts stopword table into
cache. In that case, InnoDB double frees the truncate creation
transaction. InnoDB should free the transaction which was
created inside ha_innobase::create.
2021-05-23 15:53:59 +05:30
Sergei Golubchik
f70b11c8c9 cmake: fewer Build-Depends in SRPM
don't require tar/gtar, git, getconf, groff/nroff, and ruby.
2021-05-22 21:56:51 +02:00
Sergei Golubchik
d7321893d8 CONNECT: move jar files to /usr/share and include them in DEBs 2021-05-22 21:56:51 +02:00
Marko Mäkelä
9739cf1859 MDEV-25664 Potential hang in purge for virtual columns
ha_innobase::open(): If the table is only being opened by purge
for evaluating virtual column values, avoid invoking
initialize_auto_increment(), because the purge thread may already
be holding an shared latch on the clustered index root page.
Shared latches are not recursive. The additional request would lead
to a hang if another thread has started waiting for an exclusive latch.
2021-05-21 19:12:13 +03:00
Marko Mäkelä
08b6fd9395 MDEV-25710: Dead code os_file_opendir() in the server
The functions fil_file_readdir_next_file(), os_file_opendir(),
os_file_closedir() became dead code in the server in MariaDB 10.4.0
with commit 09af00cbde (the removal of
the crash recovery logic for the TRUNCATE TABLE implementation that
was replaced in MDEV-13564).

os_file_opendir(), os_file_closedir(): Define as macros.
2021-05-18 12:13:18 +03:00
Marko Mäkelä
4240704abc Merge 10.3 into 10.4 2021-05-18 08:59:12 +03:00
Marko Mäkelä
ca3f497564 Merge 10.2 into 10.3, except MDEV-25682 2021-05-18 08:40:19 +03:00