Commit graph

188525 commits

Author SHA1 Message Date
Marko Mäkelä
4bfdba2e89 MDEV-26672 innodb_undo_log_truncate may reset transaction ID sequence
trx_rseg_header_create(): Add a parameter for the value that is
to be written to TRX_RSEG_MAX_TRX_ID. If we omit this write, then
the updated test innodb.undo_truncate will fail for the 4k, 8k, 16k
page sizes. This was broken ever since
commit 947efe17ed (MDEV-15158)
removed the writes of transaction identifiers to the TRX_SYS page.

srv_do_purge(): Truncate undo tablespaces also during slow shutdown
(innodb_fast_shutdown=0).

Thanks to Krunal Bauskar for noticing this problem.
2021-09-24 11:23:37 +03:00
Marko Mäkelä
b46cf33ab8 Merge 10.2 into 10.3 2021-09-22 18:01:41 +03:00
Marko Mäkelä
1cb218c37c MDEV-26450: Corruption due to innodb_undo_log_truncate
At least since commit 055a3334ad
(MDEV-13564) the undo log truncation in InnoDB did not work correctly.

The main issue is that during the execution of
trx_purge_truncate_history() some pages of the newly truncated
undo tablespace could be discarded.

fsp_try_extend_data_file(): Apply the peculiar rounding of
fil_space_t::size_in_header only to the system tablespace,
whose size can be expressed in megabytes in a configuration parameter.
Other files may freely grow by a number of pages.

fseg_alloc_free_page_low(): Do allow the extension of undo tablespaces,
and mention the file name in the error message.

mtr_t::commit_shrink(): Implement crash-safe shrinking of a tablespace
file. First, durably write the log, then shrink the file, and finally
release the page latches of the rebuilt tablespace. Refactored from
trx_purge_truncate_history().

log_write_and_flush_prepare(), log_write_and_flush(): New functions
to durably write log during mtr_t::commit_shrink().
2021-09-22 14:15:00 +03:00
Marko Mäkelä
21d19ed45b Silence a warning about unused Bison label 2021-09-22 14:08:51 +03:00
Daniel Ye
9fc1ef932f
MDEV-26545 Spider does not correctly handle UDF and stored function in where conds
- Handle stored function conditions correctly, with the same logic as with UDFs.
- When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
- Disable direct update/delete when a udf condition is skipped.
2021-09-22 18:55:05 +09:00
Daniel Ye
ac1c6738f9
MDEV-26545 Spider does not correctly handle UDF and stored function in where conds
- Handle stored function conditions correctly, with the same logic as with UDFs.
- When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
2021-09-22 18:54:22 +09:00
Vladislav Vaintroub
3d30458695 MDEV-26521 Remove mdev-504.test 2021-09-21 20:45:42 +02:00
Alexey Bychko
25d6bbcd51 MDEV-23506 mariadb-connector-c-devel package from standard RHEL 8 repo conflicts with MariaDB's packages
added alternative name for MariaDB-devel package to replace
mariadb-connector-c-devel from RHEL 8 distribution.
this patch is for 10.3+ on RHEL/Centos 8
2021-09-21 14:44:39 +07:00
Julius Goryavsky
f4d6d01782 MDEV-26441: Linux-dependent construct in SST scripts
SST scripts currently use Linux-specific construction
to create a temporary directory if the path prefix for
that directory is specified by the user. This does not
work with FreeBSD. This commit adds support for FreeBSD.

No separate test required.
2021-09-20 12:10:53 +02:00
Marko Mäkelä
3209bc667f MDEV-26636: InnoDB defragmentation statistics cause races on TEMPORARY TABLE
btr_defragment_save_defrag_stats_if_needed(): Do not save
defragmentation statistics for temporary tables.
They are exempt of defragmentation anyway
(ha_innobase::optimize() never invokes defragmentation for them),
and the user-visible names are not available inside InnoDB.

Furthermore, InnoDB assumes that temporary tables are never accessed
by other threads than the one that handles the session with which
the temporary table is associated with.

Furthermore, we simplify the test innodb.innodb_defrag_stats
and include a test case that demonstrates that defragmentation
statistics are no longer being saved for temporary tables.
2021-09-18 15:47:52 +03:00
Thirunarayanan Balathandayuthapani
496d3dded4 MDEV-15675 encryption.innodb_encryption failed in buildbot with timeout
Test case fail to include undo tablespace while waiting for the
encryption thread to encrypt all existing tablespace
2021-09-17 19:37:57 +05:30
Oleksandr Byelkin
dce490e9d4 Give less memory to get reliable error. 2021-09-16 17:29:37 +02:00
Daniel Black
dabc3329c3 MDEV-26622: man mysqldump - insert-ignore not insert-into
Thanks Karl Levik
2021-09-16 20:52:31 +10:00
Vladislav Vaintroub
b1351c1594 MDEV-26574 An improper locking bug due to unreleased lock in the ds_xbstream.cc
release lock in all as cases n xbstream_open, also fix the case where malloc would return NULL.
2021-09-15 14:55:45 +02:00
Vladislav Vaintroub
8937762ead MDEV-26573 : A static analyzer warning about ds_archive.cc
This file had not been compiled for long time.
Remove this from the tree.
2021-09-15 14:19:24 +02:00
Thirunarayanan Balathandayuthapani
696de6d06c MDEV-25702 Auxiliary FTS table evicts during optimize table
InnoDB could evict the fts auxiliary table in
row_fts_merge_insert(). So bulk insert could be
dealing with garbage FTS auxiliary table.Patch
should delay closing the table in row_fts_merge_insert().
2021-09-13 12:39:17 +05:30
Vladislav Vaintroub
b112c9dfaa Fix Connect build with MSVC+Ninja 2021-09-11 18:32:07 +02:00
Vladislav Vaintroub
8988e471b5 Merge branch '10.2' into 10.3
# Conflicts:
#	cmake/os/Windows.cmake
#	sql/sql_yacc.yy
2021-09-11 16:21:36 +02:00
Vladislav Vaintroub
f345172379 MDEV-26527 speedup appveyor build - 10.2 2021-09-11 15:19:42 +02:00
Vladislav Vaintroub
3504f70f7f Bison 3.7 - fix "conversion from 'ptrdiff_t' to 'ulong', possible loss of data" 2021-09-11 15:19:42 +02:00
Vladislav Vaintroub
879e21b68c Define minbuild target for 10.2 2021-09-11 15:19:42 +02:00
Vladislav Vaintroub
7e6b033507 Fix MYSQL_MAINTAINER_MODE=ERR, on Windows, with Ninja , in 10.2
A conversion warning 4267 that we want to disable(prior to 10.3),
was suppressed with  cmake VS generator for C++ and C, despite being set
only for CXX flags.

The fix is to disable the warning in C flags, too. In 10.2, this warning
is noisy, in 10.3 it is fixed.
2021-09-11 15:19:42 +02:00
Marko Mäkelä
bcd25e1066 Merge 10.2 into 10.3 2021-09-11 11:14:18 +03:00
Marko Mäkelä
ac064c2b47 Fix an occasional timeout in innodb.alter_partitioned 2021-09-11 11:12:11 +03:00
Marko Mäkelä
d09426f9e6 MDEV-26537 InnoDB corrupts files due to incorrect st_blksize calculation
The st_blksize returned by fstat(2) is not documented to be
a power of 2, like we assumed in
commit 58252fff15 (MDEV-26040).
While on Linux, the st_blksize appears to report the file system
block size (which hopefully is not smaller than the sector size
of the underlying block device), on FreeBSD we observed
st_blksize values that might have been something similar to st_size.

Also IBM AIX was affected by this. A simple test case would
lead to a crash when using the minimum innodb_buffer_pool_size=5m
on both FreeBSD and AIX:

seq -f 'create table t%g engine=innodb select * from seq_1_to_200000;' \
1 100|mysql test&
seq -f 'create table u%g engine=innodb select * from seq_1_to_200000;' \
1 100|mysql test&

We will fix this by not trusting st_blksize at all, and assuming that
the smallest allowed write size (for O_DIRECT) is 4096 bytes. We hope
that no storage systems with larger block size exist. Anything larger
than 4096 bytes should be unlikely, given that it is the minimum
virtual memory page size of many contemporary processors.

MariaDB Server on Microsoft Windows was not affected by this.

While the 512-byte sector size of the venerable Seagate ST-225 is still
in widespread use, the minimum innodb_page_size is 4096 bytes, and
innodb_log_file_size can be set in integer multiples of 65536 bytes.

The only occasion where InnoDB uses smaller data file block sizes than
4096 bytes is with ROW_FORMAT=COMPRESSED tables with KEY_BLOCK_SIZE=1
or KEY_BLOCK_SIZE=2 (or innodb_page_size=4096). For such tables,
we will from now on preallocate space in integer multiples of 4096 bytes
and let regular writes extend the file by 1024, 2048, or 3072 bytes.

The view INFORMATION_SCHEMA.INNODB_SYS_TABLESPACES.FS_BLOCK_SIZE
should report the raw st_blksize.

For page_compressed tables, the function fil_space_get_block_size()
will map to 512 any st_blksize value that is larger than 4096.

os_file_set_size(): Assume that the file system block size is 4096 bytes,
and only support extending files to integer multiples of 4096 bytes.

fil_space_extend_must_retry(): Round down the preallocation size to
an integer multiple of 4096 bytes.
2021-09-10 19:15:41 +03:00
Vladislav Vaintroub
1c378f1b95 Speedup build of the MSI package 2021-09-10 12:02:44 +02:00
Sergei Golubchik
46cb16388a Merge branch '10.2' into 10.3 2021-09-09 12:15:55 +02:00
Sergei Golubchik
c7184c470e fix main.truncate failures in --embedded 2021-09-09 09:57:39 +02:00
Sergei Golubchik
a6383a1954 Merge branch '10.2' into 10.3 2021-09-07 23:24:06 +02:00
Sergei Golubchik
0d3de06eda disable bzip2/lz4/lzo in rpm builds, distro dependent 2021-09-07 21:13:04 +02:00
Sergei Golubchik
ca2f89deac disable bzip2/lz4/lzo in bintar builds, as they always have been 2021-09-07 21:13:04 +02:00
Vicențiu Ciorbaru
b85b8348e7 Merge branch '10.2' into 10.3 2021-09-07 16:32:35 +03:00
Anel Husakovic
630d722902 MDEV-19227: mysql_plugin doesn't run bootstrap from source
Reviewed by: serg@mariadb.com
             daniel@mariadb.org
2021-09-07 14:25:56 +02:00
Haidong Ji
528abc749e MDEV-25325 built-in documentation for performance_schema tables
Improve documentation of performance_schema tables by appending COLUMN
comments to tables. Additionally improve test coverage and update corresponding
tests.
2021-09-07 08:45:19 +03:00
Rucha Deodhar
edde990e35 MDEV-23365: Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())'
failed upon killed TRUNCATE

Note: This is a backport of 1cb4caa66d from
10.3
Analysis: Assertion failure happens because less session memory is set and so
table can't be reopened. So the statement can't be used. This error goes
unreported.
Fix: Return the error state.
2021-09-06 18:12:45 +05:30
Sergei Golubchik
f17537579a disable cmake feature summary after the first run
because the summary is incorrect after the first run anyway
2021-09-06 09:49:53 +02:00
Daniel Black
38648bbbf5 MDEV-12055 binlog.binlog_stm_ctype_ucs postfix
As highlighted in https://bugs.gentoo.org/807995,
people occasionaly run tests in the 20:00->23:59:59 time range.

Fixes 265e3253f1

binlog.binlog_stm_ctype_ucs 'mix'        w1 [ fail ]
        Test ended at 2021-08-11 22:55:35

CURRENT_TEST: binlog.binlog_stm_ctype_ucs
--- /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result    2021-06-18 18:19:11.000000000 +0800
+++ /var/tmp/portage/dev-db/mariadb-10.5.11/work/mysql/mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.reject    2021-08-11 22:55:34.993447479 +0800
@@ -76,21 +76,21 @@
 /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
 DELIMITER /*!*/;
 # at #
-#YYMMDD HH:MM:SS server id #  end_log_pos # CRC32 XXX  Start: binlog v 4, server v #.##.## created YYMMDD HH:MM:SS
+#210811 22:55:34 server id #  end_log_pos # CRC32 XXX  Start: binlog v 4, server v #.##.## created 210811 22:55:34
2021-09-05 23:38:25 +00:00
Daniel Black
21d31b9970 MDEV-26529: binlog.binlog_flush_binlogs_delete_domain fails on RISC-V
Per https://bugs.gentoo.org/807995

The test failed with:

CURRENT_TEST: binlog.binlog_flush_binlogs_delete_domain
— /tmp/mariadb-10.5.11/mysql-test/suite/binlog/r/binlog_flush_binlogs_delete_domain.result 2021-06-18 18:19:11.000000000 +0800
+++ /tmp/mariadb-10.5.11/mysql-test/suite/binlog/r/binlog_flush_binlogs_delete_domain.reject 2021-09-01 22:55:29.406655479 +0800
@@ -85,6 +85,6 @@
ERROR HY000: The value of gtid domain being deleted ('4294967296') exceeds its maximum size of 32 bit unsigned integer
FLUSH BINARY LOGS DELETE_DOMAIN_ID = (4294967295);
Warnings:
-Warning 1076 The gtid domain being deleted ('4294967295') is not in the current binlog state
+Warning 1076 The gtid domain being deleted ('18446744073709551615') is not in the current binlog state
DROP TABLE t;
RESET MASTER;

mysqltest: Result length mismatch

ptr_domain_id is a uint32* so explicitly cast this when printing it out.

Thanks Marek Szuba for the bug report and testing the patch.
2021-09-06 08:47:58 +10:00
Oleksandr Byelkin
391f6b4f1e MDEV-26362: incorrect nest_level value with INTERSECT
Add DBUG_ASSERT (should be kept in merge)
Fix nest_level assignment in LEX::add_unit_in_brackets (should be ignored in merge to 10.4)
2021-09-05 10:38:12 +02:00
Jan Lindström
f55477060c MDEV-26518 ; Galera incorrectly handles primary or unique keys with any multi-byte character set
We need to set temporary buffer large enough to fit also multi-byte
characters.
2021-09-02 07:32:19 +03:00
Jan Lindström
99f6a266c8 MDEV-26517 : Galera test failure on galera_fk_cascade_delete_debug
Move --error on --reap where it belongs and take a account that
there could be different return codes.
2021-09-02 07:32:19 +03:00
Vladislav Vaintroub
5613ead49e MDEV-26521 Remove mdev-504.test
This stress tests fails on not sufficiently tweaked windows boxes,due
to rapid succession of connects and disconnects, making Windows run out
of ephemeral ports.

Approved by author of the test (the author is happy to see it removed)
2021-09-01 19:29:44 +02:00
Vladislav Vaintroub
d6b7738dcc Fix potential null pointer access after the allocation error 2021-09-01 18:21:34 +02:00
Vladislav Vaintroub
234ae43d5a Cleanup - remove confusing comment. 2021-09-01 18:21:00 +02:00
Elena Stepanova
a4a4d6a7c8 MDEV-26514 Option to build a separate test zip package on Windows
echo is needed for the tests
2021-09-01 17:02:42 +03:00
Elena Stepanova
8382c3260b MDEV-26514 Option to build a separate test zip package on Windows
Add a possibility to run MTR tests on a release zip
2021-08-31 21:13:13 +03:00
Sergei Golubchik
1597b3d76b disable bzip2, lzma, and lzo explicitly in debian release builds
we don't build debian release builds with them anyway,
but let's make it explicit, independently on what happens to be
installed on the builder.
2021-08-31 09:50:57 +02:00
Marko Mäkelä
e835cc851e Merge 10.2 into 10.3 2021-08-31 08:36:59 +03:00
Marko Mäkelä
ceb40ef45b MDEV-26504 THD::copy_db_to() fails to return true if THD::db is null
THD::copy_db_to(): Always return true if the output parameter
was left uninitialized. This fixes a regression that was caused
by commit 7d0d934ca6 (MDEV-16473).

MariaDB Server 10.3 and later were unaffected by this bug
thanks to commit a7e352b54d.

Possibly this bug only affects mysql_list_fields()
in the Embedded Server (libmysqld).

This bug was found by GCC 11.2.0 in CMAKE_BUILD_TYPE=RelWithDebInfo.
2021-08-30 14:26:27 +03:00
Marko Mäkelä
fda704c82c Fix GCC 11 -Wmaybe-uninitialized for PLUGIN_PERFSCHEMA
init_mutex_v1_t: Stop lying that the mutex parameter is const.
GCC 11.2.0 assumes that it is and could complain about any mysql_mutex_t
being uninitialized even after mysql_mutex_init() as long as
PLUGIN_PERFSCHEMA is enabled.

init_rwlock_v1_t, init_cond_v1_t: Remove untruthful const qualifiers.

Note: init_socket_v1_t is expecting that the socket fd has already
been created before PSI_SOCKET_CALL(init_socket), and therefore that
parameter really is being treated as a pointer to const.
2021-08-30 11:52:59 +03:00