buf_page_print(): Dump the buffer page 32 bytes (64 hexadecimal digits)
per line. In this way, the limitation in mtr
("Data too long for column 'line'") will not be triggered.
Also, do not bother decoding the page contents, because everything
is present in the hexadecimal output.
dict_index_find_on_id_low(): Merge to dict_index_get_if_in_cache_low().
The direct call in buf_page_print() was prone to crashing, in case the
table definition was concurrently evicted or dropped from the
data dictionary cache.
Spider supports (or at least allows) INSERT DELAYED but the
documentation does not specify spider as a storage engine that supports
"INSERT DELAYED".
Also, although not mentioned in the documentation, "INSERT DELAYED" is
not intended to be executed inside a transaction, as can be seen from
the list of supported storage engines.
The current implementation allows executing a delayed insert on a
remote transactional table and this breaks the consistency ensured by
the transaction.
We too remove "internal_delayed", one of the Spider table parameters.
Documentation says,
> Whether to transmit existence of delay to remote servers when
> executing an INSERT DELAYED statement on local server.
This table parameter is only used for "INSERT DELAYED".
Reviewed by: Nayuta Yanagisawa
Take into account that in preparation of a simple key cache for resizing no disk blocks might be assigned to it.
Reviewer: IgorBabaev <igor@mariadb.com>
Revert "TSAN: data race on vptr (ctor/dtor vs virtual call)"
This reverts commit 78084fa747.
This commit was done to please TSAN, which falsely reported an error
where there was none.
Yet as consequence, it could cause a real error, a crash in os_aio_free on
shutdown
fil_name_process(): If the recovery of a tablespace was deferred,
do invoke fil_ibd_load() even though the name in recv_spaces is
not changing. This allows us to recover from a situation where
there are many FILE_RENAME records, renaming a tablespace back
and forth, and a FILE_MODIFY record that had been written by
fil_names_clear().
Co-developed with: Thirunarayanan Balathandayuthapani
Fixed tpool timer implementation on POSIX.
Prior to this patch, under some specific rare circumstances (concurrency
related), timer callback execution might be skipped.
When attempting to recover a database with an incorrect encryption key,
the unencrypted page contents should be expected to differ from what
was written before recovery. Let us suppress some more messages.
This caused intermittent failures, depending on when the latest
log checkpoint was triggered.
trx_undo_rec_copy(): Return nullptr if the undo record is corrupted.
trx_undo_rec_get_undo_no(): Define inline with the declaration.
trx_purge_dummy_rec: Replaced with a -1 pointer.
row_undo_rec_get(), UndorecApplier::apply_undo_rec(): Check
if trx_undo_rec_copy() returned nullptr.
trx_purge_get_next_rec(): Return nullptr upon encountering any
corruption, to signal the end of purge.
On GNU/Linux, even though the C11 aligned_alloc() appeared in
GNU libc early on, some custom memory allocators did not
implement it until recently. For example, before
gperftools/gperftools@d406f22853
the free() in tcmalloc would fail to free memory that was
returned by aligned_alloc(), because the latter would map to the
built-in allocator of libc. The Linux specific memalign() has a
similar interface and is safer to use, because it has been
available for a longer time. For AddressSanitizer, we will use
aligned_alloc() so that the constraint on size can be enforced.
buf_tmp_reserve_compression_buf(): When HAVE_ALIGNED_ALLOC holds,
round up the size to be an integer multiple of the alignment.
pfs_malloc(): In the unit test stub, round up the size to be an
integer multiple of the alignment.
Table_cache_instance: Define the structure aligned at
the CPU cache line, and remove a pad[] data member.
Krunal Bauskar reported this to improve performance on ARMv8.
aligned_malloc(): Wrapper for the Microsoft _aligned_malloc()
and the ISO/IEC 9899:2011 <stdlib.h> aligned_alloc().
Note: The parameters are in the Microsoft order (size, alignment),
opposite of aligned_alloc(alignment, size).
Note: The standard defines that size must be an integer multiple
of alignment. It is enforced by AddressSanitizer but not by GNU libc
on Linux.
aligned_free(): Wrapper for the Microsoft _aligned_free() and
the standard free().
HAVE_ALIGNED_ALLOC: A new test. Unfortunately, support for
aligned_alloc() may still be missing on some platforms.
We will fall back to posix_memalign() for those cases.
HAVE_MEMALIGN: Remove, along with any use of the nonstandard memalign().
PFS_ALIGNEMENT (sic): Removed; we will use CPU_LEVEL1_DCACHE_LINESIZE.
PFS_ALIGNED: Defined using the C++11 keyword alignas.
buf_pool_t::page_hash_table::create(),
lock_sys_t::hash_table::create():
lock_sys_t::hash_table::resize(): Pad the allocation size to an
integer multiple of the alignment.
Reviewed by: Vladislav Vaintroub
There was a race condition between log_checkpoint_low() and
deleting or renaming data files. The scenario is as follows:
1. The buffer pool does not contain dirty pages.
2. A FILE_DELETE or FILE_RENAME record is written.
3. The checkpoint LSN will be moved ahead of the write of the record.
4. The server is killed before the file is actually renamed or deleted.
We will prevent this race condition by ensuring that a log checkpoint
cannot occur between the durable write and the file system operation:
1. Durably write the FILE_DELETE or FILE_RENAME record.
2. Perform the file system operation.
3. Allow any log checkpoint to proceed.
mtr_t::commit_file(): Implement the DELETE or RENAME logic.
fil_delete_tablespace(): Delegate some of the logic to
mtr_t::commit_file().
fil_space_t::rename(): Delegate some logic to mtr_t::commit_file().
Remove the debug injection point fil_rename_tablespace_failure_2
because we do test RENAME failures without any debug injection.
fil_name_write_rename_low(), fil_name_write_rename(): Remove.
Tested by Matthias Leich
This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).
Also fixed broken logic of automatic generation of the Diffie-Hellman parameters
for socat version less than 1.7.3 (which defaults to 512-bit values instead of
2048-bit ones).
Recent adventures in liburing and btrfs have shown up some kernel
version dependent bugs. Having a bug report of accurace kernel version
can start to correlate these errors sooner.
On Linux, /proc/version contains the kernel version.
FreeBSD has kern.version (per man 8 sysctl), so include that too.
Example output:
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Core pattern: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
Kernel version: Linux version 5.19.0-0.rc2.21.fc37.x86_64 (mockbuild@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 12.1.1 20220507 (Red Hat 12.1.1-1), GNU ld version 2.38-14.fc37) #1 SMP PREEMPT_DYNAMIC Mon Jun 13 15:27:24 UTC 2022
Segmentation fault (core dumped)
it starts an EXPLAIN of a multi-table join and tries to KILL it.
no sync points.
depending on how fast the hareware is and optimizer development
it might kill EXPLAIN at some random point in time (generally unrelated
to the Bug#28598 it was supposed to test) or EXPLAIN might finish
before the KILL and the test will fail.
Work around MDEV-28718 for now, but also optimize the interation
of information_schema.SYSTEM_VARIABLES.
Add test case to show that tzinfo data into bootstrap is
desired functionality.
Bug report thanks to Dan Lenski of AWS.
We do not really care about the exact result; we only care that the
statistics will be accessed. The result could change depending on
when some statistics were updated in the background or when some
committed delete-marked rows were purged from other tables on
which persistent statistics are enabled.
ha_innobase::prepare_inplace_alter_table(): If the ALTER TABLE
operation is no-op for InnoDB, do reset m_prebuilt->trx_id
so that ha_innobase::table_version() will always report either 0
or the identifier of the transaction that would commit changes
to the InnoDB data dictionary.
The failure scenario involved a completed DROP INDEX followed by
a no-op ALTER TABLE during which the server was killed.
An effort to create a reproducible test failed.
ha_partition::set_auto_increment_if_higher expects
part_share->auto_inc_initialized is true or can_use_for_auto_inc_init()
is false (but as the comment of this method says, it returns false
only if we use Spider engine with DROP TABLE or ALTER TABLE query).
However, part_share->auto_inc_initialized becomes true only after all
partitions are opened (since 6dce6aeceb).
Therefore, I added a conditional expression in order to read all
partitions when we execute REPLACE on a table that has an
AUTO_INCREMENT column.
Reviewed by: Nayuta Yanagisawa
Reviewed by: Alexey Botchkov
The bug was that build_notnull_conds_for_range_scans() did not take into
account the join_tab is not yet sorted with constant tables first.
Fixed the bug by testing explicitely if a table is a const table.
Since commit 0b47c126e3 (MDEV-13542)
we treat all-zero pages as corrupted ones.
During a stress test, a read-ahead of an all-zero page was triggered
and the page read was completed concurrently with buf_page_create_low().
This caused the assertion to fail, because buf_page_create_low() was
waiting for the page latch.
buf_page_get_low(): Only invoke buf_pool_t::corrupted_evict()
if the block was not already marked as corrupted.
buf_page_create_low(): On page identifier mismatch, retry the
buf_pool.page_hash lookup.
buf_pool_t::corrupted_evict(): Set the state of the block to FREED
so that a concurrent buf_page_get_low() will refuse to load the page.
Wait for the page latch to be vacant before proceeding to remove
the block from buf_pool.page_hash and buf_pool.LRU.
page_id_t::set_corrupted(), page_id_t::is_corrupted(): Accessors
for indicating a corrupted page identifier.
Tested by Matthias Leich
trx_purge_free_segment(): Do mark that the block will be modified.
It seems possible that this regression was introduced by the
changes to the page-freeing logic
in commit 4179f93d28 (MDEV-18976).
Tested by: Matthias Leich
- innodb_fts.sync_block doesn't make sense after MDEV-25581's patch
because fts cache syncing is done as a part of insert operation
and it leads to completion of select over insert sometimes.
This test case is not relevant any more
In commit 73fee39ea6 (MDEV-27985)
a regression was introduced that would cause bpage=nullptr to
be referenced.
buf_flush_LRU_list_batch(): Always terminate the loop upon
encountering a null pointer.
Current Debian package revision scheme when using
debian/autobake-deb.sh script is:
'1:VERSION+maria~LSBNAME'
For example if VERSION can be like 10.6.8 and LSBNAME is
buster then version and revision is:
'1:10.6.8+maria~buster'
Which can lead to problem as distro code names can be lexical unordered.
For example Debian LSBNAME's can be:
Codename Buster is Debian version 10
Codename Bookworm is Debian version 11
This happens because in ASCII table
Buster first two digits are 'Bu' and they are in hex 0x42 and 0x75
and Bookworm first digits 'Bo' are they are in hex 0x42 and 0x6F
When apt is upgrading it means that:
1:10.6.8+maria~buster is bigger than 1:10.6.8+maria~bookworm
and that leads to problems in dist-upgrade process
To solve problem revision format is changed to:
'1:VERSION+maria~(deb|ubu)LSBVERSION'
Example for Debian 11 is now:
1:10.6.8+maria~deb11
and for Ubuntu 22.04 is now:
1:10.6.8+maria~ubu2204
There are new Variables
* VERSION which contains whole version string
* LSBVERSION which contains LSB version of distro
* LSBID which contains LSB ID (Debian or Ubuntu)
added to debian/autobake-deb.sh.
Also CODENAME is change to LSBNAME as it's more declaritive
During rebuild of partition, the partitioning engine calls
alter_close_table(), which does not unlock and close some table
instances of the target table.
Then, the engine fails to rename partitions because there are table
instances that are still locked.
Closing all the table instance of the target table fixes the bug.
During rebuild of partition, the partitioning engine calls
alter_close_table(), which does not unlock and close some table
instances of the target table.
Then, the engine fails to rename partitions because there are table
instances that are still locked.
Closing all the table instance of the target table fixes the bug.
innodb_drop_database(): Use explicit TO_BINARY casts on
SYS_TABLES.NAME, which for historical reasons uses the wrong collation
latin1_swedish_ci instead of BINARY.
MariaDB codebase is huge and Lintian has lots of test than
can fire false-positive warnings which leads to situation
where real problems can't be spotted.
Suspend obvious false-positive Lintian warnings and
let Lintian problems that needs some love shine
out.
Suspends in package mariadb-test-data
Supporting BSD family needs to use '/usr/bin/env perl' and not '/usr/bin/perl'
Perl script are for testing and not for production in mariadb-test-data
package:
* incorrect-path-for-interpreter
There is several files with national-encoding which are test file so they
can't be in unicode charset
* national-encoding
Serveral test paths are intentionally repeated:
* repeated-path-segment
Suspends in package mariadb-test
Supporting BSD family needs to use '/usr/bin/env perl' and not '/usr/bin/perl'
Perl script are for testing and not for production in mariadb-test-data
package:
* incorrect-path-for-interpreter
Suspends in package source package
Remade some 'version-substvar-for-external-package' to use
regex.
MGroonga is missing source file 'jquery-ui-1.8.18.custom.js' correct
lintian suspend with regex:
* source-is-missing
There is several files with very long line lenghts. Add suspends
for those that can't be corrected in several places. Most
of them are test result files, SQL test files or intentional
long lines that can't be splitted.
* very-long-line-length-in-source-file
There is several autogenerated C++ files which probably should not
be there but they should not do any harm:
* source-contains-autogenerated-visual-c++-file
Problem:
========
InnoDB FTS requesting the fts sync of the table once the fts
cache size reaches 1/10 of innodb_ft_cache_size. But fts_sync()
releases cache lock when writing the word. By doing this, InnoDB
insert thread increases the innodb fts cache memory and
SYNC operation will take more time to complete.
Solution:
=========
Remove the fts sync operation(FTS_MSG_SYNC_TABLE) from
the fts optimize background thread. Instead of that,
allow user thread to sync the InnoDB fts cache when
the cache size exceeds 512 kb. User thread holds
cache lock while doing cache syncing, it make sure that
other threads doesn't add the docs into the cache.
Removed FTS_MSG_SYNC_TABLE and its related function
because we do remove the FTS_MSG_SYNC_TABLE message
itself.
Removed fts_sync_index_check() and all related
function because other threads doesn't add while
cache operation going on.
Test prior to this change:
CURRENT_TEST: connect.mysql
mysqltest: At line 485: query 'INSERT IGNORE INTO t3 VALUES (5),(10),(30)' failed: ER_GET_ERRMSG (1296): Got error 122 '(1062) Duplicate entry '10' for key 'PRIMARY' [INSERT INTO `t1` (`a`) VALUES (10)]' from CONNECT
So the ignore table option wasn't getting passed to the remove server.
Closes#2008
File '/usr/bin/mariadb_config' has been moved from Debian package
libmariadbd-dev to libmariadb-dev since MariaDB version 10.2
this leads to situation where upgrade will no succeed but fail
with this kind of error message
* trying to overwrite '/usr/bin/mariadb_config', which is also in package libmariadbd-dev 1:10.2.44+maria~bionic
Add libmariadbd-dev to libmariadb-dev Debian control files
'Breaks' solve situation and upgrading won't error anymore
PageConverter::update_header(): Remove an unnecessary write.
The field that was originally called FIL_PAGE_FILE_FLUSH_LSN only
made sense for the first page of the system tablespace
(initially, for the first page of each file of the system tablespace).
It never had any meaning for .ibd files, and it lost its original
meaning in MariaDB Server 10.8.1 when
commit b07920b634 (MDEV-27199)
removed the ability to start without ib_logfile0.
If the most significant 32 bits of the LSN are nonzero, this
unnecessary write would write the wrong encryption key identifier
to the page. The first page of any file is never encrypted,
so normally those bytes should be 0 for any .ibd file.
