trx_rseg_header_create(): Add a parameter for the value that is
to be written to TRX_RSEG_MAX_TRX_ID. If we omit this write, then
the updated test innodb.undo_truncate will fail for the 4k, 8k, 16k
page sizes. This was broken ever since
commit 947efe17ed (MDEV-15158)
removed the writes of transaction identifiers to the TRX_SYS page.
srv_do_purge(): Truncate undo tablespaces also during slow shutdown
(innodb_fast_shutdown=0).
Thanks to Krunal Bauskar for noticing this problem.
At least since commit 055a3334ad
(MDEV-13564) the undo log truncation in InnoDB did not work correctly.
The main issue is that during the execution of
trx_purge_truncate_history() some pages of the newly truncated
undo tablespace could be discarded.
fsp_try_extend_data_file(): Apply the peculiar rounding of
fil_space_t::size_in_header only to the system tablespace,
whose size can be expressed in megabytes in a configuration parameter.
Other files may freely grow by a number of pages.
fseg_alloc_free_page_low(): Do allow the extension of undo tablespaces,
and mention the file name in the error message.
mtr_t::commit_shrink(): Implement crash-safe shrinking of a tablespace
file. First, durably write the log, then shrink the file, and finally
release the page latches of the rebuilt tablespace. Refactored from
trx_purge_truncate_history().
log_write_and_flush_prepare(), log_write_and_flush(): New functions
to durably write log during mtr_t::commit_shrink().
btr_defragment_save_defrag_stats_if_needed(): Do not save
defragmentation statistics for temporary tables.
They are exempt of defragmentation anyway
(ha_innobase::optimize() never invokes defragmentation for them),
and the user-visible names are not available inside InnoDB.
Furthermore, InnoDB assumes that temporary tables are never accessed
by other threads than the one that handles the session with which
the temporary table is associated with.
Furthermore, we simplify the test innodb.innodb_defrag_stats
and include a test case that demonstrates that defragmentation
statistics are no longer being saved for temporary tables.
dict_index_t::clear_instant_alter(): when searhing for an AUTO_INCREMENT column
don't skip the beginning of the list because the field can be at the beginning of the list
InnoDB could evict the fts auxiliary table in
row_fts_merge_insert(). So bulk insert could be
dealing with garbage FTS auxiliary table.Patch
should delay closing the table in row_fts_merge_insert().
The st_blksize returned by fstat(2) is not documented to be
a power of 2, like we assumed in
commit 58252fff15 (MDEV-26040).
While on Linux, the st_blksize appears to report the file system
block size (which hopefully is not smaller than the sector size
of the underlying block device), on FreeBSD we observed
st_blksize values that might have been something similar to st_size.
Also IBM AIX was affected by this. A simple test case would
lead to a crash when using the minimum innodb_buffer_pool_size=5m
on both FreeBSD and AIX:
seq -f 'create table t%g engine=innodb select * from seq_1_to_200000;' \
1 100|mysql test&
seq -f 'create table u%g engine=innodb select * from seq_1_to_200000;' \
1 100|mysql test&
We will fix this by not trusting st_blksize at all, and assuming that
the smallest allowed write size (for O_DIRECT) is 4096 bytes. We hope
that no storage systems with larger block size exist. Anything larger
than 4096 bytes should be unlikely, given that it is the minimum
virtual memory page size of many contemporary processors.
MariaDB Server on Microsoft Windows was not affected by this.
While the 512-byte sector size of the venerable Seagate ST-225 is still
in widespread use, the minimum innodb_page_size is 4096 bytes, and
innodb_log_file_size can be set in integer multiples of 65536 bytes.
The only occasion where InnoDB uses smaller data file block sizes than
4096 bytes is with ROW_FORMAT=COMPRESSED tables with KEY_BLOCK_SIZE=1
or KEY_BLOCK_SIZE=2 (or innodb_page_size=4096). For such tables,
we will from now on preallocate space in integer multiples of 4096 bytes
and let regular writes extend the file by 1024, 2048, or 3072 bytes.
The view INFORMATION_SCHEMA.INNODB_SYS_TABLESPACES.FS_BLOCK_SIZE
should report the raw st_blksize.
For page_compressed tables, the function fil_space_get_block_size()
will map to 512 any st_blksize value that is larger than 4096.
os_file_set_size(): Assume that the file system block size is 4096 bytes,
and only support extending files to integer multiples of 4096 bytes.
fil_space_extend_must_retry(): Round down the preallocation size to
an integer multiple of 4096 bytes.
FTS indexes has a prefix_len=1 or prefix_len=0 as stated by comment in
mysql_prepare_create_table().
Thus, a newly added assertion should be relaxed for FTS indexes.
Bug happens when partially indexed CHAR or VARCHAR field in converted from
utf8mb3 to utf8mb4.
Fixing by relaxing assertions. For some time dict_index_t and dict_table_t
are becoming not synchronized. Namely, dict_index_t has a new prefix_len which
is a multiple of a user-provided length and charset->mbmaxlen. But
the table still have and old mbmaxlen and assertion fails. This happens only
during utf8mb3 -> utf8mb4 conversions and the magic number 4 comes from
utf8mb_4_.
At the end of ALTER TABLE (innobase_rename_or_enlarge_columns_cache())
dict_index_t and dict_table_t became synchronized
again and will stay so at all times. For, example, they will be synchronized
on table load and newly added assertion proves that.
trx_purge_rseg_get_next_history_log(): Fix a race condition that
was introduced in commit e46f76c974
(MDEV-15912). The buffer pool page contents must not be accessed
while not holding a page latch. The page latch was released by
mtr_t::commit().
This race resulted in an ASAN heap-use-after-poison during a stress test.
To avoid potential race conditions between concurrent access to
dict_table_t::freed_indexes, let us consistently use
dict_table_t::autoinc_mutex.
dict_table_remove_from_cache_low(): To avoid extensive hold time
of table->autoinc_mutex, unconditionally free the FTS data structures.
ha_innobase::check_if_supported_inplace_alter(): Do not invoke
innobase_table_is_empty() if the tablespace has been discarded.
That is, native ALTER TABLE in InnoDB will treat an empty table
in the same way as a tablespace whose tablespace has been discarded.
(Note: ALTER TABLE...ALGORITHM=COPY will fail if the tablespace
has been discarded.)
This fixes a crash that was introduced
in commit c755974775 (MDEV-19611).
Problem:
=======
The last AHI page for two indexes of an dropped table is being
freed at the same time by two threads. One thread frees the
table heap and other thread tries to access table heap again.
It leads to asan failure in btr_search_lazy_free().
Solution:
========
InnoDB uses autoinc_mutex to avoid the race condition
in btr_search_lazy_free()
Designated initializers were introduced in ISO/IEC 9899:1999 (C99),
but the C code base of MariaDB is supposed to be compatible with the
1990 version of the standard.
The InnoDB code based was switched from C to C++ in
MySQL 5.6 and MariaDB 10.0. C++ did not introduce syntax for
designated initializers until ISO/IEC 14882:2020.
Our C++ code base is still stuck with the 2011 or earlier version of
that standard.
Therefore, this check as well as the macro STRUCT_FLD are best removed.
PageConverter::update_index_page(): Always validate the PAGE_INDEX_ID.
Failure to do so could cause a crash when iterating
secondary index pages. This was caught by the 10.4 test
innodb.full_crc32_import.
Delete-marked record is on the secondary index and the clustered index
already purged the corresponding record. We cannot detect if such
record is historical and we should not: the algorithm of
row_ins_check_foreign_constraint() skips such record anyway.
If lock type is LOCK_GAP or LOCK_ORDINARY, and the transaction holds
implicit lock for the record, then explicit gap-lock will not be set for
the record, as lock_rec_convert_impl_to_expl() returns true and
lock_rec_convert_impl_to_expl() bypasses lock_rec_lock() call.
The fix converts explicit lock to implicit one if requested lock type is
not LOCK_REC_NOT_GAP.
innodb_information_schema test result is also changed as after the fix
the following statements execution:
SET autocommit=0;
INSERT INTO t1 VALUES (5,10);
SELECT * FROM t1 FOR UPDATE;
leads to additional gap lock requests.
Creation of tables by the three names mysql.user, mysql.host, mysql.db
was being blocked in the function row_mysql_is_system_table().
Since commit 4abb8216a0 (MDEV-17658),
mysql.user is a view, not a table. Since commit
ead9a34a3e (MDEV-15851), mysql.host
is not being created at all.
Let us remove the special handling of table names in InnoDB,
and allow mysql.db to be created in InnoDB. The special handling
was originally added in commit e84ef2b747
without any explanation.
Reviewed by: Sergei Golubchik
Import operation without .cfg file fails when there is mismatch of index
between metadata table and .ibd file. Moreover, MDEV-19022 shows
that InnoDB can end up with index tree where non-leaf page has only
one child page. So it is unsafe to find the secondary index root page.
This patch does the following when importing the table without .cfg file:
1) If the metadata contains more than one index then InnoDB stops
the import operation and report the user to drop all secondary
indexes before doing import operation.
2) When the metadata contain only clustered index then InnoDB finds the
index id by reading page 0 & page 3 instead of traversing the
whole tablespace.
pars_info_bind_id(): Remove the parameter copy_name. It was always
being passed as constant TRUE or true. It turns out that copying
the string is completely unnecessary. In all calls except the one
in fts_get_select_columns_str() and fts_doc_fetch_by_doc_id(),
the parameter is being passed as a compile-time constant, and therefore
the pointer cannot become stale. In that special call, the string
that is being passed is allocated from the same memory heap that
pars_info_bind_id() would have been using.
pars_info_add_id(): Remove (unused declaration).
1. rename option DEPENDENCIES in MYSQL_ADD_PLUGIN() to DEPENDS
to be consistent with other cmake commands and macros
2. use this DEPENDS option in plugins
3. add dependencies to the plugin embedded target too
4. plugins don't need to add GenError dependency explicitly,
all plugins depend on it automatically
len was containing garbage, since vctempl->mysql_col_offset was
containing old value while calling row_mysql_store_col_in_innobase_format
from innobase_get_computed_value().
It was not updated after the first ALTER TABLE call, because it's INPLACE
logic considered there's nothing to update, and exited immediately from
ha_innobase::inplace_alter_table().
However, vcol metadata needs an update, since vcols structure is changed
in mysql record.
The regression was introduced by 12614af1fe. There, refcount==1 condition
was removed, which turned out to be crucial, though racy. The idea was to
update vc_templ after each (sequencing) ALTER TABLE.
We should do the same another way, and there may be a plenty of solutions,
but the simplest one is to add a following condition:
if vcol structure is changed, drop vc_templ; it will be recreated on next
ha_innobase::open() call.
in prepare_inplace_alter_table. It is safe, since innodb inplace changes
require at least HA_ALTER_INPLACE_SHARED_LOCK_AFTER_PREPARE, which
guarantee MDL_EXCLUSIVE on this stage.
alter_templ_needs_rebuild() also has to track the columns not indexed, to
keep vc_templ correct.
Note that vc_templ is always kept constructed and available after
ha_innobase::open() call, even on INSERT, though no virtual columns are
evaluated during that statement
inside innodb.
In the test case suplied, it will be recreated on the second ALTER TABLE.
ha_innobase::prepare_inplace_alter_table(): Remove always-true conditions.
Near the start of the function, we would already have returned if
no ALTER TABLE operation flags were set that would require special
action from InnoDB.
It turns out that the conditions were redundant already when they were
introduced in mysql/mysql-server@241387a2b6
and in commit 068c61978e.
Thanks to Nikita Malyavin for noticing this.
Thanks to Nikita Malyavin for noticing this.
The dead code that was originally introduced in
mysql/mysql-server@b8bd31740c
was added in commit 2e814d4702
to this code base.
btr_scrub_start_space(): Avoid an unnecessary tablespace lookup
and related acquisition of fil_system->mutex. In MariaDB Server 10.3
we would get deadlocks between that mutex and a crypt_data mutex.
The fix was developed by Thirunarayanan Balathandayuthapani.
