fseg_create(): Initialize FSEG_FRAG_ARRY by a single MLOG_MEMSET record.
flst_zero_addr(), flst_init(): Optimize away redundant writes.
fseg_free_page_low(): Write FIL_NULL by MLOG_MEMSET.
The XDES_CLEAN_BIT is always set for every element of
the page allocation bitmap in the extent descriptor pages.
Do not bother touching it, to avoid redundant writes.
page_rec_write_field(): Remove.
dict_create_index_tree_step(): If the SYS_INDEXES.PAGE does not change,
do not update it in the data dictionary. Typically, all index page numbers
would be unchanged before and after IMPORT TABLESPACE, except if some
secondary indexes were created after loading some data.
btr_root_fseg_adjust_on_import(): Remove the redundant mtr_t* parameter.
Redo logging is disabled during the page adjustments that IMPORT TABLESPACE
is performing.
fsp_alloc_seg_inode_page(): Ever since
commit 3926673ce7
all newly allocated pages are zero-initialized.
Assert that this is the case for the FSEG_ID fields.
btr_store_big_rec_extern_fields(): Remove the redundant initialization
of the most significant 32 bits of BTR_EXTERN_LEN. InnoDB never supported
BLOBs that are longer than 4GiB. In fact, dtuple_convert_big_rec()
would write emit an error message if a clustered index record tuple would
exceed 1,000,000,000 bytes in length.
The BTR_EXTERN_LEN in the BLOB pointers in clustered index leaf page
records is zero-initialized at least since
commit 41bb3537ba
The XDES_CLEAN_BIT is always set for every element of
the page allocation bitmap in the extent descriptor pages.
Do not bother touching it, to avoid redundant writes.
xdes_set_free<bool free>(): Replaces xdes_set_bit().
The page allocation bitmaps in the extent descriptor pages
contain two bits per page: XDES_FREE_BIT and XDES_CLEAN_BIT,
which is unused. Simplify read access.
xdes_is_free(descr,mtr): Remove. Use !xdes_get_n_used(descr) instead.
xdes_is_free(): Replaces xdes_get_bit(), xdes_mtr_get_bit().
xdes_find_free(): Replaces xdes_find_bit().
fsp_seg_inode_page_get_nth_inode(): Remove the redundant parameters
physical_size, mtr.
fsp_seg_inode_page_find_used(), fsp_seg_inode_page_find_free():
Remove the redundant parameter mtr.
fsp_alloc_seg_inode_page(): Ever since
commit 3926673ce7
all newly allocated pages are zero-initialized.
Assert that this is the case for the FSEG_ID fields.
(Side note: before that fix, other parts of the pages
could contain nonzero garbage.)
btr_store_big_rec_extern_fields(): Remove the redundant initialization
of the most significant 32 bits of BTR_EXTERN_LEN. InnoDB never supported
BLOBs that are longer than 4GiB. In fact, dtuple_convert_big_rec()
would write emit an error message if a clustered index record tuple would
exceed 1,000,000,000 bytes in length.
The function mach_read_ulint() is a wrapper for the lower-level
functions mach_read_from_1(), mach_read_from_2(), mach_read_from_8().
Invoke those functions directly, for better readability of the code.
mtr_t::read_ulint(), mtr_read_ulint(): Remove. Yes, we will lose the
ability to assert that the read is covered by the mini-transaction.
We would still check that on writes, and any writes that
wrongly bypass mini-transaction logging would likely be caught by
stress testing with Mariabackup.
A conflict between MDEV-19514 (b42294bc64)
and MDEV-20934 (d7a2401750)
was resolved. We will not invoke the function ibuf_delete_recs()
from ibuf_merge_or_delete_for_page(). Instead, we will add that
logic to the function ibuf_read_merge_pages().
dict_index_add_to_cache(): Make the 'index' a reference to a pointer,
so that the caller will avoid the expensive call to
dict_index_get_if_in_cache_low().
Due to a data corruption bug that may have occurred a long time earlier
(possibly involving physical backup and MySQL Bug #69122, which was
addressed in commit f166ec71b7)
it seems possible that the InnoDB change buffer might end up containing
entries, while no buffered changes exist according to the change buffer
bitmap pages in the .ibd files.
ibuf_delete_recs(): New function, to be invoked on slow shutdown only.
Remove all buffered changes for a specific page.
ibuf_merge_or_delete_for_page(): If the change buffer bitmap is clean
and a slow shutdown is in progress, invoke ibuf_delete_recs().
We do not want to do that during normal operation, due to the additional
overhead that is involved. The bitmap page should be consistent with
the change buffer in the first place.
InnoDB: Assertion failure in file .../dict/dict0dict.cc line ...
InnoDB: Failing assertion: table->can_be_evicted
This fixes a regression that was caused by the fix of MDEV-20621
(commit a41d429765).
MySQL 5.6 (and MariaDB 10.0) introduced eviction of tables from
the InnoDB data dictionary cache. Tables that are connected to
FOREIGN KEY constraints or FULLTEXT INDEX are exempt of the eviction.
With the problematic change, a table that would already be exempt
from eviction due to FOREIGN KEY would cause the problem if there
also was a FULLTEXT INDEX defined on it.
dict_load_table(): Only prevent eviction if table->can_be_evicted holds.
Don't save/restore HP_INFO as it could be changed by a concurrent thread.
different parts of HP_INFO are protected by different mutexes and
the mutex that protect most of the HP_INFO does not protect its open_list
data.
As a bonus, make heap_check_heap() to take const HP_INFO* and not
make any changes there whatsoever.
Something was wrong with the removal of pointer indirection,
because on 32-bit Windows we got crash recovery failures.
Curiously, WITH_ASAN of a 32-bit debug build did not notice anything.
This reverts a part of commit b7fc2c899e.
We have a 2MiB recv_sys.buf for the initial buffering. The minimum size
of log_sys.buf would be 16MiB, and that buffer should be practically
unused during recovery. If the buffer pool size is measured in gigabytes,
it would indeed make sense to use the buffer pool for the recovered log
records, perhaps after we have run out of log_sys.buf.
FIXME: allow the entire buf_block_t::frame to be used for buffered
log records, and remove MEM_HEAP_FOR_RECV_SYS. For example, use
buf_page_t::list or buf_page_t::LRU for keeping track of memory that
was allocated for recovery? Most members of buf_block_t
(except buf_block_t::frame) are unused when
block->page.state == BUF_BLOCK_MEMORY.
dict_table_rename_in_cache(): Use strcpy() instead of strncpy(),
because they are known to be equivalent in this case (the length
of old_name was already validated).
mariabackup: Invoke strncpy() with one less than the buffer size,
and explicitly add NUL as the last byte of the buffer.
recv_t, recv_t::data_t: Define constructors that copy the log records
Ideally, we should remove recv_sys.buf, RECV_DATA_BLOCK_SIZE,
and recv_sys_justify_left_parsing_buf(), and let the
recv_sys.pages point directly to a buffer of parsed redo log
records.
The RECV_PARSING_BUF_SIZE (size of recv_sys.buf) is only 2MiB,
while the minimum innodb_log_buffer_size (size of log_sys.buf) is 16MiB,
and log_sys.buf is unused during redo log apply!
Except for fil_name_process(), which invokes os_normalize_path(),
the redo log record parser will not modify the redo log records.
Add const qualifiers accordingly.
storage/innobase/log/log0recv.cc|1760 col 35 error| 'void* memcpy(void*, const void*, size_t)' writing to an object of type 'struct recv_t' with no trivial copy-assignment [-Werror=class-memaccess]
- Add SLEEP() calls to the testcase to make it really test that the
time doesn't change.
- Always use .frm file creation time as a creation timestamp (attempts
to re-use older create_time when the table DDL changes are not good
because then create_time will change after server restart)
- Use the same method names as the upstream patch does
- Use std::atomic for m_update_time
page_recv_t: Replaces recv_sys_t::recs_t.
page_recv_t::state is not private, even though some accessors exist.
page_recv_t::log: A singly-linked list of log_rec_t* with STL decoration
and the custom operations trim() and append(). The list members are private.
recv_t::data_t: Replaces recv_data_t.
recv_t::data: Remove the pointer indirection for the first log chunk,
and copy the first chunk directly after the record. Adjust the
definition of RECV_DATA_BLOCK_SIZE accordingly.
In the collections of Standard Template Library,
empty() is a predicate and clear() empties a collection.
Let us rename recv_sys.empty() to recv_sys.clear() to avoid confusion.
innobase_drop_foreign_try(): Don't evict and reload the dict_foreign_t
during instant ALTER TABLE if the FOREIGN KEY constraint is being
dropped.
The MDEV-19630 fix (commit 07b1a26c33)
was incomplete, because it did not cover a case where the
FOREIGN KEY constraint is being dropped.
Apply the changes to InnoDB and XtraDB that had been
inadvertently skipped in the merge
commit ae476868a5
That merge failure sabotaged part of MDEV-20127:
>Revert a problematic auto_increment_increment 'fix' from 2014.
>This involves replacing the MDEV-8827 fix and in 10.1,
>removing some WSREP instrumentation.
The code changes were re-merged manually by executing the following:
# Get the parent of the problematic merge.
git checkout ae476868a5394041a00e75a29c7d45917e8dfae8^
# Perform the merge again.
git merge ae476868a5394041a00e75a29c7d45917e8dfae8^2
# Get the conflict resolution from that merge.
git checkout ae476868a5 .
# Note: Any changes to these files were removed (empty diff)!
git diff HEAD storage/{innobase,xtradb}/handler/ha_innodb.cc
# Apply the code changes:
git diff cf40393471b10ca68cc1d2804c22ab9203900978^2..MERGE_HEAD \
storage/{innobase,xtradb}/handler/ha_innodb.cc|
patch -p1
In debug build, whenever MEMORY table instance gets closed it performs
consistency check without protection. It may cause server crash if
executed concurrently with DML.
Moved consistency check to ha_heap::external_lock(F_UNLCK), so that it
is protected by THR_LOCK.
Lock wait can happen on secondary index when doing FK checks for wsrep.
We should just return error to upper layer and applier will retry
operation when needed.
ut_strlcpy(): Replace with the standard function strncpy().
ut_strlcpy_rev(): Define in the same compilation unit where
the only caller resides. Avoid unnecessary definition
in non-debug builds.
TODO: do not use fil_* functions for redo log files.
log_t::checkpoint_lock: remove this lock which was used to wait for
async I/O completion.
checkpoint_lock_key
checkpoint_lock: remove now unneeded globals
log_write_checkpoint_info(): remove sync argument because all checkpoint
writes are synchronous now
log_write_checkpoint_info(): remove sync argument
log_group_checkpoint(): merge with the only caller
log_complete_checkpoint(): merge with the only caller
log_t::complete_checkpoint(): remove by merging with the only caller.
mem_heap_dup(): Avoid mem_heap_alloc() and memcpy() of data=NULL, len=0.
trx_undo_report_insert_virtual(), trx_undo_page_report_insert(),
trx_undo_page_report_modify(): Avoid memcpy(ptr, NULL, 0).
dfield_data_is_binary_equal(): Correctly handle data=NULL, len=0.
rec_init_offsets_temp(): Do allow def_val=NULL in the interface.
This clean-up was motivated by WITH_UBSAN, and no bug related to this
was observed in the wild. It should be noted that undefined behaviour
such as memcpy(ptr, NULL, 0) could allow compilers to perform unsafe
optimizations, like it was the case in
commit fc168c3a5e (MDEV-15587).
mem_heap_dup(): Avoid mem_heap_alloc() and memcpy() of data=NULL, len=0.
trx_undo_report_insert_virtual(), trx_undo_page_report_insert(),
trx_undo_page_report_modify(): Avoid memcpy(ptr, NULL, 0).
dfield_data_is_binary_equal(): Correctly handle data=NULL, len=0.
This clean-up was motivated by WITH_UBSAN, and no bug related to this
was observed in the wild. It should be noted that undefined behaviour
such as memcpy(ptr, NULL, 0) could allow compilers to perform unsafe
optimizations, like it was the case in
commit fc168c3a5e (MDEV-15587).
Ignore GetDiskFreeSpace() errors in os_file_get_status_win32
The call is only used to calculate filesystem block size, and this in
turn is only shown in information_schema.sys_tablespaces.FS_BLOCK_SIZE.
There is no other use of this field, it does not affect any Innodb
functionality
Historically, InnoDB split the redo log into at least 2 files.
MDEV-12061 allowed the minimum to be innodb_log_files_in_group=1,
but it kept the default at innodb_log_files_in_group=2.
Because performance seems to be slightly better with only one log file,
and because implementing an append-only variant of the log would require
a single file, let us define the default to be 1, and have
innodb_log_file_size=96M, to retain the same default total size.
- fts_optimize_thread() uses dict_table_t object instead of table id.
So that it doesn't acquire dict_sys->mutex. It leads to remove the
hang of dict_sys->mutex between fts_optimize_thread() and other threads.
- in_queue to indicate whether the table is in fts_optimize_queue. It
is protected by fts_optimize_wq->mutex to avoid any race condition.
- fts_optimize_init() adds the fts table to the fts_optimize_wq
InnoDB stores synced_doc_id + 1 value in FTS_CONFIG table. But
while reading the synced doc id from FTS_CONFIG table after restart,
InnoDB should read synced_doc_id - 1 to get the actual synced
doc id value.
Based on the performance testing that was conducted in MDEV-17492,
the InnoDB adaptive hash index could only help performance in specific,
almost-read-only workloads. It could slow down all kinds of workloads
(especially DROP TABLE, TRUNCATE TABLE, ALTER TABLE, or DROP INDEX
operations), and it can become corrupted, causing crashes (such as
MDEV-18815, MDEV-20203) and possibly data corruption. Furthermore,
the adaptive hash index consumes space from the InnoDB buffer pool,
which could hurt performance when the working set would almost fit
in the buffer pool.
Given all this, it is best to disable the adaptive hash index by default.
This can happen if one uses a backup where not all aria_log.* files
are copied or if the last one is too short. In this case the data
files will contain data that is not in the logs and recovery will fail.
Other things:
- Fixed tprint() to not print extra new line to debug trace
To diagnose a hang in slow shutdown (innodb_fast_shutdown=0),
let us introduce a Boolean startup option in debug builds
that will cause the contents of the InnoDB change buffer
to be dumped to the server error log at startup.
Reduce the scope of some variables, remove a goto and a redundant
assertion.
For B-tree secondary indexes, this function can remove a delete-marked
purgeable record, in case a row rollback of the INSERT was initiated
due to an error in an earlier secondary index.
The BtrBulk class, which was introduced in MySQL 5.7, is by design
the exclusive writer to an index. It is therefore unnecessary to
acquire the dict_index_t::lock in that code.
Holding the dict_index_t::lock would unnecessarily block other threads
(SQL connections and the InnoDB purge threads) from buffering concurrent
modifications to being-created secondary indexes.
This fix is motivated by a change in MySQL 5.7.28:
Bug #29008298 MYSQLD CRASHES ITSELF WHEN CREATING INDEX
mysql/mysql-server@f9fb96c20f
PageBulk::init(), PageBulk::latch(): Never acquire m_index->lock.
PageBulk::storeExt(): Remove some pointer indirection, and improve
a debug assertion that seems to prove that some code is redundant.
BtrBulk::pageCommit(): Assert that m_index->lock is not being held.
btr_blob_log_check_t: Do not acquire m_index->lock if
m_op == BTR_STORE_INSERT_BULK. Add UNIV_UNLIKELY hints around
that condition.
btr_store_big_rec_extern_fields(): Allow index->lock not to be held
while op == BTR_STORE_INSERT_BULK. Add UNIV_UNLIKELY hints around
that condition.
MDEV-18451 Server crashes in maria_create_trn_for_mysql
upon ALTER TABLE
Problem was that when table was locked many times, not all
instances where removed from the transaction by
_ma_remove_table_from_trnman()
MDEV-16210 original case was wrongly allowed versioned DELETE from
referenced table where reference is by non-primary key. InnoDB UPDATE
has optimization for new rows not changing its clustered index
position. In this case InnoDB doesn't update all secondary indexes and
misses the one holding the referenced key. The fix was to disable this
optimization for versioned DELETE. In case of versioned DELETE we
forcely update all secondary indexes and therefore check them for
constraints.
But the above fix raised another problem with versioned DELETE on
foreign table side. In case when there was no corresponding record in
referenced table (illegal foreign reference can be done with "set
foreign_key_checks=off") there was spurious constraint check (because
versioned DELETE is actually UPDATE) and hence the operation failed
with constraint error.
MDEV-16210 tried to fix the above problem by checking foreign table
instead of referenced table and that at least was illegal.
Constraint check is done by row_ins_check_foreign_constraint() no
matter what kind of table is checked, referenced or foreign
(controlled by check_ref argument).
Referenced table is checked by row_upd_check_references_constraints().
Foreign table is checked by row_ins_check_foreign_constraints().
Current fix rolls back the wrong fix for the above problem and
disables referenced table check for DELETE on foreign side by
introducing `check_foreign` argument which when set to *false* skips
row_ins_check_foreign_constraints() call.
fil_crypt_rotate_page(): Skip the key rotation for pages that carry 0
in FIL_PAGE_TYPE. This avoids not only unnecessary writes, but also
failures of the recently added debug assertion in
buf_flush_init_for_writing() that the FIL_PAGE_TYPE should be nonzero.
Note: the debug assertion can fail if the file was originally created
before MySQL 5.5. In old InnoDB versions, FIL_PAGE_TYPE was only
initialized for B-tree pages, to FIL_PAGE_INDEX. For any other pages,
the field could be garbage, including FIL_PAGE_INDEX. In MariaDB 10.2
and later, buf_flush_init_for_writing() would initialize the
FIL_PAGE_TYPE on such old pages, but only after passing the debug
assertion that insists that pages have a nonzero FIL_PAGE_TYPE.
Thus, the debug assertion at the start of buf_flush_init_for_writing()
can fail when upgrading from very old debug files. This assertion is
only present in debug builds, not release builds.
