ReadFile/WriteFile operations.
Innodb opens files with FILE_FLAG_OVERLAPPED. lpNumberOfBytesRead/Written
are documented to be potentially inaccurate in this case,
(possibly even if async operations complete synchronously?)
The fix is to always pass NULL for the correspondng parameters,
as recommended by MSDN. Read the actual counts with
GetQueuedCompletionStatus() or GetOverlappedResult().
os_file_get_size(): Use fstat() instead of calling lseek() 3 times.
In this way, concurrent calls to this function should not interfere
with each other.
Suggested by Vladislav Vaintroub.
The predicate page_is_root() would not hold if btr_create() fails
before the root page is fully initialized. Move the debug assertion
from btr_free_root_invalidate() to its other caller,
btr_free_if_exists(). In that caller, we actually already checked
for page_is_root().
os_file_set_size(): Sometimes the file already is large enough.
Avoid calling posix_fallocate() with a non-positive argument.
Also, add a missing space to an error message.
With this patch, parameters passed to posix_fallocate() will be
the same as they were prior to refactoring in commit b731a5bcf2
In particular, 'offset' parameter for posix_fallocate is again current_file_size
and 'length' is new_file_size - current_file_size.
This seems to fix crashes on ancient Linux (kernel 2.6).
The assertion failure was relaxed in
commit 02ba15a9da
but it was not enough, because another assertion would fail.
trx_undo_free_prepared(): Allow any undo log state. For transactions
that were resurrected in TRX_STATE_COMMITTED_IN_MEMORY
the undo log state would have been reset by trx_undo_set_state_at_finish().
The redo log record MLOG_UNDO_HDR_DISCARD is never written, and
the function trx_undo_discard_latest_update_undo() is never called
outside crash recovery.
Starting with MariaDB 10.2.2, crash recovery from an older InnoDB
version is refused. Therefore, we can safely remove the code for
parsing this long-unused redo log record type.
The use of MLOG_UNDO_HDR_DISCARD was removed by Heikki Tuuri,
the original designer and author of InnoDB, more than 13 years ago:
commit 3caab0f327
Author: unknown <heikki@hundin.mysql.fi>
Date: Thu Mar 18 14:57:22 2004 +0200
trx0undo.h, trx0undo.c, trx0trx.c:
Update an obsolete comment about trx commit: we can no longer call
trx_undo_update_cleanup_by_discard(), and actually, the idea to
call it was erroneous, it cannot work
trx_free_prepared(): The function trx_resurrect_insert()
can resurrect a committed transaction when the TRX_UNDO_STATE
field is not TRX_UNDO_ACTIVE or TRX_UNDO_PREPARED.
(In the examined failure of mariabackup.xb_compressed_encrypted,
assertion failure in --apply-log shutdown at line 25,
the TRX_UNDO_STATE strangely contained the value 0xffff.)
In any case, the assertion must allow the state
TRX_STATE_COMMITTED_IN_MEMORY for resurrected transactions.
Replace all references in InnoDB and XtraDB error log messages
to bugs.mysql.com with references to https://jira.mariadb.org/.
The original merge
commit 4274d0bf57
was accidentally reverted by the subsequent merge
commit 3b35d745c3
row_upd_changes_field_size_or_external(): Always return TRUE
if an instantly added column is being updated such that the
column value is absent from the record.
Also, avoid some unnecessary computations.
MariaDB did not import the Oracle Bug #23481444
(commit 6ca4f693c1ce472e2b1bf7392607c2d1124b4293) from MySQL.
Initially, the code changes were disabled, and later removed.
Let us remove the last remaining references.
MDEV-14021 Failing assertion: table->get_ref_count() == 0 on ALTER TABLE with implicit ALGORITHM=INPLACE
Post-merge fix: Remove the now-useless use of table reference count.
The dict_operation_lock S-latch whose use was introduced in the 10.0
fix will prevent the table from disappearing due to concurrent
DDL operations (DROP, table-rebuilding ALTER, TRUNCATE).
New test cases
innodb-page-cleaners
Modified test cases
innodb_page_cleaners_basic
New function buf_flush_set_page_cleaner_thread_cnt
Increase or decrease the amount of page cleaner worker threads.
In case of increase this function creates based on current
abount and requested amount how many new threads should be
created. In case of decrease this function sets up the
requested amount of threads and uses is_requested event
to signal workers. Then we wait until all new treads
are started, old threads that should exit signal
is_finished or shutdown has marked that page cleaner
should finish.
buf_flush_page_cleaner_worker
Store current thread id and thread_no and then signal
event is_finished. If number of used page cleaner threads
decrease we shut down those threads that have thread_no
greater or equal than number of page configured page
cleaners - 1 (note that there will be always page cleaner
coordinator). Before exiting we signal is_finished.
New function innodb_page_cleaners_threads_update
Update function for innodb-page-cleaners system variable.
innobase_start_or_create_for_mysql
If more than one page cleaner threads is configured
we use new function buf_flush_set_page_cleaner_thread_cnt
to set up the requested threads (-1 coordinator).
InnoDB was writing unnecessary information to the
update undo log records. Most notably, if an indexed column is updated,
the old value of the column would be logged twice: first as part of
the update vector, and then another time because it is an indexed column.
Because the InnoDB undo log record must fit in a single page,
this would cause unnecessary failure of certain updates.
Even after this fix, InnoDB still seems to be unnecessarily logging
indexed column values for non-updated columns. It seems that non-updated
secondary index columns only need to be logged when a PRIMARY KEY
column is updated. To reduce risk, we are not fixing this remaining flaw
in GA versions.
trx_undo_page_report_modify(): Log updated indexed columns only once.
remove remnants of 10.0 bugfix, incorrectly merged into 10.2
Using col_names[i] was obviously, wrong, must've been col_names[ifield->col_no].
incorrect column name resulted in innodb having index unique_id2(id1),
while the server thought it's unique_id2(id4).
But col_names[ifield->col_no] is wrong too, because `table` has non-renamed
columns, so the correct column name is always dict_table_get_col_name(table, ifield->col_no)
i_s_sys_tables_fill_table_stats(): Acquire dict_operation_lock
S-latch before acquiring dict_sys->mutex, to prevent the table
from being removed from the data dictionary cache and from
being freed while i_s_dict_fill_sys_tablestats() is accessing
the table handle.
Issue
=====
The original issue was that the size of a fil_per_table tablespace was calculated
incorrectly during truncate in the presence of an fts index. This incorrect calculation
was fixed as part of BUG#25053705 along with a testcase to reproduce the bug. The
assert that was added as part of it to reproduce the bug was wrong and resulted in
this bug.
Fix
===
Although the assert was removed earlier in a seperate commit as it was blocking the
ntest, this patch replaces the other parts of the code that were added to reproduce
the bug and replaces it with code that tries to reproduce the bug in a different way.
The new code basically tries to tweak conditions so as to simulate the random read
where a page that doesn't exist is tried to be read.
RB: 15890
Reviewed-by: Jimmy Yang <Jimmy.Yang@oracle.com>
Reviewed-by: Satya Bodapati <satya.bodapati@oracle.com>
The ownership of the field query->intersection usually transfers
to query->doc_ids. In some error scenario, it could be possible
that fts_query_free() would be invoked with query->intersection!=NULL.
Let us handle that case, instead of intentionally crashing the server.
MySQL 5.7 added code to push down the LIMIT to fulltext search
in InnoDB:
commit 2cd0ebf97e1b265e2282d7ceb5d8dfb663ffc48f
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date: Fri May 27 13:49:28 2016 +0530
Bug #22709692 FTS QUERY EXCEEDS RESULT CACHE LIMIT
The code was disabled when MySQL 5.7.9 was merged to MariaDB 10.2.2.
We shall remove the disabled code and unnecessary variables.
When MySQL 5.6.10 introduced innodb_read_only mode, it skipped the
creation of the InnoDB buffer pool dump/restore subsystem in that mode.
Attempts to set the variable innodb_buf_pool_dump_now would have
no effect in innodb_read_only mode, but the corresponding condition
was forgotten in from the other two update functions.
MySQL 5.7.20 would fix the innodb_buffer_pool_load_now,
but not innodb_buffer_pool_load_abort. Let us fix both in MariaDB.
fts_get_next_doc_id(): Assign the first and subsequent FTS_DOC_ID
in the same way: by post-incrementing the cached value.
If there is a user-specified FTS_DOC_ID, do not touch the internal
sequence.
There are two bugs related to failed ADD INDEX and
the InnoDB table cache eviction.
dict_table_close(): Try dropping failed ADD INDEX when releasing
the last table handle, not when releasing the last-but-one.
dict_table_remove_from_cache_low(): Do not invoke
row_merge_drop_indexes() after freeing all index metadata.
Instead, directly invoke row_merge_drop_indexes_dict() to
remove the metadata from the persistent data dictionary
and to free the index pages.
