Bug #11766501: Multiple RBS break the get rseg with mininum trx_t::no code during purge
Bug# 59291 changes:
Main problem is that truncating the UNDO log at the completion of every
trx_purge() call is expensive as the number of rollback segments is increased.
We truncate after a configurable amount of pages. The innodb_purge_batch_size
parameter is used to control when InnoDB does the actual truncate. The truncate
is done once after 128 (or TRX_SYS_N_RSEGS iterations). In other words we
truncate after purge 128 * innodb_purge_batch_size. The smaller the batch
size the quicker we truncate.
Introduce a new parameter that allows how many rollback segments to use for
storing REDO information. This is really step 1 in allowing complete control
to the user over rollback space management.
New parameters:
i) innodb_rollback_segments = number of rollback_segments to use
(default is now 128) dynamic parameter, can be changed anytime.
Currently there is little benefit in changing it from the default.
Optimisations in the patch.
i. Change the O(n) behaviour of trx_rseg_get_on_id() to O(log n)
Backported from 5.6. Refactor some of the binary heap code.
Create a new include/ut0bh.ic file.
ii. Avoid truncating the rollback segments after every purge.
Related changes that were moved to a separate patch:
i. Purge should not do any flushing, only wait for space to be free so that
it only does purging of records unless it is held up by a long running
transaction that is preventing it from progressing.
ii. Give the purge thread preference over transactions when acquiring the
rseg->mutex during commit. This to avoid purge blocking unnecessarily
when getting the next rollback segment to purge.
Bug #11766501 changes:
Add the rseg to the min binary heap under the cover of the kernel mutex and
the binary heap mutex. This ensures the ordering of the min binary heap.
The two changes have to be committed together because they share the same
that fixes both issues.
rb://567 Approved by: Inaam Rana.
rb://566
approved by: Sunny
When using native aio on linux each IO helper thread should be able to
handle upto 256 IO requests. The number 256 is the same which is used
for simulated aio as well. In case of windows where we also use native
aio this limit is 32 because of OS constraints. It seems that we are
using the limit of 32 for all the platforms where we are using native
aio. The fix is to use 256 on all platforms except windows (when native
aio is enabled on windows)
The bug would cause a crash of InnoDB if a non-standard or unknown table
flags existed in a SYS_TABLES record. This is important because the next
file version, Cheetah, will identify itself by expanding this field. So
unless this is fixed, an older engine that tries to open a table in a
tablespace with a newer file version will crash instead of report an error
and refuse to open the table, as it should do.
Reviewed at RB://583. Approved by Marko.
trx_get_trx_by_xid(): Invalidate trx->xid after a successful lookup,
so that subsequent callers will not find the same transaction.
The only callers of trx_get_trx_by_xid() will be invoking
innobase_commit_low() or innobase_rollback_trx(), and those code paths
should not depend on trx->xid.
rb://584 approved by Jimmy Yang
that implement add_index
The problem was that ALTER TABLE blocked reads on an InnoDB table
while adding a secondary index, even if this was not needed. It is
only needed for the final step where the .frm file is updated.
The reason queries were blocked, was that ALTER TABLE upgraded the
metadata lock from MDL_SHARED_NO_WRITE (which blocks writes) to
MDL_EXCLUSIVE (which blocks all accesses) before index creation.
The way the server handles index creation, is that storage engines
publish their capabilities to the server and the server determines
which of the following three ways this can be handled: 1) build a
new version of the table; 2) change the existing table but with
exclusive metadata lock; 3) change the existing table but without
metadata lock upgrade.
For InnoDB and secondary index creation, option 3) should have been
selected. However this failed for two reasons. First, InnoDB did
not publish this capability properly.
Second, the ALTER TABLE code failed to made proper use of the
information supplied by the storage engine. A variable
need_lock_for_indexes was set accordingly, but was not later used.
This patch fixes this problem by only doing metadata lock upgrade
before index creation/deletion if this variable has been set.
This patch also changes some of the related terminology used
in the code. Specifically the use of "fast" and "online" with
respect to ALTER TABLE. "Fast" was used to indicate that an
ALTER TABLE operation could be done without involving a
temporary table. "Fast" has been renamed "in-place" to more
accurately describe the behavior.
"Online" meant that the operation could be done without taking
a table lock. However, in the current implementation writes
are always prohibited during ALTER TABLE and an exclusive
metadata lock is held while updating the .frm, so ALTER TABLE
is not completely online. This patch replaces "online" with
"in-place", with additional comments indicating if concurrent
reads are allowed during index creation/deletion or not.
An important part of this update of terminology is renaming
of the handler flags used by handlers to indicate if index
creation/deletion can be done in-place and if concurrent reads
are allowed. For example, the HA_ONLINE_ADD_INDEX_NO_WRITES
flag has been renamed to HA_INPLACE_ADD_INDEX_NO_READ_WRITE,
while HA_ONLINE_ADD_INDEX is now HA_INPLACE_ADD_INDEX_NO_WRITE.
Note that this is a rename to clarify current behavior, the
flag values have not changed and no flags have been removed or
added.
Test case added to innodb_mysql_sync.test.
over the mutex list and free each mutex. When UNIV_MEM_DEBUG is defined, we
need skip the hash mutex.
It is a minor bug affecting only UNIV_SYNC_DEBUG builds, found by Michael.
There are two main pain points, one is lookup by thread id for sync_thread_t
and the other is to do a lookup by latch or level in sync_thread_t::levels.
Changed the sync_thread_t::levels lookup and reserve operation from O(N)
to O(1).
Pure lookups are still O(N), the main change for pure lookup is that we no
longer need to search up to SYNC_THREAD_N_LEVELS but only up to the number
of slots actually ever used ie. it is possible some were used in the past
but are now on the free list. If the in_use count drops to 0 we reset the
free list too.
Overload the sync_level_t::level field to track the free list. If
sync_thread_t::latch == NULL then sync_thread_t::level contains the ordinal
value of the previous free entry.
rb://580 Approved by Jimmy Yang.
rw_lock_debug_print(): Add parameter FILE* for specifying the output stream.
rw_lock_list_print_info(): Invoke rw_lock_debug_print() on file, not stderr.
"rows examined" estimates". This change implements "innodb_stats_method"
with options of "nulls_equal", "nulls_unequal" and "null_ignored".
rb://553 approved by Marko
Check whether the master and purge thread are active after creating them. Do
not proceed until both threads have started. We do this by checking whether a
slot has been reserved by both the respective threads.
Add srv_thread_has_reserved_slot() returns slot no or ULINT_UNDEFINED.
rb://536 Approved by Jimmy
