Change the read ahead parameter name to innodb_read_ahead_threshold.
Change the meaning of this parameter to signify the number of pages
that must be sequentially accessed for InnoDB to trigger a readahead
request.
Suggested by: Ken
This patch introduces heuristics based flushing rate of dirty pages to
avoid IO bursts at checkpoint.
1) log_capacity / log_generated per second gives us number of seconds
in which ALL dirty pages need to be flushed. Based on this rough
assumption we can say that
n_dirty_pages / (log_capacity / log_generation_rate) = desired_flush_rate
2) We use weighted averages (hard coded to 20 seconds) of
log_generation_rate to avoid resonance.
3) From the desired_flush_rate we subtract the number of pages that have
been flushed due to LRU flushing. That gives us pages that we should
flush as part of flush_list cleanup. And that is the number (capped by
maximum io_capacity) that we try to flush from the master thread.
Knobs:
======
innodb_adaptive_flushing: boolean, global, dynamic, default TRUE.
Since this heuristic is very experimental and has the potential to
dramatically change the IO pattern I think it is a good idea to leave a
knob to turn it off.
Approved by: Heikki
The current implementation is to try to flush the neighbors of every
page that we flush. This patch makes the following distinction:
1) If the flush is from flush_list AND
2) If the flush is intended to move the oldest_modification LSN ahead
(this happens when a user thread sees little space in the log file and
attempts to flush pages from the buffer pool so that a checkpoint can
be made)
THEN
Do not try to flush the neighbors. Just focus on flushing dirty pages at
the end of flush_list
Approved by: Heikki
Fix Mantis Issue#244 fix bug in linear read ahead (no check on access pattern)
The changes are:
1) Take into account access pattern when deciding whether or not to do linear
read ahead.
2) Expose a knob innodb_read_ahead_factor = [0-64] default (8), dynamic,
global to control linear read ahead behvior
3) Disable random read ahead. Keep the code for now.
Submitted by: Inaam (rb://122)
Approved by: Heikki (rb://122)
/*********************************
comments to Doxygen /** style like this:
/*****************************//**
This conversion was performed by the following command:
perl -i -e 'while(<ARGV>){if (m|^/\*{30}\**$|) {
s|\*{4}$|//**| if ++$com>1; $_ .= "\@file $ARGV\n" if $com==2}
print; if(eof){$.=0;undef $com}}' */*[ch] include/univ.i
This patch was created by running the following commands:
for i in */*[ch]; do doxygenify.pl $i; done
perl -i -pe 's#\*{3} \*/$#****/#' */*[ch]
where doxygenify.pl is
https://svn.innodb.com/svn/misc/trunk/tools/doxygenify.pl r510
Verified the consistency as follows:
(0) not too many /* in: */ or /* out: */ comments left in the code:
grep -l '/\*\s*\(in\|out\)[,:/]' */*[ch]
(1) no difference when ignoring blank lines, after stripping all
C90-style /* comments */, including multi-line ones, before and after
applying this patch:
perl -i -e 'undef $/;while(<ARGV>){s#/\*(.*?)\*/##gs;print}' */*[ch]
diff -I'^\s*$' --exclude .svn -ru TREE1 TREE2
(2) after stripping @return comments and !<, generated a diff and omitted
the hunks where /* out: */ function return comments were removed:
perl -i -e'undef $/;while(<ARGV>){s#!<##g;s#\n\@return\t.*?\*/# \*/#gs;print}'\
*/*[ch]
svn diff|
perl -e 'undef $/;$_=<>;s#\n-\s*/\* out[:,]([^\n]*?)(\n-[^\n]*?)*\*/##gs;print'
Some unintended changes were left. These will be removed in a
subsequent patch.
SHOW ENGINE INNODB MUTEX shows all mutexes and rw_locks. This can
be overwhelming particularly when the buffer pool is very large
(note that each block in buffer pool has at least one mutex, one
rw_lock and an additional mutex if rw_lock does not use atomics).
With this patch status of following mutexes and rw-locks is not shown:
1) block->mutex
2) block->lock
3) block->lock->mutex (if applicable)
4) All other mutexes and rw-locks for which number of os-waits are zero
Addresses issue# 179 rb://99
Approved by: Marko
* Remove old Innobase copyright lines from C source files
* Add a reference to the GPLv2 license as recommended by the lawyers
at Oracle Legal
[Step 1/28]
To the files touched by the Google patch from c4144 (excluding
include/os0sync.ic because later we removed Google code from that file):
* Remove the Google license
* Remove old Innobase copyright lines
* Add a reference to the Google license and to the GPLv2 license at the top,
as recommended by the lawyers at Oracle Legal.
This patch changes the innodb mutexes and rw_locks implementation.
On supported platforms it uses GCC builtin atomics. These changes
are based on the patch sent by Mark Callaghan of Google under BSD
license. More technical discussion can be found at rb://30
Approved by: Heikki
Both are minor changes:
1) Compiler error introduced in r4072: double ';' at the end.
2) Warning introduced in r3613: \mem\mem0pool.c(481) :
warning C4098: 'mem_area_free' : 'void' function returning a value
Approved by: Sunny (IM)
btr_search_disabled: Rename to btr_search_enabled and change the type
to char, so that it can be directly linked to the MySQL parameters.
Note that the variable is protected by btr_search_latch and
btr_search_enabled_mutex, a new mutex introduced in this patch.
btr_search_enabled_mutex: A new mutex, to protect btr_search_enabled
together with btr_search_latch.
buf_pool_drop_hash_index(): New function, to be called from
btr_search_disable().
btr_search_disable(), btr_search_enable(): Fix bugs. These functions
were previously unused.
btr_search_guess_on_hash(), btr_search_build_page_hash_index():
Check btr_search_enabled once more, while holding btr_search_latch.
btr_cur_search_to_nth_level(): Note that the reads of btr_search_enabled
may be dirty and explain why it should not be a problem.
innobase_adaptive_hash_index: Remove. The variable btr_search_enabled will be used directly instead.
innodb_adaptive_hash_index_update(): New function, an update callback for
innodb_adaptive_hash_index. This will call either btr_search_disable()
or btr_search_enable() when the value is assigned. The functions will
be called even if the value does not appear to be changed, e.g., when
setting from TRUE to TRUE or FALSE to FALSE.
rb://85 approved by Heikki Tuuri. This addresses Issue #163.
within UNIV_DEBUG. The two remaining callers in non-debug builds,
btr_search_guess_on_hash() and btr_search_validate(), were rewritten
to call buf_page_hash_get().
To implement support for a resizeable buffer pool, the function
buf_block_align() had been rewritten to perform a page hash lookup in
the buffer pool. The caller was also made responsible for holding the
buffer pool mutex.
Because the page hash lookup is expensive and it has to be done while
holding the buffer pool mutex, implement buf_block_align() by pointer
arithmetics again, and make btr_search_guess_on_hash() call it. Note
that this will have to be adjusted if the interface to the resizeable
buffer pool is actually implemented.
rb://83 approved by Heikki Tuuri, to address Issue #161.
As a deviation from the approved patch, this patch also makes
btr_search_validate() (invoked by CHECK TABLE) check that
buf_pool->page_hash is consistent with buf_block_align().
buf_LRU_block_free_non_file_page(): ut_ad(block) before dereferencing block.
buf_LRU_block_remove_hashed_page(): Forbid buf_pool_mutex_exit() while
calling buf_buddy_free(). Callers of buf_LRU_block_remove_hashed_page()
assume that the buffer pool mutex will not be released and reacquired.
satisfy some conditions when UNIV_DEBUG is defined.
HASH_SEARCH(): New parameter: ASSERTION. All users will pass an appropriate
ut_ad() or nothing.
dict_table_add_to_columns(): Assert that the table being added to the data
dictionary cache is not already being pointed to by the name_hash and
id_hash tables.
HASH_SEARCH_ALL(): New macro, for use in dict_table_add_to_columns().
dict_mem_table_free(): Set ut_d(table->cached = FALSE), so that we can
check ut_ad(table->cached) when traversing the hash tables, as in
HASH_SEARCH(name_hash, dict_sys->table_hash, ...) and
HASH_SEARCH(id_hash, dict_sys->table_id_hash, ...).
dict_table_get_low(), dict_table_get_on_id_low(): Assert
ut_ad(!table || table->cached).
fil_space_get_by_id(): Check ut_ad(space->magic_n == FIL_SPACE_MAGIC_N)
in HASH_SEARCH(hash, fil_system->spaces, ...).
fil_space_get_by_name(): Check ut_ad(space->magic_n == FIL_SPACE_MAGIC_N)
in HASH_SEARCH(name_hash, fil_system->name_hash, ...).
buf_buddy_block_free(): Check that the blocks are in valid state in
HASH_SEARCH(hash, buf_pool->zip_hash, ...).
buf_page_hash_get(): Check that the blocks are in valid state in
HASH_SEARCH(hash, buf_pool->page_hash, ...).
get_share(), free_share(): Check ut_ad(share->use_count > 0) in
HASH_SEARCH(table_name_hash, innobase_open_tables, ...).
This was posted as rb://75 for tracking down errors similar to Issue #153.
buf_flush_try_page(): Rename to buf_flush_page(), and change the
return type to void. Replace the parameters space, offset with bpage,
and remove the second page hash lookup. Note and assert that both
buf_pool_mutex and block_mutex must now be held upon entering the
function. They will still be released by this function.
buf_flush_try_neighbors(): Replace buf_flush_try_page() with
buf_flush_page(). Make the logic easier to follow by not negating the
precondition of buf_flush_page().
rb://73 approved by Sunny Bains. This is related to Issue #157.
remove dead code.
buf_flush_ready_for_flush(), buf_flush_try_page(): Assert that
flush_type is one of BUF_FLUSH_LRU or BUF_FLUSH_LIST. The flush_type
comes from buf_flush_batch(), which already asserts this. The
assertion holds for all calls in the source code.
buf_flush_try_page(): Remove the dead case BUF_FLUSH_SINGLE_PAGE
of switch (flush_type).
buf_page_struct: Note that space and offset are also protected by
buf_pool_mutex. They are only assigned to by
buf_block_set_file_page(). Thus, it suffices for buf_flush_batch() to
hold just buf_pool_mutex when checking these fields.
buf_flush_try_page(): Rename "locked" to "is_s_latched", per Heikki's request.
buf_flush_batch(): Move the common statement mutex_exit(block_mutex)
from all if-else if-else branches before the if block. Remove the
redundant test (buf_pool->init_flush[flush_type] == FALSE) that was
apparently copied from buf_flush_write_complete().
buf_flush_write_block_low(): Note why it is safe not to hold buf_pool_mutex
or block_mutex. Enumerate the assumptions in debug assertions.
