This patch introduces heuristics based flushing rate of dirty pages to
avoid IO bursts at checkpoint.
1) log_capacity / log_generated per second gives us number of seconds
in which ALL dirty pages need to be flushed. Based on this rough
assumption we can say that
n_dirty_pages / (log_capacity / log_generation_rate) = desired_flush_rate
2) We use weighted averages (hard coded to 20 seconds) of
log_generation_rate to avoid resonance.
3) From the desired_flush_rate we subtract the number of pages that have
been flushed due to LRU flushing. That gives us pages that we should
flush as part of flush_list cleanup. And that is the number (capped by
maximum io_capacity) that we try to flush from the master thread.
Knobs:
======
innodb_adaptive_flushing: boolean, global, dynamic, default TRUE.
Since this heuristic is very experimental and has the potential to
dramatically change the IO pattern I think it is a good idea to leave a
knob to turn it off.
Approved by: Heikki
/*********************************
comments to Doxygen /** style like this:
/*****************************//**
This conversion was performed by the following command:
perl -i -e 'while(<ARGV>){if (m|^/\*{30}\**$|) {
s|\*{4}$|//**| if ++$com>1; $_ .= "\@file $ARGV\n" if $com==2}
print; if(eof){$.=0;undef $com}}' */*[ch] include/univ.i
This patch was created by running the following commands:
for i in */*[ch]; do doxygenify.pl $i; done
perl -i -pe 's#\*{3} \*/$#****/#' */*[ch]
where doxygenify.pl is
https://svn.innodb.com/svn/misc/trunk/tools/doxygenify.pl r510
Verified the consistency as follows:
(0) not too many /* in: */ or /* out: */ comments left in the code:
grep -l '/\*\s*\(in\|out\)[,:/]' */*[ch]
(1) no difference when ignoring blank lines, after stripping all
C90-style /* comments */, including multi-line ones, before and after
applying this patch:
perl -i -e 'undef $/;while(<ARGV>){s#/\*(.*?)\*/##gs;print}' */*[ch]
diff -I'^\s*$' --exclude .svn -ru TREE1 TREE2
(2) after stripping @return comments and !<, generated a diff and omitted
the hunks where /* out: */ function return comments were removed:
perl -i -e'undef $/;while(<ARGV>){s#!<##g;s#\n\@return\t.*?\*/# \*/#gs;print}'\
*/*[ch]
svn diff|
perl -e 'undef $/;$_=<>;s#\n-\s*/\* out[:,]([^\n]*?)(\n-[^\n]*?)*\*/##gs;print'
Some unintended changes were left. These will be removed in a
subsequent patch.
* Remove old Innobase copyright lines from C source files
* Add a reference to the GPLv2 license as recommended by the lawyers
at Oracle Legal
[Step 5/28]
buf_flush_insert_sorted_into_flush_list(): Remove unused code.
Change the parameter to buf_block_t* block and assert that
block->state == BUF_BLOCK_FILE_PAGE. This is part of Issue #155.
in *.h files, so that the function signatures in the *.h and *.c files fully
match each other.
ut_dulint_sort(): Add a UNIV_INTERN qualifier also to the function definition.
page checksum of compressed-only blocks. Pass the compressed page frame
to fil_io() when needed.
page_zip_calc_checksum(): Skip also FIL_PAGE_LSN and FIL_PAGE_FILE_FLUSH_LSN.
buf_flush_init_for_writing(): Expect page to be non-NULL.
and to be flushed to disk.
buf_LRU_free_block(): Enable the freeing of uncompressed pages of
compressed tablespaces.
trx_doublewrite->buf_block_arr[]: Change the type from buf_block_t*
to buf_page_t*.
buf_flush_ready_for_flush(): Add debug assertion.
buf_flush_buffered_writes(), buf_flush_try_page(): Support blocks of type
BUF_BLOCK_ZIP_DIRTY.
buf_flush_post_to_doublewrite_buf(), buf_flush_write_block_low():
Change the type of the parameter from buf_block_t* to buf_page_t*.
buf_flush_init_for_writing(): Allow page to be NULL if page_zip_ is non-NULL.
to BUF_BLOCK_ZIP_DIRTY (discarding the uncompressed page corresponding
to a modified compressed page that has not been flushed to disk).
buf_page_can_relocate(): New function, modelled after
buf_flush_ready_for_replace().
buf_LRU_free_block(): Allow the transition to BUF_BLOCK_ZIP_DIRTY.
buf_flush_insert_into_flush_list(): Make the prototype public.
buf_buddy_relocate(): Remove an unnecessary switch statement.
Use buf_page_can_relocate() instead of buf_flush_ready_for_replace().
buf_page_peek(): Made UNIV_INLINE.
Document UNIV_ZIP_DEBUG.
buf_flush_remove(): New function, for removing a block from the flush list.
Sliced from buf_flush_write_complete().
buf_page_set_state(): Allow transitions between BUF_BLOCK_ZIP_PAGE
and BUF_BLOCK_ZIP_DIRTY.
(state == BUF_BLOCK_ZIP_PAGE). Make use of buf_page_in_file()
and buf_page_get_mutex().
buf_block_get_newest_modification(): Rename to
buf_page_get_newest_modification().
mysql.patch: Patch to change or add variables to MySQL
innodb.patch: Patch to make the master thread poll requests to resize
the buffer pool.
Replace srv_pool_size and innobase_buffer_pool_size
with srv_buf_pool_size, srv_buf_pool_old_size,
and srv_buf_pool_curr_size.
Add buf_chunk_t, a collection of buf_block_t.
buf_flush_init_for_writing(): Remove parameters space, page_no.
fsp_init_file_page_low(): Wriet space_id and page_no to the page.
fil_create_new_single_table_tablespace(): Write space_id to the page.
btr_create(): page_zip_compress() returns FALSE on failure.
page_zip_write_header(): Write to page_zip->data[] instead of page_zip[].
buf_flush_init_for_writing(): Add parameter page_zip and set the fields
also in the header of the compressed page.
btr_cur_search_to_nth_level(): Add ut_ad() on page_zip_validate().
