Change the read ahead parameter name to innodb_read_ahead_threshold.
Change the meaning of this parameter to signify the number of pages
that must be sequentially accessed for InnoDB to trigger a readahead
request.
Suggested by: Ken
This patch introduces heuristics based flushing rate of dirty pages to
avoid IO bursts at checkpoint.
1) log_capacity / log_generated per second gives us number of seconds
in which ALL dirty pages need to be flushed. Based on this rough
assumption we can say that
n_dirty_pages / (log_capacity / log_generation_rate) = desired_flush_rate
2) We use weighted averages (hard coded to 20 seconds) of
log_generation_rate to avoid resonance.
3) From the desired_flush_rate we subtract the number of pages that have
been flushed due to LRU flushing. That gives us pages that we should
flush as part of flush_list cleanup. And that is the number (capped by
maximum io_capacity) that we try to flush from the master thread.
Knobs:
======
innodb_adaptive_flushing: boolean, global, dynamic, default TRUE.
Since this heuristic is very experimental and has the potential to
dramatically change the IO pattern I think it is a good idea to leave a
knob to turn it off.
Approved by: Heikki
Based on contribution from Google Inc.
This patch introduces a new parameter innodb_io_capacity to control the
rate at which master threads performs various tasks. The default value
is 200 and higher values imply more aggressive flushing and ibuf merges
from within the master thread.
This patch also changes the ibuf merge from synchronous to asynchronous.
Another minor change is not to force the master thread to wait for a
log flush to complete every second.
Approved by: Heikki
This patch changes the following defaults:
max_dirty_pages_pct: default from 90 to 75. max allowed from 100 to 99
additional_mem_pool_size: default from 1 to 8 MB
buffer_pool_size: default from 8 to 128 MB
log_buffer_size: default from 1 to 8 MB
read_io_threads/write_io_threads: default from 1 to 4
The log file sizes are untouched because of upgrade issues
Reviewed by: Heikki
Enable functionality to have multiple background io helper threads.
This patch is based on percona contributions.
More details about this patch will be written at:
https://svn.innodb.com/innobase/MultipleBackgroundThreads
The patch essentially does the following:
expose following knobs:
innodb_read_io_threads = [1 - 64] default 1
innodb_write_io_threads = [1 - 64] default 1
deprecate innodb_file_io_threads (this parameter was relevant only on windows)
Internally it allows multiple segments for read and write IO request arrays
where one thread works on one segement.
Submitted by: Inaam (rb://124)
Approved by: Heikki (rb://124)
Fix Mantis Issue#244 fix bug in linear read ahead (no check on access pattern)
The changes are:
1) Take into account access pattern when deciding whether or not to do linear
read ahead.
2) Expose a knob innodb_read_ahead_factor = [0-64] default (8), dynamic,
global to control linear read ahead behvior
3) Disable random read ahead. Keep the code for now.
Submitted by: Inaam (rb://122)
Approved by: Heikki (rb://122)
/*********************************
comments to Doxygen /** style like this:
/*****************************//**
This conversion was performed by the following command:
perl -i -e 'while(<ARGV>){if (m|^/\*{30}\**$|) {
s|\*{4}$|//**| if ++$com>1; $_ .= "\@file $ARGV\n" if $com==2}
print; if(eof){$.=0;undef $com}}' */*[ch] include/univ.i
This patch was created by running the following commands:
for i in */*[ch]; do doxygenify.pl $i; done
perl -i -pe 's#\*{3} \*/$#****/#' */*[ch]
where doxygenify.pl is
https://svn.innodb.com/svn/misc/trunk/tools/doxygenify.pl r510
Verified the consistency as follows:
(0) not too many /* in: */ or /* out: */ comments left in the code:
grep -l '/\*\s*\(in\|out\)[,:/]' */*[ch]
(1) no difference when ignoring blank lines, after stripping all
C90-style /* comments */, including multi-line ones, before and after
applying this patch:
perl -i -e 'undef $/;while(<ARGV>){s#/\*(.*?)\*/##gs;print}' */*[ch]
diff -I'^\s*$' --exclude .svn -ru TREE1 TREE2
(2) after stripping @return comments and !<, generated a diff and omitted
the hunks where /* out: */ function return comments were removed:
perl -i -e'undef $/;while(<ARGV>){s#!<##g;s#\n\@return\t.*?\*/# \*/#gs;print}'\
*/*[ch]
svn diff|
perl -e 'undef $/;$_=<>;s#\n-\s*/\* out[:,]([^\n]*?)(\n-[^\n]*?)*\*/##gs;print'
Some unintended changes were left. These will be removed in a
subsequent patch.
------------------------------------------------------------------------
r4994 | marko | 2009-05-14 15:04:55 +0300 (Thu, 14 May 2009) | 18 lines
branches/5.1: Prevent a race condition in innobase_commit() by ensuring
that innodb_commit_concurrency>0 remains constant at run time. (Bug #42101)
srv_commit_concurrency: Make this a static variable in ha_innodb.cc.
innobase_commit_concurrency_validate(): Check that innodb_commit_concurrency
is not changed from or to 0 at run time. This is needed, because
innobase_commit() assumes that innodb_commit_concurrency>0 remains constant.
Without this limitation, the checks for innodb_commit_concurrency>0
in innobase_commit() should be removed and that function would have to
acquire and release commit_cond_m at least twice per invocation.
Normally, innodb_commit_concurrency=0, and introducing the mutex operations
would mean significant overhead.
innodb_bug42101.test, innodb_bug42101-nonzero.test: Test cases.
rb://123 approved by Heikki Tuuri
------------------------------------------------------------------------
access
Mapping the atomic operations to Windows Interlocked functions:
os_compare_and_swap_* to InterlockedCompareExchange(64)
os_atomic_increment_* to InterlockedExchangeAdd(64)
os_atomic_test_and_set_byte to InterlockedExchange
In this patch, the legacy code under UNIV_CAN_USE_X86_ASSEMBLER is
removed all together, and add HAVE_WINDOWS_ATOMICS and
INNODB_RW_LOCKS_USE_ATOMICS to CMakeLists.txt
This is to address mantis issue#194.
rb://113
Approved by: Marko
Enable atomics on solaris (using the libc functions as defined in
atomic.h) if GCC atomic builtins are not present.
There still remains some work to be done (by Vasil?). This patch
makes changes to plug.in to check pthread_t size and presence of
atomic functions when running on solaris. The same has to become
a part of the generated Makefile.in when we bake our source.
Reviewed by: Heikki rb://106
As suggested by Ken, print a message that says that the Google SMP patch
(GCC atomics) is disabled if it is. Also extend the message when the patch
is partially enabled to make it clear that it is partially enabled.
again, to finally put Issue #181 to rest.
Revert some parts of r4274. It is best not to call ut_malloc() before
srv_general_init().
mem_init(): Do not call ut_mem_init().
srv_general_init(): Initialize the memory subsystem in two phases:
first ut_mem_init(), then mem_init(). This is because os_sync_init()
and sync_init() depend on ut_mem_init() and mem_init() depends on
os_sync_init() or sync_init().
srv_parse_data_file_paths_and_sizes(),
srv_parse_log_group_home_dirs(): Remove the output parameters. Assign
to the global variables directly. Allocate memory with malloc()
instead of ut_malloc(), because these functions will be called before
srv_general_init().
srv_free_paths_and_sizes(): New function, for cleaning up after
srv_parse_data_file_paths_and_sizes() and
srv_parse_log_group_home_dirs().
rb://92 approved by Sunny Bains
outside Valgrind, with innodb_use_sys_malloc set to 0 and 1.
mem_init(): Invoke ut_mem_init() before mem_pool_create(), because
the latter one will invoke ut_malloc().
srv_general_init(): Do not initialize the memory subsystem (mem_init()).
innobase_init(): Initialize the memory subsystem (mem_init()) before
calling srv_parse_data_file_paths_and_sizes(), which needs ut_malloc().
Call ut_free_all_mem() in error handling to clean up after the mem_init().
* Remove old Innobase copyright lines from C source files
* Add a reference to the GPLv2 license as recommended by the lawyers
at Oracle Legal
[Step 26/28]
To the files touched by the Google patch from c4144 (excluding
include/os0sync.ic because later we removed Google code from that file):
* Remove the Google license
* Remove old Innobase copyright lines
* Add a reference to the Google license and to the GPLv2 license at the top,
as recommended by the lawyers at Oracle Legal.
This patch changes the innodb mutexes and rw_locks implementation.
On supported platforms it uses GCC builtin atomics. These changes
are based on the patch sent by Mark Callaghan of Google under BSD
license. More technical discussion can be found at rb://30
Approved by: Heikki
innodb_buffer_pool_size >= 4096M on x64 Windows
All three srv_buf_pool related variables are defined as ulong, which is
32-bit on 64-bit Windows. They are changed to 64-bit ulint. Also
system_info.dwPageSize appears to be 32-bit only. Casting to 64-bit
is required.
Approved by: Marko (on IM)
Merge 2929:3458 from branches/5.1 (resolving conflict in c3257,
note also that r3363 reverted r2933 so there are not changes in
mysql-test/innodb-autoinc.result with the current merge):
------------------------------------------------------------------------
r2933 | calvin | 2008-10-30 02:57:31 +0200 (Thu, 30 Oct 2008) | 10 lines
Changed paths:
M /branches/5.1/mysql-test/innodb-autoinc.result
branches/5.1: correct the result file innodb-autoinc.result
Change the followings:
auto_increment_increment
auto_increment_offset
to
auto-increment-increment
auto-increment-offset
------------------------------------------------------------------------
r2981 | marko | 2008-11-07 14:54:10 +0200 (Fri, 07 Nov 2008) | 5 lines
Changed paths:
M /branches/5.1/row/row0mysql.c
branches/5.0: row_mysql_store_col_in_innobase_format(): Correct a misleading
comment. In the UTF-8 encoding, ASCII takes 1 byte per character, while
the "latin1" character set (normally ISO-8859-1, but in MySQL it actually
refers to the Windows Code Page 1252 a.k.a. CP1252, WinLatin1)
takes 1 to 3 bytes (1 to 2 bytes for the ISO-8859-1 subset).
------------------------------------------------------------------------
r3114 | calvin | 2008-11-14 20:31:48 +0200 (Fri, 14 Nov 2008) | 8 lines
Changed paths:
M /branches/5.1/handler/ha_innodb.cc
branches/5.1: fix bug#40386: Not flushing query cache after truncate
ha_statistics.records can not be 0 unless the table is empty, set to
1 instead. The original problem of bug#29507 is fixed in the server.
Additional test was done with the fix of bug#29507 in the server.
Approved by: Heikki (on IM)
------------------------------------------------------------------------
r3257 | inaam | 2008-11-24 22:06:50 +0200 (Mon, 24 Nov 2008) | 13 lines
Changed paths:
M /branches/5.1/handler/ha_innodb.cc
M /branches/5.1/srv/srv0srv.c
M /branches/5.1/trx/trx0trx.c
branches/5.1 bug#40760
The config param innodb_thread_concurrency is dynamically set and is
read when a thread enters/exits innodb. If the value is changed between
the enter and exit time the behaviour becomes erratic.
The fix is not to use srv_thread_concurrency when exiting, instead use
the flag trx->declared_to_be_inside_innodb.
rb://57
Approved by: Marko
------------------------------------------------------------------------
r3363 | calvin | 2008-12-04 19:00:20 +0200 (Thu, 04 Dec 2008) | 13 lines
Changed paths:
M /branches/5.1/mysql-test/innodb-autoinc.result
branches/5.1: revert the changes in r2933
The changes in r2933 causes test failure on Linux.
More investigation is needed for Windows.
Change the followings in innodb-autoinc.result:
auto-increment-increment
auto-increment-offset
back to:
auto_increment_increment
auto_increment_offset
------------------------------------------------------------------------
r3412 | vasil | 2008-12-05 10:46:18 +0200 (Fri, 05 Dec 2008) | 7 lines
Changed paths:
M /branches/5.1/trx/trx0undo.c
branches/5.1:
Add the traditional 2 spaces after the timestamp so the message does
not look like:
070223 13:26:01InnoDB: Warning: canno....
------------------------------------------------------------------------
r3458 | vasil | 2008-12-09 11:21:08 +0200 (Tue, 09 Dec 2008) | 51 lines
Changed paths:
M /branches/5.1/mysql-test/innodb_bug34300.test
branches/5.1:
Merge a change from MySQL to fix the failing innodb_bug34300 mysql-test:
main.innodb_bug34300 [ fail ]
mysqltest: At line 11: query 'SET @@max_allowed_packet=16777216' failed: 1621: SESSION variable 'max_allowed_packet' is read-only. Use SET GLOBAL to assign the value
Aborting: main.innodb_bug34300 failed in default mode.
The changeset is this:
------------------------------------------------------------
revno: 2709.1.10
committer: Staale Smedseng <staale.smedseng@sun.com>
branch nick: b22891-51-bugteam
timestamp: Thu 2008-11-20 08:51:48 +0100
message:
A fix for Bug#22891 "session level max_allowed_packet can be
set but is ignored".
This patch makes @@session.max_allowed_packed and
@@session.net_buffer_length read-only as suggested in the bug
report. The user will have to use SET GLOBAL (and reconnect)
to alter the session values of these variables.
The error string ER_VARIABLE_IS_READONLY is introduced.
Tests are modified accordingly.
modified:
mysql-test/r/func_compress.result
mysql-test/r/max_allowed_packet_basic.result
mysql-test/r/max_allowed_packet_func.result
mysql-test/r/net_buffer_length_basic.result
mysql-test/r/packet.result
mysql-test/r/union.result
mysql-test/r/variables.result
mysql-test/t/func_compress.test
mysql-test/t/innodb_bug34300.test
mysql-test/t/max_allowed_packet_basic.test
mysql-test/t/max_allowed_packet_func.test
mysql-test/t/net_buffer_length_basic.test
mysql-test/t/packet.test
mysql-test/t/union.test
mysql-test/t/variables.test
sql/set_var.cc
sql/set_var.h
sql/share/errmsg.txt
------------------------------------------------------------
------------------------------------------------------------------------
except r2924, which was merged from branches/zip r2866 to branches/5.1
and except r2879 which was merged separately by Calvin:
------------------------------------------------------------------------
r2902 | vasil | 2008-10-28 12:10:25 +0200 (Tue, 28 Oct 2008) | 10 lines
branches/5.1:
Fix Bug#38189 innodb_stats_on_metadata missing
Make the variable innodb_stats_on_metadata visible to the users and
also settable at runtime. Previously it was only "visible" as a command
line startup option to mysqld.
Approved by: Marko (https://svn.innodb.com/rb/r/36)
------------------------------------------------------------------------
r2929 | marko | 2008-10-29 21:26:14 +0200 (Wed, 29 Oct 2008) | 13 lines
branches/5.1: dtype_get_sql_null_size(): return the correct storage
size of a SQL NULL column. (Bug #40369)
When MySQL Bug #20877 was fixed in r834, this function was
accidentally modified to return 0 or 1. Apparently, the only impact of
this bug is that fixed-length columns cannot be updated in-place from
or to SQL NULL, even in ROW_FORMAT=REDUNDANT. After this fix,
fixed-length columns in ROW_FORMAT=REDUNDANT will have a constant
storage size as they should, no matter if NULL or non-NULL. The bug
caused fixed-length NULL columns to occupy 1 byte.
rb://37 approved by Heikki over IM.
------------------------------------------------------------------------
(Bug #36285, rb://9).
innodb-index.test, innodb-index.result: Set innodb_lock_wait_timeout as
a session variable instead of relying on the global value.
innodb-index-master.opt: Remove.
innodb-timeout.test: Test that setting the innodb_lock_wait_timeout
works as advertised.
thd_lock_wait_timeout(): New function, to retrieve the lock wait timeout
for a given MySQL client connection (thd), or the global value (thd==NULL).
srv_lock_wait_timeout, innobase_lock_wait_timeout: Remove.
Replace MYSQL_SYSVAR_LONG(lock_wait_timeout)
with MYSQL_THDVAR_ULONG(lock_wait_timeout).
