Commit graph

182 commits

Author SHA1 Message Date
marko
68a1ee9960 branches/zip: Add some Doxygen comments for many structs, typedefs,
#defines and global variables.  Many are still missing.
2009-05-26 12:28:49 +00:00
marko
11ff89d994 branches/zip: Add @file comments, and convert decorative
/*********************************
comments to Doxygen /** style like this:
/*****************************//**

This conversion was performed by the following command:

perl -i -e 'while(<ARGV>){if (m|^/\*{30}\**$|) {
s|\*{4}$|//**| if ++$com>1; $_ .= "\@file $ARGV\n" if $com==2}
print; if(eof){$.=0;undef $com}}' */*[ch] include/univ.i
2009-05-25 09:52:29 +00:00
marko
e49dee377b branches/zip: Convert the function comments to Doxygen format.
This patch was created by running the following commands:

for i in */*[ch]; do doxygenify.pl $i; done
perl -i -pe 's#\*{3} \*/$#****/#' */*[ch]

where doxygenify.pl is
https://svn.innodb.com/svn/misc/trunk/tools/doxygenify.pl r510

Verified the consistency as follows:

(0) not too many /* in: */ or /* out: */ comments left in the code:
grep -l '/\*\s*\(in\|out\)[,:/]' */*[ch]

(1) no difference when ignoring blank lines, after stripping all
C90-style /* comments */, including multi-line ones, before and after
applying this patch:

perl -i -e 'undef $/;while(<ARGV>){s#/\*(.*?)\*/##gs;print}' */*[ch]
diff -I'^\s*$' --exclude .svn -ru TREE1 TREE2

(2) after stripping @return comments and !<, generated a diff and omitted
the hunks where /* out: */ function return comments were removed:

perl -i -e'undef $/;while(<ARGV>){s#!<##g;s#\n\@return\t.*?\*/# \*/#gs;print}'\
 */*[ch]
svn diff|
perl -e 'undef $/;$_=<>;s#\n-\s*/\* out[:,]([^\n]*?)(\n-[^\n]*?)*\*/##gs;print'

Some unintended changes were left.  These will be removed in a
subsequent patch.
2009-05-25 05:30:14 +00:00
marko
79362a389d branches/zip: Remove bogus out: comments of functions returning void. 2009-05-19 07:00:51 +00:00
marko
b10bc48d35 branches/zip: Add some missing out: comments to buf0buf.h, buf0buf.c. 2009-05-18 12:36:10 +00:00
marko
a512de6783 branches/zip: buf_validate(): Add missing out: comment. 2009-05-18 12:29:51 +00:00
marko
a0714b182c branches/zip: univ.i: Define REFMAN as the base URL of the
MySQL Reference Manual and use it in every string.
This fixes Issue #221.
2009-04-16 12:02:27 +00:00
inaam
b76aa20cbc branches/zip
SHOW ENGINE INNODB MUTEX shows all mutexes and rw_locks. This can
be overwhelming particularly when the buffer pool is very large
(note that each block in buffer pool has at least one mutex, one
rw_lock and an additional mutex if rw_lock does not use atomics).
With this patch status of following mutexes and rw-locks is not shown:

1) block->mutex
2) block->lock
3) block->lock->mutex (if applicable)
4) All other mutexes and rw-locks for which number of os-waits are zero

Addresses issue# 179 rb://99

Approved by: Marko
2009-03-25 17:18:33 +00:00
marko
d90bea085a branches/zip: Remove unneeded definitions and dependencies
from UNIV_HOTBACKUP builds.
2009-03-23 14:21:34 +00:00
marko
83e98148b5 branches/zip: buf_page_print(): Clean up the code #ifdef UNIV_HOTBACKUP. 2009-03-23 10:05:47 +00:00
marko
94ca1bb10d branches/zip: recv_recover_page(): Remove compile-time constant parameters. 2009-03-20 12:48:26 +00:00
marko
ee2b2dc739 branches/zip: UT_LIST_VALIDATE(): Add the parameter ASSERTION and
adjust all callers.
2009-03-16 09:43:34 +00:00
vasil
c6232c06fa branches/zip:
To the files touched by the Google patch from c4144 (excluding
include/os0sync.ic because later we removed Google code from that file):

* Remove the Google license
* Remove old Innobase copyright lines
* Add a reference to the Google license and to the GPLv2 license at the top,
as recommended by the lawyers at Oracle Legal.
2009-02-17 07:56:33 +00:00
inaam
a1bb700fd2 branches/zip rb://30
This patch changes the innodb mutexes and rw_locks implementation.
On supported platforms it uses GCC builtin atomics. These changes
are based on the patch sent by Mark Callaghan of Google under BSD
license. More technical discussion can be found at rb://30

Approved by: Heikki
2009-02-09 23:36:25 +00:00
marko
fd6df2ba78 branches/zip: buf_block_align(): Fix a bogus debug assertion
that was introduced in r4036, to address Issue #161.
2009-02-04 14:09:24 +00:00
calvin
f43b923008 branches/zip: fix a compiler error and a warning
Both are minor changes:
1) Compiler error introduced in r4072: double ';' at the end.
2) Warning introduced in r3613: \mem\mem0pool.c(481) : 
warning C4098: 'mem_area_free' : 'void' function returning a value

Approved by: Sunny (IM)
2009-02-03 00:35:56 +00:00
marko
6aab391620 branches/zip: Make innodb_adaptive_hash_index settable.
btr_search_disabled: Rename to btr_search_enabled and change the type
to char, so that it can be directly linked to the MySQL parameters.
Note that the variable is protected by btr_search_latch and
btr_search_enabled_mutex, a new mutex introduced in this patch.

btr_search_enabled_mutex: A new mutex, to protect btr_search_enabled
together with btr_search_latch.

buf_pool_drop_hash_index(): New function, to be called from
btr_search_disable().

btr_search_disable(), btr_search_enable(): Fix bugs.  These functions
were previously unused.

btr_search_guess_on_hash(), btr_search_build_page_hash_index():
Check btr_search_enabled once more, while holding btr_search_latch.

btr_cur_search_to_nth_level(): Note that the reads of btr_search_enabled
may be dirty and explain why it should not be a problem.

innobase_adaptive_hash_index: Remove. The variable btr_search_enabled will be used directly instead.

innodb_adaptive_hash_index_update(): New function, an update callback for
innodb_adaptive_hash_index.  This will call either btr_search_disable()
or btr_search_enable() when the value is assigned.  The functions will
be called even if the value does not appear to be changed, e.g., when
setting from TRUE to TRUE or FALSE to FALSE.

rb://85 approved by Heikki Tuuri.  This addresses Issue #163.
2009-01-30 21:30:29 +00:00
marko
b707b2e3b1 branches/zip: In r988, the function buf_block_align() was enclosed
within UNIV_DEBUG. The two remaining callers in non-debug builds,
btr_search_guess_on_hash() and btr_search_validate(), were rewritten
to call buf_page_hash_get().

To implement support for a resizeable buffer pool, the function
buf_block_align() had been rewritten to perform a page hash lookup in
the buffer pool. The caller was also made responsible for holding the
buffer pool mutex.

Because the page hash lookup is expensive and it has to be done while
holding the buffer pool mutex, implement buf_block_align() by pointer
arithmetics again, and make btr_search_guess_on_hash() call it. Note
that this will have to be adjusted if the interface to the resizeable
buffer pool is actually implemented.

rb://83 approved by Heikki Tuuri, to address Issue #161.

As a deviation from the approved patch, this patch also makes
btr_search_validate() (invoked by CHECK TABLE) check that
buf_pool->page_hash is consistent with buf_block_align().
2009-01-26 20:33:20 +00:00
marko
0dfed5a8f3 branches/zip: buf_page_get_gen(): Fix a "possibly uninitialized" warning
that was introduced in r4030.
2009-01-26 14:16:39 +00:00
marko
5dceabc908 branches/zip: buf_page_get_gen(): Remove the unused mode BUF_GET_NOWAIT.
This was noticed while investigating Issue #160.
2009-01-23 13:31:36 +00:00
marko
bef98e3442 branches/zip: Remove some redundant #include statements. 2009-01-23 09:04:49 +00:00
marko
7683ebd044 branches/zip: Fix some comments to say buf_pool_mutex. 2009-01-13 08:32:24 +00:00
marko
b0ab87862c branches/zip: Introduce UNIV_AHI_DEBUG for debugging the adaptive hash
index without enabling UNIV_DEBUG.
2008-12-17 09:19:56 +00:00
marko
1297a39ffe branches/zip: buf_page_init_for_read(): Use common code for error exit. 2008-12-16 10:13:31 +00:00
marko
d32ffdd1a3 branches/zip: Clean up the insert buffer subsystem.
Originally, there were provisions in InnoDB for multiple insert buffer
B-trees, apparently one for each tablespace.

When Heikki implemented innodb_file_per_table (multiple InnoDB
tablespaces) in MySQL 4.1, he made the insert buffer live only in the
system tablespace (space 0) but left the provisions in the code.

When Osku Salerma implemented delete buffering, he also cleaned up the
insert buffer subsystem so that only one insert buffer B-tree exists.
This patch applies the clean-up to the InnoDB Plugin.

Having a separate patch of the insert buffer clean-up should help us
better compare the essential changes of the InnoDB Plugin and InnoDB+
and to track down bugs that are specific to InnoDB+.

IBUF_SPACE_ID: New constant, defined as 0.

ibuf_data_t: Remove.

ibuf_t: Add the applicable fields from ibuf_data_t.  There is only one
insert buffer tree from now on.

ibuf_page_low(), ibuf_page(): Merge to a single function ibuf_page().

fil_space_t: Remove ibuf_data.

fil_space_get_ibuf_data(): Remove.  There is only one ibuf_data, for
space IBUF_SPACE_ID.

fil_ibuf_init_at_db_start(): Remove.

ibuf_init_at_db_start(): Fuse with ibuf_data_init_for_space().

ibuf_validate_low(): Remove.  There is only one ibuf tree.

ibuf_free_excess_pages(), ibuf_header_page_get(),
ibuf_free_excess_pages(): Remove the parameter space, which was always
0.

ibuf_tree_root_get(): Remove the parameters space and data.  There is
only one ibuf tree, for space IBUF_SPACE_ID.

ibuf_data_sizes_update(): Rename to ibuf_size_update(), and remove the
parameter data.  There is only one ibuf data struct.

ibuf_build_entry_pre_4_1_x(): New function, refactored from
ibuf_build_entry_from_ibuf_rec().

ibuf_data_enough_free_for_insert(), ibuf_data_too_much_free(): Remove
the parameter data.  There is only one insert buffer tree.

ibuf_add_free_page(), ibuf_remove_free_page(): Remove the parameters
space and data.  There is only one insert buffer tree.

ibuf_get_merge_page_nos(): Add parenthesis, to reduce diffs to
branches/innodb+.

ibuf_contract_ext(): Do not pick an insert buffer tree at random.
There is only one.

ibuf_print(): Print the single insert buffer tree.

rb://19 approved by Heikki on IM
2008-12-12 14:08:23 +00:00
marko
0e613f0c92 branches/zip: buf_LRU_free_block(), buf_page_try_get_func(): Assert that
there is nothing in the insert buffer for the page.  This is for
tracking down Issue #128.
2008-11-21 14:24:31 +00:00
marko
d9317eb699 branches/zip: Non-functional change:
buf_block_dbg_add_level(block, level): Define as an empty macro when
UNIV_SYNC_DEBUG is not defined.  Remove #ifdef UNIV_SYNC_DEBUG around
all invocations.
2008-09-22 06:59:58 +00:00
marko
0d8f5237dd branches/zip: Introduce UNIV_LRU_DEBUG for debugging the LRU buffer pool
cache, especially buf_pool->LRU_old and bpage->old.

buf_LRU_old_adjust_len(), buf_LRU_remove_block(): Check that blocks in
buf_pool->LRU_old have the "old" flag set and the blocks preceding
buf_pool->LRU_old have the "old" flag clear.

buf_LRU_add_block_low(), buf_relocate(): Check that buf_pool->LRU_old
is the first block in the LRU list whose "old" flag is set.

buf_LRU_free_block(): When replacing a control block in the LRU list
with a control block for a compressed page, assert that the "old"
flags in the neighboring LRU list entries grow monotonically.

buf_page_set_old(): Assert that the "old" flags in the neighboring LRU
list entries grow monotonically.
2008-08-18 18:28:20 +00:00
marko
9c0d868a8d branches/zip: Correct some typos in comments. 2008-08-06 12:45:35 +00:00
marko
cd124b7524 branches/zip: Always display a newline after ut_print_buf(stderr, ...). 2008-08-06 08:10:21 +00:00
sunny
33c5bfc756 branches/zip: Cover the diagnostic print statement with a suitable lock. It
caused an assertion failure during testing.
2008-06-17 19:15:37 +00:00
vasil
b21c94dc01 branches/zip: Merge 2437:2485 from branches/5.1:
(r2478 was skipped for the obvious reason)

  ------------------------------------------------------------------------
  r2464 | vasil | 2008-05-19 17:59:42 +0300 (Mon, 19 May 2008) | 9 lines
  
  branches/5.1:
  
  Fix Bug#36600 SHOW STATUS takes a lot of CPU in buf_get_latched_pages_number
  
  by removing the Innodb_buffer_pool_pages_latched variable from
  SHOW STATUS output in non-UNIV_DEBUG compilation.
   
  Approved by:	Heikki
  
  ------------------------------------------------------------------------
  r2466 | calvin | 2008-05-20 01:37:14 +0300 (Tue, 20 May 2008) | 12 lines
  
  branches/5.1:
  
  Fix Bug#11894 innodb_file_per_table crashes w/ Windows .sym symbolic
  link hack
  
  The crash was due to un-handled error 3 (path not found). In the case
  of file per table, change the call to os_file_handle_error_no_exit()
  from os_file_handle_error(). Also, checks for full path pattern during
  table create (Windows only), which is used in symbolic link and temp
  table creation.
  
  Approved by:	Heikki
  ------------------------------------------------------------------------
  r2478 | sunny | 2008-05-23 08:29:08 +0300 (Fri, 23 May 2008) | 3 lines
  
  branches/5.1: Fix for bug# 36793. This is a back port from branches/zip. This
  code has been tested on a big-endian machine too.
  
  ------------------------------------------------------------------------
  r2480 | vasil | 2008-05-27 11:40:07 +0300 (Tue, 27 May 2008) | 11 lines
  
  branches/5.1:
  
  Fix Bug#36819 ut_usectime does not handle errors from gettimeofday
  
  by retrying gettimeofday() several times if it fails in ut_usectime().
  If it fails on all calls then return error to the caller to be handled
  at higher level.
  
  Update the variable innodb_row_lock_time_max in SHOW STATUS output only
  if ut_usectime() was successful.
  
  ------------------------------------------------------------------------
  r2482 | sunny | 2008-05-28 12:18:35 +0300 (Wed, 28 May 2008) | 5 lines
  
  branches/5.1: Fix for Bug#35602, "Failed to read auto-increment value from
  storage engine". The test for REPLACE was an error of ommission since it's
  classified as a simple INSERT. For REPLACE statements we don't acquire
  the special AUTOINC lock for AUTOINC_NEW_STYLE_LOCKING with this fix.
  
  ------------------------------------------------------------------------
  r2485 | vasil | 2008-05-28 16:01:14 +0300 (Wed, 28 May 2008) | 9 lines
  
  branches/5.1:
  
  Fix Bug#36149 Read buffer overflow in srv0start.c found during "make test"
  
  Use strncmp(3) instead of memcmp(3) to avoid reading past end of the string
  if it is empty (*str == '\0'). This bug is _not_ a buffer overflow.
  
  Discussed with:	Sunny (via IM)
  
  ------------------------------------------------------------------------
2008-06-06 15:24:58 +00:00
sunny
fd41c6a315 branches/zip: The block can't be in io fix state WRITE and we check for
that in the fix but the earlier check fails because it tries to be too
strict and assumes that if the block is on the clean list then it can
only be in state io fix NONE.
2008-04-12 10:04:28 +00:00
marko
77d8c98735 branches/zip: Rename ib_ulonglong to ib_uint64_t and ib_longlong to ib_int64_t. 2008-03-17 14:19:04 +00:00
marko
9d58e3ff99 branches/zip: Improve the LRU algorithm with a separate unzip_LRU list of
blocks that contains uncompressed and compressed frames.  This patch was
designed by Heikki and Inaam, implemented by Inaam, and refined and reviewed
by Marko and Sunny.

buf_buddy_n_frames, buf_buddy_min_n_frames, buf_buddy_max_n_frames: Remove.

buf_page_belongs_to_unzip_LRU(): New predicate:
bpage->zip.data && buf_page_get_state(bpage) == BUF_BLOCK_FILE_PAGE.

buf_pool_t, buf_block_t: Add the linked list unzip_LRU.  A block in the
regular LRU list is in unzip_LRU iff buf_page_belongs_to_unzip_LRU() holds.

buf_LRU_free_block(): Add a third return value to refine the case
"cannot free the block".

buf_LRU_search_and_free_block(): Update the documentation to reflect the
implementation.

buf_LRU_stat_t, buf_LRU_stat_cur, buf_LRU_stat_sum, buf_LRU_stat_arr[]:
Statistics for the unzip_LRU algorithm.

buf_LRU_stat_update(): New function: Update the statistics.  Called once
per second by srv_error_monitor_thread().

buf_LRU_validate(): Validate the unzip_LRU list as well.

buf_LRU_evict_from_unzip_LRU(): New predicate: Use the unzip_LRU before
falling back to the regular LRU?

buf_LRU_free_from_unzip_LRU_list(), buf_LRU_free_from_common_LRU_list():
Subfunctions of buf_LRU_search_and_free_block().

buf_LRU_search_and_free_block(): Reimplement.  Try to evict an uncompressed
page from the unzip_LRU list before falling back to evicting an entire block
from the common LRU list.

buf_unzip_LRU_remove_block_if_needed(): New function.

buf_unzip_LRU_add_block(): New function: Add a block to the unzip_LRU list.
2008-03-03 12:48:38 +00:00
marko
f7595d4278 branches/zip: buf_relocate(): Document and assert that bpage must be
in one of the states BUF_BLOCK_ZIP_DIRTY or BUF_BLOCK_ZIP_PAGE.
Note that the caller will have to relocate bpage->list.
2008-02-29 08:20:17 +00:00
marko
2c2b06ad75 branches/zip: Introduce UNIV_INTERN, a linkage specifier for InnoDB-global
symbols.  Use it for all definitions of non-static variables and functions.

lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static.  It is
referenced from pars0grm.c.

Actually, according to
	nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:

* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs

The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.

The class ha_innodb is renamed from class ha_innobase by a #define.  Thus,
there will be no clash with the builtin InnoDB.  However, there will be some
overhead for invoking virtual methods of class ha_innodb.  Ideas for making
the vtable hidden are welcome.  -fvisibility=hidden is not available in GCC 3.
2008-02-06 14:17:36 +00:00
marko
473d7c16a9 branches/zip: Introduce the page type code FIL_PAGE_TYPE_ZBLOB2 for
continuation pages containing compressed BLOBs.  The first compressed
BLOB page will be of type FIL_PAGE_TYPE_ZBLOB.
2008-01-24 08:12:02 +00:00
marko
66791e5a1b branches/zip: Free the buffer pool at shutdown.
buf_pool_free(): New function: Free all chunks of the buffer pool.

innobase_shutdown_for_mysql(): Call buf_pool_free() right before
ut_free_all_mem().
2008-01-16 12:44:44 +00:00
marko
7d18bba5e7 branches/zip: Add comments about the lock and latch protection of externally
stored columns (BLOBs).

btr_copy_blob_prefix(), btr_copy_zblob_prefix(),
btr_copy_externally_stored_field_prefix_low(),
btr_copy_externally_stored_field_prefix(),
btr_copy_externally_stored_field(),
btr_rec_copy_externally_stored_field():
Note that the page containing the clustered index record that points to
the BLOB must be latched.

btr_copy_zblob_prefix(): Note that there is no latch on the page, and thus
all accesses to a given page via this function must be covered by the same
set of locks or latches.

btr_copy_zblob_prefix(): Note that the block acquired by
buf_page_get_zip() is protected by an exclusive table lock or
or by a latch on the clustered index record.
2008-01-16 10:45:14 +00:00
marko
5c95ec4035 branches/zip: buf_pool_init(): Initialize buf_pool_zip_mutex before
acquiring buf_pool_mutex.  This avoids triggering the debug assertion
that was added in r2227.
2008-01-15 09:43:14 +00:00
marko
523a8a0e8d branches/zip: Add instrumentation for prohibiting the release of
the buffer pool mutex.  The instrumentation can be activated by
defining UNIV_DEBUG or UNIV_BUF_DEBUG at compilation time.

buf_pool_mutex_exit_forbidden: New variable.  When this is nonzero,
an assertion will fail in buf_pool_mutex_exit().

buf_pool_mutex_exit_forbid(): Macro for declaring that the buffer pool
mutex must not be released.  Calls may be nested.

buf_pool_mutex_exit_allow(): Macro for declaring that the buffer pool
mutex may be released.  Calls may be nested.

buf_LRU_search_and_free_block(): Prohibit buf_pool_mutex_exit() in the
scope of the function.

buf_LRU_free_block(): Prohibit buf_pool_mutex_exit() in buf_buddy_alloc()
and buf_buddy_free().

buf_LRU_block_remove_hashed_page(): Prohibit buf_pool_mutex_exit()
in buf_buddy_free().
2008-01-10 09:51:57 +00:00
marko
3bb405841f branches/zip: Implement wrappers for all operations on the buffer pool mutex.
buf_pool->mutex: Rename to buf_pool_mutex, so that the wrappers will have
to be used when changes are merged from other source trees.

buf_pool->zip_mutex: Rename to buf_pool_zip_mutex.

buf_pool_mutex_own(), buf_pool_mutex_enter(), buf_pool_mutex_exit():
Wrappers for buf_pool_mutex.
2008-01-10 09:37:13 +00:00
vasil
815bb2ccf4 branches/zip:
Non-functional change: add "out:" comment for the return value.
2007-12-17 10:03:15 +00:00
marko
a563fbc3d6 branches/zip: Clarify that buf_buddy_alloc() should only be used for
allocating compressed page frames or their control blocks.  Also note
that if buf_buddy_alloc() is used for allocating a control block,
it must be initialized before releasing buf_pool->mutex.

buf_page_init_for_read(): When the page hash check fails after
buf_buddy_alloc(), free the uninitialized control block before freeing
the compressed page frame.  This fixes a potential error in
buf_buddy_relocate_block().
2007-12-12 14:12:52 +00:00
marko
902615e374 branches/zip: Document how the data structures of the buddy allocator
are interfaced with the buffer pool.
2007-12-12 13:42:03 +00:00
marko
40dce922cc branches/zip: buf_page_get_gen(): Check the return status of
buf_zip_decompress() and return NULL on decompression failure.
2007-12-10 12:54:53 +00:00
marko
6f306fad26 branches/zip: buf_buddy_alloc(): Assign *lru = TRUE whenever the buffer pool
mutex is temporarily released.

buf_LRU_free_block(), buf_buddy_alloc_clean(): Add an output parameter that
will be assigned TRUE when the buffer pool mutex is released.

This bug was spotted by and fix provided by Sunny.
2007-12-10 09:48:28 +00:00
marko
7a1d5d91b7 branches/zip: Fix a bug that was introduced in r2123.
buf_block_is_uncompressed(): Check that the pointer is aligned.  Use the
C modulus operator % instead of ut_align_offset(), because sizeof(buf_block_t)
is not guaranteed to be a power of 2.
2007-11-28 13:45:22 +00:00
marko
f6a5cc1d47 branches/zip: buf_page_get_gen(): Note that the guessed block may also
point to a buffer pool chunk that has been released when resizing the
buffer pool.

buf_block_is_uncompressed(): Check that the pointer is aligned.  Thanks
to this check, it is safe to pass an arbitrary pointer as a guess
to buf_page_get_gen().
2007-11-28 11:31:12 +00:00