Commit graph

231 commits

Author SHA1 Message Date
marko
e8574ccb03 branches/zip: buf_LRU_search_and_free_block(): Do not forbid the release
of the buffer pool mutex.  Apparently, it is temporarily released also
in older versions of MySQL/InnoDB for the duration of the
btr_search_drop_page_hash_index() call [in buf_LRU_free_block()].
2008-01-10 12:34:25 +00:00
marko
523a8a0e8d branches/zip: Add instrumentation for prohibiting the release of
the buffer pool mutex.  The instrumentation can be activated by
defining UNIV_DEBUG or UNIV_BUF_DEBUG at compilation time.

buf_pool_mutex_exit_forbidden: New variable.  When this is nonzero,
an assertion will fail in buf_pool_mutex_exit().

buf_pool_mutex_exit_forbid(): Macro for declaring that the buffer pool
mutex must not be released.  Calls may be nested.

buf_pool_mutex_exit_allow(): Macro for declaring that the buffer pool
mutex may be released.  Calls may be nested.

buf_LRU_search_and_free_block(): Prohibit buf_pool_mutex_exit() in the
scope of the function.

buf_LRU_free_block(): Prohibit buf_pool_mutex_exit() in buf_buddy_alloc()
and buf_buddy_free().

buf_LRU_block_remove_hashed_page(): Prohibit buf_pool_mutex_exit()
in buf_buddy_free().
2008-01-10 09:51:57 +00:00
marko
3bb405841f branches/zip: Implement wrappers for all operations on the buffer pool mutex.
buf_pool->mutex: Rename to buf_pool_mutex, so that the wrappers will have
to be used when changes are merged from other source trees.

buf_pool->zip_mutex: Rename to buf_pool_zip_mutex.

buf_pool_mutex_own(), buf_pool_mutex_enter(), buf_pool_mutex_exit():
Wrappers for buf_pool_mutex.
2008-01-10 09:37:13 +00:00
marko
051ee49116 branches/zip: Split the source-only configuration parameter
buf_buddy_min_n_frames into two, also buf_buddy_max_n_frames.
Set the default values in such a way that a strict LRU policy will apply
for replacing compressed or uncompressed page frames in the buffer pool.

These parameters have not yet been exposed to the MySQL layer.
2008-01-04 14:08:41 +00:00
vasil
815bb2ccf4 branches/zip:
Non-functional change: add "out:" comment for the return value.
2007-12-17 10:03:15 +00:00
marko
a563fbc3d6 branches/zip: Clarify that buf_buddy_alloc() should only be used for
allocating compressed page frames or their control blocks.  Also note
that if buf_buddy_alloc() is used for allocating a control block,
it must be initialized before releasing buf_pool->mutex.

buf_page_init_for_read(): When the page hash check fails after
buf_buddy_alloc(), free the uninitialized control block before freeing
the compressed page frame.  This fixes a potential error in
buf_buddy_relocate_block().
2007-12-12 14:12:52 +00:00
marko
902615e374 branches/zip: Document how the data structures of the buddy allocator
are interfaced with the buffer pool.
2007-12-12 13:42:03 +00:00
marko
40dce922cc branches/zip: buf_page_get_gen(): Check the return status of
buf_zip_decompress() and return NULL on decompression failure.
2007-12-10 12:54:53 +00:00
marko
6f306fad26 branches/zip: buf_buddy_alloc(): Assign *lru = TRUE whenever the buffer pool
mutex is temporarily released.

buf_LRU_free_block(), buf_buddy_alloc_clean(): Add an output parameter that
will be assigned TRUE when the buffer pool mutex is released.

This bug was spotted by and fix provided by Sunny.
2007-12-10 09:48:28 +00:00
marko
7a1d5d91b7 branches/zip: Fix a bug that was introduced in r2123.
buf_block_is_uncompressed(): Check that the pointer is aligned.  Use the
C modulus operator % instead of ut_align_offset(), because sizeof(buf_block_t)
is not guaranteed to be a power of 2.
2007-11-28 13:45:22 +00:00
marko
f6a5cc1d47 branches/zip: buf_page_get_gen(): Note that the guessed block may also
point to a buffer pool chunk that has been released when resizing the
buffer pool.

buf_block_is_uncompressed(): Check that the pointer is aligned.  Thanks
to this check, it is safe to pass an arbitrary pointer as a guess
to buf_page_get_gen().
2007-11-28 11:31:12 +00:00
marko
7aedcf308e branches/zip: buf_page_get_gen(): Ignore "guess" if it does not point to
buf_pool->chunks[]->blocks[].

buf_block_is_uncompressed(): New function, to detect if a block points to
buf_pool->chunks[]->blocks[].
2007-11-05 10:31:41 +00:00
marko
bcd672cae7 branches/zip: buf_page_init_low(): Initialize also flush_type. 2007-10-31 22:02:23 +00:00
marko
e00f417abb branches/zip: buf_LRU_block_remove_hashed_page(): Add some Valgrind
instrumentation.
2007-10-31 10:40:41 +00:00
marko
d321ea1141 branches/zip: buf_buddy_relocate(): The src block may be partially freed.
Remove the Valgrind check about it.
2007-10-31 10:40:09 +00:00
marko
2d77ed4f40 branches/zip: Improve Valgrind instrumentation.
buf_buddy_relocate(): Allow the source block to contain uninitialized data.

buf0buddy.c: Replace the remaining VALGRIND_CHECK_ macros with the wrappers
defined in univ.i.
2007-10-31 09:00:08 +00:00
marko
7319444c08 branches/zip: Improve Valgrind instrumentation.
buf_LRU_free_block(): Check that the block descriptor contains valid data.

buf_buddy_relocate(): Check that the source block contains valid data.

buf_page_get_gen(): Do not dereference bpage after calling buf_relocate().
This avoids a bogus Valgrind warning; the memory itself was valid.

buf_page_hash_get(): Check that the returned block descriptor contains
valid data.
2007-10-30 09:27:09 +00:00
marko
ada2f71ab7 branches/zip: Reduce WAIT_FOR_READ from 20 to 5 milliseconds, to correspond
to the time in which modern disks can serve a random disk read.
2007-10-26 12:34:53 +00:00
marko
a4b26c4ffb branches/zip: buf_pool_t: Add n_pend_unzip. Display it in buf_print(). 2007-10-26 08:53:36 +00:00
marko
1890ec59c7 branches/zip: buf_page_get_gen(): Improve the comments about
wait_until_unfixed.
2007-10-26 08:48:01 +00:00
marko
aa8c8329f2 branches/zip: buf_pool_init(): Allocate buf_pool with mem_zalloc(). 2007-10-25 12:54:18 +00:00
marko
a14eb7445c branches/zip: buf_page_get_gen(): Reinitialize guess when re-entering the loop. 2007-10-25 12:51:18 +00:00
marko
37309546e0 branches/zip: Cast void* pointers to byte* when doing pointer arithmetics. 2007-10-25 07:03:02 +00:00
marko
0349b9d235 branches/zip: buf_chunk_init(): Note that the function does not always
return the requested amount of memory.
2007-10-15 11:26:58 +00:00
marko
3d57033064 branches/zip: Optimize buf_page_try_get_func().
buf_block_hash_get(): New function, similar to buf_page_hash_get().

buf_page_get_block(): Remove the const qualifiers.  This is a low-level
function, and the operations on block->mutex are non-const.

buf_page_try_get_func(): Implement with lower-level predicates, somewhat
similar to buf_page_get_known_nowait().

lock_rec_print(): Remove the unused variable zip_size and the
call to fil_space_get_zip_size().  Adapt to buf_page_try_get() returning
a const pointer.
2007-10-01 07:32:45 +00:00
marko
1110b563b2 branches/zip: Introduce the function buf_block_fix_dec(), similar to
buf_block_fix_inc(), and use it.
2007-10-01 07:25:02 +00:00
vasil
e05b8e86b9 branches/zip:
Move part of the code from lock_rec_print() in a separate function
buf_page_try_get() because the same functionality is needed in
INFORMATION SCHEMA code.

Approved by:	Heikki
2007-09-27 14:35:18 +00:00
inaam
ae1bf9e3e9 branches/zip: undo changes made in r1763.
These should have gone in branches/fts

Spotted by: Marko and Ken
2007-09-11 00:38:13 +00:00
marko
372a25240a branches/zip: Improve Valgrind instrumentation.
UNIV_MEM_ASSERT_RW(): New macro, to check that the contents of a memory
area is defined.

UNIV_MEM_ASSERT_W(): New macro, to check that a memory area is writable.

UNIV_MEM_ASSERT_AND_FREE(): New macro, to check that the memory is
writable before declaring it free (unwritable).  This replaces UNIV_MEM_FREE()
in many places.

mem_init_buf(): Declare the memory undefined.

mem_erase_buf(): Declare the memory freed.
2007-09-06 11:36:35 +00:00
inaam
8d96873f9d branches/zip: The freed_page_clock for both buf_pool and block starts ticking
only after the buffer_cache has warmed up. During the initial warmup
phase these values are set to zero. We should recommend to move the
block to the start of the LRU in this case.

Also added some comments to buf0buf.c about the LRU logic.

reviewed by: Sunny
2007-08-27 19:41:56 +00:00
marko
d662c3c656 branches/zip: Merge 1575:1664 from trunk. 2007-08-01 11:18:43 +00:00
marko
54113d6d9f branches/zip: HASH_SEARCH(): Add the parameter TYPE, for the type of DATA.
C++ requires explicit type casts when converting from void*, and *.ic files
are now inlined in ha_innodb.cc.

fil_space_get_by_id(), fil_space_get_by_name(): New inline functions, to
replace occurrences of the HASH_SEARCH() macro in fil0fil.c.  This should
improve readability.
2007-08-01 08:13:22 +00:00
marko
e7284367dc branches/zip: Merge revisions 1402:1493 from trunk.
Remove mysql.patch, because the configuration parameter interface has changed.
2007-05-14 09:07:15 +00:00
marko
fbc6f37cb1 branches/zip: Fix two bugs.
mtr_commit(): Do not acquire the log mutex if no log records are to be written.

ibuf_set_free_bits_func(): Do not clear mtr.modifications, because that would
prevent the dirty insert buffer bitmap block from being moved to the flush
list.  Instead, assume that mtr_commit() will not acquire log_sys->mutex
because redo logging is disabled.

buf_page_init_for_read(): I/O-fix the block before buf_buddy_alloc(), because
the newly allocated block could otherwise be reused by the compressed page.
2007-05-07 05:53:42 +00:00
marko
dd529ee402 branches/zip: When reading pages for merging buffered inserts, request that
the pages be decompressed.

buf_read_page_low(), buf_page_init_for_read(): Add parameter "unzip".
2007-05-06 13:21:59 +00:00
marko
1cd528122e branches/zip: buf_LRU_block_free_non_file_page(): Flag block->frame
allocated before overwriting some fields and flagging the frame freed.
Some of the buffer frame may have been flagged freed by the user
(such as mem_heap).
2007-03-28 11:07:28 +00:00
marko
e4910d4775 branches/zip: Add the pseudo table INFORMATION_SCHEMA.INNODB_BUDDY
for displaying buddy allocator statistics.

buf_buddy_used[], buf_buddy_relocated[]: Declare as global symbols.

mysql_declare_plugin(innobase): Add MYSQL_INFORMATION_SCHEMA_PLUGIN
for innobase_is_buddy ("INNODB_BUDDY").
2007-03-12 12:01:45 +00:00
marko
2b060715ef branches/zip: Split UNIV_IBUF_COUNT_DEBUG from UNIV_IBUF_DEBUG. 2007-02-27 11:56:38 +00:00
marko
37b0f7bf1c branches/zip: Enable the insert buffer on compressed tablespaces.
page_zip_max_ins_size(): New function.

btr_cur_optimistic_insert(), btr_cur_optimistic_delete(),
btr_page_split_and_insert(), btr_compress(): Do not update the
ibuf free bits for non-leaf pages or pages belonging to a clustered index.
The insert buffer only covers operations on leaf pages of secondary indexes.
For pages covered by the insert buffer, limit the max_ins_size to
page_zip_max_ins_size().

buf_page_get_gen(): Merge the insert buffer after decompressing the page.

buf_page_io_complete(): Relax the assertion about ibuf_count.  For
compressed-only pages, the insert buffer merge takes place
in buf_page_get_gen().

ibuf_index_page_calc_free_bits(), ibuf_index_page_calc_free_from_bits(),
ibuf_index_page_calc_free(), ibuf_update_free_bits_if_full(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(),
ibuf_set_free_bits_low(): Add the parameter zip_size.  Limit the maximum
insert size to page_zip_max_ins_size().
2007-02-19 20:32:06 +00:00
marko
9b861942ac branches/zip: buf_LRU_search_and_free_block(): Do not try to preserve
compressed pages, because doing so would require a call to
buf_buddy_alloc(), which in turn can invoke buf_LRU_get_free_block()
and buf_LRU_search_and_free_block(), potentially infinite recursion.
2007-02-19 14:23:32 +00:00
marko
d56453be8e branches/zip: buf_LRU_search_and_free_block(): Avoid freeing compressed-only
pages when they do not occupy too much of the buffer pool.

buf0buddy.c, buf0buddy.h: Export buf_buddy_n_frames and buf_buddy_min_n_frames.
2007-02-16 09:22:50 +00:00
marko
e67d5c761c branches/zip: Add statistic counters to the buddy allocator.
buf_buddy_n_frames: Number of buffer frames allocated for the buddy system
buf_buddy_used[]: Counts of allocated blocks, by buddy size.
buf_buddy_relocated[]: Counts of blocks relocated, bu buddy size.

buf_buddy_alloc_clean(): Note the added configuration parameter
buf_buddy_min_n_frames: Preferred minimum buf_buddy_n_frames
2007-02-13 12:41:02 +00:00
marko
1a103f0324 branches/zip: buf_LRU_get_free_block(): Add an UNIV_UNLIKELY hint
around zip_size.  zip_size will be 0 most of the time.
2007-02-13 11:21:57 +00:00
marko
d3a813fd69 branches/zip: buf_page_get_gen(): After allocating an uncompressed page for
decompression, check that the compressed block has not been
buffer-fixed or I/O-fixed.  This race condition was introduced in r1286.
2007-02-13 11:15:27 +00:00
marko
eb03297439 branches/zip: buf_page_init_for_read(): Defer the costly calls to
buf_buddy_alloc() until after the block has been found not to exist
in the buffer pool.

buf_buddy_alloc(), buf_buddy_alloc_low(): Make lru an output parameter.
Improve documentation.

buf_page_init_for_read(): Simplify the code.  Check buf_page_hash_get()
again if buf_buddy_alloc() released buf_pool->mutex.
2007-02-08 14:19:55 +00:00
marko
b36300e1ba branches/zip: buf_page_create(): Do not IO-fix or X-latch the block
when not invoking buf_buddy_alloc().
2007-02-08 12:33:54 +00:00
marko
5ac6f3217d branches/zip: buf_page_init_for_read(): Do not allocate an uncompressed page
frame when reading compressed pages, unless crash recovery is in progress.

buf_page_read_low(): Adapt for buf_page_init_for_read() returning
buf_page_t* instead of buf_block_t*.
2007-02-08 12:03:03 +00:00
marko
7315b06291 branches/zip: buf_page_init_for_read(), buf_page_create(): Note in the
function comments that buf_page_init_for_read() does not perform the
state transition to FILE_PAGE.  This should have been done in r1286.
2007-02-08 09:08:11 +00:00
marko
329c469e70 branches/zip: Do not decompress blocks in buf_page_init_for_read(),
but in buf_page_get_gen().  Adjust comments.  This prevents
unnecessary decompression in read-ahead when the compressed block
exists in the buffer pool.

If the block does not exist in the buffer pool, read-ahead will still
allocate an uncompressed page and decompress the block.

Move buf_block_init_low() and buf_zip_decompress() earlier in the file,
because some compilers are unable to inline functions that are defined
after the invocation.

buf_page_get_gen(): Decompress the block if needed.

buf_page_init_for_read(): Do not decompress.
2007-02-07 13:59:43 +00:00
marko
13efcc13fc branches/zip: Add UNIV_HASH_DEBUG for detecting multiple invocations
of HASH_DELETE().

HASH_ASSERT_VALID(), HASH_INVALIDATE(): New macros, used in HASH_DELETE()
and HASH_SEARCH().

buf_page_init_low(): Add HASH_INVALIDATE(bpage, hash);

buf_page_init(): Call buf_page_init_low() before HASH_INSERT(), so that
bpage->hash will not be invalidated.
2007-01-24 11:41:19 +00:00