Commit graph

132 commits

Author SHA1 Message Date
marko
a8635f7e29 branches/zip: buf_page_get_gen(): Ignore "guess" if it does not point to
buf_pool->chunks[]->blocks[].

buf_block_is_uncompressed(): New function, to detect if a block points to
buf_pool->chunks[]->blocks[].
2007-11-05 10:31:41 +00:00
marko
66e193b4ba branches/zip: buf_page_init_low(): Initialize also flush_type. 2007-10-31 22:02:23 +00:00
marko
902893c719 branches/zip: Improve Valgrind instrumentation.
buf_LRU_free_block(): Check that the block descriptor contains valid data.

buf_buddy_relocate(): Check that the source block contains valid data.

buf_page_get_gen(): Do not dereference bpage after calling buf_relocate().
This avoids a bogus Valgrind warning; the memory itself was valid.

buf_page_hash_get(): Check that the returned block descriptor contains
valid data.
2007-10-30 09:27:09 +00:00
marko
fca2f0c678 branches/zip: Reduce WAIT_FOR_READ from 20 to 5 milliseconds, to correspond
to the time in which modern disks can serve a random disk read.
2007-10-26 12:34:53 +00:00
marko
fcd34fa208 branches/zip: buf_pool_t: Add n_pend_unzip. Display it in buf_print(). 2007-10-26 08:53:36 +00:00
marko
d7deb7fc2e branches/zip: buf_page_get_gen(): Improve the comments about
wait_until_unfixed.
2007-10-26 08:48:01 +00:00
marko
8bee0c8358 branches/zip: buf_pool_init(): Allocate buf_pool with mem_zalloc(). 2007-10-25 12:54:18 +00:00
marko
6b3be8a613 branches/zip: buf_page_get_gen(): Reinitialize guess when re-entering the loop. 2007-10-25 12:51:18 +00:00
marko
43becedf54 branches/zip: Cast void* pointers to byte* when doing pointer arithmetics. 2007-10-25 07:03:02 +00:00
marko
415a1045b2 branches/zip: buf_chunk_init(): Note that the function does not always
return the requested amount of memory.
2007-10-15 11:26:58 +00:00
marko
6759a929ad branches/zip: Optimize buf_page_try_get_func().
buf_block_hash_get(): New function, similar to buf_page_hash_get().

buf_page_get_block(): Remove the const qualifiers.  This is a low-level
function, and the operations on block->mutex are non-const.

buf_page_try_get_func(): Implement with lower-level predicates, somewhat
similar to buf_page_get_known_nowait().

lock_rec_print(): Remove the unused variable zip_size and the
call to fil_space_get_zip_size().  Adapt to buf_page_try_get() returning
a const pointer.
2007-10-01 07:32:45 +00:00
marko
aafdd9ab80 branches/zip: Introduce the function buf_block_fix_dec(), similar to
buf_block_fix_inc(), and use it.
2007-10-01 07:25:02 +00:00
vasil
f38752862a branches/zip:
Move part of the code from lock_rec_print() in a separate function
buf_page_try_get() because the same functionality is needed in
INFORMATION SCHEMA code.

Approved by:	Heikki
2007-09-27 14:35:18 +00:00
inaam
04fb05b13e branches/zip: undo changes made in r1763.
These should have gone in branches/fts

Spotted by: Marko and Ken
2007-09-11 00:38:13 +00:00
inaam
98c1aefe7c branches/zip: The freed_page_clock for both buf_pool and block starts ticking
only after the buffer_cache has warmed up. During the initial warmup
phase these values are set to zero. We should recommend to move the
block to the start of the LRU in this case.

Also added some comments to buf0buf.c about the LRU logic.

reviewed by: Sunny
2007-08-27 19:41:56 +00:00
marko
0a780db994 branches/zip: Merge 1575:1664 from trunk. 2007-08-01 11:18:43 +00:00
marko
fa4f5fcfe5 branches/zip: Fix two bugs.
mtr_commit(): Do not acquire the log mutex if no log records are to be written.

ibuf_set_free_bits_func(): Do not clear mtr.modifications, because that would
prevent the dirty insert buffer bitmap block from being moved to the flush
list.  Instead, assume that mtr_commit() will not acquire log_sys->mutex
because redo logging is disabled.

buf_page_init_for_read(): I/O-fix the block before buf_buddy_alloc(), because
the newly allocated block could otherwise be reused by the compressed page.
2007-05-07 05:53:42 +00:00
marko
eec9380475 branches/zip: When reading pages for merging buffered inserts, request that
the pages be decompressed.

buf_read_page_low(), buf_page_init_for_read(): Add parameter "unzip".
2007-05-06 13:21:59 +00:00
marko
eddd572abd branches/zip: Split UNIV_IBUF_COUNT_DEBUG from UNIV_IBUF_DEBUG. 2007-02-27 11:56:38 +00:00
marko
efd79c7a86 branches/zip: Enable the insert buffer on compressed tablespaces.
page_zip_max_ins_size(): New function.

btr_cur_optimistic_insert(), btr_cur_optimistic_delete(),
btr_page_split_and_insert(), btr_compress(): Do not update the
ibuf free bits for non-leaf pages or pages belonging to a clustered index.
The insert buffer only covers operations on leaf pages of secondary indexes.
For pages covered by the insert buffer, limit the max_ins_size to
page_zip_max_ins_size().

buf_page_get_gen(): Merge the insert buffer after decompressing the page.

buf_page_io_complete(): Relax the assertion about ibuf_count.  For
compressed-only pages, the insert buffer merge takes place
in buf_page_get_gen().

ibuf_index_page_calc_free_bits(), ibuf_index_page_calc_free_from_bits(),
ibuf_index_page_calc_free(), ibuf_update_free_bits_if_full(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(),
ibuf_set_free_bits_low(): Add the parameter zip_size.  Limit the maximum
insert size to page_zip_max_ins_size().
2007-02-19 20:32:06 +00:00
marko
0aff0fdfd2 branches/zip: buf_page_get_gen(): After allocating an uncompressed page for
decompression, check that the compressed block has not been
buffer-fixed or I/O-fixed.  This race condition was introduced in r1286.
2007-02-13 11:15:27 +00:00
marko
75cf6705ac branches/zip: buf_page_init_for_read(): Defer the costly calls to
buf_buddy_alloc() until after the block has been found not to exist
in the buffer pool.

buf_buddy_alloc(), buf_buddy_alloc_low(): Make lru an output parameter.
Improve documentation.

buf_page_init_for_read(): Simplify the code.  Check buf_page_hash_get()
again if buf_buddy_alloc() released buf_pool->mutex.
2007-02-08 14:19:55 +00:00
marko
5035ff1af4 branches/zip: buf_page_create(): Do not IO-fix or X-latch the block
when not invoking buf_buddy_alloc().
2007-02-08 12:33:54 +00:00
marko
76fa1cfbdd branches/zip: buf_page_init_for_read(): Do not allocate an uncompressed page
frame when reading compressed pages, unless crash recovery is in progress.

buf_page_read_low(): Adapt for buf_page_init_for_read() returning
buf_page_t* instead of buf_block_t*.
2007-02-08 12:03:03 +00:00
marko
bedc6a976b branches/zip: buf_page_init_for_read(), buf_page_create(): Note in the
function comments that buf_page_init_for_read() does not perform the
state transition to FILE_PAGE.  This should have been done in r1286.
2007-02-08 09:08:11 +00:00
marko
1f0b739f4f branches/zip: Do not decompress blocks in buf_page_init_for_read(),
but in buf_page_get_gen().  Adjust comments.  This prevents
unnecessary decompression in read-ahead when the compressed block
exists in the buffer pool.

If the block does not exist in the buffer pool, read-ahead will still
allocate an uncompressed page and decompress the block.

Move buf_block_init_low() and buf_zip_decompress() earlier in the file,
because some compilers are unable to inline functions that are defined
after the invocation.

buf_page_get_gen(): Decompress the block if needed.

buf_page_init_for_read(): Do not decompress.
2007-02-07 13:59:43 +00:00
marko
73bb34156f branches/zip: Add UNIV_HASH_DEBUG for detecting multiple invocations
of HASH_DELETE().

HASH_ASSERT_VALID(), HASH_INVALIDATE(): New macros, used in HASH_DELETE()
and HASH_SEARCH().

buf_page_init_low(): Add HASH_INVALIDATE(bpage, hash);

buf_page_init(): Call buf_page_init_low() before HASH_INSERT(), so that
bpage->hash will not be invalidated.
2007-01-24 11:41:19 +00:00
marko
ae0ecb11aa branches/zip: buf_page_t: Add the in_page_hash and in_zip_hash
for extra validation in UNIV_DEBUG builds.
2007-01-24 10:36:05 +00:00
marko
87defea2bb branches/zip: buf_page_init_for_read(): Decrement, do not reset
block->page.buf_fix_count.  This mistake was made in r1255.
When access to the block is requested by some other thread,
buf_fix_count can be incremented during the execution of
buf_zip_decompress().
2007-01-23 16:12:14 +00:00
marko
7ebd17e0e4 branches/zip: buf_page_init_for_read(): When decompressing a compressed block,
initialize all the fields of the control block.

buf_block_init_low(), buf_page_init_low(): New functions, used in
buf_page_init_for_backup_restore(), buf_page_init(), and
buf_page_init_for_read().
2007-01-23 14:08:37 +00:00
marko
9ef14edbce branches/zip: Fix a bug in the buddy allocator.
buf_buddy_relocate_block(): New function, split from buf_buddy_relocate().
Return FALSE when buf_page_can_relocate() returns FALSE.

Fix some formatting in buf_buddy_free_low() and buf_page_init_for_read().
2007-01-19 12:27:47 +00:00
marko
adbd5c3a85 branches/zip: When retrieving compressed BLOBs, avoid allocating
uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().
2007-01-18 23:10:49 +00:00
marko
f56b6cf4cb branches/zip: Fix bugs.
buf_LRU_free_block(): Do not allocate the control block when
a compressed-only page is being freed.

buf_validate(): Note that buf_pool->zip_clean may contain blocks
with io_fix == BUF_IO_READ.
2007-01-18 20:29:35 +00:00
marko
5fc003b65d branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG. 2007-01-18 18:29:12 +00:00
marko
f116afa724 branches/zip: Remove the fil_space_get_zip_size() call from
buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.
2007-01-18 09:59:00 +00:00
marko
b68c70121e branches/zip: buf_page_init_for_read(): X-latch the block before
invoking buf_buddy_alloc().  An IO-fixed block must also be x-latched
when block->mutex and buf_pool->mutex are released.

buf_page_create(): Protect the block during buf_buddy_alloc() by
IO-fixing and X-latching it.
2007-01-16 13:45:52 +00:00
marko
8efb68f1c9 branches/zip: buf_page_init_for_read(): IO-fix the block during
buf_zip_decompress().  This will prevent btr_cur_search_to_nth_level()
from accessing a half-decompressed leaf page.  Internal B-tree nodes
(non-leaf pages) in B-tree searches are only protected by the tree
latch (index->lock) and by buffer-fixing.
2007-01-16 12:40:15 +00:00
marko
6604943597 branches/zip: When adding a page to the buffer pool, add it to
buf_pool->page_hash and buf_pool->LRU before releasing buf_pool->mutex.

buf_page_init_for_read(), buf_page_create(): Allocate the compressed
page after the block has been added to the buffer pool.  Document the
reason for this.
2007-01-15 16:51:39 +00:00
marko
ef4abdf10d branches/zip: Make Valgrind diagnostics more accurate.
buf_page_init(): Declare block->frame valid only in the system tablespace.

trx_sysf_create(): Declare the rest of the page valid, although it is
uninitialized.
2007-01-13 21:15:55 +00:00
marko
2b82ae8cb4 branches/zip: Improve Valgrind instrumentation of allocated memory.
UNIV_MEM_FREE(): Declare a memory area free.
UNIV_MEM_ALLOC(): Declare a memory area allocated (but uninitialized).
UNIV_MEM_DESC(): Associate a memory area with a control block.
UNIV_MEM_UNDESC(): Unassociate a control block.

trx_sys_create_doublewrite_buf(): Clear the buffer with memset().

buf_page_init(): Add a bogus UNIV_MEM_VALID(block->frame) to silence
valid warnings about InnoDB data pages containing uninitialized data.

buf_LRU_get_free_only(): Add UNIV_MEM_ALLOC(block->frame).

buf_LRU_get_free_block(): Add UNIV_MEM_DESC(block->page.zip.data, block).

buf_LRU_free_block(): Add UNIV_MEM_DESC(b->zip.data, b) when allocating
a compressed-only control block for a compressed page.

buf_LRU_block_free_non_file_page(): Replace UNIV_MEM_INVALID() with
UNIV_MEM_FREE().

buf_LRU_block_remove_hashed_page(): Add UNIV_MEM_UNDESC(bpage) when
deallocating a compressed-only control block.  Add
UNIV_MEM_INVALID(block->frame).  (The frame should be flagged free
by buf_LRU_block_free_non_file_page() moments later.)

buf0buddy.c: Disable some extra checks in Valgrind-instrumented builds.
Add UNIV_MEM_VALID(), UNIV_MEM_INVALID(), UNIV_MEM_FREE(), UNIV_MEM_ALLOC()
as necessary.
2007-01-12 12:36:40 +00:00
marko
9478ec03e9 branches/zip: buf_relocate(): Copy the block. Invalidate the source block.
buf_buddy_block_free(), buf_buddy_free_low(): Flag the freed bytes
uninitialized.

buf_page_init_for_read(), buf_buddy_relocate(): Do not refer to bpage
after invoking buf_relocate().

buf_buddy_relocate(): Flag src uninitialized.
2007-01-12 09:14:15 +00:00
marko
225e82c112 branches/zip: buf_page_io_complete(): Change the parameter type from
buf_block_t* to buf_page_t*.  Handle compressed-only blocks.
2007-01-10 14:36:39 +00:00
marko
8c0f676265 branches/zip: buf_page_init(): Improve diagnostic output in case of
buf_pool->page_hash corruption.
2007-01-10 14:36:15 +00:00
marko
020b6035be branches/zip: buf_page_init_for_read(): Increment the buf_fix_count
during the buf_zip_decompress() call.  Otherwise, the block may end
up being freed by another thread.  This bug was caught by the
page_zip_validate() check in buf_LRU_block_remove_hashed_page().
2007-01-10 12:50:10 +00:00
marko
1f7bc8e232 branches/zip: Remove a race condition.
buf_page_init_for_read(): Protect the buf_zip_decompress() call with
an X-lock on the block.

page_zip_validate(): Move the page_is_comp() assertion after the
page header comparison.
2007-01-10 10:08:34 +00:00
marko
d3917bb5c3 branches/zip: buf_page_t: Introduce the field in_flush_list #ifdef UNIV_DEBUG.
buf_flush_batch(): Remove the local variable "found".
2007-01-09 12:35:42 +00:00
marko
5a094438a5 branches/zip: Maintain a valid checksum for compressed-only pages kept in
the buffer pool.

buf_LRU_free_block(): When preserving the compressed page, compute the
checksum.  Also assert page_zip_validate().

buf_zip_decompress(): Add parameter "check" for enabling checksum tests.
Before decompressing a compressed-only page from the buffer pool, verify
the compressed page checksum.
2007-01-08 17:08:57 +00:00
marko
0583d98529 branches/zip: buf_get_latched_pages_number(): Count compressed-only blocks too. 2007-01-05 21:00:07 +00:00
marko
27ded6f71e branches/zip: Support compressed-only pages in buf_page_create().
buf_page_get_gen(): Note that buf_read_page() may uncompress
a compressed-only page.

buf_page_create(): Test buf_page_in_file() instead of BUF_BLOCK_FILE_PAGE.
2007-01-05 14:03:44 +00:00
marko
13d6355676 branches/zip: buf_validate(): Validate also the compressed-only blocks. 2007-01-05 12:13:28 +00:00