Commit graph

201 commits

Author SHA1 Message Date
marko
0a780db994 branches/zip: Merge 1575:1664 from trunk. 2007-08-01 11:18:43 +00:00
marko
08c1fc7aa7 branches/zip: HASH_SEARCH(): Add the parameter TYPE, for the type of DATA.
C++ requires explicit type casts when converting from void*, and *.ic files
are now inlined in ha_innodb.cc.

fil_space_get_by_id(), fil_space_get_by_name(): New inline functions, to
replace occurrences of the HASH_SEARCH() macro in fil0fil.c.  This should
improve readability.
2007-08-01 08:13:22 +00:00
marko
88df83bb3b branches/zip: Merge revisions 1402:1493 from trunk.
Remove mysql.patch, because the configuration parameter interface has changed.
2007-05-14 09:07:15 +00:00
marko
fa4f5fcfe5 branches/zip: Fix two bugs.
mtr_commit(): Do not acquire the log mutex if no log records are to be written.

ibuf_set_free_bits_func(): Do not clear mtr.modifications, because that would
prevent the dirty insert buffer bitmap block from being moved to the flush
list.  Instead, assume that mtr_commit() will not acquire log_sys->mutex
because redo logging is disabled.

buf_page_init_for_read(): I/O-fix the block before buf_buddy_alloc(), because
the newly allocated block could otherwise be reused by the compressed page.
2007-05-07 05:53:42 +00:00
marko
eec9380475 branches/zip: When reading pages for merging buffered inserts, request that
the pages be decompressed.

buf_read_page_low(), buf_page_init_for_read(): Add parameter "unzip".
2007-05-06 13:21:59 +00:00
marko
b4715b5d5e branches/zip: buf_LRU_block_free_non_file_page(): Flag block->frame
allocated before overwriting some fields and flagging the frame freed.
Some of the buffer frame may have been flagged freed by the user
(such as mem_heap).
2007-03-28 11:07:28 +00:00
marko
72b15b170a branches/zip: Add the pseudo table INFORMATION_SCHEMA.INNODB_BUDDY
for displaying buddy allocator statistics.

buf_buddy_used[], buf_buddy_relocated[]: Declare as global symbols.

mysql_declare_plugin(innobase): Add MYSQL_INFORMATION_SCHEMA_PLUGIN
for innobase_is_buddy ("INNODB_BUDDY").
2007-03-12 12:01:45 +00:00
marko
eddd572abd branches/zip: Split UNIV_IBUF_COUNT_DEBUG from UNIV_IBUF_DEBUG. 2007-02-27 11:56:38 +00:00
marko
efd79c7a86 branches/zip: Enable the insert buffer on compressed tablespaces.
page_zip_max_ins_size(): New function.

btr_cur_optimistic_insert(), btr_cur_optimistic_delete(),
btr_page_split_and_insert(), btr_compress(): Do not update the
ibuf free bits for non-leaf pages or pages belonging to a clustered index.
The insert buffer only covers operations on leaf pages of secondary indexes.
For pages covered by the insert buffer, limit the max_ins_size to
page_zip_max_ins_size().

buf_page_get_gen(): Merge the insert buffer after decompressing the page.

buf_page_io_complete(): Relax the assertion about ibuf_count.  For
compressed-only pages, the insert buffer merge takes place
in buf_page_get_gen().

ibuf_index_page_calc_free_bits(), ibuf_index_page_calc_free_from_bits(),
ibuf_index_page_calc_free(), ibuf_update_free_bits_if_full(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(),
ibuf_set_free_bits_low(): Add the parameter zip_size.  Limit the maximum
insert size to page_zip_max_ins_size().
2007-02-19 20:32:06 +00:00
marko
df5a9376ea branches/zip: buf_LRU_search_and_free_block(): Do not try to preserve
compressed pages, because doing so would require a call to
buf_buddy_alloc(), which in turn can invoke buf_LRU_get_free_block()
and buf_LRU_search_and_free_block(), potentially infinite recursion.
2007-02-19 14:23:32 +00:00
marko
9c663a0c46 branches/zip: buf_LRU_search_and_free_block(): Avoid freeing compressed-only
pages when they do not occupy too much of the buffer pool.

buf0buddy.c, buf0buddy.h: Export buf_buddy_n_frames and buf_buddy_min_n_frames.
2007-02-16 09:22:50 +00:00
marko
5e0dcfae3f branches/zip: Add statistic counters to the buddy allocator.
buf_buddy_n_frames: Number of buffer frames allocated for the buddy system
buf_buddy_used[]: Counts of allocated blocks, by buddy size.
buf_buddy_relocated[]: Counts of blocks relocated, bu buddy size.

buf_buddy_alloc_clean(): Note the added configuration parameter
buf_buddy_min_n_frames: Preferred minimum buf_buddy_n_frames
2007-02-13 12:41:02 +00:00
marko
6f6aefb553 branches/zip: buf_LRU_get_free_block(): Add an UNIV_UNLIKELY hint
around zip_size.  zip_size will be 0 most of the time.
2007-02-13 11:21:57 +00:00
marko
0aff0fdfd2 branches/zip: buf_page_get_gen(): After allocating an uncompressed page for
decompression, check that the compressed block has not been
buffer-fixed or I/O-fixed.  This race condition was introduced in r1286.
2007-02-13 11:15:27 +00:00
marko
75cf6705ac branches/zip: buf_page_init_for_read(): Defer the costly calls to
buf_buddy_alloc() until after the block has been found not to exist
in the buffer pool.

buf_buddy_alloc(), buf_buddy_alloc_low(): Make lru an output parameter.
Improve documentation.

buf_page_init_for_read(): Simplify the code.  Check buf_page_hash_get()
again if buf_buddy_alloc() released buf_pool->mutex.
2007-02-08 14:19:55 +00:00
marko
5035ff1af4 branches/zip: buf_page_create(): Do not IO-fix or X-latch the block
when not invoking buf_buddy_alloc().
2007-02-08 12:33:54 +00:00
marko
76fa1cfbdd branches/zip: buf_page_init_for_read(): Do not allocate an uncompressed page
frame when reading compressed pages, unless crash recovery is in progress.

buf_page_read_low(): Adapt for buf_page_init_for_read() returning
buf_page_t* instead of buf_block_t*.
2007-02-08 12:03:03 +00:00
marko
bedc6a976b branches/zip: buf_page_init_for_read(), buf_page_create(): Note in the
function comments that buf_page_init_for_read() does not perform the
state transition to FILE_PAGE.  This should have been done in r1286.
2007-02-08 09:08:11 +00:00
marko
1f0b739f4f branches/zip: Do not decompress blocks in buf_page_init_for_read(),
but in buf_page_get_gen().  Adjust comments.  This prevents
unnecessary decompression in read-ahead when the compressed block
exists in the buffer pool.

If the block does not exist in the buffer pool, read-ahead will still
allocate an uncompressed page and decompress the block.

Move buf_block_init_low() and buf_zip_decompress() earlier in the file,
because some compilers are unable to inline functions that are defined
after the invocation.

buf_page_get_gen(): Decompress the block if needed.

buf_page_init_for_read(): Do not decompress.
2007-02-07 13:59:43 +00:00
marko
73bb34156f branches/zip: Add UNIV_HASH_DEBUG for detecting multiple invocations
of HASH_DELETE().

HASH_ASSERT_VALID(), HASH_INVALIDATE(): New macros, used in HASH_DELETE()
and HASH_SEARCH().

buf_page_init_low(): Add HASH_INVALIDATE(bpage, hash);

buf_page_init(): Call buf_page_init_low() before HASH_INSERT(), so that
bpage->hash will not be invalidated.
2007-01-24 11:41:19 +00:00
marko
ae0ecb11aa branches/zip: buf_page_t: Add the in_page_hash and in_zip_hash
for extra validation in UNIV_DEBUG builds.
2007-01-24 10:36:05 +00:00
marko
87defea2bb branches/zip: buf_page_init_for_read(): Decrement, do not reset
block->page.buf_fix_count.  This mistake was made in r1255.
When access to the block is requested by some other thread,
buf_fix_count can be incremented during the execution of
buf_zip_decompress().
2007-01-23 16:12:14 +00:00
marko
7ebd17e0e4 branches/zip: buf_page_init_for_read(): When decompressing a compressed block,
initialize all the fields of the control block.

buf_block_init_low(), buf_page_init_low(): New functions, used in
buf_page_init_for_backup_restore(), buf_page_init(), and
buf_page_init_for_read().
2007-01-23 14:08:37 +00:00
marko
9ef14edbce branches/zip: Fix a bug in the buddy allocator.
buf_buddy_relocate_block(): New function, split from buf_buddy_relocate().
Return FALSE when buf_page_can_relocate() returns FALSE.

Fix some formatting in buf_buddy_free_low() and buf_page_init_for_read().
2007-01-19 12:27:47 +00:00
marko
adbd5c3a85 branches/zip: When retrieving compressed BLOBs, avoid allocating
uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().
2007-01-18 23:10:49 +00:00
marko
f56b6cf4cb branches/zip: Fix bugs.
buf_LRU_free_block(): Do not allocate the control block when
a compressed-only page is being freed.

buf_validate(): Note that buf_pool->zip_clean may contain blocks
with io_fix == BUF_IO_READ.
2007-01-18 20:29:35 +00:00
marko
5fc003b65d branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG. 2007-01-18 18:29:12 +00:00
marko
ca662b43e6 branches/zip: buf_LRU_free_block(): Allocate a control block for the
compressed page also when freeing an unmodified uncompressed page.
2007-01-18 15:43:59 +00:00
marko
8f65d7a661 branches/zip: Merge revisions 1165:1206 from trunk. 2007-01-18 12:58:39 +00:00
marko
f116afa724 branches/zip: Remove the fil_space_get_zip_size() call from
buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.
2007-01-18 09:59:00 +00:00
marko
d3d728955b branches/zip: buf_flush_write_block_low(): When flushing a compressed page
that also exists in uncompressed form in the buffer pool, pass the
compressed frame to fil_io() instead of the uncompressed one when
not using the doublewrite buffer.
2007-01-16 22:19:29 +00:00
marko
e77a82a054 branches/zip: buf_flush_write_block_low(): Avoid recomputing the compressed
page checksum of compressed-only blocks.  Pass the compressed page frame
to fil_io() when needed.

page_zip_calc_checksum(): Skip also FIL_PAGE_LSN and FIL_PAGE_FILE_FLUSH_LSN.

buf_flush_init_for_writing(): Expect page to be non-NULL.
2007-01-16 21:51:40 +00:00
marko
a85fd29415 branches/zip: Allow dirty compressed-only blocks to exist in the buffer pool
and to be flushed to disk.

buf_LRU_free_block(): Enable the freeing of uncompressed pages of
compressed tablespaces.

trx_doublewrite->buf_block_arr[]: Change the type from buf_block_t*
to buf_page_t*.

buf_flush_ready_for_flush(): Add debug assertion.

buf_flush_buffered_writes(), buf_flush_try_page(): Support blocks of type
BUF_BLOCK_ZIP_DIRTY.

buf_flush_post_to_doublewrite_buf(), buf_flush_write_block_low():
Change the type of the parameter from buf_block_t* to buf_page_t*.

buf_flush_init_for_writing(): Allow page to be NULL if page_zip_ is non-NULL.
2007-01-16 18:02:49 +00:00
marko
94e49a71b2 branches/zip: Note that b (the newly created control block for the
compressed page) is not protected by block_mutex.  Before releasing
buf_pool->mutex, we can simply assign b->io_fix.  After reacquiring
buf_pool->mutex, we also have to acquire buf_pool->zip_mutex.
2007-01-16 16:39:53 +00:00
marko
9fe82e80e3 branches/zip: buf_LRU_free_block(): When preserving the compressed page
of a block, do not release buf_pool->mutex between the time the old control
block is removed from buf_pool->page_hash and the new control block is
added to it.  Prevent operations on the compressed-only block while calling
btr_search_drop_page_hash_index() or page_zip_calc_checksum().

buf_LRU_invalidate_tablespace(): Revert the change done in r1223.
buf_pool->zip_mutex will have been released by
buf_LRU_block_remove_hashed_page() when it returns BUF_BLOCK_ZIP_FREE.
2007-01-16 16:24:47 +00:00
marko
c3599a3ab1 branches/zip: buf_flush_remove(): buf_page_get_mutex(bpage) must have been
acquired by the caller.

buf_LRU_invalidate_tablespace(): Do not leak buf_pool->zip_mutex when
rescanning the LRU list.
2007-01-16 14:37:40 +00:00
marko
91546e7d10 branches/zip: buf_LRU_free_block(): When freeing the uncompressed page
of a dirty block, relocate the control blocks on buf_pool->flush_list
instead of calling buf_flush_insert_into_flush_list().
2007-01-16 14:11:21 +00:00
marko
b68c70121e branches/zip: buf_page_init_for_read(): X-latch the block before
invoking buf_buddy_alloc().  An IO-fixed block must also be x-latched
when block->mutex and buf_pool->mutex are released.

buf_page_create(): Protect the block during buf_buddy_alloc() by
IO-fixing and X-latching it.
2007-01-16 13:45:52 +00:00
marko
8efb68f1c9 branches/zip: buf_page_init_for_read(): IO-fix the block during
buf_zip_decompress().  This will prevent btr_cur_search_to_nth_level()
from accessing a half-decompressed leaf page.  Internal B-tree nodes
(non-leaf pages) in B-tree searches are only protected by the tree
latch (index->lock) and by buffer-fixing.
2007-01-16 12:40:15 +00:00
marko
82f4ef9596 branches/zip: buf_LRU_block_remove_hashed_page(): When preserving the
compressed page, copy the contents from the uncompressed page if necessary.
2007-01-16 10:24:13 +00:00
marko
69f0212f4b branches/zip: buf_buddy_alloc_clean(): Remove bogus assertion ut_a(bpage)
and update the comments.
2007-01-16 08:59:39 +00:00
marko
6604943597 branches/zip: When adding a page to the buffer pool, add it to
buf_pool->page_hash and buf_pool->LRU before releasing buf_pool->mutex.

buf_page_init_for_read(), buf_page_create(): Allocate the compressed
page after the block has been added to the buffer pool.  Document the
reason for this.
2007-01-15 16:51:39 +00:00
marko
ac24d86c49 branches/zip: buf_LRU_free_block(): Flag the buffer frame fully initialized
when calling btr_search_drop_page_hash_index() and uninitialized after that.
2007-01-14 20:32:44 +00:00
marko
ef4abdf10d branches/zip: Make Valgrind diagnostics more accurate.
buf_page_init(): Declare block->frame valid only in the system tablespace.

trx_sysf_create(): Declare the rest of the page valid, although it is
uninitialized.
2007-01-13 21:15:55 +00:00
marko
6df0fc42ed branches/zip: Improve Valgrind diagnostics.
buf_buddy_relocate(): Document that src may have been split and some of
it may be in buf_pool->zip_free[j] for some j<i, thus mach_read_from_4()
may be reading from free memory.  Check buffer pages only for addressability,
not for being completely initialized.  Ensure that block descriptors are
completely initialized.

buf_buddy_remove_from_free(): Check prev->state and next->state in
UNIV_DEBUG && UNIV_DEBUG_VALGRIND builds.

buf_buddy_free_low(): Flag the entire block valid prior to invoking
buf_buddy_remove_from_free() in order to avoid a Valgrind warning about
the debug assertion on bpage->state.  Remove the Valgrind check for
buddy being completely allocated.  If it has been split to smaller blocks,
some of those blocks may be free.
2007-01-12 22:23:12 +00:00
marko
5559afc6d4 branches/zip: buf_buddy_relocate(): Add Valgrind checks. 2007-01-12 14:30:18 +00:00
marko
1ca38a9404 branches/zip: Add the buf_pool->zip_free[] accessor functions
buf_buddy_add_to_free() and buf_buddy_remove_from_free().
Remove some bogus Valgrind warnings.  Improve Valgrind instrumentation.
2007-01-12 14:18:22 +00:00
marko
2b82ae8cb4 branches/zip: Improve Valgrind instrumentation of allocated memory.
UNIV_MEM_FREE(): Declare a memory area free.
UNIV_MEM_ALLOC(): Declare a memory area allocated (but uninitialized).
UNIV_MEM_DESC(): Associate a memory area with a control block.
UNIV_MEM_UNDESC(): Unassociate a control block.

trx_sys_create_doublewrite_buf(): Clear the buffer with memset().

buf_page_init(): Add a bogus UNIV_MEM_VALID(block->frame) to silence
valid warnings about InnoDB data pages containing uninitialized data.

buf_LRU_get_free_only(): Add UNIV_MEM_ALLOC(block->frame).

buf_LRU_get_free_block(): Add UNIV_MEM_DESC(block->page.zip.data, block).

buf_LRU_free_block(): Add UNIV_MEM_DESC(b->zip.data, b) when allocating
a compressed-only control block for a compressed page.

buf_LRU_block_free_non_file_page(): Replace UNIV_MEM_INVALID() with
UNIV_MEM_FREE().

buf_LRU_block_remove_hashed_page(): Add UNIV_MEM_UNDESC(bpage) when
deallocating a compressed-only control block.  Add
UNIV_MEM_INVALID(block->frame).  (The frame should be flagged free
by buf_LRU_block_free_non_file_page() moments later.)

buf0buddy.c: Disable some extra checks in Valgrind-instrumented builds.
Add UNIV_MEM_VALID(), UNIV_MEM_INVALID(), UNIV_MEM_FREE(), UNIV_MEM_ALLOC()
as necessary.
2007-01-12 12:36:40 +00:00
marko
9478ec03e9 branches/zip: buf_relocate(): Copy the block. Invalidate the source block.
buf_buddy_block_free(), buf_buddy_free_low(): Flag the freed bytes
uninitialized.

buf_page_init_for_read(), buf_buddy_relocate(): Do not refer to bpage
after invoking buf_relocate().

buf_buddy_relocate(): Flag src uninitialized.
2007-01-12 09:14:15 +00:00
marko
225e82c112 branches/zip: buf_page_io_complete(): Change the parameter type from
buf_block_t* to buf_page_t*.  Handle compressed-only blocks.
2007-01-10 14:36:39 +00:00