Commit graph

608 commits

Author SHA1 Message Date
marko
9c663a0c46 branches/zip: buf_LRU_search_and_free_block(): Avoid freeing compressed-only
pages when they do not occupy too much of the buffer pool.

buf0buddy.c, buf0buddy.h: Export buf_buddy_n_frames and buf_buddy_min_n_frames.
2007-02-16 09:22:50 +00:00
marko
5e0dcfae3f branches/zip: Add statistic counters to the buddy allocator.
buf_buddy_n_frames: Number of buffer frames allocated for the buddy system
buf_buddy_used[]: Counts of allocated blocks, by buddy size.
buf_buddy_relocated[]: Counts of blocks relocated, bu buddy size.

buf_buddy_alloc_clean(): Note the added configuration parameter
buf_buddy_min_n_frames: Preferred minimum buf_buddy_n_frames
2007-02-13 12:41:02 +00:00
marko
6f6aefb553 branches/zip: buf_LRU_get_free_block(): Add an UNIV_UNLIKELY hint
around zip_size.  zip_size will be 0 most of the time.
2007-02-13 11:21:57 +00:00
marko
0aff0fdfd2 branches/zip: buf_page_get_gen(): After allocating an uncompressed page for
decompression, check that the compressed block has not been
buffer-fixed or I/O-fixed.  This race condition was introduced in r1286.
2007-02-13 11:15:27 +00:00
marko
9b2643705a branches/zip: btr_cur_optimistic_insert(): After dtuple_convert_big_rec(),
invoke btr_cur_add_ext() so that rec_size will be computed correctly.
Prepare to free the memory heap when handling errors.
2007-02-12 12:04:49 +00:00
marko
75cf6705ac branches/zip: buf_page_init_for_read(): Defer the costly calls to
buf_buddy_alloc() until after the block has been found not to exist
in the buffer pool.

buf_buddy_alloc(), buf_buddy_alloc_low(): Make lru an output parameter.
Improve documentation.

buf_page_init_for_read(): Simplify the code.  Check buf_page_hash_get()
again if buf_buddy_alloc() released buf_pool->mutex.
2007-02-08 14:19:55 +00:00
marko
5035ff1af4 branches/zip: buf_page_create(): Do not IO-fix or X-latch the block
when not invoking buf_buddy_alloc().
2007-02-08 12:33:54 +00:00
marko
76fa1cfbdd branches/zip: buf_page_init_for_read(): Do not allocate an uncompressed page
frame when reading compressed pages, unless crash recovery is in progress.

buf_page_read_low(): Adapt for buf_page_init_for_read() returning
buf_page_t* instead of buf_block_t*.
2007-02-08 12:03:03 +00:00
marko
bedc6a976b branches/zip: buf_page_init_for_read(), buf_page_create(): Note in the
function comments that buf_page_init_for_read() does not perform the
state transition to FILE_PAGE.  This should have been done in r1286.
2007-02-08 09:08:11 +00:00
marko
1f0b739f4f branches/zip: Do not decompress blocks in buf_page_init_for_read(),
but in buf_page_get_gen().  Adjust comments.  This prevents
unnecessary decompression in read-ahead when the compressed block
exists in the buffer pool.

If the block does not exist in the buffer pool, read-ahead will still
allocate an uncompressed page and decompress the block.

Move buf_block_init_low() and buf_zip_decompress() earlier in the file,
because some compilers are unable to inline functions that are defined
after the invocation.

buf_page_get_gen(): Decompress the block if needed.

buf_page_init_for_read(): Do not decompress.
2007-02-07 13:59:43 +00:00
marko
52f7bb36ca branches/zip: page_zip_write_rec(): Treat records containing externally
stored columns as a special case.

page_zip_write_rec_ext(): New function for writing records containing
externally stored columns.
2007-02-05 14:05:39 +00:00
marko
8fb1f51344 branches/zip: page_zip_decompress_clust(): Optimize the case when the
record does not contain externally stored columns.

page_zip_decompress_clust_ext(): New function for decompressing records
containing externally stored columns.
2007-02-05 13:20:00 +00:00
marko
2633b08830 branches/zip: ha_innobase::create(): Do not test create_info->used_fields
before testing create_info->avg_row_length, because otherwise the table
would be converted to uncompressed format by CREATE INDEX and possibly
other statements.
2007-02-05 12:17:11 +00:00
marko
ec06cfdb00 branches/zip: Introduce separate functions for handling records containing
externally stored columns.

page_zip_compress_clust_ext(), page_zip_apply_log_ext():
New functions.

page_zip_compress_clust(), page_zip_apply_log(): Check rec_offs_any_extern()
and avoid invoking the costly loop in most cases.

Similar optimizations can be made in page_zip_decompress_clust() and
page_zip_write_rec().
2007-02-05 11:19:25 +00:00
marko
8ef3075630 branches/zip: Speed up rec_offs_any_extern() and rec_offs_n_extern()
by adding the REC_OFFS_EXTERNAL flag to rec_offs_base(offsets)[0].

This reduces the processor usage of page_zip_write_rec() by about 40%
in one test case.  The code could be sped up further by testing
rec_offs_any_extern() outside of loops that check rec_offs_nth_extern().
The vast majority of records does not contain any externally stored columns.
2007-02-02 16:20:48 +00:00
marko
645b392294 branches/zip: Merge revisions 1206:1271 from trunk. 2007-02-02 11:31:29 +00:00
marko
b09ce29aeb branches/zip: Simplify lock_mode_stronger_or_eq() and lock_mode_compatible()
and introduce enum lock_mode.

lock_mode_stronger_or_eq(), lock_mode_compatible(): Replace if-then-else
chain with a bitwise and against a constant.
2007-02-01 20:56:23 +00:00
marko
fad9e98767 branches/zip: Split the function page_cur_insert_rec_low().
page_cur_insert_rec_zip_reorg(): New function: Recompress or
reorganize a compressed page.

page_cur_insert_rec_zip(): New function: insert a record to
a compressed page.

page_cur_insert_rec_low(): Only handle inserts to uncompressed pages.
2007-02-01 09:53:26 +00:00
marko
6fa5b727d8 branches/zip: Minor optimization.
trx_undo_page_report_modify(), trx_undo_report_row_operation():
Add const qualifier to the parameter rec.  Remove some local variables.

trx_undo_report_row_operation(): Invoke rec_get_offsets() only once.
2007-01-31 15:11:28 +00:00
marko
9a94888273 branches/zip: btr_search_guess_on_hash(): Eliminate two local variables
and optimize one test.
2007-01-31 14:28:37 +00:00
marko
36734f3420 branches/zip: btr_cur_optimistic_insert(): Optimize some tests,
eliminate some local variables, and add branching hints.
2007-01-31 14:12:57 +00:00
marko
463aaf28ba branches/zip: Avoid memory fragmentation when adding column definitions
to tables.

dict_mem_table_add_col(): Add the parameter "heap" for temporary memory
allocation.  Allow it and "name" to be NULL.  These parameters are NULL
when creating dummy indexes.

dict_add_col_name(): Remove calls to ut_malloc() and ut_free().

dict_table_get_col_name(): Allow table->col_names to be NULL.

dict_table_add_system_columns(), dict_table_add_to_cache():
Add the parameter "heap".
2007-01-30 09:24:18 +00:00
marko
6e2c59d421 branches/zip: Make zlib use InnoDB memory heaps when processing BLOBs.
page_zip_set_alloc(): New function, to configure zlib to use a memory heap.

btr_store_big_rec_extern_fields(): Reduce memLevel, so that a heap of
256 kilobytes will suffice.  Use page_zip_set_alloc().

btr_copy_externally_stored_field_prefix_low(): Use page_zip_set_alloc().

page_zip_compress(), page_zip_decompress(): Use page_zip_set_alloc().
2007-01-29 08:51:20 +00:00
marko
73bb34156f branches/zip: Add UNIV_HASH_DEBUG for detecting multiple invocations
of HASH_DELETE().

HASH_ASSERT_VALID(), HASH_INVALIDATE(): New macros, used in HASH_DELETE()
and HASH_SEARCH().

buf_page_init_low(): Add HASH_INVALIDATE(bpage, hash);

buf_page_init(): Call buf_page_init_low() before HASH_INSERT(), so that
bpage->hash will not be invalidated.
2007-01-24 11:41:19 +00:00
marko
ae0ecb11aa branches/zip: buf_page_t: Add the in_page_hash and in_zip_hash
for extra validation in UNIV_DEBUG builds.
2007-01-24 10:36:05 +00:00
marko
87defea2bb branches/zip: buf_page_init_for_read(): Decrement, do not reset
block->page.buf_fix_count.  This mistake was made in r1255.
When access to the block is requested by some other thread,
buf_fix_count can be incremented during the execution of
buf_zip_decompress().
2007-01-23 16:12:14 +00:00
marko
f0154795ad branches/zip: lock0lock.c: Minor cleanup.
Replace some while loops with do..while or for loops.

lock_move_rec_list_start(): Replace 2 with PAGE_HEAP_NO_USER_LOW.
2007-01-23 14:10:48 +00:00
marko
7ebd17e0e4 branches/zip: buf_page_init_for_read(): When decompressing a compressed block,
initialize all the fields of the control block.

buf_block_init_low(), buf_page_init_low(): New functions, used in
buf_page_init_for_backup_restore(), buf_page_init(), and
buf_page_init_for_read().
2007-01-23 14:08:37 +00:00
marko
208421f186 branches/zip: lock0lock.c: Add UNIV_UNLIKELY hints to LOCK_WAIT checks.
Declare some local variables in narrower scope.

lock_move_rec_list_start(): Check for orphan locks #ifdef UNIV_DEBUG
2007-01-23 10:02:02 +00:00
marko
4b43c8968a branches/zip: lock_move_reorganize_page(): Move the UNIV_DEBUG check for
orphaned locks outside the "for" loop.  This mistake was made in r1252.
2007-01-22 15:46:27 +00:00
marko
73ea67b0e8 branches/zip: lock_move_reorganize_page(): Clean up the code a little,
and implement UNIV_DEBUG diagnostics for orphaned locks.
2007-01-22 14:44:34 +00:00
marko
63055ee052 branches/zip: Enclose ut_ad(lock_rec_validate_page(...))
inside #ifdef UNIV_DEBUG_LOCK_VALIDATE.

lock_rec_validate_page(): Assert that page_find_rec_with_heap_no()
returns non-NULL.
2007-01-22 10:24:21 +00:00
marko
96de1cd1de branches/zip: Enable assertions on ut_ad(lock_rec_validate_page()).
lock_rec_get_nth_bit(), lock_rec_set_nth_bit(), lock_rec_reset_nth_bit(),
lock_rec_copy(): Simplify the code.
2007-01-22 08:48:00 +00:00
marko
9ef14edbce branches/zip: Fix a bug in the buddy allocator.
buf_buddy_relocate_block(): New function, split from buf_buddy_relocate().
Return FALSE when buf_page_can_relocate() returns FALSE.

Fix some formatting in buf_buddy_free_low() and buf_page_init_for_read().
2007-01-19 12:27:47 +00:00
marko
adbd5c3a85 branches/zip: When retrieving compressed BLOBs, avoid allocating
uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().
2007-01-18 23:10:49 +00:00
marko
f56b6cf4cb branches/zip: Fix bugs.
buf_LRU_free_block(): Do not allocate the control block when
a compressed-only page is being freed.

buf_validate(): Note that buf_pool->zip_clean may contain blocks
with io_fix == BUF_IO_READ.
2007-01-18 20:29:35 +00:00
marko
5fc003b65d branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG. 2007-01-18 18:29:12 +00:00
marko
ca662b43e6 branches/zip: buf_LRU_free_block(): Allocate a control block for the
compressed page also when freeing an unmodified uncompressed page.
2007-01-18 15:43:59 +00:00
marko
ffc0bc3eaf branches/zip: Free buffer blocks that are no longer needed for BLOB storage.
btr_blob_free(): New function to commit a mini-transaction and to free
an uncompressed BLOB block, or the entire block.

btr_store_big_rec_extern_fields(): Replace the existing code with
btr_blob_free().  The old code may have contained a race condition.

btr_free_externally_stored_field(): Completely free the buffer blocks
allocated for the freed BLOB.
2007-01-18 14:02:56 +00:00
marko
8f65d7a661 branches/zip: Merge revisions 1165:1206 from trunk. 2007-01-18 12:58:39 +00:00
marko
f116afa724 branches/zip: Remove the fil_space_get_zip_size() call from
buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.
2007-01-18 09:59:00 +00:00
marko
fd687bbedd branches/zip: btr_cur_pessimistic_delete(): Correct a comment. 2007-01-18 08:35:57 +00:00
marko
ccbb93c016 branches/zip: btr_copy_externally_stored_field_prefix_low():
Split the code to subroutines.

btr_copy_blob_prefix(): New function: copy the prefix of the externally
stored part of an uncompressed BLOB.

btr_copy_zblob_prefix(): New function: copy the prefix of the externally
stored part of a compressed BLOB.

btr_blob_get_part_len(), btr_blob_get_next_page_no(): Add const qualifier
to the parameter.
2007-01-17 09:07:20 +00:00
marko
d3d728955b branches/zip: buf_flush_write_block_low(): When flushing a compressed page
that also exists in uncompressed form in the buffer pool, pass the
compressed frame to fil_io() instead of the uncompressed one when
not using the doublewrite buffer.
2007-01-16 22:19:29 +00:00
marko
e77a82a054 branches/zip: buf_flush_write_block_low(): Avoid recomputing the compressed
page checksum of compressed-only blocks.  Pass the compressed page frame
to fil_io() when needed.

page_zip_calc_checksum(): Skip also FIL_PAGE_LSN and FIL_PAGE_FILE_FLUSH_LSN.

buf_flush_init_for_writing(): Expect page to be non-NULL.
2007-01-16 21:51:40 +00:00
marko
a85fd29415 branches/zip: Allow dirty compressed-only blocks to exist in the buffer pool
and to be flushed to disk.

buf_LRU_free_block(): Enable the freeing of uncompressed pages of
compressed tablespaces.

trx_doublewrite->buf_block_arr[]: Change the type from buf_block_t*
to buf_page_t*.

buf_flush_ready_for_flush(): Add debug assertion.

buf_flush_buffered_writes(), buf_flush_try_page(): Support blocks of type
BUF_BLOCK_ZIP_DIRTY.

buf_flush_post_to_doublewrite_buf(), buf_flush_write_block_low():
Change the type of the parameter from buf_block_t* to buf_page_t*.

buf_flush_init_for_writing(): Allow page to be NULL if page_zip_ is non-NULL.
2007-01-16 18:02:49 +00:00
marko
94e49a71b2 branches/zip: Note that b (the newly created control block for the
compressed page) is not protected by block_mutex.  Before releasing
buf_pool->mutex, we can simply assign b->io_fix.  After reacquiring
buf_pool->mutex, we also have to acquire buf_pool->zip_mutex.
2007-01-16 16:39:53 +00:00
marko
9fe82e80e3 branches/zip: buf_LRU_free_block(): When preserving the compressed page
of a block, do not release buf_pool->mutex between the time the old control
block is removed from buf_pool->page_hash and the new control block is
added to it.  Prevent operations on the compressed-only block while calling
btr_search_drop_page_hash_index() or page_zip_calc_checksum().

buf_LRU_invalidate_tablespace(): Revert the change done in r1223.
buf_pool->zip_mutex will have been released by
buf_LRU_block_remove_hashed_page() when it returns BUF_BLOCK_ZIP_FREE.
2007-01-16 16:24:47 +00:00
marko
c3599a3ab1 branches/zip: buf_flush_remove(): buf_page_get_mutex(bpage) must have been
acquired by the caller.

buf_LRU_invalidate_tablespace(): Do not leak buf_pool->zip_mutex when
rescanning the LRU list.
2007-01-16 14:37:40 +00:00
marko
91546e7d10 branches/zip: buf_LRU_free_block(): When freeing the uncompressed page
of a dirty block, relocate the control blocks on buf_pool->flush_list
instead of calling buf_flush_insert_into_flush_list().
2007-01-16 14:11:21 +00:00