Commit graph

517 commits

Author SHA1 Message Date
marko
b6452b4182 branches/zip: btr_cur_insert_if_possible(): Update the free bits on
compressed secondary index pages after attempting the insert.

btr_cur_update_alloc_zip(): Reset the insert buffer free bits if there is
insufficient space available on secondary index pages.  Do not update
the bits after successful operation.

btr_cur_update_in_place(): Update the free bits in the index buffer.

btr_cur_optimistic_update(): First check rec_offs_any_extern(), because
checking it is cheaper than checking the "external storage" flags of the
update vector.  Update the insert buffer free bits in any case after
successfully calling btr_cur_update_alloc_zip().
2007-02-27 07:25:24 +00:00
marko
22ad99fb0c branches/zip: Update the insert buffer free bits when recompressing
a page during update-in-place.

page_zip_alloc(): Rename to btr_cur_update_alloc_zip().
Invoke ibuf_update_free_bits_if_full().
2007-02-26 09:35:02 +00:00
marko
705dfc7cfd branches/zip: Many places: Avoid re-reorganizing compressed pages after
failing insert.  Reorganization will have been attempted in
page_cur_tuple_insert() or page_cur_rec_insert().

page_zip_reorganize(): Recompute the insert buffer free bits for
leaf pages of secondary indexes.

ibuf_data_enough_free_for_insert(): Simplify.
2007-02-20 15:01:47 +00:00
marko
e08004c553 branches/zip: page_cur_rec_field_extends(): Fix compilation errors that
were introduced when reducing the memory footprint of the data dictionary
cache (Bug #20877), around r834.  This function is only compiled if
PAGE_CUR_LE_OR_EXTENDS is defined in page0cur.h.
2007-02-20 13:50:39 +00:00
marko
3e463b8bc0 branches/zip: Fix bugs related to the insert buffering of
compressed tablespaces.

btr_page_reorganize_low(): Recompute the insert buffer free bits
for compressed pages belonging to a non-clustered index.

btr_cur_optimistic_insert(): Determine if the page was reorganized by
page_cur_tuple_insert(). Do not attempt to reorganize compressed pages,
because that will be attempted by page_cur_tuple_insert().
2007-02-20 13:36:17 +00:00
marko
df5da79d26 branches/zip: btr_cur_optimistic_insert(): If btr_page_reorganize()
fails, reset the insert buffer free bits for the page.
2007-02-20 11:30:13 +00:00
marko
37b0f7bf1c branches/zip: Enable the insert buffer on compressed tablespaces.
page_zip_max_ins_size(): New function.

btr_cur_optimistic_insert(), btr_cur_optimistic_delete(),
btr_page_split_and_insert(), btr_compress(): Do not update the
ibuf free bits for non-leaf pages or pages belonging to a clustered index.
The insert buffer only covers operations on leaf pages of secondary indexes.
For pages covered by the insert buffer, limit the max_ins_size to
page_zip_max_ins_size().

buf_page_get_gen(): Merge the insert buffer after decompressing the page.

buf_page_io_complete(): Relax the assertion about ibuf_count.  For
compressed-only pages, the insert buffer merge takes place
in buf_page_get_gen().

ibuf_index_page_calc_free_bits(), ibuf_index_page_calc_free_from_bits(),
ibuf_index_page_calc_free(), ibuf_update_free_bits_if_full(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(),
ibuf_set_free_bits_low(): Add the parameter zip_size.  Limit the maximum
insert size to page_zip_max_ins_size().
2007-02-19 20:32:06 +00:00
marko
fed27de38b branches/zip: page_zip_get_trailer_len(), page_zip_available():
Replace the parameter "dict_index_t* index" with "ibool is_clust".

dict_index_is_clust(): Add __attribute__((pure)).
2007-02-19 14:51:44 +00:00
marko
9b861942ac branches/zip: buf_LRU_search_and_free_block(): Do not try to preserve
compressed pages, because doing so would require a call to
buf_buddy_alloc(), which in turn can invoke buf_LRU_get_free_block()
and buf_LRU_search_and_free_block(), potentially infinite recursion.
2007-02-19 14:23:32 +00:00
marko
d56453be8e branches/zip: buf_LRU_search_and_free_block(): Avoid freeing compressed-only
pages when they do not occupy too much of the buffer pool.

buf0buddy.c, buf0buddy.h: Export buf_buddy_n_frames and buf_buddy_min_n_frames.
2007-02-16 09:22:50 +00:00
marko
e67d5c761c branches/zip: Add statistic counters to the buddy allocator.
buf_buddy_n_frames: Number of buffer frames allocated for the buddy system
buf_buddy_used[]: Counts of allocated blocks, by buddy size.
buf_buddy_relocated[]: Counts of blocks relocated, bu buddy size.

buf_buddy_alloc_clean(): Note the added configuration parameter
buf_buddy_min_n_frames: Preferred minimum buf_buddy_n_frames
2007-02-13 12:41:02 +00:00
marko
1a103f0324 branches/zip: buf_LRU_get_free_block(): Add an UNIV_UNLIKELY hint
around zip_size.  zip_size will be 0 most of the time.
2007-02-13 11:21:57 +00:00
marko
d3a813fd69 branches/zip: buf_page_get_gen(): After allocating an uncompressed page for
decompression, check that the compressed block has not been
buffer-fixed or I/O-fixed.  This race condition was introduced in r1286.
2007-02-13 11:15:27 +00:00
marko
745404846f branches/zip: btr_cur_optimistic_insert(): After dtuple_convert_big_rec(),
invoke btr_cur_add_ext() so that rec_size will be computed correctly.
Prepare to free the memory heap when handling errors.
2007-02-12 12:04:49 +00:00
marko
eb03297439 branches/zip: buf_page_init_for_read(): Defer the costly calls to
buf_buddy_alloc() until after the block has been found not to exist
in the buffer pool.

buf_buddy_alloc(), buf_buddy_alloc_low(): Make lru an output parameter.
Improve documentation.

buf_page_init_for_read(): Simplify the code.  Check buf_page_hash_get()
again if buf_buddy_alloc() released buf_pool->mutex.
2007-02-08 14:19:55 +00:00
marko
b36300e1ba branches/zip: buf_page_create(): Do not IO-fix or X-latch the block
when not invoking buf_buddy_alloc().
2007-02-08 12:33:54 +00:00
marko
5ac6f3217d branches/zip: buf_page_init_for_read(): Do not allocate an uncompressed page
frame when reading compressed pages, unless crash recovery is in progress.

buf_page_read_low(): Adapt for buf_page_init_for_read() returning
buf_page_t* instead of buf_block_t*.
2007-02-08 12:03:03 +00:00
marko
7315b06291 branches/zip: buf_page_init_for_read(), buf_page_create(): Note in the
function comments that buf_page_init_for_read() does not perform the
state transition to FILE_PAGE.  This should have been done in r1286.
2007-02-08 09:08:11 +00:00
marko
329c469e70 branches/zip: Do not decompress blocks in buf_page_init_for_read(),
but in buf_page_get_gen().  Adjust comments.  This prevents
unnecessary decompression in read-ahead when the compressed block
exists in the buffer pool.

If the block does not exist in the buffer pool, read-ahead will still
allocate an uncompressed page and decompress the block.

Move buf_block_init_low() and buf_zip_decompress() earlier in the file,
because some compilers are unable to inline functions that are defined
after the invocation.

buf_page_get_gen(): Decompress the block if needed.

buf_page_init_for_read(): Do not decompress.
2007-02-07 13:59:43 +00:00
marko
8edd94a767 branches/zip: page_zip_write_rec(): Treat records containing externally
stored columns as a special case.

page_zip_write_rec_ext(): New function for writing records containing
externally stored columns.
2007-02-05 14:05:39 +00:00
marko
a18381a9ca branches/zip: page_zip_decompress_clust(): Optimize the case when the
record does not contain externally stored columns.

page_zip_decompress_clust_ext(): New function for decompressing records
containing externally stored columns.
2007-02-05 13:20:00 +00:00
marko
0af6d909ba branches/zip: ha_innobase::create(): Do not test create_info->used_fields
before testing create_info->avg_row_length, because otherwise the table
would be converted to uncompressed format by CREATE INDEX and possibly
other statements.
2007-02-05 12:17:11 +00:00
marko
a53aa6ac07 branches/zip: Introduce separate functions for handling records containing
externally stored columns.

page_zip_compress_clust_ext(), page_zip_apply_log_ext():
New functions.

page_zip_compress_clust(), page_zip_apply_log(): Check rec_offs_any_extern()
and avoid invoking the costly loop in most cases.

Similar optimizations can be made in page_zip_decompress_clust() and
page_zip_write_rec().
2007-02-05 11:19:25 +00:00
marko
e45f4cdaeb branches/zip: Speed up rec_offs_any_extern() and rec_offs_n_extern()
by adding the REC_OFFS_EXTERNAL flag to rec_offs_base(offsets)[0].

This reduces the processor usage of page_zip_write_rec() by about 40%
in one test case.  The code could be sped up further by testing
rec_offs_any_extern() outside of loops that check rec_offs_nth_extern().
The vast majority of records does not contain any externally stored columns.
2007-02-02 16:20:48 +00:00
marko
16ffd00dc1 branches/zip: Merge revisions 1206:1271 from trunk. 2007-02-02 11:31:29 +00:00
marko
35f83441a0 branches/zip: Simplify lock_mode_stronger_or_eq() and lock_mode_compatible()
and introduce enum lock_mode.

lock_mode_stronger_or_eq(), lock_mode_compatible(): Replace if-then-else
chain with a bitwise and against a constant.
2007-02-01 20:56:23 +00:00
marko
f05b0766d9 branches/zip: Split the function page_cur_insert_rec_low().
page_cur_insert_rec_zip_reorg(): New function: Recompress or
reorganize a compressed page.

page_cur_insert_rec_zip(): New function: insert a record to
a compressed page.

page_cur_insert_rec_low(): Only handle inserts to uncompressed pages.
2007-02-01 09:53:26 +00:00
marko
10bb43856c branches/zip: Minor optimization.
trx_undo_page_report_modify(), trx_undo_report_row_operation():
Add const qualifier to the parameter rec.  Remove some local variables.

trx_undo_report_row_operation(): Invoke rec_get_offsets() only once.
2007-01-31 15:11:28 +00:00
marko
853fe3537f branches/zip: btr_search_guess_on_hash(): Eliminate two local variables
and optimize one test.
2007-01-31 14:28:37 +00:00
marko
e773660dab branches/zip: btr_cur_optimistic_insert(): Optimize some tests,
eliminate some local variables, and add branching hints.
2007-01-31 14:12:57 +00:00
marko
0178dba161 branches/zip: Avoid memory fragmentation when adding column definitions
to tables.

dict_mem_table_add_col(): Add the parameter "heap" for temporary memory
allocation.  Allow it and "name" to be NULL.  These parameters are NULL
when creating dummy indexes.

dict_add_col_name(): Remove calls to ut_malloc() and ut_free().

dict_table_get_col_name(): Allow table->col_names to be NULL.

dict_table_add_system_columns(), dict_table_add_to_cache():
Add the parameter "heap".
2007-01-30 09:24:18 +00:00
marko
4b07b77f5a branches/zip: Make zlib use InnoDB memory heaps when processing BLOBs.
page_zip_set_alloc(): New function, to configure zlib to use a memory heap.

btr_store_big_rec_extern_fields(): Reduce memLevel, so that a heap of
256 kilobytes will suffice.  Use page_zip_set_alloc().

btr_copy_externally_stored_field_prefix_low(): Use page_zip_set_alloc().

page_zip_compress(), page_zip_decompress(): Use page_zip_set_alloc().
2007-01-29 08:51:20 +00:00
marko
13efcc13fc branches/zip: Add UNIV_HASH_DEBUG for detecting multiple invocations
of HASH_DELETE().

HASH_ASSERT_VALID(), HASH_INVALIDATE(): New macros, used in HASH_DELETE()
and HASH_SEARCH().

buf_page_init_low(): Add HASH_INVALIDATE(bpage, hash);

buf_page_init(): Call buf_page_init_low() before HASH_INSERT(), so that
bpage->hash will not be invalidated.
2007-01-24 11:41:19 +00:00
marko
3b5c872916 branches/zip: buf_page_t: Add the in_page_hash and in_zip_hash
for extra validation in UNIV_DEBUG builds.
2007-01-24 10:36:05 +00:00
marko
6473df1f55 branches/zip: buf_page_init_for_read(): Decrement, do not reset
block->page.buf_fix_count.  This mistake was made in r1255.
When access to the block is requested by some other thread,
buf_fix_count can be incremented during the execution of
buf_zip_decompress().
2007-01-23 16:12:14 +00:00
marko
46e544a082 branches/zip: lock0lock.c: Minor cleanup.
Replace some while loops with do..while or for loops.

lock_move_rec_list_start(): Replace 2 with PAGE_HEAP_NO_USER_LOW.
2007-01-23 14:10:48 +00:00
marko
9dca485c35 branches/zip: buf_page_init_for_read(): When decompressing a compressed block,
initialize all the fields of the control block.

buf_block_init_low(), buf_page_init_low(): New functions, used in
buf_page_init_for_backup_restore(), buf_page_init(), and
buf_page_init_for_read().
2007-01-23 14:08:37 +00:00
marko
14ed331720 branches/zip: lock0lock.c: Add UNIV_UNLIKELY hints to LOCK_WAIT checks.
Declare some local variables in narrower scope.

lock_move_rec_list_start(): Check for orphan locks #ifdef UNIV_DEBUG
2007-01-23 10:02:02 +00:00
marko
cdcffb28ca branches/zip: lock_move_reorganize_page(): Move the UNIV_DEBUG check for
orphaned locks outside the "for" loop.  This mistake was made in r1252.
2007-01-22 15:46:27 +00:00
marko
2ac3bfd883 branches/zip: lock_move_reorganize_page(): Clean up the code a little,
and implement UNIV_DEBUG diagnostics for orphaned locks.
2007-01-22 14:44:34 +00:00
marko
d48c3388df branches/zip: Enclose ut_ad(lock_rec_validate_page(...))
inside #ifdef UNIV_DEBUG_LOCK_VALIDATE.

lock_rec_validate_page(): Assert that page_find_rec_with_heap_no()
returns non-NULL.
2007-01-22 10:24:21 +00:00
marko
ecff2cb1bb branches/zip: Enable assertions on ut_ad(lock_rec_validate_page()).
lock_rec_get_nth_bit(), lock_rec_set_nth_bit(), lock_rec_reset_nth_bit(),
lock_rec_copy(): Simplify the code.
2007-01-22 08:48:00 +00:00
marko
f5652a6a1e branches/zip: Fix a bug in the buddy allocator.
buf_buddy_relocate_block(): New function, split from buf_buddy_relocate().
Return FALSE when buf_page_can_relocate() returns FALSE.

Fix some formatting in buf_buddy_free_low() and buf_page_init_for_read().
2007-01-19 12:27:47 +00:00
marko
88efbdb335 branches/zip: When retrieving compressed BLOBs, avoid allocating
uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().
2007-01-18 23:10:49 +00:00
marko
22147cce7c branches/zip: Fix bugs.
buf_LRU_free_block(): Do not allocate the control block when
a compressed-only page is being freed.

buf_validate(): Note that buf_pool->zip_clean may contain blocks
with io_fix == BUF_IO_READ.
2007-01-18 20:29:35 +00:00
marko
d8536b0de8 branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG. 2007-01-18 18:29:12 +00:00
marko
fc5428caf3 branches/zip: buf_LRU_free_block(): Allocate a control block for the
compressed page also when freeing an unmodified uncompressed page.
2007-01-18 15:43:59 +00:00
marko
bee672c0e1 branches/zip: Free buffer blocks that are no longer needed for BLOB storage.
btr_blob_free(): New function to commit a mini-transaction and to free
an uncompressed BLOB block, or the entire block.

btr_store_big_rec_extern_fields(): Replace the existing code with
btr_blob_free().  The old code may have contained a race condition.

btr_free_externally_stored_field(): Completely free the buffer blocks
allocated for the freed BLOB.
2007-01-18 14:02:56 +00:00
marko
c712838401 branches/zip: Merge revisions 1165:1206 from trunk. 2007-01-18 12:58:39 +00:00
marko
fff329cd3d branches/zip: Remove the fil_space_get_zip_size() call from
buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.
2007-01-18 09:59:00 +00:00