Commit graph

127 commits

Author SHA1 Message Date
marko
cf2cce5dfd branches/zip: Use mem_calloc and mem_heap_calloc when possible.
row_ext_create(): After a memset() inside UNIV_DEBUG, flag the memory
uninitialized with UNIV_MEM_ALLOC().
2007-08-13 14:07:59 +00:00
marko
fbdd5b437c branches/zip: btr_cur_mark_dtuple_inherited_extern(): Initialize len.
This was broken in r1591.
2007-08-01 07:53:27 +00:00
marko
5288876e23 branches/zip: Make merge sort handle externally stored columns.
Some things still fail in innodb-index.test, and there seems to be
a race condition (data dictionary lock wait) when running with --valgrind.

dfield_t: Add an "external storage" flag, dfield->ext.

dfield_is_null(), dfield_is_ext(), dfield_set_ext(), dfield_set_null():
New functions.

dfield_copy(), dfield_copy_data(): Add const qualifiers, fix in/out comments.

data_write_sql_null(): Use memset().

big_rec_field_t: Replace byte* data with const void* data.

ut_ulint_sort(): Remove.

upd_field_t: Remove extern_storage.

upd_node_t: Replace ext_vec, n_ext_vec with n_ext.

row_merge_copy_blobs(): New function.

row_ins_index_entry(): Add the parameter "ibool foreign" for suppressing
foreign key checks during fast index creation or when inserting into
secondary indexes.

btr_page_insert_fits(): Add const qualifiers.

btr_cur_add_ext(), upd_ext_vec_contains(): Remove.

dfield_print_also_hex(), dfield_print(): Replace if...else if with switch.
Observe dfield_is_ext().
2007-06-21 09:43:15 +00:00
marko
d7dbb4608c branches/zip: btr_rec_copy_externally_stored_field(): Add const qualifier
to rec_t*.  Remove the ut_ad(rec_offs_validate()), because this function
will be called from row0merge.c on a record that lacks the
REC_N_NEW_EXTRA_BYTES.
2007-06-21 08:58:41 +00:00
marko
d62b28beeb branches/zip: Add some const qualifiers, mainly to upd_t*.
upd_get_n_fields(), upd_get_nth_field(): Add const qualifiers.
2007-06-19 12:44:45 +00:00
marko
6613d53767 branches/zip: btr_cur_optimistic_insert(): Set the insert buffer free bits
exactly.  Rename the local variable "ulint level" to "ibool leaf".
Document that if the function returns DB_SUCCESS on a compressed page that
is covered by the insert buffer, the mini-transaction must be committed
before latching any further pages.  Verify that this is the case on all
execution paths.
2007-05-16 12:01:31 +00:00
marko
36e875f7a5 branches/zip: Document and obey the rules for modifying the free bits in
the insert buffer bitmap.

ibuf_set_free_bits_func(): Never disable redo logging.

ibuf_update_free_bits_zip(): Remove.

btr_page_reorganize_low(), page_zip_reorganize(): Do not update the insert
buffer bitmap.  Instead, document that callers will have to take care of it,
and adapt the callers.

btr_compress(): On error, reset the insert buffer free bits.

btr_cur_insert_if_possible(): Do not modify the insert buffer bitmap.

btr_compress(), btr_cur_optimistic_insert(): On compressed pages,
reset the insert buffer bitmap.  Document why.

btr_cur_update_alloc_zip(): Document why it is necessary and sufficient
to reset the insert buffer free bits.

btr_cur_update_in_place(), btr_cur_optimistic_update(),
btr_cur_pessimistic_update(): Update the free bits in the same
mini-transaction.  Document that the mini-transaction must be
committed before latching any further pages.  Verify that this
is the case in all execution paths.

row_ins_sec_index_entry_by_modify(), row_ins_clust_index_entry_by_modify(),
row_undo_mod_clust_low(): Because these functions call
btr_cur_update_in_place(), btr_cur_optimistic_update(), or
btr_cur_pessimistic_update(), document that the mini-transaction must be
committed before latching any further pages.  Verify that this is the case
in all execution paths.
2007-05-16 09:23:53 +00:00
marko
2e73922495 branches/zip: ibuf_set_free_bits_func(): Disable redo logging when
crash recovery is in progress.  This avoids a hang when
btr_parse_page_reorganize(), called from an I/O handler thread,
attempts to acquire log_sys->mutex while it is being held by
the main thread (the one that runs innobase_init()).  This change
was committed accidentally.  It may be unsafe to clear
mtr.modifications, because buf_page_release() at mtr_commit() may
forget to put modified pages to the flush list.

Cleanup: Remove the "type" parameter from many ibuf functions.
Let the caller check that !dict_index_is_clust().  This should avoid
function calls and register spilling.

ibuf_set_free_bits_func(), ibuf_set_free_bits(): Remove the parameter "type".

ibuf_reset_free_bits_with_type(): Rename to ibuf_reset_free_bits().
Remove the parameter "type".

ibuf_update_free_bits_if_full(), ibuf_update_free_bits_zip(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low():
Remove the parameter "index".
2007-05-06 12:39:46 +00:00
marko
750e130ce9 branches/zip: btr_cur_pessimistic_update(): Add parameter heap.
Previously, when big_rec was returned, the fields would point to
freed memory.  The memory heap was allocated locally, and the data tuple
was allocated from the heap, and the big_rec would point to some fields
in the data tuple.

row_ins_clust_index_entry_by_modify(): Add parameter heap,
for the same reason.
2007-03-28 19:35:52 +00:00
marko
6e8da36bba branches/zip: Minor cleanup.
btr_rec_free_updated_extern_fields(): Remove the parameter
do_not_free_inherited, which is always passed as TRUE.

btr_cur_unmark_extern_fields(): Invoke rec_offs_any_extern().

btr_cur_unmark_dtuple_extern_fields(): Remove a local variable.
2007-03-14 12:34:55 +00:00
marko
9311391ec2 branches/zip: ibuf_update_free_bits_if_full(): Remove the zip_size
parameter.  Add ibuf_update_free_bits_zip() for compressed pages.
2007-03-01 11:28:30 +00:00
marko
79d6b61fb9 btr_cur_optimistic_insert(): When inserting into a compressed page,
pass max_ins_size=UNIV_PAGE_SIZE and increase=ULINT_UNDEFINED to
ibuf_update_free_bits_if_full().  Otherwise, a bogus assertion failure
may occur when the page is recompressed and there is more space available
in the modification log after inserting the record.
2007-02-28 13:19:13 +00:00
marko
1b6e514342 branches/zip: Note that page_cur_tuple_insert() can fail without attempting
to reorganize a compressed page.  This mistake was made in r1308.
2007-02-28 11:08:59 +00:00
marko
58c45a0695 branches/zip: Prepare for a negative return from page_zip_max_ins_size(). 2007-02-27 10:21:44 +00:00
marko
b6452b4182 branches/zip: btr_cur_insert_if_possible(): Update the free bits on
compressed secondary index pages after attempting the insert.

btr_cur_update_alloc_zip(): Reset the insert buffer free bits if there is
insufficient space available on secondary index pages.  Do not update
the bits after successful operation.

btr_cur_update_in_place(): Update the free bits in the index buffer.

btr_cur_optimistic_update(): First check rec_offs_any_extern(), because
checking it is cheaper than checking the "external storage" flags of the
update vector.  Update the insert buffer free bits in any case after
successfully calling btr_cur_update_alloc_zip().
2007-02-27 07:25:24 +00:00
marko
22ad99fb0c branches/zip: Update the insert buffer free bits when recompressing
a page during update-in-place.

page_zip_alloc(): Rename to btr_cur_update_alloc_zip().
Invoke ibuf_update_free_bits_if_full().
2007-02-26 09:35:02 +00:00
marko
705dfc7cfd branches/zip: Many places: Avoid re-reorganizing compressed pages after
failing insert.  Reorganization will have been attempted in
page_cur_tuple_insert() or page_cur_rec_insert().

page_zip_reorganize(): Recompute the insert buffer free bits for
leaf pages of secondary indexes.

ibuf_data_enough_free_for_insert(): Simplify.
2007-02-20 15:01:47 +00:00
marko
3e463b8bc0 branches/zip: Fix bugs related to the insert buffering of
compressed tablespaces.

btr_page_reorganize_low(): Recompute the insert buffer free bits
for compressed pages belonging to a non-clustered index.

btr_cur_optimistic_insert(): Determine if the page was reorganized by
page_cur_tuple_insert(). Do not attempt to reorganize compressed pages,
because that will be attempted by page_cur_tuple_insert().
2007-02-20 13:36:17 +00:00
marko
df5da79d26 branches/zip: btr_cur_optimistic_insert(): If btr_page_reorganize()
fails, reset the insert buffer free bits for the page.
2007-02-20 11:30:13 +00:00
marko
37b0f7bf1c branches/zip: Enable the insert buffer on compressed tablespaces.
page_zip_max_ins_size(): New function.

btr_cur_optimistic_insert(), btr_cur_optimistic_delete(),
btr_page_split_and_insert(), btr_compress(): Do not update the
ibuf free bits for non-leaf pages or pages belonging to a clustered index.
The insert buffer only covers operations on leaf pages of secondary indexes.
For pages covered by the insert buffer, limit the max_ins_size to
page_zip_max_ins_size().

buf_page_get_gen(): Merge the insert buffer after decompressing the page.

buf_page_io_complete(): Relax the assertion about ibuf_count.  For
compressed-only pages, the insert buffer merge takes place
in buf_page_get_gen().

ibuf_index_page_calc_free_bits(), ibuf_index_page_calc_free_from_bits(),
ibuf_index_page_calc_free(), ibuf_update_free_bits_if_full(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(),
ibuf_set_free_bits_low(): Add the parameter zip_size.  Limit the maximum
insert size to page_zip_max_ins_size().
2007-02-19 20:32:06 +00:00
marko
745404846f branches/zip: btr_cur_optimistic_insert(): After dtuple_convert_big_rec(),
invoke btr_cur_add_ext() so that rec_size will be computed correctly.
Prepare to free the memory heap when handling errors.
2007-02-12 12:04:49 +00:00
marko
e773660dab branches/zip: btr_cur_optimistic_insert(): Optimize some tests,
eliminate some local variables, and add branching hints.
2007-01-31 14:12:57 +00:00
marko
4b07b77f5a branches/zip: Make zlib use InnoDB memory heaps when processing BLOBs.
page_zip_set_alloc(): New function, to configure zlib to use a memory heap.

btr_store_big_rec_extern_fields(): Reduce memLevel, so that a heap of
256 kilobytes will suffice.  Use page_zip_set_alloc().

btr_copy_externally_stored_field_prefix_low(): Use page_zip_set_alloc().

page_zip_compress(), page_zip_decompress(): Use page_zip_set_alloc().
2007-01-29 08:51:20 +00:00
marko
88efbdb335 branches/zip: When retrieving compressed BLOBs, avoid allocating
uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().
2007-01-18 23:10:49 +00:00
marko
bee672c0e1 branches/zip: Free buffer blocks that are no longer needed for BLOB storage.
btr_blob_free(): New function to commit a mini-transaction and to free
an uncompressed BLOB block, or the entire block.

btr_store_big_rec_extern_fields(): Replace the existing code with
btr_blob_free().  The old code may have contained a race condition.

btr_free_externally_stored_field(): Completely free the buffer blocks
allocated for the freed BLOB.
2007-01-18 14:02:56 +00:00
marko
fff329cd3d branches/zip: Remove the fil_space_get_zip_size() call from
buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.
2007-01-18 09:59:00 +00:00
marko
65f5052455 branches/zip: btr_cur_pessimistic_delete(): Correct a comment. 2007-01-18 08:35:57 +00:00
marko
60c96ae359 branches/zip: btr_copy_externally_stored_field_prefix_low():
Split the code to subroutines.

btr_copy_blob_prefix(): New function: copy the prefix of the externally
stored part of an uncompressed BLOB.

btr_copy_zblob_prefix(): New function: copy the prefix of the externally
stored part of a compressed BLOB.

btr_blob_get_part_len(), btr_blob_get_next_page_no(): Add const qualifier
to the parameter.
2007-01-17 09:07:20 +00:00
marko
581a10d3f9 branches/zip: btr0cur.c: Add missing #include "buf0lru.h" that
should have been added in r1218.
2007-01-16 13:23:10 +00:00
marko
e96ed0fb5a branches/zip: btr_store_big_rec_extern_fields(): Release the uncompressed
page frame to save memory.
2007-01-16 11:56:33 +00:00
marko
82dfd0f6c7 branches/zip: page_zip_des_t: Rename "size" to "ssize" and reduce the
storage size from 16 to 3 bits.

page_zip_get_size(), page_zip_set_size(): New functions.

Replace direct references to page_zip_des_t:size with calls to
buf_block_get_zip_size(), page_zip_get_size(), and page_zip_set_size().
2006-11-27 13:44:32 +00:00
marko
a658b00597 branches/zip: When shrinking the buffer pool, disable and purge the adaptive
hash index, because it might occupy the chunk we would like to free.

TODO: In btr_search_check_free_space_in_heap(), release the block if
btr_search_latch is not immediately available.

buf_pool_shrink(): Split from buf_pool_resize().

btr_search_disabled: New variable, similar to srv_use_adaptive_hash_indexes
that was removed earlier.

btr_search_disable(): New function: disable and purge the adaptive hash index.

btr_search_enable(): New function: enable the adaptive hash index.

ha_clear(): New function: Empty a hash table and free the memory heaps.
2006-11-21 14:40:14 +00:00
marko
8478f3824c branches/zip: btr_store_big_rec_extern_fields(): Remove the incomplete
statement that was accidentally checked in in r1022.
2006-11-16 08:41:10 +00:00
marko
ebeee34510 btr_store_big_rec_extern_fields(): Write the (space_id, page_no) of the
record on every compressed BLOB page header, so that compressed BLOBs
can be relocated or the chain of BLOB pages can be validated.
2006-11-15 21:58:01 +00:00
marko
69fc883be2 branches/zip: Remove support for AWE (Intel PAE on Win32).
Removal of the variable innobase_buffer_pool_awe_mem_mb requires
coordination with MySQL AB.
2006-10-30 13:48:08 +00:00
marko
dfb8add7b3 branches/zip: btr_store_big_rec_extern_fields(): Replace
page_get_page_no(page_align(field_ref)) calls with a local variable
rec_page_no = buf_block_get_page_no(rec_block).
2006-10-26 08:47:00 +00:00
marko
9ac3df7687 branches/zip: Remove a few buf_block_align() calls from debug builds.
btr_check_node_ptr(): Replace page_t* parameter with buf_block_t*.

btr_free_externally_stored_field(): Add const qualifier to rec.
Remove an explicit buf_block_align() call, but replace an
mtr_memo_contains() with mtr_memo_contains_page().

row_upd_rec_sys_fields(): Reorder an assertion containing buf_block_align()
so that the costly call can be avoided in some cases.
2006-10-25 08:52:43 +00:00
marko
42dcad50b8 branches/zip: Remove all references to buf_block_align() except those
from the adaptive hash index [btr_search_guess_on_hash() and
btr_search_validate()].  Some references to buf_block_align() remain
in debug builds.

btr_store_big_rec_extern_fields(): Add the parameter rec_block.

page_rec_get_next_low(): Do not assume that the page has been
allocated from the buffer pool when printing the diagnostic information.

page_cur_insert_rec_low(): Replace the parameter page_zip_des_t* page_zip
with the parameter buf_block_t* block.
2006-10-24 14:06:31 +00:00
marko
9b0a89867e branches/zip: Remove all buf_block_align() calls from lock0lock.c.
Replace all page_t* parameters with buf_block_t*, and replace many
rec_t* parameters with heap_no.  This eliminates also many
rec_get_heap_no() calls, which became more expensive with the
introduction of ROW_FORMAT=COMPACT in MySQL/InnoDB 5.0.3.

page_rec_get_heap_no(), lock_get_min_heap_no(): New functions.
2006-10-24 06:45:52 +00:00
marko
7e8c129370 branches/zip: buf_page_get_release_on_io(), buf_page_get_gen():
Replace buf_frame_t* guess with buf_block_t* guess in order to avoid
a buf_block_align() call.

trx_undo_t: Replace page_t* guess_page with buf_block_t* guess_block.

btr_search_t: Replace page_t* root_guess with buf_block_t* root_guess.
2006-10-23 19:34:45 +00:00
marko
009533b471 branches/zip: Minor cleanup.
page_set_max_trx_id(): Replace page_t* parameter with buf_block_t*,
to avoid a buf_block_align() call.

Replace some occurrences of page_get_page_no() with buf_block_get_page_no().

page_cur_delete_rec(): Replace buf_block_align() with page_cur_get_block().
2006-10-23 18:26:10 +00:00
marko
f97f00ee60 branches/zip: Fix some bugs.
btr_cur_pessimistic_insert(): Remove local variable "block".  Replace
an extraneous call to btr_cur_position() [with wrong "block", in r948]
with a debug assertion that the cursor is positioned on the preceding
record after the page split.

btr_pcur_restore_position(): Do not use btr_pcur_get_page(), because
the debug assertion there could fail.  This bug was introduced in r928.
2006-10-23 11:38:32 +00:00
marko
8562e752bc branches/zip: Remove some more buf_block_align() calls.
btr_cur_t: Move page_block to page_cur_t::block.

page_cur_get_block(), page_cur_get_page_zip(): New functions.

page_cur_position(): Add parameter block.

Remove many page_zip parameters, now that there is page_cur_get_page_zip().
Replace some page, page_zip parameters with block.

Add some const qualifiers to function parameters and remove casts.

PAGE_HEAP_NO_INFIMUM, PAGE_HEAP_NO_SUPREMUM, PAGE_HEAP_NO_USER_LOW:
New constants.

Replace some cursor code in low-level diagnostic functions with
direct management of rec, because buf_block_t::buf_fix_count may be 0
when the functions are called, and debug assertions would fail.
2006-10-20 12:45:53 +00:00
marko
5a5e163f50 branches/zip: btr_cur_open_at_index_side(): Correct a debug assertion
added in r928.  Do not call btr_cur_get_block() in the debug assertion,
because the debug assertion there would fail.
2006-10-20 11:54:13 +00:00
marko
98a829723c branches/zip: Add const qualifiers to function parameters, mostly dtuple_t. 2006-10-20 08:30:07 +00:00
marko
91f8a3dde5 branches/zip: Remove many fil_space_get_zip_size() calls.
ibuf_page(), ibuf_page_low(), ibuf_free_excess_pages(), ibuf_insert(),
buf_read_page(), buf_read_ahead_linear(), buf_read_recv_pages():
Add parameter zip_size.
2006-10-19 11:07:50 +00:00
marko
e8a779e73e branches/zip: Remove the casts introduced in r920 and r925. Replace the
accessors returning pointers with macros that preserve const qualifiers.
In UNIV_DEBUG builds, retain the accessors and cast away constness there.

dfield_get_type(), dfield_get_data(), dtuple_get_nth_field(),
dict_table_get_nth_col(), dict_table_get_sys_col(): Implement as macro
unless #ifdef UNIV_DEBUG.

rec_get_nth_field(): Replace with rec_get_nth_field_offs() that does not
do pointer arithmetics.  Implement rec_get_nth_field() as a macro.
2006-10-19 07:27:26 +00:00
marko
ab0456cb5e branches/zip: btr_cur_t: replace left_page with left_block, to avoid
calling buf_block_align() in btr_pcur_move_backward_from_page().
Also, enclose some assertions about page_is_comp() in #ifdef UNIV_BTR_DEBUG.
2006-10-18 17:43:04 +00:00
marko
c659b59a5d branches/zip: Eliminate many buf_block_align() calls.
Replace page_t* or page_zip_des_t* parameters in some functions with
buf_block_t*.

buf_frame_get_page_zip(): Disable unless #ifdef UNIV_DEBUG || UNIV_ZIP_DEBUG.

btr_cur_t: Add buf_block_t* page_block.

btr_pcur_get_block(), btr_cur_get_block(), btr_cur_get_page_zip():
New functions.

btr_cur_position(): Add the parameter block.
2006-10-18 11:39:31 +00:00
marko
6bf52fdaca branches/zip: Introduce const qualifiers to many read-only parameters
and modify some functions to return const pointers.  Add const qualifiers
to local variable declarations or casts to remove the const qualifier
in those places where write access is needed.
2006-10-17 12:24:13 +00:00