Commit graph

116 commits

Author SHA1 Message Date
marko
75e478a01b branches/zip: page_zip_empty_size(): New function to compute the
guaranteed free space available for inserting one record.

btr_page_get_sure_split_rec(), btr_cur_pessimistic_insert():
Use page_zip_empty_size().

btr_page_split_and_insert(): Relax a debug assertion that there should
be at least two user records on the page.  On compressed pages, we may
be able to write only one record.
2006-09-20 14:26:53 +00:00
marko
c12e237a09 branches/zip: Instrument the buffer pool allocator and deallocator
for more accurate Valgrind debugging.

univ.i: Introduce UNIV_DEBUG_VALGRIND, UNIV_MEM_VALID, and UNIV_MEM_INVALID.

buf_LRU_block_free_non_file_page(): Invalidate the buffer frame
with UNIV_MEM_INVALID().

buf_LRU_get_free_block(): Declare the buffer frame valid
with UNIV_MEM_VALID().

Other memory is allocated and deallocated via malloc() and free(),
which are already overridden by Valgrind.  Without the added
instrumentation, accesses to free pages in the buffer pool cannot
be caught.

The diagnostics could probably be improved further by declaring all
non-latched buffer frames invalid.
2006-09-20 12:42:08 +00:00
marko
9a16aac783 branches/zip: Merge revisions 804:838 from trunk. 2006-09-19 10:14:07 +00:00
marko
e4dc19ba38 branches/zip: Merge revisions 789:804 from trunk. 2006-09-12 07:29:57 +00:00
marko
78d1dd326c branches/zip: Merge revisions 767:789 from trunk. 2006-09-04 19:47:47 +00:00
marko
6429e2e682 branches/zip: dtuple_convert_big_rec(): Shorten the locally stored part
of externally stored columns to 108 bytes when possible.

This will unmask a bug in the handling of BLOBs on compressed tablespaces,
especially of small page sizes.  Apparently, the locally stored part must
be less than 235 bytes in size for the bug to occur.

dict_col_struct: Add field min_prefix.

dict_index_build_internal_non_clust(): Adjust min_prefix.
2006-08-31 11:00:02 +00:00
marko
6c932eefb7 branches/zip: Merge revisions 765:767 from trunk. 2006-08-29 12:17:50 +00:00
marko
64f3cee67f branches/zip: Merge revisions 720:765 from trunk and reindent the code
as per revisions r763:765.
2006-08-29 09:30:31 +00:00
marko
20de0a2ad7 branches/zip: Assert that only leaf pages of clustered indexes may contain
externally stored columns.  Make some use of the fact in compression and
decompression.

rec_init_offsets(): Add ut_ad(dict_index_is_clust(index)) before flagging
external columns.

page_zip_get_trailer_len(): Assert ut_ad(!page_zip->n_blobs) unless
dict_index_is_clust(index).

page_zip_get_n_prev_extern(): Add ut_ad(dict_index_is_clust(index)).

page_zip_compress(), page_zip_decompress_low(): Add dict_index_is_clust()
assertions.  Allow completely external storage of columns.  Simplify
the computation of "externs".

page_zip_write_rec(): Store leaf node records of non-clustered indexes
with one memcpy(), avoiding rec_offs_n_extern() and the for loop.

page_zip_write_blob_ptr(): Add ut_ad(dict_index_is_clust(index)).
Simplify the computation of "externs".

page_zip_clear_rec(): Only attempt to zero out BLOB pointers when
the record belongs to a leaf page of a clustered index.

page_zip_dir_delete(): Take a fast path when the record is
not on a leaf page of a clustered index.
Simplify the computation of "externs".

page_zip_copy(): Assert that n_blobs is zero on anything else than
leaf pages of clustered indexes.
2006-08-21 10:27:15 +00:00
marko
98b296946f branches/zip: page_zip_dir_insert(): Remove unused parameters index, offsets. 2006-08-21 09:27:44 +00:00
marko
7291e6579e branches/zip: dtuple_convert_big_rec(): Do not attempt to shorten records
of secondary indexes.  (This is just an optimization; the logic in the
search loop would never store such records externally.)
2006-08-21 08:41:05 +00:00
marko
2c3b9102f8 branches/zip: page_cur_insert_rec_low(): Replace page_zip_dir_rewrite()
with page_zip_dir_insert().  Pass page_zip to rec_set_n_owned_new()
and page_dir_slot_set_n_owned().

page_zip_dir_insert(): New function.  Shift the dense page directory and
write the inserted record there.
2006-08-18 19:30:55 +00:00
marko
0ebbc851b1 branches/zip: Correct the description of DB_TOO_BIG_RECORD. 2006-08-18 10:02:50 +00:00
marko
ea4572cecf branches/zip: btr_cur_pessimistic_insert(): Prior to splitting, ensure that
the record will fit on an empty compressed page.

buf_block_alloc(), buf_block_free(), buf_frame_alloc(), buf_frame_free():
Make inlined.
2006-08-17 11:57:51 +00:00
marko
ae8901c8c9 branches/zip: Account for the compressed page size when checking if a
record will fit or need external storage.

btr_page_get_sure_split_rec(): Estimate the free space of an empty
compressed page.

page_zip_rec_needs_ext(): New function, to replace existing tests whether
external storage is needed.
2006-08-17 08:14:38 +00:00
marko
5e19de2b95 branches/zip: Clean up debug code.
struct mtr_struct: Omit unused field "state" unless #ifdef UNIV_DEBUG.

buf_block_buf_fix_inc_debug(): Remove redundant __attribute__((unused))
and #ifdef UNIV_SYNC_DEBUG.
2006-08-16 09:35:36 +00:00
marko
cbb6e0ca14 branches/zip: Merge revisions 713:720 from trunk. 2006-08-15 11:55:06 +00:00
marko
37f03d23d4 branches/zip: Introduce the symbol UNIV_DEBUG_PRINT for enabling debug
print functions.

buf_LRU_print(), buf_print(): Replace #ifdef UNIV_DEBUG
with #ifdef UNIV_DEBUG_PRINT.
2006-08-15 11:54:20 +00:00
marko
df74f57e24 branches/zip: btr0btr.ic: Add #include "page0zip.h" so that the code
will compile with inlining disabled.
2006-08-15 08:37:54 +00:00
marko
a013fd9c49 branches/zip: Allow the code to be built with function inlining enabled.
page0page.h: Add #include "page0zip.h"

page_dir_slot_get_rec(): Avoid void* arithmetics.

trx0undo.ic, trx0sys.ic: Remove unnecessary #include "page0zip.h"

page_zip_write_header(): Move prototype from page0types.h to page0zip.h
2006-08-14 08:25:46 +00:00
marko
36ed778816 branches/zip: Merge revisions 634:713 from trunk. 2006-08-11 11:49:38 +00:00
marko
441c29e94f branches/zip: Fix some crash recovery bugs.
dict_load_table(): Initialize table->flags with zip_size.

mlog_parse_nbytes(), mlog_parse_string(): Add parameter page_zip and
write the changes also to the compressed page if one is specified.
Assert that these functions are not called on FIL_PAGE_INDEX pages.

buf_page_io_complete(): Replace block->frame with frame where appropriate.

recv_parse_or_apply_log_rec_body(): Add ut_a(!page_zip) where appropriate.

page_parse_delete_rec_list(): Add parameter page_zip.
2006-08-02 05:49:15 +00:00
marko
62adcfabeb branches/zip: Fix a bug in consistent reads of rows containing BLOB data.
btr_rec_copy_externally_stored_field(): Add parameter zip_size.
Do not call buf_block_align(rec), because rec can also be in
dynamically allocated memory.  buf_block_align() can only be invoked
on addresses inside the buffer pool.

page_zip_clear_rec(): Improve formatting.
2006-07-31 06:43:25 +00:00
marko
5ac7833038 branches/zip: page_zip_validate(): Replace buf_frame_alloc() with
ut_malloc() to avoid breaking the latching order.

page_rec_check(), page_dir_slot_get_rec(): Replace buf_frame_align()
with ut_align_down() to avoid assertion failures in page_zip_validate().
2006-07-28 07:34:33 +00:00
marko
81fca4d1ea branches/zip: btr_parse_page_reorganize(): Add parameter page_zip. 2006-07-04 21:15:30 +00:00
marko
51fb42bfa8 branches/zip: page_cur_insert_rec_low(): Replace page_zip_alloc()
with page_zip_available() in order to avoid an extra page_zip_compress().

page_zip_available(): Make the function public.

page0zip.ic: Sort the inline function declarations in order to avoid
forward references, which do not work on some compilers.
2006-07-03 19:12:21 +00:00
marko
7279a72289 branches/zip: Improve assertions related to the linked lists of records
on B-tree index pages.

page_rec_set_next(): Assert that rec != next.

rec_get_next_ptr(), rec_get_next_offs(): On compact pages, assert that
there are at least REC_N_NEW_EXTRA_BYTES + 1 between records.

page_cur_insert_rec_write_log(): Replace a buf_frame_align() call
with ut_align_offset().

page_cur_insert_rec_low(): Assert that current_rec != insert_rec.
2006-06-30 09:49:32 +00:00
marko
9972604267 branches/zip: Eliminate page corruption in btr_compress() when
page_zip_reorganize() was invoked.

btr_compress(): Obtain orig_pred and orig_succ after copying the records.
Add a debug assertion about FIL_PAGE_PREV.

page_copy_rec_list_end(), page_copy_rec_list_start(): Change the return
type from ibool to rec_t.  Adjust the return value after invoking
page_zip_reorganize().
2006-06-28 12:01:29 +00:00
marko
7bc09e8c18 branches/zip: Make page_zip_validate() independent of UNIV_DEBUG.
Replace most occurrences of #if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG
with #ifdef UNIV_ZIP_DEBUG.
2006-06-20 19:35:59 +00:00
marko
0d5392310b branches/zip: fil_space_truncate_start(): Fix an incorrect assertion,
disable unless #ifdef UNIV_LOG_ARCHIVE.
2006-06-20 11:35:41 +00:00
marko
3e02b8dbe5 branches/zip: page_zip_reorganize(): On success, write to the redo log. 2006-06-19 11:07:21 +00:00
marko
be7add3b56 branches/zip: Add parameter zip_size to fil_io().
fil_read(), fil_write(): Make these inlined functions in fil0fil.c.

fil_write_lsn_and_arch_no_to_file(): Remove the parameter space_id and
note that this function is to be called on the system tablespace, which
is uncompressed.
2006-06-15 07:27:13 +00:00
marko
77fa0fd682 branches/zip: Improve some assertions.
recv_parse_or_apply_log_rec_body(): Assert that mlog_parse_nbytes() is
never called on compressed B-tree pages.

page_zip_simple_validate(): Correct a typo in a debug assertion.

xdes_calc_descriptor_page(): Fix an incorrect debug assertion.
2006-06-14 11:19:08 +00:00
marko
9bc27fe8b3 branches/zip: Fix some bugs in the crash recovery of compressed tablespaces.
fil_node_open_file(): Set node->size correctly for compressed tablespaces.

fsp_header_write_space_id(): Rename to fsp_header_write_fields(),
add parameter zip_size.

fil_io(): Add UNIV_UNLIKELY hints to assertion-like tests.
2006-06-14 06:29:41 +00:00
marko
645e56909f branches/zip: Merge revisions 583:634 from trunk. 2006-06-13 20:23:26 +00:00
marko
0260ba1f0f branches/zip: Cleanup suggested by Osku.
ut_is_2pow(): New function for testing if a number is zero or a power of two.
Use this function instead of bitwise arithmetics or ut_2_power_up() where
possible.
2006-06-07 11:23:21 +00:00
marko
0e6236a625 branches/zip: Replace all ut_ad(page_zip_validate()) assertions with
ut_a(page_zip_validate()) that are enabled if UNIV_DEBUG or
UNIV_ZIP_DEBUG is defined.
2006-06-07 11:06:12 +00:00
marko
4cda2f9d0c branches/zip: Remove remaining occurrences of XDES_DESCRIBED_PER_PAGE.
mlog_parse_string(): Remove an off-by-one error in an assertion.

fil_extend_space_to_desired_size(): fsp_header_get_free_limit(),
fsp_try_extend_data_file(): Replace UNIV_PAGE_SIZE with the actual page size.

PAGE_ZIP_MIN_SIZE: New constant, to be used in assertions.

FSP_SEG_INODES_PER_PAGE: Add parameter zip_size.

fsp_seg_inode_page_get_nth_inode(), fsp_seg_inode_page_find_used(),
fsp_seg_inode_page_find_free(): Add parameter zip_size.
2006-06-07 10:41:58 +00:00
marko
5e6324b7e7 branches/zip: Remove some more references to XDES_DESCRIBED_PER_PAGE.
ibuf_parse_bitmap_init(), ibuf_bitmap_page_init(),
ibuf_bitmap_page_get_bits(), ibuf_bitmap_set_bits(),
ibuf_bitmap_page_no_calc(), ibuf_bitmap_get_map_page(),
xdes_calc_descriptor_page(), xdes_calc_descriptor_index(),
fsp_descr_page(): Add parameter zip_size.
2006-06-06 07:42:04 +00:00
marko
fe5474851a branches/zip: When identifying insert buffer bitmap pages, note that the
pages occur every zip_size pages in compressed tablespaces, instead of
UNIV_PAGESIZE (or XDES_DESCRIBED_PER_PAGE).

ibuf_bitmap_page(): Add parameter zip_size.

ibuf_fixed_addr_page(): Add parameter space.
2006-06-05 09:04:19 +00:00
marko
831916878d branches/zip: Write compressed pages to disk.
os_aio_simulated_handle(): Temporarily disable os_file_check_page_trailers(),
which cannot be invoked on compressed pages.

dict_table_add_system_columns(): New function, split from
dict_table_add_to_cache().

mlog_parse_index(): Add system columns to the dummy table and identify
DB_TRX_ID and DB_ROLL_PTR in the dummy index.

buf_LRU_get_free_block(): Note that page_zip->data should be allocated from
an aligned memory pool.

buf_flush_buffered_writes(): Write compressed pages to disk.

buf_flush_post_to_doublewrite_buf(): Copy compressed pages to the
doublewrite buffer.  Zero fill any excess space.

buf_flush_init_for_writing(): Treat all compressed pages the same.

buf_read_page_low(): Read compressed pages from disk.

buf_page_io_complete(): Process compressed pages.

trx_sys_doublewrite_init_or_restore_page(): Process compressed pages.

mlog_write_initial_log_record_fast(): Enable a debug printout
#ifdef UNIV_LOG_DEBUG.

fsp_header_init(), fsp_fill_free_list(): Pass the compressed page size
to buf_page_create().

page_zip_compress_write_log(): Flatten the if-else if-else logic.

page_zip_parse_write_blob_ptr(): Do not test page_zip if page==NULL.

page_zip_parse_write_node_ptr(): Do not test page_zip if page==NULL.
Invoke mlog_close() correctly.

row_sel_store_row_id_to_prebuilt(): Add UNIV_UNLIKELY hint to an
assertion-like test.
2006-05-30 09:04:57 +00:00
marko
c432dc4b9c branches/zip: Merge revisions 560:583 from trunk. 2006-05-22 09:30:34 +00:00
marko
6c3c7271c9 branches/zip: Try to reorganize the page when compression fails.
page_zip_compress_write_log(): Make static.

page_zip_compress(): Add optional parameter mtr for redo logging.

page_zip_reorganize(): Low-level counterpart of btr_page_reorganize().

page_zip_copy(): Add debug assertions about mtr_memo_contains.

page_cur_insert_rec_low(): Try page_zip_reorganize() and seek to the
new position of insert_rec if it succeeds.

page_copy_rec_list_end(), page_copy_rec_list_start():
Try page_zip_reorganize().

page_move_rec_list_end(): Remove bogus comment.
2006-05-16 14:17:43 +00:00
marko
00b55da692 branches/zip: Shorten the log record MLOG_ZIP_PAGE_COMPRESS.
page_zip_copy(), page_zip_compress_write_log(): Add parameter 'index'.

page_zip_parse_write_header(): Check for !page_zip only if page != NULL.

page_zip_compress_write_log(), page_zip_parse_compress(): Omit some
fields in the page header.  Omit the unused bytes between the modification
log and the page trailer.

parse_or_apply_log_rec_body(): Remove a bogus debug assertion.
2006-05-16 07:58:10 +00:00
marko
0aa8b2cc74 branches/zip: btr_page_split_and_insert(): Implement fallbacks for
page_move_rec_list_start() and page_move_rec_list_end().  Relax the
conditions on insert_will_fit if page_zip is set.

page_delete_rec_list_start(): Make public.

page_zip_copy(): Clear REC_INFO_MIN_REC_FLAG if FIL_PAGE_PREV != FIL_NULL.
2006-05-15 11:43:35 +00:00
marko
93fe682c17 branches/zip: After merge fix, and cleanup.
btr_validate_level(): Add missing parameter to added buf_page_print() calls.

btr_root_raise_and_insert(): Replace low-level code with page_zip_copy().

page_zip_copy(): New function to copy the data payload of a compressed page,
byte for byte.  The file page header and trailer are not copied, to avoid
overwriting the fields FIL_PAGE_OFFSET, FIL_PAGE_PREV and FIL_PAGE_NEXT.
2006-05-11 18:23:10 +00:00
marko
2433ba7648 branches/zip: Merge revisions 558:560 from trunk. 2006-05-11 17:00:43 +00:00
marko
fefd14cbbc branches/zip: Merge revisions 536:558 from trunk. 2006-05-11 12:47:23 +00:00
marko
17eb5eab4d branches/zip: Minor cleanup.
btr_cur_compress_if_useful(): Replace if () return(); return() with return.

page_rec_get_next_low(): New function.

page_rec_get_prev(): Invoke page_is_comp() outside the loop.
Replace page_rec_get_next() with loop-specific instances of
page_rec_get_next_low().

page_copy_rec_list_end(): Add some debug assertions.
2006-05-11 12:31:22 +00:00
marko
6748a82b84 branches/zip: Remove GCC 4.0.4 warnings from
make AM_CFLAGS='-std=c89 -Wall -W -pedantic'
2006-05-10 13:58:35 +00:00