Commit graph

96 commits

Author SHA1 Message Date
marko
36ed778816 branches/zip: Merge revisions 634:713 from trunk. 2006-08-11 11:49:38 +00:00
marko
441c29e94f branches/zip: Fix some crash recovery bugs.
dict_load_table(): Initialize table->flags with zip_size.

mlog_parse_nbytes(), mlog_parse_string(): Add parameter page_zip and
write the changes also to the compressed page if one is specified.
Assert that these functions are not called on FIL_PAGE_INDEX pages.

buf_page_io_complete(): Replace block->frame with frame where appropriate.

recv_parse_or_apply_log_rec_body(): Add ut_a(!page_zip) where appropriate.

page_parse_delete_rec_list(): Add parameter page_zip.
2006-08-02 05:49:15 +00:00
marko
62adcfabeb branches/zip: Fix a bug in consistent reads of rows containing BLOB data.
btr_rec_copy_externally_stored_field(): Add parameter zip_size.
Do not call buf_block_align(rec), because rec can also be in
dynamically allocated memory.  buf_block_align() can only be invoked
on addresses inside the buffer pool.

page_zip_clear_rec(): Improve formatting.
2006-07-31 06:43:25 +00:00
marko
5ac7833038 branches/zip: page_zip_validate(): Replace buf_frame_alloc() with
ut_malloc() to avoid breaking the latching order.

page_rec_check(), page_dir_slot_get_rec(): Replace buf_frame_align()
with ut_align_down() to avoid assertion failures in page_zip_validate().
2006-07-28 07:34:33 +00:00
marko
81fca4d1ea branches/zip: btr_parse_page_reorganize(): Add parameter page_zip. 2006-07-04 21:15:30 +00:00
marko
51fb42bfa8 branches/zip: page_cur_insert_rec_low(): Replace page_zip_alloc()
with page_zip_available() in order to avoid an extra page_zip_compress().

page_zip_available(): Make the function public.

page0zip.ic: Sort the inline function declarations in order to avoid
forward references, which do not work on some compilers.
2006-07-03 19:12:21 +00:00
marko
7279a72289 branches/zip: Improve assertions related to the linked lists of records
on B-tree index pages.

page_rec_set_next(): Assert that rec != next.

rec_get_next_ptr(), rec_get_next_offs(): On compact pages, assert that
there are at least REC_N_NEW_EXTRA_BYTES + 1 between records.

page_cur_insert_rec_write_log(): Replace a buf_frame_align() call
with ut_align_offset().

page_cur_insert_rec_low(): Assert that current_rec != insert_rec.
2006-06-30 09:49:32 +00:00
marko
9972604267 branches/zip: Eliminate page corruption in btr_compress() when
page_zip_reorganize() was invoked.

btr_compress(): Obtain orig_pred and orig_succ after copying the records.
Add a debug assertion about FIL_PAGE_PREV.

page_copy_rec_list_end(), page_copy_rec_list_start(): Change the return
type from ibool to rec_t.  Adjust the return value after invoking
page_zip_reorganize().
2006-06-28 12:01:29 +00:00
marko
7bc09e8c18 branches/zip: Make page_zip_validate() independent of UNIV_DEBUG.
Replace most occurrences of #if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG
with #ifdef UNIV_ZIP_DEBUG.
2006-06-20 19:35:59 +00:00
marko
0d5392310b branches/zip: fil_space_truncate_start(): Fix an incorrect assertion,
disable unless #ifdef UNIV_LOG_ARCHIVE.
2006-06-20 11:35:41 +00:00
marko
3e02b8dbe5 branches/zip: page_zip_reorganize(): On success, write to the redo log. 2006-06-19 11:07:21 +00:00
marko
be7add3b56 branches/zip: Add parameter zip_size to fil_io().
fil_read(), fil_write(): Make these inlined functions in fil0fil.c.

fil_write_lsn_and_arch_no_to_file(): Remove the parameter space_id and
note that this function is to be called on the system tablespace, which
is uncompressed.
2006-06-15 07:27:13 +00:00
marko
77fa0fd682 branches/zip: Improve some assertions.
recv_parse_or_apply_log_rec_body(): Assert that mlog_parse_nbytes() is
never called on compressed B-tree pages.

page_zip_simple_validate(): Correct a typo in a debug assertion.

xdes_calc_descriptor_page(): Fix an incorrect debug assertion.
2006-06-14 11:19:08 +00:00
marko
9bc27fe8b3 branches/zip: Fix some bugs in the crash recovery of compressed tablespaces.
fil_node_open_file(): Set node->size correctly for compressed tablespaces.

fsp_header_write_space_id(): Rename to fsp_header_write_fields(),
add parameter zip_size.

fil_io(): Add UNIV_UNLIKELY hints to assertion-like tests.
2006-06-14 06:29:41 +00:00
marko
645e56909f branches/zip: Merge revisions 583:634 from trunk. 2006-06-13 20:23:26 +00:00
marko
0260ba1f0f branches/zip: Cleanup suggested by Osku.
ut_is_2pow(): New function for testing if a number is zero or a power of two.
Use this function instead of bitwise arithmetics or ut_2_power_up() where
possible.
2006-06-07 11:23:21 +00:00
marko
0e6236a625 branches/zip: Replace all ut_ad(page_zip_validate()) assertions with
ut_a(page_zip_validate()) that are enabled if UNIV_DEBUG or
UNIV_ZIP_DEBUG is defined.
2006-06-07 11:06:12 +00:00
marko
4cda2f9d0c branches/zip: Remove remaining occurrences of XDES_DESCRIBED_PER_PAGE.
mlog_parse_string(): Remove an off-by-one error in an assertion.

fil_extend_space_to_desired_size(): fsp_header_get_free_limit(),
fsp_try_extend_data_file(): Replace UNIV_PAGE_SIZE with the actual page size.

PAGE_ZIP_MIN_SIZE: New constant, to be used in assertions.

FSP_SEG_INODES_PER_PAGE: Add parameter zip_size.

fsp_seg_inode_page_get_nth_inode(), fsp_seg_inode_page_find_used(),
fsp_seg_inode_page_find_free(): Add parameter zip_size.
2006-06-07 10:41:58 +00:00
marko
5e6324b7e7 branches/zip: Remove some more references to XDES_DESCRIBED_PER_PAGE.
ibuf_parse_bitmap_init(), ibuf_bitmap_page_init(),
ibuf_bitmap_page_get_bits(), ibuf_bitmap_set_bits(),
ibuf_bitmap_page_no_calc(), ibuf_bitmap_get_map_page(),
xdes_calc_descriptor_page(), xdes_calc_descriptor_index(),
fsp_descr_page(): Add parameter zip_size.
2006-06-06 07:42:04 +00:00
marko
fe5474851a branches/zip: When identifying insert buffer bitmap pages, note that the
pages occur every zip_size pages in compressed tablespaces, instead of
UNIV_PAGESIZE (or XDES_DESCRIBED_PER_PAGE).

ibuf_bitmap_page(): Add parameter zip_size.

ibuf_fixed_addr_page(): Add parameter space.
2006-06-05 09:04:19 +00:00
marko
831916878d branches/zip: Write compressed pages to disk.
os_aio_simulated_handle(): Temporarily disable os_file_check_page_trailers(),
which cannot be invoked on compressed pages.

dict_table_add_system_columns(): New function, split from
dict_table_add_to_cache().

mlog_parse_index(): Add system columns to the dummy table and identify
DB_TRX_ID and DB_ROLL_PTR in the dummy index.

buf_LRU_get_free_block(): Note that page_zip->data should be allocated from
an aligned memory pool.

buf_flush_buffered_writes(): Write compressed pages to disk.

buf_flush_post_to_doublewrite_buf(): Copy compressed pages to the
doublewrite buffer.  Zero fill any excess space.

buf_flush_init_for_writing(): Treat all compressed pages the same.

buf_read_page_low(): Read compressed pages from disk.

buf_page_io_complete(): Process compressed pages.

trx_sys_doublewrite_init_or_restore_page(): Process compressed pages.

mlog_write_initial_log_record_fast(): Enable a debug printout
#ifdef UNIV_LOG_DEBUG.

fsp_header_init(), fsp_fill_free_list(): Pass the compressed page size
to buf_page_create().

page_zip_compress_write_log(): Flatten the if-else if-else logic.

page_zip_parse_write_blob_ptr(): Do not test page_zip if page==NULL.

page_zip_parse_write_node_ptr(): Do not test page_zip if page==NULL.
Invoke mlog_close() correctly.

row_sel_store_row_id_to_prebuilt(): Add UNIV_UNLIKELY hint to an
assertion-like test.
2006-05-30 09:04:57 +00:00
marko
c432dc4b9c branches/zip: Merge revisions 560:583 from trunk. 2006-05-22 09:30:34 +00:00
marko
6c3c7271c9 branches/zip: Try to reorganize the page when compression fails.
page_zip_compress_write_log(): Make static.

page_zip_compress(): Add optional parameter mtr for redo logging.

page_zip_reorganize(): Low-level counterpart of btr_page_reorganize().

page_zip_copy(): Add debug assertions about mtr_memo_contains.

page_cur_insert_rec_low(): Try page_zip_reorganize() and seek to the
new position of insert_rec if it succeeds.

page_copy_rec_list_end(), page_copy_rec_list_start():
Try page_zip_reorganize().

page_move_rec_list_end(): Remove bogus comment.
2006-05-16 14:17:43 +00:00
marko
00b55da692 branches/zip: Shorten the log record MLOG_ZIP_PAGE_COMPRESS.
page_zip_copy(), page_zip_compress_write_log(): Add parameter 'index'.

page_zip_parse_write_header(): Check for !page_zip only if page != NULL.

page_zip_compress_write_log(), page_zip_parse_compress(): Omit some
fields in the page header.  Omit the unused bytes between the modification
log and the page trailer.

parse_or_apply_log_rec_body(): Remove a bogus debug assertion.
2006-05-16 07:58:10 +00:00
marko
0aa8b2cc74 branches/zip: btr_page_split_and_insert(): Implement fallbacks for
page_move_rec_list_start() and page_move_rec_list_end().  Relax the
conditions on insert_will_fit if page_zip is set.

page_delete_rec_list_start(): Make public.

page_zip_copy(): Clear REC_INFO_MIN_REC_FLAG if FIL_PAGE_PREV != FIL_NULL.
2006-05-15 11:43:35 +00:00
marko
93fe682c17 branches/zip: After merge fix, and cleanup.
btr_validate_level(): Add missing parameter to added buf_page_print() calls.

btr_root_raise_and_insert(): Replace low-level code with page_zip_copy().

page_zip_copy(): New function to copy the data payload of a compressed page,
byte for byte.  The file page header and trailer are not copied, to avoid
overwriting the fields FIL_PAGE_OFFSET, FIL_PAGE_PREV and FIL_PAGE_NEXT.
2006-05-11 18:23:10 +00:00
marko
2433ba7648 branches/zip: Merge revisions 558:560 from trunk. 2006-05-11 17:00:43 +00:00
marko
fefd14cbbc branches/zip: Merge revisions 536:558 from trunk. 2006-05-11 12:47:23 +00:00
marko
17eb5eab4d branches/zip: Minor cleanup.
btr_cur_compress_if_useful(): Replace if () return(); return() with return.

page_rec_get_next_low(): New function.

page_rec_get_prev(): Invoke page_is_comp() outside the loop.
Replace page_rec_get_next() with loop-specific instances of
page_rec_get_next_low().

page_copy_rec_list_end(): Add some debug assertions.
2006-05-11 12:31:22 +00:00
marko
6748a82b84 branches/zip: Remove GCC 4.0.4 warnings from
make AM_CFLAGS='-std=c89 -Wall -W -pedantic'
2006-05-10 13:58:35 +00:00
marko
cfc945a954 branches/zip: Merge revisions 465:532 from trunk. 2006-05-08 06:18:59 +00:00
marko
f258491a26 branches/zip: Minor improvements.
Introduce FIL_PAGE_ZBLOB_DATA as a synonym for FIL_PAGE_FILE_FLUSH_LSN.

btr_store_big_rec_extern_fields(): Make the assertion about
dict_table_zip_size() more accurate.

buf_LRU_get_free_block(), buf_block_alloc(): Add parameter zip_size.

buf_calc_zblob_page_checksum(): Remove. Replace with page_zip_calc_checksum().

buf_page_init(): Remove parameter zip_size.

buf_page_io_complete(): Add a placeholder for handling compressed pages.

trx_doublewrite_page_inside(): Remove redundant function.

page_zip_write_rec(): Relax an overly tight assertion about blob_no.
2006-05-04 11:44:49 +00:00
marko
7bd3aca4ad branches/zip: Prepare for writing compressed and reading pages.
buf_page_print(): Print also compressed pages.  Add parameter zip_size.

buf_flush_init_for_writing(): Stamp the fields on a compressed B-tree index
page.

Add the header field FIL_PAGE_ZBLOB_SPACE_ID as an alias of FIL_PAGE_PREV.

page_zip_calc_checksum(): New function.

page_zip_compress(): Avoid copying the fields that are written in
buf_flush_init_for_writing().

page_zip_header_cmp(): New function for comparing those fields of the
page header that will not be written in buf_flush_init_for_writing().
2006-05-02 11:44:39 +00:00
marko
b0ae76c51a branches/zip: Minor cleanup and bug fixes
btr_page_reorganize_low(): Rename new_page to temp_page.

btr_store_big_rec_extern_fields(): FIL_PAGE_TYPE is 2 bytes, not 4.

buf_page_init(), buf_page_create(), buf_read_page_low(),
buf_page_init_for_read(): Add parameter zip_size.

buf_page_init_for_backup_restore(),
recv_apply_log_recs_for_backup(): Enclose in #ifdef UNIV_HOTBACKUP.

Enclose some debug code in #ifdef UNIV_LOG_REPLICATE.

page_zip_write_header_log(): Replace page_zip with a pointer to
the uncompressed page.

page_zip_write_rec(): Relax an assertion about blob_no + n_ext.

page_copy_rec_list_to_created_page_write_log(): Allow logging to be disabled.
2006-04-26 09:35:18 +00:00
marko
121623c1b4 branches/zip: Write the compressed page size to SYS_TABLES.TYPE
and to the file space header (FSP_PAGE_ZIP_SIZE, renamed from
FSP_LOWEST_NO_WRITE).

fil_space_struct: Add zip_size.

dict_table_struct: Embed zip_size in flags.

dict_table_zip_size(): Infer zip_size from table->flags.

dict_sys_tables_get_zip_size(): Read zip_size from SYS_TABLES.TYPE.

fil_space_get_zip_size(): Read zip_size from the file space header.

Add the redo log entry type MLOG_ZIP_FILE_CREATE.
2006-04-25 07:12:32 +00:00
marko
f72f850b79 branches/zip: Merge revisions 459:465 from trunk. 2006-04-12 12:39:58 +00:00
marko
450bbd7c3b branches/zip: Merge revisions 265:459 from trunk. 2006-04-12 09:32:17 +00:00
marko
cb6a8cc100 branches/zip: Fix some crash recovery bugs.
page_zip_alloc(): Add parameter "mtr" and log successful calls
to page_zip_compress().

page_zip_write_blob_ptr(), page_zip_write_node_ptr(): Write the offset on
the uncompressed page, because mlog_write_initial_log_record_fast()
does not do so.

page_zip_write_header_log(), page_zip_parse_write_header(): Encode the
offset in one byte.
2006-04-11 18:40:44 +00:00
marko
829229ce73 branches/zip: Add the redo log type MLOG_ZIP_PAGE_COMPRESS. Remove
MLOG_ZIP_LIST_START_COPY and MLOG_ZIP_LIST_END_COPY.

btr_compress(): Simplify a debug assertion.

page_zip_compress_write_log(), page_zip_parse_compress(): New functions.

page_cur_parse_insert_rec(): Simplify the code.

page_parse_create_zip(): Removed.

page_create_zip(), page_copy_rec_list_end(),
page_copy_rec_list_start(): Invoke page_zip_compress_write_log().
2006-04-11 12:27:06 +00:00
marko
64d5043f9e branches/zip: Minor cleanup. Remove MLOG_ZIP_ROOT_RAISE.
btr_page_create(): Remove parameters "prev" and "next".  The pointers
to adjacent B-tree pages will be set in the caller before any user
records are inserted.

Remove the redo log entry type MLOG_ZIP_ROOT_RAISE, since
btr_root_raise_and_insert() will not defer modifications to
the compressed page.

btr_attach_half_pages(): Add parameter new_page_zip.
2006-04-11 09:02:27 +00:00
marko
16dddacb6f branches/zip: Replace the redo log entry types
MLOG_ZIP_COMPRESS and MLOG_ZIP_DECOMPRESS with higher-level entry types.
Implement the logging and crash recovery of MLOG_ZIP_PAGE_CREATE.

page_create_zip(): New function for creating a compressed B-tree page.

page_parse_create_zip(): New function for applying a MLOG_ZIP_PAGE_CREATE
redo log record.

btr_page_create(): Remove the prototype.  Add parameters page_zip, level,
prev, and next.

btr0btr.c: Eliminate page_zip_compress() calls where possible.

page_zip_alloc(), page_zip_compress(), page_zip_decompress(),
page_zip_clear_rec(): Remove parameter mtr.

recv_parse_or_apply_log_rec_body(): Handle MLOG_ZIP_PAGE_CREATE.
Add TODO comments for the other added redo log entry types.
2006-04-10 19:48:37 +00:00
marko
1f8b0f3835 branches/zip: Implement crash recovery of writing BLOB pointers.
page_zip_parse_write_blob_ptr(): New function for applying the redo log
record MLOG_ZIP_WRITE_BLOB_PTR.

page_zip_write_blob_ptr(): Write the necessary information to the redo log.

page0zip.c: Tighten the assertions to ensure that blob_ptr < page_zip->n_blobs.

page_zip_write_node_ptr(): Use memcpy() instead of mach_write_to_4().
2006-04-07 12:56:22 +00:00
marko
817ca61f77 branches/zip: Implement crash recovery of writing to the page header.
page_zip_parse_write_header(): New function for applying the redo log of
MLOG_ZIP_WRITE_HEADER.

page_zip_parse_write_node_ptr(): Mark the log corrupted if !page_zip.
2006-04-07 11:44:14 +00:00
marko
684651ce2e branches/zip: Implement the crash recovery of MLOG_ZIP_WRITE_NODE_PTR.
page_zip_parse_write_node_ptr(): New function to apply a redo log of
MLOG_ZIP_WRITE_NODE_PTR.

page_zip_write_node_ptr(): Write all needed information to the redo log.

page_zip_write_header_log(): Write all necessary information to the redo log.
2006-04-07 11:04:08 +00:00
marko
f4c776f518 branches/zip: Initial steps towards disk-based storage of compressed pages.
dict_mem_table_create(): Account for DICT_TF_COMPRESSED in a debug assertion.

btr_store_big_rec_extern_fields(), btr_free_externally_stored_field(),
btr_copy_externally_stored_field(): Implement the disk format for
compressed BLOB pages.

btr_copy_externally_stored_field(): Improve error reporting and handling
when decompressing BLOB pages.

buf_flush_init_for_writing(), buf_page_is_corrupted(), buf_page_print():
Account for compressed BLOB pages (FIL_PAGE_TYPE_ZBLOB).

buf_calc_zblob_page_checksum(): New function.
2006-04-05 13:41:12 +00:00
marko
ac5b886f4f branches/zip: Fix some BLOB handling bugs.
btr_store_big_rec_extern_fields(): Assert that page_zip is non-NULL
if and only if dict_table_is_zip() holds.

btr_free_externally_stored_field(): Observe dict_table_is_zip().
Allow page_zip==NULL even if dict_table_is_zip().  Remove the
related TODO comment in row_purge_upd_exist_or_extern().

page_zip_available(): uncompressed_size already includes
PAGE_ZIP_DIR_SLOT_SIZE.

page_zip_decompress(): Remove bogus assertion d_stream.next_out == last.
Do not subtract BTR_EXTERN_FIELD_REF_SIZE from d_stream.avail_in when
decompressing records, because the records may be deleted later in
page_zip_apply_log(), and no BLOB pointers are allocated for deleted
records.
2006-04-03 20:33:31 +00:00
marko
e2b4592d6d branches/zip: Do not clear the records in page_delete_rec_list_end().
page_delete_rec_list_end(): Do not invoke page_zip_clear_rec().

page_zip_clear_rec(): Remove the global declaration.  Add an assertion
that the record must be on the free list.  Copy onl rec_offs_data_size()
bytes before attempting to clear the record and to compress the page.

page_zip_compress(), page_zip_write_rec(): Remove debug assertions that
trx_id, roll_ptr and node_ptr of deleted records be zero-filled.
2006-03-23 08:13:08 +00:00
marko
b03053a633 branches/zip: Minor fixes
Restore page_zip_clear_rec() to the global scope.
Invoke it in page_delete_rec_list_end().

Add debug assertions to page0zip.c to guard against overwriting data
on the compressed page.
2006-03-17 14:11:57 +00:00
marko
f3f76d249a branches/zip: Improve the clearing of deleted records. Try to support
operations on BLOB columns.  There are some bugs in the code, because
test-insert and a few other tests fail.

page_mem_free(): Add parameter index.  Decrement PAGE_N_RECS here.
Move some operations to page_zip_dir_delete().

page_zip_clear_rec(): Make this a static function.

page_zip_dir_delete(): Add parameters index and offsets.
Decrement PAGE_N_RECS and clear info_bits and n_owned.

page_zip_get_n_prev_extern(): Correct the synopsis and the algorithm.
Add parameter page_zip.  Search the records in heap_no order instead
of collation order.

page_zip_compress(), page_zip_decompress(): Only copy BLOB pointers
and increment n_blobs for records that have not been deleted.

page_zip_clear_rec(): Clear trx_id and roll_ptr on the compressed page.

page_zip_dir_delete(): Decrement PAGE_N_RECS.  Shift the array of
BLOB pointers.  Call page_zip_clear_rec().

page_zip_dir_add_slot(): Shift the array of BLOB pointers to make
space of roll_ptr and trx_id.

page_cur_delete_rec(): Do not decrement PAGE_N_RECS or call
page_zip_clear_rec(), as page_mem_free() already does it.
2006-03-16 14:02:22 +00:00
marko
df85f7287c branches/zip: Minor fixes
trx-undo_prev_version_build(): Pass offsets==NULL to
rec_set_field_extern_bits().

rec_set_field_extern_bits(), rec_set_field_extern_bits_new():
Accept offsets==NULL.

row_upd_rec_in_place(): Remove the bogus comment that the function
would only be invoked on a clustered index.  Remove the related
debug assertion.
2006-03-16 09:44:52 +00:00