Commit graph

692 commits

Author SHA1 Message Date
marko
8378c349e9 branches/zip: Replace all ut_ad(page_zip_validate()) assertions with
ut_a(page_zip_validate()) that are enabled if UNIV_DEBUG or
UNIV_ZIP_DEBUG is defined.
2006-06-07 11:06:12 +00:00
marko
49d252397a branches/zip: Remove remaining occurrences of XDES_DESCRIBED_PER_PAGE.
mlog_parse_string(): Remove an off-by-one error in an assertion.

fil_extend_space_to_desired_size(): fsp_header_get_free_limit(),
fsp_try_extend_data_file(): Replace UNIV_PAGE_SIZE with the actual page size.

PAGE_ZIP_MIN_SIZE: New constant, to be used in assertions.

FSP_SEG_INODES_PER_PAGE: Add parameter zip_size.

fsp_seg_inode_page_get_nth_inode(), fsp_seg_inode_page_find_used(),
fsp_seg_inode_page_find_free(): Add parameter zip_size.
2006-06-07 10:41:58 +00:00
marko
8b14f39765 branches/zip: Remove some more references to XDES_DESCRIBED_PER_PAGE.
ibuf_parse_bitmap_init(), ibuf_bitmap_page_init(),
ibuf_bitmap_page_get_bits(), ibuf_bitmap_set_bits(),
ibuf_bitmap_page_no_calc(), ibuf_bitmap_get_map_page(),
xdes_calc_descriptor_page(), xdes_calc_descriptor_index(),
fsp_descr_page(): Add parameter zip_size.
2006-06-06 07:42:04 +00:00
marko
a782c1cced branches/zip: When identifying insert buffer bitmap pages, note that the
pages occur every zip_size pages in compressed tablespaces, instead of
UNIV_PAGESIZE (or XDES_DESCRIBED_PER_PAGE).

ibuf_bitmap_page(): Add parameter zip_size.

ibuf_fixed_addr_page(): Add parameter space.
2006-06-05 09:04:19 +00:00
marko
1dd4d56863 branches/zip: Write compressed pages to disk.
os_aio_simulated_handle(): Temporarily disable os_file_check_page_trailers(),
which cannot be invoked on compressed pages.

dict_table_add_system_columns(): New function, split from
dict_table_add_to_cache().

mlog_parse_index(): Add system columns to the dummy table and identify
DB_TRX_ID and DB_ROLL_PTR in the dummy index.

buf_LRU_get_free_block(): Note that page_zip->data should be allocated from
an aligned memory pool.

buf_flush_buffered_writes(): Write compressed pages to disk.

buf_flush_post_to_doublewrite_buf(): Copy compressed pages to the
doublewrite buffer.  Zero fill any excess space.

buf_flush_init_for_writing(): Treat all compressed pages the same.

buf_read_page_low(): Read compressed pages from disk.

buf_page_io_complete(): Process compressed pages.

trx_sys_doublewrite_init_or_restore_page(): Process compressed pages.

mlog_write_initial_log_record_fast(): Enable a debug printout
#ifdef UNIV_LOG_DEBUG.

fsp_header_init(), fsp_fill_free_list(): Pass the compressed page size
to buf_page_create().

page_zip_compress_write_log(): Flatten the if-else if-else logic.

page_zip_parse_write_blob_ptr(): Do not test page_zip if page==NULL.

page_zip_parse_write_node_ptr(): Do not test page_zip if page==NULL.
Invoke mlog_close() correctly.

row_sel_store_row_id_to_prebuilt(): Add UNIV_UNLIKELY hint to an
assertion-like test.
2006-05-30 09:04:57 +00:00
marko
662318ba2c branches/zip: buf_flush_buffered_writes(): Disable the checks on compressed
pages.

buf_flush_post_to_doublewrite_buf(): Add a TODO comment about compressed pages.
2006-05-24 14:35:22 +00:00
marko
80915cdf91 branches/zip: Merge revisions 560:583 from trunk. 2006-05-22 09:30:34 +00:00
marko
927ed2fd56 branches/zip: buf_flush_buffered_writes(): Replace four occurrences of
mach_read_from_4(...) != mach_read_from_4(...) with memcmp(..., ..., 4).
2006-05-22 09:23:30 +00:00
marko
b15d26d6e2 branches/zip: Write the compressed page size to SYS_TABLES.TYPE and adjust
a few places accordingly.

os_aio_simulated_handle(): Add TODO comments about skipping the write
checks for compressed pages.

dict_create_sys_tables_tuple(): Write the compressed page size to
the TYPE column.

open_or_create_data_files(): Simplify the fil_node_create() call.

fil_node_create(): Do not touch space->zip_size.  It was already initialized
by fil_space_create().

fil_reset_too_high_lsns(), buf_flush_buffered_writes(): Add TODO comment
about compressed pages.

buf_flush_init_for_writing(): Handle pages of type FIL_PAGE_INODE,
FIL_PAGE_IBUF_BITMAP, and FIL_PAGE_TYPE_FSP_HDR as uncompressed ones.
2006-05-17 12:03:32 +00:00
marko
168fa020f3 branches/zip: Try to reorganize the page when compression fails.
page_zip_compress_write_log(): Make static.

page_zip_compress(): Add optional parameter mtr for redo logging.

page_zip_reorganize(): Low-level counterpart of btr_page_reorganize().

page_zip_copy(): Add debug assertions about mtr_memo_contains.

page_cur_insert_rec_low(): Try page_zip_reorganize() and seek to the
new position of insert_rec if it succeeds.

page_copy_rec_list_end(), page_copy_rec_list_start():
Try page_zip_reorganize().

page_move_rec_list_end(): Remove bogus comment.
2006-05-16 14:17:43 +00:00
marko
2779bdc7ff branches/zip: btr_root_raise_and_insert(): Remove a bogus assertion. 2006-05-16 12:01:20 +00:00
marko
2c3b224796 branches/zip: Shorten the log record MLOG_ZIP_PAGE_COMPRESS.
page_zip_copy(), page_zip_compress_write_log(): Add parameter 'index'.

page_zip_parse_write_header(): Check for !page_zip only if page != NULL.

page_zip_compress_write_log(), page_zip_parse_compress(): Omit some
fields in the page header.  Omit the unused bytes between the modification
log and the page trailer.

parse_or_apply_log_rec_body(): Remove a bogus debug assertion.
2006-05-16 07:58:10 +00:00
marko
64678b8bc9 branches/zip: btr_page_split_and_insert(): Implement fallbacks for
page_move_rec_list_start() and page_move_rec_list_end().  Relax the
conditions on insert_will_fit if page_zip is set.

page_delete_rec_list_start(): Make public.

page_zip_copy(): Clear REC_INFO_MIN_REC_FLAG if FIL_PAGE_PREV != FIL_NULL.
2006-05-15 11:43:35 +00:00
marko
9c7913d92a branches/zip: btr_compress(): When merging a compressed page to the right,
set the FIL_PAGE_PREV field of merge_page to FIL_NULL before copying the
records in order not to break the assumption of page_zip_compress() that
min_rec_mark is always set on the first user record of a non-leaf page
whose FIL_PAGE_PREV field is FIL_NULL, and never otherwise.
2006-05-12 13:49:04 +00:00
marko
1d406a014d branches/zip: Fix a bug that was triggered during purge.
btr_compress(): Defer any tree modifications until the records
have successfully been copied.

row_purge_remove_sec_if_poss_low(): Simplify the setting of
the return status.
2006-05-12 11:36:17 +00:00
marko
27d21c0f89 branches/zip: Remove a potential assertion failure.
btr_lift_page_up(): If page_copy_rec_list_end() fails, resort to
page_zip_copy().

page_zip_copy(): Validate the copy of the page.
2006-05-12 09:35:56 +00:00
marko
3b21e1478b branches/zip: After merge fix, and cleanup.
btr_validate_level(): Add missing parameter to added buf_page_print() calls.

btr_root_raise_and_insert(): Replace low-level code with page_zip_copy().

page_zip_copy(): New function to copy the data payload of a compressed page,
byte for byte.  The file page header and trailer are not copied, to avoid
overwriting the fields FIL_PAGE_OFFSET, FIL_PAGE_PREV and FIL_PAGE_NEXT.
2006-05-11 18:23:10 +00:00
marko
24dce67db4 branches/zip: Merge revisions 558:560 from trunk. 2006-05-11 17:00:43 +00:00
marko
e47742cbef branches/zip: Merge revisions 536:558 from trunk. 2006-05-11 12:47:23 +00:00
marko
e69d2f5763 branches/zip: Minor cleanup.
btr_cur_compress_if_useful(): Replace if () return(); return() with return.

page_rec_get_next_low(): New function.

page_rec_get_prev(): Invoke page_is_comp() outside the loop.
Replace page_rec_get_next() with loop-specific instances of
page_rec_get_next_low().

page_copy_rec_list_end(): Add some debug assertions.
2006-05-11 12:31:22 +00:00
marko
5073bb5015 branches/zip: btr_compress(): Do not modify the node pointer until the
records have successfully been copied.
2006-05-10 18:01:41 +00:00
marko
aa852a3b72 branches/zip: Remove GCC 4.0.4 warnings from
make AM_CFLAGS='-std=c89 -Wall -W -pedantic'
2006-05-10 13:58:35 +00:00
marko
3547f43d22 branches/zip: Fix bugs.
btr_root_raise_and_insert(): When copying root to new_page byte for byte,
restore the page number of new_page afterwards.

buf_flush_init_for_writing(): For FIL_PAGE_INDEX, write the page number
and space id also to the uncompressed page.
2006-05-10 11:28:14 +00:00
marko
b829657046 branches/zip: Fix bugs triggered by running out of space.
btr_root_raise_and_insert(): If page_copy_rec_list_end() fails,
copy the pages byte for byte.

page_zip_compress(): Ensure that the uncompressed storage area will
fit on the compressed page.
2006-05-10 07:15:07 +00:00
marko
8099eba891 branches/zip: Add detailed debugging output to page_zip_compress(). 2006-05-09 16:09:09 +00:00
marko
32e6f3f292 branches/zip: btr_cur_optimistic_insert(): Fail gracefully if
reorganizing the page fails when compressed pages are enabled.
2006-05-08 13:30:46 +00:00
marko
57df0547fd branches/zip: Temporarily allow the compressed page size to be specified
in CREATE TABLE and ALTER TABLE with AVG_ROW_LENGTH={1,2,4,8,16}.
2006-05-08 12:20:34 +00:00
marko
5d8a41d9b5 branches/zip: Merge revisions 532:536 from trunk. 2006-05-08 09:37:03 +00:00
marko
d30e44bc32 branches/zip: Merge revisions 465:532 from trunk. 2006-05-08 06:18:59 +00:00
marko
a9ee619a9b branches/zip: Minor improvements.
Introduce FIL_PAGE_ZBLOB_DATA as a synonym for FIL_PAGE_FILE_FLUSH_LSN.

btr_store_big_rec_extern_fields(): Make the assertion about
dict_table_zip_size() more accurate.

buf_LRU_get_free_block(), buf_block_alloc(): Add parameter zip_size.

buf_calc_zblob_page_checksum(): Remove. Replace with page_zip_calc_checksum().

buf_page_init(): Remove parameter zip_size.

buf_page_io_complete(): Add a placeholder for handling compressed pages.

trx_doublewrite_page_inside(): Remove redundant function.

page_zip_write_rec(): Relax an overly tight assertion about blob_no.
2006-05-04 11:44:49 +00:00
marko
f118a9c19f branches/zip: Prepare for writing compressed and reading pages.
buf_page_print(): Print also compressed pages.  Add parameter zip_size.

buf_flush_init_for_writing(): Stamp the fields on a compressed B-tree index
page.

Add the header field FIL_PAGE_ZBLOB_SPACE_ID as an alias of FIL_PAGE_PREV.

page_zip_calc_checksum(): New function.

page_zip_compress(): Avoid copying the fields that are written in
buf_flush_init_for_writing().

page_zip_header_cmp(): New function for comparing those fields of the
page header that will not be written in buf_flush_init_for_writing().
2006-05-02 11:44:39 +00:00
marko
7d6a4e7058 branches/zip: page_cur_insert_rec_low(): Write a log record of
recompressing the page.
2006-04-28 10:53:24 +00:00
marko
bb6387683a branches/zip: Minor improvements.
buf_flush_init_for_writing(): Calculate the checksum with the actual zip_size.

buf_calc_zblob_page_checksum(): Skip the field FIL_PAGE_SPACE_OR_CHKSUM.

trx_sys_doublewrite_init_or_restore_page(): Use the actual zip_size.

page_cur_insert_rec_low(): If page_zip_alloc() fails, try compressing the
whole page afterwards.
2006-04-28 09:30:40 +00:00
marko
5d591705bb branches/zip: btr_lift_page_up(): Set level before page_zip_compress(). 2006-04-27 10:48:12 +00:00
marko
ed354ce80c branches/zip: Minor cleanup and bug fixes
btr_page_reorganize_low(): Rename new_page to temp_page.

btr_store_big_rec_extern_fields(): FIL_PAGE_TYPE is 2 bytes, not 4.

buf_page_init(), buf_page_create(), buf_read_page_low(),
buf_page_init_for_read(): Add parameter zip_size.

buf_page_init_for_backup_restore(),
recv_apply_log_recs_for_backup(): Enclose in #ifdef UNIV_HOTBACKUP.

Enclose some debug code in #ifdef UNIV_LOG_REPLICATE.

page_zip_write_header_log(): Replace page_zip with a pointer to
the uncompressed page.

page_zip_write_rec(): Relax an assertion about blob_no + n_ext.

page_copy_rec_list_to_created_page_write_log(): Allow logging to be disabled.
2006-04-26 09:35:18 +00:00
marko
6dd1d8c4fe branches/zip: Write the compressed page size to SYS_TABLES.TYPE
and to the file space header (FSP_PAGE_ZIP_SIZE, renamed from
FSP_LOWEST_NO_WRITE).

fil_space_struct: Add zip_size.

dict_table_struct: Embed zip_size in flags.

dict_table_zip_size(): Infer zip_size from table->flags.

dict_sys_tables_get_zip_size(): Read zip_size from SYS_TABLES.TYPE.

fil_space_get_zip_size(): Read zip_size from the file space header.

Add the redo log entry type MLOG_ZIP_FILE_CREATE.
2006-04-25 07:12:32 +00:00
marko
802c09ad0a branches/zip: Merge revisions 459:465 from trunk. 2006-04-12 12:39:58 +00:00
marko
0c4dfda3be branches/zip: Merge revisions 265:459 from trunk. 2006-04-12 09:32:17 +00:00
marko
483faa7720 branches/zip: Fix some crash recovery bugs.
page_zip_alloc(): Add parameter "mtr" and log successful calls
to page_zip_compress().

page_zip_write_blob_ptr(), page_zip_write_node_ptr(): Write the offset on
the uncompressed page, because mlog_write_initial_log_record_fast()
does not do so.

page_zip_write_header_log(), page_zip_parse_write_header(): Encode the
offset in one byte.
2006-04-11 18:40:44 +00:00
marko
f119108ff3 branches/zip: Add the redo log type MLOG_ZIP_PAGE_COMPRESS. Remove
MLOG_ZIP_LIST_START_COPY and MLOG_ZIP_LIST_END_COPY.

btr_compress(): Simplify a debug assertion.

page_zip_compress_write_log(), page_zip_parse_compress(): New functions.

page_cur_parse_insert_rec(): Simplify the code.

page_parse_create_zip(): Removed.

page_create_zip(), page_copy_rec_list_end(),
page_copy_rec_list_start(): Invoke page_zip_compress_write_log().
2006-04-11 12:27:06 +00:00
marko
c65c50ae69 branches/zip: Minor cleanup. Remove MLOG_ZIP_ROOT_RAISE.
btr_page_create(): Remove parameters "prev" and "next".  The pointers
to adjacent B-tree pages will be set in the caller before any user
records are inserted.

Remove the redo log entry type MLOG_ZIP_ROOT_RAISE, since
btr_root_raise_and_insert() will not defer modifications to
the compressed page.

btr_attach_half_pages(): Add parameter new_page_zip.
2006-04-11 09:02:27 +00:00
marko
e20dfbc0ce branches/zip: Disable accidentally committed debug code that would break
crash recovery of uncompressed tables.
2006-04-11 08:12:39 +00:00
marko
9b6200978d branches/zip: Replace the redo log entry types
MLOG_ZIP_COMPRESS and MLOG_ZIP_DECOMPRESS with higher-level entry types.
Implement the logging and crash recovery of MLOG_ZIP_PAGE_CREATE.

page_create_zip(): New function for creating a compressed B-tree page.

page_parse_create_zip(): New function for applying a MLOG_ZIP_PAGE_CREATE
redo log record.

btr_page_create(): Remove the prototype.  Add parameters page_zip, level,
prev, and next.

btr0btr.c: Eliminate page_zip_compress() calls where possible.

page_zip_alloc(), page_zip_compress(), page_zip_decompress(),
page_zip_clear_rec(): Remove parameter mtr.

recv_parse_or_apply_log_rec_body(): Handle MLOG_ZIP_PAGE_CREATE.
Add TODO comments for the other added redo log entry types.
2006-04-10 19:48:37 +00:00
marko
b4fe38d5f4 branches/zip: btr_lift_page_up(): Remove return value;
assume that the operation always succeeds.
2006-04-10 06:54:14 +00:00
marko
0906cd0cae branches/zip: Implement crash recovery of writing BLOB pointers.
page_zip_parse_write_blob_ptr(): New function for applying the redo log
record MLOG_ZIP_WRITE_BLOB_PTR.

page_zip_write_blob_ptr(): Write the necessary information to the redo log.

page0zip.c: Tighten the assertions to ensure that blob_ptr < page_zip->n_blobs.

page_zip_write_node_ptr(): Use memcpy() instead of mach_write_to_4().
2006-04-07 12:56:22 +00:00
marko
44b22ebec4 branches/zip: Implement crash recovery of writing to the page header.
page_zip_parse_write_header(): New function for applying the redo log of
MLOG_ZIP_WRITE_HEADER.

page_zip_parse_write_node_ptr(): Mark the log corrupted if !page_zip.
2006-04-07 11:44:14 +00:00
marko
52b7eb50c2 branches/zip: Implement the crash recovery of MLOG_ZIP_WRITE_NODE_PTR.
page_zip_parse_write_node_ptr(): New function to apply a redo log of
MLOG_ZIP_WRITE_NODE_PTR.

page_zip_write_node_ptr(): Write all needed information to the redo log.

page_zip_write_header_log(): Write all necessary information to the redo log.
2006-04-07 11:04:08 +00:00
marko
a1661efb8d branches/zip: Initial steps towards disk-based storage of compressed pages.
dict_mem_table_create(): Account for DICT_TF_COMPRESSED in a debug assertion.

btr_store_big_rec_extern_fields(), btr_free_externally_stored_field(),
btr_copy_externally_stored_field(): Implement the disk format for
compressed BLOB pages.

btr_copy_externally_stored_field(): Improve error reporting and handling
when decompressing BLOB pages.

buf_flush_init_for_writing(), buf_page_is_corrupted(), buf_page_print():
Account for compressed BLOB pages (FIL_PAGE_TYPE_ZBLOB).

buf_calc_zblob_page_checksum(): New function.
2006-04-05 13:41:12 +00:00
marko
7b57df2298 branches/zip: Minor cleanup.
Replace btr_page_get_level() with page_is_leaf() where possible.

row_purge_upd_exist_or_extern(): Remove obsolete TODO comment.

dtuple_convert_big_rec(): Replace a flag variable with goto.
2006-04-04 10:42:05 +00:00
marko
e14e718664 branches/zip: Fix some BLOB handling bugs.
btr_store_big_rec_extern_fields(): Assert that page_zip is non-NULL
if and only if dict_table_is_zip() holds.

btr_free_externally_stored_field(): Observe dict_table_is_zip().
Allow page_zip==NULL even if dict_table_is_zip().  Remove the
related TODO comment in row_purge_upd_exist_or_extern().

page_zip_available(): uncompressed_size already includes
PAGE_ZIP_DIR_SLOT_SIZE.

page_zip_decompress(): Remove bogus assertion d_stream.next_out == last.
Do not subtract BTR_EXTERN_FIELD_REF_SIZE from d_stream.avail_in when
decompressing records, because the records may be deleted later in
page_zip_apply_log(), and no BLOB pointers are allocated for deleted
records.
2006-04-03 20:33:31 +00:00