Commit graph

250 commits

Author SHA1 Message Date
marko
5d5bdb1b43 branches/zip: page_zip_dir_size(): Remove bogus debug assertion.
The function will be invoked in page_zip_decompress() before setting
page_zip->m_start or page_zip->m_end.
2006-06-30 11:59:28 +00:00
marko
7279a72289 branches/zip: Improve assertions related to the linked lists of records
on B-tree index pages.

page_rec_set_next(): Assert that rec != next.

rec_get_next_ptr(), rec_get_next_offs(): On compact pages, assert that
there are at least REC_N_NEW_EXTRA_BYTES + 1 between records.

page_cur_insert_rec_write_log(): Replace a buf_frame_align() call
with ut_align_offset().

page_cur_insert_rec_low(): Assert that current_rec != insert_rec.
2006-06-30 09:49:32 +00:00
marko
9972604267 branches/zip: Eliminate page corruption in btr_compress() when
page_zip_reorganize() was invoked.

btr_compress(): Obtain orig_pred and orig_succ after copying the records.
Add a debug assertion about FIL_PAGE_PREV.

page_copy_rec_list_end(), page_copy_rec_list_start(): Change the return
type from ibool to rec_t.  Adjust the return value after invoking
page_zip_reorganize().
2006-06-28 12:01:29 +00:00
marko
f7e2b57faa branches/zip: Tighten the assertions about page_zip->m_end. 2006-06-21 11:24:47 +00:00
marko
bbd6d812d3 branches/zip: Add debug assertions about page_zip->m_end < paeg_zip->m_size. 2006-06-21 09:15:09 +00:00
marko
7bc09e8c18 branches/zip: Make page_zip_validate() independent of UNIV_DEBUG.
Replace most occurrences of #if defined UNIV_DEBUG || defined UNIV_ZIP_DEBUG
with #ifdef UNIV_ZIP_DEBUG.
2006-06-20 19:35:59 +00:00
marko
3e02b8dbe5 branches/zip: page_zip_reorganize(): On success, write to the redo log. 2006-06-19 11:07:21 +00:00
marko
fa846a7be5 branches/zip: page_zip_validate(): First compare the page headers
and obey the added flag page_zip_validate_header_only.
2006-06-14 08:15:44 +00:00
marko
781f1ca5b0 branches/zip: Add page_zip_validate() assertions and remove a bogus
page_zip_assertion() failure in page_copy_rec_list_end().

btr_root_raise_and_insert(), btr_lift_page_up(), btr_compress():
Add page_zip_validate() assertions.

btr_compress(): Only copy FIL_PAGE_PREV when UNIV_BTR_DEBUG is defined.

page_cur_delete_rec(): Document why the page_zip_validate() assertion
was removed.

page_copy_rec_list_end(): Remove the page_zip_validate() assertion and
document why.

page_move_rec_list_end(): Add page_zip_validate() assertion.
2006-06-13 08:24:26 +00:00
marko
d228053315 branches/zip: Add page_zip_validate() checks.
page_cur_delete_rec(): Do not call page_zip_validate() in the beginning,
because btr_set_min_rec_mark() in btr_cur_pessimistic_delete() will
cause a temporary mismatch.

Document temporary mismatches caused by btr_set_min_rec_mark() calls
and explain why they will not cause any problems.
2006-06-12 12:37:54 +00:00
marko
0e6236a625 branches/zip: Replace all ut_ad(page_zip_validate()) assertions with
ut_a(page_zip_validate()) that are enabled if UNIV_DEBUG or
UNIV_ZIP_DEBUG is defined.
2006-06-07 11:06:12 +00:00
marko
831916878d branches/zip: Write compressed pages to disk.
os_aio_simulated_handle(): Temporarily disable os_file_check_page_trailers(),
which cannot be invoked on compressed pages.

dict_table_add_system_columns(): New function, split from
dict_table_add_to_cache().

mlog_parse_index(): Add system columns to the dummy table and identify
DB_TRX_ID and DB_ROLL_PTR in the dummy index.

buf_LRU_get_free_block(): Note that page_zip->data should be allocated from
an aligned memory pool.

buf_flush_buffered_writes(): Write compressed pages to disk.

buf_flush_post_to_doublewrite_buf(): Copy compressed pages to the
doublewrite buffer.  Zero fill any excess space.

buf_flush_init_for_writing(): Treat all compressed pages the same.

buf_read_page_low(): Read compressed pages from disk.

buf_page_io_complete(): Process compressed pages.

trx_sys_doublewrite_init_or_restore_page(): Process compressed pages.

mlog_write_initial_log_record_fast(): Enable a debug printout
#ifdef UNIV_LOG_DEBUG.

fsp_header_init(), fsp_fill_free_list(): Pass the compressed page size
to buf_page_create().

page_zip_compress_write_log(): Flatten the if-else if-else logic.

page_zip_parse_write_blob_ptr(): Do not test page_zip if page==NULL.

page_zip_parse_write_node_ptr(): Do not test page_zip if page==NULL.
Invoke mlog_close() correctly.

row_sel_store_row_id_to_prebuilt(): Add UNIV_UNLIKELY hint to an
assertion-like test.
2006-05-30 09:04:57 +00:00
marko
6c3c7271c9 branches/zip: Try to reorganize the page when compression fails.
page_zip_compress_write_log(): Make static.

page_zip_compress(): Add optional parameter mtr for redo logging.

page_zip_reorganize(): Low-level counterpart of btr_page_reorganize().

page_zip_copy(): Add debug assertions about mtr_memo_contains.

page_cur_insert_rec_low(): Try page_zip_reorganize() and seek to the
new position of insert_rec if it succeeds.

page_copy_rec_list_end(), page_copy_rec_list_start():
Try page_zip_reorganize().

page_move_rec_list_end(): Remove bogus comment.
2006-05-16 14:17:43 +00:00
marko
00b55da692 branches/zip: Shorten the log record MLOG_ZIP_PAGE_COMPRESS.
page_zip_copy(), page_zip_compress_write_log(): Add parameter 'index'.

page_zip_parse_write_header(): Check for !page_zip only if page != NULL.

page_zip_compress_write_log(), page_zip_parse_compress(): Omit some
fields in the page header.  Omit the unused bytes between the modification
log and the page trailer.

parse_or_apply_log_rec_body(): Remove a bogus debug assertion.
2006-05-16 07:58:10 +00:00
marko
0aa8b2cc74 branches/zip: btr_page_split_and_insert(): Implement fallbacks for
page_move_rec_list_start() and page_move_rec_list_end().  Relax the
conditions on insert_will_fit if page_zip is set.

page_delete_rec_list_start(): Make public.

page_zip_copy(): Clear REC_INFO_MIN_REC_FLAG if FIL_PAGE_PREV != FIL_NULL.
2006-05-15 11:43:35 +00:00
marko
d02cde9a66 branches/zip: Remove a potential assertion failure.
btr_lift_page_up(): If page_copy_rec_list_end() fails, resort to
page_zip_copy().

page_zip_copy(): Validate the copy of the page.
2006-05-12 09:35:56 +00:00
marko
93fe682c17 branches/zip: After merge fix, and cleanup.
btr_validate_level(): Add missing parameter to added buf_page_print() calls.

btr_root_raise_and_insert(): Replace low-level code with page_zip_copy().

page_zip_copy(): New function to copy the data payload of a compressed page,
byte for byte.  The file page header and trailer are not copied, to avoid
overwriting the fields FIL_PAGE_OFFSET, FIL_PAGE_PREV and FIL_PAGE_NEXT.
2006-05-11 18:23:10 +00:00
marko
17eb5eab4d branches/zip: Minor cleanup.
btr_cur_compress_if_useful(): Replace if () return(); return() with return.

page_rec_get_next_low(): New function.

page_rec_get_prev(): Invoke page_is_comp() outside the loop.
Replace page_rec_get_next() with loop-specific instances of
page_rec_get_next_low().

page_copy_rec_list_end(): Add some debug assertions.
2006-05-11 12:31:22 +00:00
marko
6748a82b84 branches/zip: Remove GCC 4.0.4 warnings from
make AM_CFLAGS='-std=c89 -Wall -W -pedantic'
2006-05-10 13:58:35 +00:00
marko
6bde840828 branches/zip: Fix bugs triggered by running out of space.
btr_root_raise_and_insert(): If page_copy_rec_list_end() fails,
copy the pages byte for byte.

page_zip_compress(): Ensure that the uncompressed storage area will
fit on the compressed page.
2006-05-10 07:15:07 +00:00
marko
e1dbb801e3 branches/zip: Add detailed debugging output to page_zip_compress(). 2006-05-09 16:09:09 +00:00
marko
f258491a26 branches/zip: Minor improvements.
Introduce FIL_PAGE_ZBLOB_DATA as a synonym for FIL_PAGE_FILE_FLUSH_LSN.

btr_store_big_rec_extern_fields(): Make the assertion about
dict_table_zip_size() more accurate.

buf_LRU_get_free_block(), buf_block_alloc(): Add parameter zip_size.

buf_calc_zblob_page_checksum(): Remove. Replace with page_zip_calc_checksum().

buf_page_init(): Remove parameter zip_size.

buf_page_io_complete(): Add a placeholder for handling compressed pages.

trx_doublewrite_page_inside(): Remove redundant function.

page_zip_write_rec(): Relax an overly tight assertion about blob_no.
2006-05-04 11:44:49 +00:00
marko
7bd3aca4ad branches/zip: Prepare for writing compressed and reading pages.
buf_page_print(): Print also compressed pages.  Add parameter zip_size.

buf_flush_init_for_writing(): Stamp the fields on a compressed B-tree index
page.

Add the header field FIL_PAGE_ZBLOB_SPACE_ID as an alias of FIL_PAGE_PREV.

page_zip_calc_checksum(): New function.

page_zip_compress(): Avoid copying the fields that are written in
buf_flush_init_for_writing().

page_zip_header_cmp(): New function for comparing those fields of the
page header that will not be written in buf_flush_init_for_writing().
2006-05-02 11:44:39 +00:00
marko
7a54aa4d02 branches/zip: page_cur_insert_rec_low(): Write a log record of
recompressing the page.
2006-04-28 10:53:24 +00:00
marko
0cb4c58fa2 branches/zip: Minor improvements.
buf_flush_init_for_writing(): Calculate the checksum with the actual zip_size.

buf_calc_zblob_page_checksum(): Skip the field FIL_PAGE_SPACE_OR_CHKSUM.

trx_sys_doublewrite_init_or_restore_page(): Use the actual zip_size.

page_cur_insert_rec_low(): If page_zip_alloc() fails, try compressing the
whole page afterwards.
2006-04-28 09:30:40 +00:00
marko
b0ae76c51a branches/zip: Minor cleanup and bug fixes
btr_page_reorganize_low(): Rename new_page to temp_page.

btr_store_big_rec_extern_fields(): FIL_PAGE_TYPE is 2 bytes, not 4.

buf_page_init(), buf_page_create(), buf_read_page_low(),
buf_page_init_for_read(): Add parameter zip_size.

buf_page_init_for_backup_restore(),
recv_apply_log_recs_for_backup(): Enclose in #ifdef UNIV_HOTBACKUP.

Enclose some debug code in #ifdef UNIV_LOG_REPLICATE.

page_zip_write_header_log(): Replace page_zip with a pointer to
the uncompressed page.

page_zip_write_rec(): Relax an assertion about blob_no + n_ext.

page_copy_rec_list_to_created_page_write_log(): Allow logging to be disabled.
2006-04-26 09:35:18 +00:00
marko
121623c1b4 branches/zip: Write the compressed page size to SYS_TABLES.TYPE
and to the file space header (FSP_PAGE_ZIP_SIZE, renamed from
FSP_LOWEST_NO_WRITE).

fil_space_struct: Add zip_size.

dict_table_struct: Embed zip_size in flags.

dict_table_zip_size(): Infer zip_size from table->flags.

dict_sys_tables_get_zip_size(): Read zip_size from SYS_TABLES.TYPE.

fil_space_get_zip_size(): Read zip_size from the file space header.

Add the redo log entry type MLOG_ZIP_FILE_CREATE.
2006-04-25 07:12:32 +00:00
marko
cb6a8cc100 branches/zip: Fix some crash recovery bugs.
page_zip_alloc(): Add parameter "mtr" and log successful calls
to page_zip_compress().

page_zip_write_blob_ptr(), page_zip_write_node_ptr(): Write the offset on
the uncompressed page, because mlog_write_initial_log_record_fast()
does not do so.

page_zip_write_header_log(), page_zip_parse_write_header(): Encode the
offset in one byte.
2006-04-11 18:40:44 +00:00
marko
829229ce73 branches/zip: Add the redo log type MLOG_ZIP_PAGE_COMPRESS. Remove
MLOG_ZIP_LIST_START_COPY and MLOG_ZIP_LIST_END_COPY.

btr_compress(): Simplify a debug assertion.

page_zip_compress_write_log(), page_zip_parse_compress(): New functions.

page_cur_parse_insert_rec(): Simplify the code.

page_parse_create_zip(): Removed.

page_create_zip(), page_copy_rec_list_end(),
page_copy_rec_list_start(): Invoke page_zip_compress_write_log().
2006-04-11 12:27:06 +00:00
marko
16dddacb6f branches/zip: Replace the redo log entry types
MLOG_ZIP_COMPRESS and MLOG_ZIP_DECOMPRESS with higher-level entry types.
Implement the logging and crash recovery of MLOG_ZIP_PAGE_CREATE.

page_create_zip(): New function for creating a compressed B-tree page.

page_parse_create_zip(): New function for applying a MLOG_ZIP_PAGE_CREATE
redo log record.

btr_page_create(): Remove the prototype.  Add parameters page_zip, level,
prev, and next.

btr0btr.c: Eliminate page_zip_compress() calls where possible.

page_zip_alloc(), page_zip_compress(), page_zip_decompress(),
page_zip_clear_rec(): Remove parameter mtr.

recv_parse_or_apply_log_rec_body(): Handle MLOG_ZIP_PAGE_CREATE.
Add TODO comments for the other added redo log entry types.
2006-04-10 19:48:37 +00:00
marko
1f8b0f3835 branches/zip: Implement crash recovery of writing BLOB pointers.
page_zip_parse_write_blob_ptr(): New function for applying the redo log
record MLOG_ZIP_WRITE_BLOB_PTR.

page_zip_write_blob_ptr(): Write the necessary information to the redo log.

page0zip.c: Tighten the assertions to ensure that blob_ptr < page_zip->n_blobs.

page_zip_write_node_ptr(): Use memcpy() instead of mach_write_to_4().
2006-04-07 12:56:22 +00:00
marko
817ca61f77 branches/zip: Implement crash recovery of writing to the page header.
page_zip_parse_write_header(): New function for applying the redo log of
MLOG_ZIP_WRITE_HEADER.

page_zip_parse_write_node_ptr(): Mark the log corrupted if !page_zip.
2006-04-07 11:44:14 +00:00
marko
684651ce2e branches/zip: Implement the crash recovery of MLOG_ZIP_WRITE_NODE_PTR.
page_zip_parse_write_node_ptr(): New function to apply a redo log of
MLOG_ZIP_WRITE_NODE_PTR.

page_zip_write_node_ptr(): Write all needed information to the redo log.

page_zip_write_header_log(): Write all necessary information to the redo log.
2006-04-07 11:04:08 +00:00
marko
f4c776f518 branches/zip: Initial steps towards disk-based storage of compressed pages.
dict_mem_table_create(): Account for DICT_TF_COMPRESSED in a debug assertion.

btr_store_big_rec_extern_fields(), btr_free_externally_stored_field(),
btr_copy_externally_stored_field(): Implement the disk format for
compressed BLOB pages.

btr_copy_externally_stored_field(): Improve error reporting and handling
when decompressing BLOB pages.

buf_flush_init_for_writing(), buf_page_is_corrupted(), buf_page_print():
Account for compressed BLOB pages (FIL_PAGE_TYPE_ZBLOB).

buf_calc_zblob_page_checksum(): New function.
2006-04-05 13:41:12 +00:00
marko
ed3d577301 branches/zip: Minor cleanup.
Replace btr_page_get_level() with page_is_leaf() where possible.

row_purge_upd_exist_or_extern(): Remove obsolete TODO comment.

dtuple_convert_big_rec(): Replace a flag variable with goto.
2006-04-04 10:42:05 +00:00
marko
ac5b886f4f branches/zip: Fix some BLOB handling bugs.
btr_store_big_rec_extern_fields(): Assert that page_zip is non-NULL
if and only if dict_table_is_zip() holds.

btr_free_externally_stored_field(): Observe dict_table_is_zip().
Allow page_zip==NULL even if dict_table_is_zip().  Remove the
related TODO comment in row_purge_upd_exist_or_extern().

page_zip_available(): uncompressed_size already includes
PAGE_ZIP_DIR_SLOT_SIZE.

page_zip_decompress(): Remove bogus assertion d_stream.next_out == last.
Do not subtract BTR_EXTERN_FIELD_REF_SIZE from d_stream.avail_in when
decompressing records, because the records may be deleted later in
page_zip_apply_log(), and no BLOB pointers are allocated for deleted
records.
2006-04-03 20:33:31 +00:00
marko
5a6c5d4010 branches/zip: Fix some bugs
btr_page_split_and_insert(): Avoid dereferencing pointers to garbage on
the old page.

btr_cur_pessimistic_insert(): Pass pointer to big_rec_vec to
btr_cur_optimistic_insert().

trx_undo_prev_version_build(): Only invoke rec_set_field_extern_bits()
if n_ext_vect > 0.

row_ins_index_entry_low(): Simplify a debug assertion.

page_copy_rec_list_end_no_locks(): Make the loop slightly more readable.

page_delete_rec_list_end(): Delete records on compressed pages one by one.
2006-04-03 11:19:01 +00:00
marko
e7165957a1 branches/zip: Minor fixes for BLOB handling.
page_zip_get_n_prev_extern(): Ignore deleted records.

page_zip_write_rec(): Remove bogus debug assertion about the BLOB pointer
being zero-filled.  It would fail when reallocating records from the page
heap or when updating records in place.
2006-03-30 11:47:10 +00:00
marko
23dc2e55c2 branches/zip: Correctly identify deleted records when deciding whether to
copy BLOB pointers.

page_zip_dir_find_free_low(): New function,
split from page_zip_dir_find_free().

Add comments about processing the records in heap_no order.

Fix some typographic errors in comments and improve formatting.

page_zip_decompress(): Initialize (clear) the BLOB pointers in deleted records.

page_zip_clear_rec(): Relocate page_zip_validate() assertions, so that they
will not fail if a record containing BLOB pointers is being deleted.

Note that page_zip_validate() will fail if page_zip_clear_rec() is unable
to clear the record.
2006-03-29 14:08:57 +00:00
marko
d21b11ba95 branches/zip: Fix two bugs.
page_zip_decompress(): Pass size = d_stream.avail_in + 1
to page_zip_apply_log(), since the terminating NUL byte is not included
in the space reserved for the compressed data stream.

page_zip_clear_rec(): Clear also node pointer fields.
2006-03-28 07:57:47 +00:00
marko
a179b68f9b Rename the static page_zip_alloc() to page_zip_malloc() to resolve a clash. 2006-03-27 12:30:25 +00:00
marko
dc4d373274 branches/zip: Implement custom memory management for zlib.
Wrap ut_malloc() and ut_free() as page_zip_alloc() and page_zip_free().
2006-03-27 12:27:53 +00:00
marko
e2b4592d6d branches/zip: Do not clear the records in page_delete_rec_list_end().
page_delete_rec_list_end(): Do not invoke page_zip_clear_rec().

page_zip_clear_rec(): Remove the global declaration.  Add an assertion
that the record must be on the free list.  Copy onl rec_offs_data_size()
bytes before attempting to clear the record and to compress the page.

page_zip_compress(), page_zip_write_rec(): Remove debug assertions that
trx_id, roll_ptr and node_ptr of deleted records be zero-filled.
2006-03-23 08:13:08 +00:00
marko
96954515be branches/zip: Remove a bogus assertion.
page_zip_write_rec(): Allow the record to be delete-marked.
2006-03-22 14:10:09 +00:00
marko
9903b729fd branches/zip: Bug fix
page_zip_write_trx_id_and_roll_ptr(): Write roll_ptr after trx_id,
not over it.
2006-03-22 10:31:18 +00:00
marko
48dd3e3f5a branches/zip: Minor improvements.
page_cur_search_with_match(): Add a page_zip_validate() debug assertion.

page_validate(): Add a missing space to a fprintf() format string.
2006-03-22 08:05:06 +00:00
marko
0a0e2ff560 branches/zip: Minor improvements.
page_zip_compress(): Add a debug check that the trx_id and roll_ptr columns
of deleted records are zero-filled.  This should provide some detection
against misdirected writes.

page_zip_decompress(): Correct typos in comments.
2006-03-20 15:14:38 +00:00
marko
a7e3a7f7c9 branches/zip: Minor improvements
btr_page_get_father_for_rec(): Remove redundant assertion.
Add UNIV_UNLIKELY to an assertion-like test.

btr_validate_level(): Add debug assertions on page_zip_validate().

page_zip_compress(): Remove bogus assertion.

page_zip_write_rec(): Assert that the record written is neither delete-marked
nor deleted.
2006-03-20 13:21:11 +00:00
marko
f8794abbf1 branches/zip: Minor bug fix. test-insert still does not pass.
page_zip_compress(): Assert that trx_id and roll_ptr are zero for
deleted records.

page_zip_clear_rec(): Remove comment about need for redo logging.
This operationg is already logged on a higher level.

page_delete_rec_list_end(): Always invoke page_zip_clear_rec()
on compressed pages.
2006-03-17 15:36:25 +00:00
marko
b03053a633 branches/zip: Minor fixes
Restore page_zip_clear_rec() to the global scope.
Invoke it in page_delete_rec_list_end().

Add debug assertions to page0zip.c to guard against overwriting data
on the compressed page.
2006-03-17 14:11:57 +00:00