os_aio_simulated_handle(): Temporarily disable os_file_check_page_trailers(),
which cannot be invoked on compressed pages.
dict_table_add_system_columns(): New function, split from
dict_table_add_to_cache().
mlog_parse_index(): Add system columns to the dummy table and identify
DB_TRX_ID and DB_ROLL_PTR in the dummy index.
buf_LRU_get_free_block(): Note that page_zip->data should be allocated from
an aligned memory pool.
buf_flush_buffered_writes(): Write compressed pages to disk.
buf_flush_post_to_doublewrite_buf(): Copy compressed pages to the
doublewrite buffer. Zero fill any excess space.
buf_flush_init_for_writing(): Treat all compressed pages the same.
buf_read_page_low(): Read compressed pages from disk.
buf_page_io_complete(): Process compressed pages.
trx_sys_doublewrite_init_or_restore_page(): Process compressed pages.
mlog_write_initial_log_record_fast(): Enable a debug printout
#ifdef UNIV_LOG_DEBUG.
fsp_header_init(), fsp_fill_free_list(): Pass the compressed page size
to buf_page_create().
page_zip_compress_write_log(): Flatten the if-else if-else logic.
page_zip_parse_write_blob_ptr(): Do not test page_zip if page==NULL.
page_zip_parse_write_node_ptr(): Do not test page_zip if page==NULL.
Invoke mlog_close() correctly.
row_sel_store_row_id_to_prebuilt(): Add UNIV_UNLIKELY hint to an
assertion-like test.
page_zip_compress_write_log(): Make static.
page_zip_compress(): Add optional parameter mtr for redo logging.
page_zip_reorganize(): Low-level counterpart of btr_page_reorganize().
page_zip_copy(): Add debug assertions about mtr_memo_contains.
page_cur_insert_rec_low(): Try page_zip_reorganize() and seek to the
new position of insert_rec if it succeeds.
page_copy_rec_list_end(), page_copy_rec_list_start():
Try page_zip_reorganize().
page_move_rec_list_end(): Remove bogus comment.
page_zip_copy(), page_zip_compress_write_log(): Add parameter 'index'.
page_zip_parse_write_header(): Check for !page_zip only if page != NULL.
page_zip_compress_write_log(), page_zip_parse_compress(): Omit some
fields in the page header. Omit the unused bytes between the modification
log and the page trailer.
parse_or_apply_log_rec_body(): Remove a bogus debug assertion.
page_move_rec_list_start() and page_move_rec_list_end(). Relax the
conditions on insert_will_fit if page_zip is set.
page_delete_rec_list_start(): Make public.
page_zip_copy(): Clear REC_INFO_MIN_REC_FLAG if FIL_PAGE_PREV != FIL_NULL.
btr_validate_level(): Add missing parameter to added buf_page_print() calls.
btr_root_raise_and_insert(): Replace low-level code with page_zip_copy().
page_zip_copy(): New function to copy the data payload of a compressed page,
byte for byte. The file page header and trailer are not copied, to avoid
overwriting the fields FIL_PAGE_OFFSET, FIL_PAGE_PREV and FIL_PAGE_NEXT.
btr_cur_compress_if_useful(): Replace if () return(); return() with return.
page_rec_get_next_low(): New function.
page_rec_get_prev(): Invoke page_is_comp() outside the loop.
Replace page_rec_get_next() with loop-specific instances of
page_rec_get_next_low().
page_copy_rec_list_end(): Add some debug assertions.
btr_root_raise_and_insert(): If page_copy_rec_list_end() fails,
copy the pages byte for byte.
page_zip_compress(): Ensure that the uncompressed storage area will
fit on the compressed page.
Introduce FIL_PAGE_ZBLOB_DATA as a synonym for FIL_PAGE_FILE_FLUSH_LSN.
btr_store_big_rec_extern_fields(): Make the assertion about
dict_table_zip_size() more accurate.
buf_LRU_get_free_block(), buf_block_alloc(): Add parameter zip_size.
buf_calc_zblob_page_checksum(): Remove. Replace with page_zip_calc_checksum().
buf_page_init(): Remove parameter zip_size.
buf_page_io_complete(): Add a placeholder for handling compressed pages.
trx_doublewrite_page_inside(): Remove redundant function.
page_zip_write_rec(): Relax an overly tight assertion about blob_no.
buf_page_print(): Print also compressed pages. Add parameter zip_size.
buf_flush_init_for_writing(): Stamp the fields on a compressed B-tree index
page.
Add the header field FIL_PAGE_ZBLOB_SPACE_ID as an alias of FIL_PAGE_PREV.
page_zip_calc_checksum(): New function.
page_zip_compress(): Avoid copying the fields that are written in
buf_flush_init_for_writing().
page_zip_header_cmp(): New function for comparing those fields of the
page header that will not be written in buf_flush_init_for_writing().
buf_flush_init_for_writing(): Calculate the checksum with the actual zip_size.
buf_calc_zblob_page_checksum(): Skip the field FIL_PAGE_SPACE_OR_CHKSUM.
trx_sys_doublewrite_init_or_restore_page(): Use the actual zip_size.
page_cur_insert_rec_low(): If page_zip_alloc() fails, try compressing the
whole page afterwards.
btr_page_reorganize_low(): Rename new_page to temp_page.
btr_store_big_rec_extern_fields(): FIL_PAGE_TYPE is 2 bytes, not 4.
buf_page_init(), buf_page_create(), buf_read_page_low(),
buf_page_init_for_read(): Add parameter zip_size.
buf_page_init_for_backup_restore(),
recv_apply_log_recs_for_backup(): Enclose in #ifdef UNIV_HOTBACKUP.
Enclose some debug code in #ifdef UNIV_LOG_REPLICATE.
page_zip_write_header_log(): Replace page_zip with a pointer to
the uncompressed page.
page_zip_write_rec(): Relax an assertion about blob_no + n_ext.
page_copy_rec_list_to_created_page_write_log(): Allow logging to be disabled.
and to the file space header (FSP_PAGE_ZIP_SIZE, renamed from
FSP_LOWEST_NO_WRITE).
fil_space_struct: Add zip_size.
dict_table_struct: Embed zip_size in flags.
dict_table_zip_size(): Infer zip_size from table->flags.
dict_sys_tables_get_zip_size(): Read zip_size from SYS_TABLES.TYPE.
fil_space_get_zip_size(): Read zip_size from the file space header.
Add the redo log entry type MLOG_ZIP_FILE_CREATE.
page_zip_alloc(): Add parameter "mtr" and log successful calls
to page_zip_compress().
page_zip_write_blob_ptr(), page_zip_write_node_ptr(): Write the offset on
the uncompressed page, because mlog_write_initial_log_record_fast()
does not do so.
page_zip_write_header_log(), page_zip_parse_write_header(): Encode the
offset in one byte.
MLOG_ZIP_COMPRESS and MLOG_ZIP_DECOMPRESS with higher-level entry types.
Implement the logging and crash recovery of MLOG_ZIP_PAGE_CREATE.
page_create_zip(): New function for creating a compressed B-tree page.
page_parse_create_zip(): New function for applying a MLOG_ZIP_PAGE_CREATE
redo log record.
btr_page_create(): Remove the prototype. Add parameters page_zip, level,
prev, and next.
btr0btr.c: Eliminate page_zip_compress() calls where possible.
page_zip_alloc(), page_zip_compress(), page_zip_decompress(),
page_zip_clear_rec(): Remove parameter mtr.
recv_parse_or_apply_log_rec_body(): Handle MLOG_ZIP_PAGE_CREATE.
Add TODO comments for the other added redo log entry types.
page_zip_parse_write_blob_ptr(): New function for applying the redo log
record MLOG_ZIP_WRITE_BLOB_PTR.
page_zip_write_blob_ptr(): Write the necessary information to the redo log.
page0zip.c: Tighten the assertions to ensure that blob_ptr < page_zip->n_blobs.
page_zip_write_node_ptr(): Use memcpy() instead of mach_write_to_4().
page_zip_parse_write_header(): New function for applying the redo log of
MLOG_ZIP_WRITE_HEADER.
page_zip_parse_write_node_ptr(): Mark the log corrupted if !page_zip.
page_zip_parse_write_node_ptr(): New function to apply a redo log of
MLOG_ZIP_WRITE_NODE_PTR.
page_zip_write_node_ptr(): Write all needed information to the redo log.
page_zip_write_header_log(): Write all necessary information to the redo log.
dict_mem_table_create(): Account for DICT_TF_COMPRESSED in a debug assertion.
btr_store_big_rec_extern_fields(), btr_free_externally_stored_field(),
btr_copy_externally_stored_field(): Implement the disk format for
compressed BLOB pages.
btr_copy_externally_stored_field(): Improve error reporting and handling
when decompressing BLOB pages.
buf_flush_init_for_writing(), buf_page_is_corrupted(), buf_page_print():
Account for compressed BLOB pages (FIL_PAGE_TYPE_ZBLOB).
buf_calc_zblob_page_checksum(): New function.
Replace btr_page_get_level() with page_is_leaf() where possible.
row_purge_upd_exist_or_extern(): Remove obsolete TODO comment.
dtuple_convert_big_rec(): Replace a flag variable with goto.
btr_store_big_rec_extern_fields(): Assert that page_zip is non-NULL
if and only if dict_table_is_zip() holds.
btr_free_externally_stored_field(): Observe dict_table_is_zip().
Allow page_zip==NULL even if dict_table_is_zip(). Remove the
related TODO comment in row_purge_upd_exist_or_extern().
page_zip_available(): uncompressed_size already includes
PAGE_ZIP_DIR_SLOT_SIZE.
page_zip_decompress(): Remove bogus assertion d_stream.next_out == last.
Do not subtract BTR_EXTERN_FIELD_REF_SIZE from d_stream.avail_in when
decompressing records, because the records may be deleted later in
page_zip_apply_log(), and no BLOB pointers are allocated for deleted
records.
btr_page_split_and_insert(): Avoid dereferencing pointers to garbage on
the old page.
btr_cur_pessimistic_insert(): Pass pointer to big_rec_vec to
btr_cur_optimistic_insert().
trx_undo_prev_version_build(): Only invoke rec_set_field_extern_bits()
if n_ext_vect > 0.
row_ins_index_entry_low(): Simplify a debug assertion.
page_copy_rec_list_end_no_locks(): Make the loop slightly more readable.
page_delete_rec_list_end(): Delete records on compressed pages one by one.
page_zip_get_n_prev_extern(): Ignore deleted records.
page_zip_write_rec(): Remove bogus debug assertion about the BLOB pointer
being zero-filled. It would fail when reallocating records from the page
heap or when updating records in place.
copy BLOB pointers.
page_zip_dir_find_free_low(): New function,
split from page_zip_dir_find_free().
Add comments about processing the records in heap_no order.
Fix some typographic errors in comments and improve formatting.
page_zip_decompress(): Initialize (clear) the BLOB pointers in deleted records.
page_zip_clear_rec(): Relocate page_zip_validate() assertions, so that they
will not fail if a record containing BLOB pointers is being deleted.
Note that page_zip_validate() will fail if page_zip_clear_rec() is unable
to clear the record.
page_zip_decompress(): Pass size = d_stream.avail_in + 1
to page_zip_apply_log(), since the terminating NUL byte is not included
in the space reserved for the compressed data stream.
page_zip_clear_rec(): Clear also node pointer fields.
page_delete_rec_list_end(): Do not invoke page_zip_clear_rec().
page_zip_clear_rec(): Remove the global declaration. Add an assertion
that the record must be on the free list. Copy onl rec_offs_data_size()
bytes before attempting to clear the record and to compress the page.
page_zip_compress(), page_zip_write_rec(): Remove debug assertions that
trx_id, roll_ptr and node_ptr of deleted records be zero-filled.
page_zip_compress(): Add a debug check that the trx_id and roll_ptr columns
of deleted records are zero-filled. This should provide some detection
against misdirected writes.
page_zip_decompress(): Correct typos in comments.
btr_page_get_father_for_rec(): Remove redundant assertion.
Add UNIV_UNLIKELY to an assertion-like test.
btr_validate_level(): Add debug assertions on page_zip_validate().
page_zip_compress(): Remove bogus assertion.
page_zip_write_rec(): Assert that the record written is neither delete-marked
nor deleted.
page_zip_compress(): Assert that trx_id and roll_ptr are zero for
deleted records.
page_zip_clear_rec(): Remove comment about need for redo logging.
This operationg is already logged on a higher level.
page_delete_rec_list_end(): Always invoke page_zip_clear_rec()
on compressed pages.
Restore page_zip_clear_rec() to the global scope.
Invoke it in page_delete_rec_list_end().
Add debug assertions to page0zip.c to guard against overwriting data
on the compressed page.
operations on BLOB columns. There are some bugs in the code, because
test-insert and a few other tests fail.
page_mem_free(): Add parameter index. Decrement PAGE_N_RECS here.
Move some operations to page_zip_dir_delete().
page_zip_clear_rec(): Make this a static function.
page_zip_dir_delete(): Add parameters index and offsets.
Decrement PAGE_N_RECS and clear info_bits and n_owned.
page_zip_get_n_prev_extern(): Correct the synopsis and the algorithm.
Add parameter page_zip. Search the records in heap_no order instead
of collation order.
page_zip_compress(), page_zip_decompress(): Only copy BLOB pointers
and increment n_blobs for records that have not been deleted.
page_zip_clear_rec(): Clear trx_id and roll_ptr on the compressed page.
page_zip_dir_delete(): Decrement PAGE_N_RECS. Shift the array of
BLOB pointers. Call page_zip_clear_rec().
page_zip_dir_add_slot(): Shift the array of BLOB pointers to make
space of roll_ptr and trx_id.
page_cur_delete_rec(): Do not decrement PAGE_N_RECS or call
page_zip_clear_rec(), as page_mem_free() already does it.
page_zip_dir_delete() will need to handle BLOBs.
rec_set_field_extern_bits(), rec_set_field_extern_bits_new():
Add parameter offsets.
rec_offs_set_nth_extern(): New function to set an extern bit in offsets.
This will be called when an extern bit is set in a record.
page_cur_rec_insert(), page_cur_insert_rec_low(): Document that the
parameter "offsets" is in/out.
page_zip_dir_delete(): Note that the array of BLOB pointers will need
to be shifted.
page0zip.ic: Document the entry type for clearing a record.
page_zip_available(): Add parameter "index". Remove parameters
"is_leaf" and "is_clustered".
page_zip_get_trailer_len(): New function for computing the trailer length
of the compressed page.
page_zip_apply_log(): Implement the modification log entry type for
clearing the data bytes of a record.
page_zip_decompress(): Initialize n_blobs when actually copying the
BLOB pointers to place.
page_zip_validate(): Add diagnostic messages for failures. Check
also m_start, m_end, and n_blobs.
page_zip_write_blob_ptr(): Add page_zip_validate() assertion.
There are still some bugs in the code.
btr_store_big_rec_extern_fields(): Remove assertion on dict_table_is_zip()
to ease testing.
btr_free_externally_stored_field(): Test page_zip instead of
dict_table_is_zip().
page_zip_write_rec(): Add parameter "create". Try to handle externally
stored columns.
rec_offs_any_extern(): Correct the function comment.
Add rec_offs_n_extern() and page_zip_get_n_prev_extern().
page_zip_dir_decode(): Replace assertion with if (...) return(FALSE).
page_zip_decompress(): Do not clear page_zip->n_blobs after counting the
BLOBs.
page_zip_write_blob_ptr(): Use page_zip_get_n_prev_extern().
Correct an off-by-one error in memcpy().
btr_cur_pessimistic_update(): Remove extraneous page_zip_write_rec() call.
btr_cur_set_ownership_of_extern_field(): Simplify the logic.
row_upd_rec_in_place(): Make use of parameter "index" in debug assertions.
page_zip_write_rec(): Remove TODO comment about redo log record.
The write will already be covered by higher-level log entries.
btr_compress(): Pass the correct page_zip to btr_node_ptr_set_child_page_no().
page_mem_free(): Remove parameter mtr. Do not call page_zip_clear_rec() here.
page_cur_delete_rec(): Call page_zip_clear_rec() here. Decrement PAGE_N_RECS
only after calling page_mem_free(). Assert page_zip_validate().
of clustered indexes. Previously, parts of the code assumed that these
columns would exist on all leaf pages. Simplify the update-in-place of
these columns.
Add inline function dict_index_is_clust() to replace all tests
index->type & DICT_CLUSTERED.
Remove the redo log entry types MLOG_ZIP_WRITE_TRX_ID and
MLOG_ZIP_WRITE_ROLL_PTR, because the modifications to these columns
are covered by logical logging.
Fuse page_zip_write_trx_id() and page_zip_write_roll_ptr() into
page_zip_write_trx_id_and_roll_ptr().
page_zip_dir_add_slot(), page_zip_available(): Add flag "is_clustered",
so that no space will be reserved for TRX_ID and ROLL_PTR on leaf pages
of secondary indexes.
page_zip_apply_log(): Flag an error when val==0 is encoded with two bytes.
page_zip_write_rec(): Add debug assertions that there is enough space
available for the entry before copying the data bytes of the record.
page_dir_split_slot(): Do not pass page_zip to page_dir_slot_set_n_owned(),
because the caller will rewrite the dense page directory later. At this
point of execution, the dense directory will not contain the newly inserted
record, which may end up being written to new_slot.
btr_root_raise_and_insert(): Because btr_page_set_level() must not
change level from non-zero to zero on compressed pages, invoke
btr_page_set_level() with page_zip==NULL and compress the entire
root page after creating it from the scratch.
btr_attach_half_pages(): Simplify the computation of lower_page_zip
and upper_page_zip. Invoke btr_node_ptr_set_child_page_no() with
the correct page_zip.
page0page.h: Add __attribute__((const)) to functions testing for
infimum or supremum.
page_zip_dir_delete(): Note that the third parameter may be NULL.
Correct offset errors.
page_zip_available(): Use n_heap instead of n_recs.
page_zip_dir_find(), page_zip_dir_find_free(): Fix off-by-one error.
page_zip_fields_encode(), page_zip_fields_decode(): Encode and decode
index->n_nullable for non-leaf pages.
page_zip_apply_log(): Write REC_NEW_HEAP_NO before calling
rec_offs_make_valid().
page_zip_write_node_ptr(): Correct off-by-one error.
page_cur_search_with_match(): Make use of page_is_leaf().
page_dir_add_slots(): Replaced with page_dir_add_slot(). Use memmove().
page_zip_fixed_field_encode(), page_zip_fields_encode(),
page_zip_fields_decode(): Do not waste one bit when encoding quantities
larger than 127.
page_zip_apply_log(), page_zip_write_rec(): Decode and encode
heap_no >= 127 correctly.
