dict_mem_table_create(): Account for DICT_TF_COMPRESSED in a debug assertion.
btr_store_big_rec_extern_fields(), btr_free_externally_stored_field(),
btr_copy_externally_stored_field(): Implement the disk format for
compressed BLOB pages.
btr_copy_externally_stored_field(): Improve error reporting and handling
when decompressing BLOB pages.
buf_flush_init_for_writing(), buf_page_is_corrupted(), buf_page_print():
Account for compressed BLOB pages (FIL_PAGE_TYPE_ZBLOB).
buf_calc_zblob_page_checksum(): New function.
btr_store_big_rec_extern_fields(): Assert that page_zip is non-NULL
if and only if dict_table_is_zip() holds.
btr_free_externally_stored_field(): Observe dict_table_is_zip().
Allow page_zip==NULL even if dict_table_is_zip(). Remove the
related TODO comment in row_purge_upd_exist_or_extern().
page_zip_available(): uncompressed_size already includes
PAGE_ZIP_DIR_SLOT_SIZE.
page_zip_decompress(): Remove bogus assertion d_stream.next_out == last.
Do not subtract BTR_EXTERN_FIELD_REF_SIZE from d_stream.avail_in when
decompressing records, because the records may be deleted later in
page_zip_apply_log(), and no BLOB pointers are allocated for deleted
records.
page_delete_rec_list_end(): Do not invoke page_zip_clear_rec().
page_zip_clear_rec(): Remove the global declaration. Add an assertion
that the record must be on the free list. Copy onl rec_offs_data_size()
bytes before attempting to clear the record and to compress the page.
page_zip_compress(), page_zip_write_rec(): Remove debug assertions that
trx_id, roll_ptr and node_ptr of deleted records be zero-filled.
Restore page_zip_clear_rec() to the global scope.
Invoke it in page_delete_rec_list_end().
Add debug assertions to page0zip.c to guard against overwriting data
on the compressed page.
operations on BLOB columns. There are some bugs in the code, because
test-insert and a few other tests fail.
page_mem_free(): Add parameter index. Decrement PAGE_N_RECS here.
Move some operations to page_zip_dir_delete().
page_zip_clear_rec(): Make this a static function.
page_zip_dir_delete(): Add parameters index and offsets.
Decrement PAGE_N_RECS and clear info_bits and n_owned.
page_zip_get_n_prev_extern(): Correct the synopsis and the algorithm.
Add parameter page_zip. Search the records in heap_no order instead
of collation order.
page_zip_compress(), page_zip_decompress(): Only copy BLOB pointers
and increment n_blobs for records that have not been deleted.
page_zip_clear_rec(): Clear trx_id and roll_ptr on the compressed page.
page_zip_dir_delete(): Decrement PAGE_N_RECS. Shift the array of
BLOB pointers. Call page_zip_clear_rec().
page_zip_dir_add_slot(): Shift the array of BLOB pointers to make
space of roll_ptr and trx_id.
page_cur_delete_rec(): Do not decrement PAGE_N_RECS or call
page_zip_clear_rec(), as page_mem_free() already does it.
trx-undo_prev_version_build(): Pass offsets==NULL to
rec_set_field_extern_bits().
rec_set_field_extern_bits(), rec_set_field_extern_bits_new():
Accept offsets==NULL.
row_upd_rec_in_place(): Remove the bogus comment that the function
would only be invoked on a clustered index. Remove the related
debug assertion.
page_zip_dir_delete() will need to handle BLOBs.
rec_set_field_extern_bits(), rec_set_field_extern_bits_new():
Add parameter offsets.
rec_offs_set_nth_extern(): New function to set an extern bit in offsets.
This will be called when an extern bit is set in a record.
page_cur_rec_insert(), page_cur_insert_rec_low(): Document that the
parameter "offsets" is in/out.
page_zip_dir_delete(): Note that the array of BLOB pointers will need
to be shifted.
page0zip.ic: Document the entry type for clearing a record.
page_zip_available(): Add parameter "index". Remove parameters
"is_leaf" and "is_clustered".
page_zip_get_trailer_len(): New function for computing the trailer length
of the compressed page.
page_zip_apply_log(): Implement the modification log entry type for
clearing the data bytes of a record.
page_zip_decompress(): Initialize n_blobs when actually copying the
BLOB pointers to place.
page_zip_validate(): Add diagnostic messages for failures. Check
also m_start, m_end, and n_blobs.
page_zip_write_blob_ptr(): Add page_zip_validate() assertion.
There are still some bugs in the code.
btr_store_big_rec_extern_fields(): Remove assertion on dict_table_is_zip()
to ease testing.
btr_free_externally_stored_field(): Test page_zip instead of
dict_table_is_zip().
page_zip_write_rec(): Add parameter "create". Try to handle externally
stored columns.
rec_offs_any_extern(): Correct the function comment.
Add rec_offs_n_extern() and page_zip_get_n_prev_extern().
page_zip_dir_decode(): Replace assertion with if (...) return(FALSE).
page_zip_decompress(): Do not clear page_zip->n_blobs after counting the
BLOBs.
page_zip_write_blob_ptr(): Use page_zip_get_n_prev_extern().
Correct an off-by-one error in memcpy().
btr_compress(): Pass the correct page_zip to btr_node_ptr_set_child_page_no().
page_mem_free(): Remove parameter mtr. Do not call page_zip_clear_rec() here.
page_cur_delete_rec(): Call page_zip_clear_rec() here. Decrement PAGE_N_RECS
only after calling page_mem_free(). Assert page_zip_validate().
of clustered indexes. Previously, parts of the code assumed that these
columns would exist on all leaf pages. Simplify the update-in-place of
these columns.
Add inline function dict_index_is_clust() to replace all tests
index->type & DICT_CLUSTERED.
Remove the redo log entry types MLOG_ZIP_WRITE_TRX_ID and
MLOG_ZIP_WRITE_ROLL_PTR, because the modifications to these columns
are covered by logical logging.
Fuse page_zip_write_trx_id() and page_zip_write_roll_ptr() into
page_zip_write_trx_id_and_roll_ptr().
page_zip_dir_add_slot(), page_zip_available(): Add flag "is_clustered",
so that no space will be reserved for TRX_ID and ROLL_PTR on leaf pages
of secondary indexes.
page_zip_apply_log(): Flag an error when val==0 is encoded with two bytes.
page_zip_write_rec(): Add debug assertions that there is enough space
available for the entry before copying the data bytes of the record.
btr_root_raise_and_insert(): Because btr_page_set_level() must not
change level from non-zero to zero on compressed pages, invoke
btr_page_set_level() with page_zip==NULL and compress the entire
root page after creating it from the scratch.
btr_attach_half_pages(): Simplify the computation of lower_page_zip
and upper_page_zip. Invoke btr_node_ptr_set_child_page_no() with
the correct page_zip.
page0page.h: Add __attribute__((const)) to functions testing for
infimum or supremum.
page_zip_dir_delete(): Note that the third parameter may be NULL.
Correct offset errors.
page_zip_available(): Use n_heap instead of n_recs.
page_zip_dir_find(), page_zip_dir_find_free(): Fix off-by-one error.
page_zip_fields_encode(), page_zip_fields_decode(): Encode and decode
index->n_nullable for non-leaf pages.
page_zip_apply_log(): Write REC_NEW_HEAP_NO before calling
rec_offs_make_valid().
page_zip_write_node_ptr(): Correct off-by-one error.
page_cur_search_with_match(): Make use of page_is_leaf().
page_dir_add_slots(): Replaced with page_dir_add_slot(). Use memmove().
row_upd_rec_in_place(), page_zip_write_rec(): Add parameter "index".
page_dir_set_n_heap(): Add a debug assertion that on compressed
pages, n_heap will always be incremented by one. Improve code formatting.
page_zip_dir_add_slot(): New function, called from
page_cur_insert_rec_low() after page_mem_alloc_heap().
rec_set_n_owned_new(): Do not call page_zip_rec_set_owned()
on the supremum record.
rec_offs_make_valid(): Add debug assertions.
page_zip_dir_user_size(): Correct an off-by-one error in the debug assertion.
page_zip_apply_log(): Add parameter trx_id_col. Skip trx_id and roll_ptr.
page_zip_decompress(): Simplify the handling of "storage" in the loop that
copies the uncompressed fields.
page_zip_write_rec(): Store trx_id and roll_ptr separately.
page_zip_write_trx_id(), page_zip_write_roll_ptr(): Fix off-by-one errors.
page_cur_insert_rec_low(): Call page_zip_dir_add_slot() after
page_mem_alloc_heap(). Remove some redundant assertions.
Pass page_zip to page_dir_split_slot().
btr_create(): page_zip_compress() returns FALSE on failure.
page_zip_write_header(): Write to page_zip->data[] instead of page_zip[].
buf_flush_init_for_writing(): Add parameter page_zip and set the fields
also in the header of the compressed page.
btr_cur_search_to_nth_level(): Add ut_ad() on page_zip_validate().
operation succeeds.
page_zip_dir_set(): Remove, as the slots will be written to a temporary
buffer.
page_zip_dir_encode(): Replace page_zip with buf. Make some assertions
more accurate. Write the dense page directory to buf.
page_zip_compress(): Allocate a temporary buffer for the compressed page
except the page header. Make some assertions more accurate.
page_zip_compress(), page_zip_decompress(): Remove 2 bytes of empty space
adjacent to the area reserved for storing uncompressed columns.
on a compressed page.
page_zip_get_n_relocated(), page_zip_set_n_relocated(),
page_zip_get_relocated(): Remove.
page0zip.c, page0zip.ic: Remove all references to n_relocated.
page_cur_insert_rec_low(): On pages that are compressed,
never relocate records that are allocated from the free list.
do not allow extra_size to decrease on compressed pages.
Split page_mem_alloc() to page_mem_alloc_free() and page_mem_alloc_heap().
page_cur_insert_rec_low(): Remove parameter "tuple". Implement some of the
logic from page_mem_alloc().
page_cur_tuple_insert(): Convert the tuple to a record and calculate offsets.
page_zip_validate(): Assert that the page is in compact format.
compressed pages.
btr_root_raise_and_insert(): Distinguish root_page_zip and new_page_zip.
btr_cur_set_ownership_of_extern_field(): Do not log the write on the
uncompressed page if it will be logged for page_zip.
lock_rec_insert_check_and_lock(), lock_sec_rec_modify_check_and_lock():
Update the max_trx_id field also on the compressed page.
mlog_write_ulint(): Add UNIV_UNLIKELY hints. Remove trailing white space.
mlog_log_string(): Remove trailing white space.
rec_set_field_extern_bits(): Remove parameter mtr, as the write will either
occur in the heap, or it will be logged at a higher level.
recv_parse_or_apply_log_rec_body(),
page_zip_write_header(): Add log record type MLOG_ZIP_WRITE_HEADER.
page_header_set_field(): Pass mtr=NULL to page_zip_write_header().
page_header_reset_last_insert(): Pass mtr to page_zip_write_header().
btr_page_set_index_id(), btr_page_set_level(),
btr_page_set_next(), btr_page_set_prev(): Pass mtr to page_zip_write_header().
row_upd_rec_sys_fields(): Pass mtr=NULL to page_zip_write_trx_id() and
page_zip_write_roll_ptr(), since the write will be logged at a higher level.
page_zip_write_header(): Add parameter mtr.
page_zip_write_header_log(): New function.
Remove rec_set_nth_field_extern_bit().
Make rec_set_nth_field_extern_bit_old() static.
Rename rec_set_nth_field_extern_bit_new()
to rec_set_field_extern_bits_new() and make it static.
row_ins_index_entry_low(): Remove bogus TODO comment.
Fix the way how btr_free_externally_stored_field() is called in purge.
btr_free_externally_stored_field(): Add parameter field_ref that points
directly to the BLOB reference. Use rec, offsets, page_zip, and i
only for the page_zip_write_blob_ptr() call.
row_purge_upd_exist_or_extern(): Do not assume that the undo log contains
the entire record. Only pass the BLOB reference to
btr_free_externally_stored_field().
This has not been extensively tested yet, because some other part of the
code breaks in "ibtestblob".
btr_free_page_low(): Add parameters "space" and "page_no", because they
are omitted from compressed BLOB pages.
btr0cur.c: Implement the compression and decompression of BLOB columns,
enabled at compile-time (#define ZIP_BLOB TRUE) for now.
btr_rec_free_externally_stored_fields(),
btr_copy_externally_stored_field(): Made static
mlog_log_string(): New function, split from mlog_write_string(), allows
to avoid a dummy memcpy() of compressed BLOB pages.
flag of records. The flags may only be updated in heap-allocated
copies of records.
btr_root_raise_and_insert(),
btr_page_split_and_insert(),
btr_cur_insert_if_possible(),
btr_cur_optimistic_insert(),
btr_cur_pessimistic_insert(),
page_cur_tuple_insert(),
page_cur_insert_rec_low(): Add parameters "ext" and "n_ext".
dtuple_convert_big_rec(): Make parameter "ext" const.
BLOB pointers, trx_id, and roll_ptr.
btr_empty(), btr_create(), page_create(): Add parameter "index", as some
index information will be encoded on the compressed page.
Define REC_NODE_PTR_SIZE as 4.
Allow btr_page_reorganize() and btr_page_reorganize_low() to fail.
Define the error code DB_ZIP_OVERFLOW.
Make row_ins_index_entry_low() static.
page0zip: Encode the index, log reorganized records, and store uncompressed
fields separately from the compressed data stream.
that will require complete index information.
dict_create_index_step(): invoke dict_index_add_to_cache() before btr_create()
dict_index_remove_from_cache(): make public
dict_index_get_if_in_cache_low(): new function, for holding dict_sys->mutex
buf_flush_init_for_writing(): remove the temporary hook to page_zip_compress()
page_create(): add temporary hook to page_zip_compress()
buf_flush_init_for_writing(): Improve the diagnostics and make the
condition for skipping pages accurate.
univ.i: Introduce UNIV_ZIP_DEBUG for enabling some page_zip_validate() tests.
page0zip.h, page0zip.c: Define and use page_zip_validate() in
page_zip_compress() and page_zip_write() if UNIV_ZIP_DEBUG or UNIV_DEBUG
is defined.
decompresses properly.
page_mem_free(): Adjust the temporary condition for zeroing data.
page_zip_compress(): Do not accept Z_STREAM_END from non-final deflate().
Tighten the assertion on c_stream.avail_in.
page_zip_dir_sort(): Note that the high bound is not inclusive.
page_zip_dir_decode(): Remove off-by-one error in page_zip_dir_sort() call.
page_zip_set_extra_bytes(): Add the termination condition that was forgotten
from the second for loop. Simplify the code a little.
page_zip_decompress(): Do not accept Z_STREAM_END from non-final inflate().
Tighten the assertion on d_stream.avail_out. Increment heap_status correctly.
A page with multiple records or deleted records still does not compress
or decompress properly.
buf_flush_init_for_writing(): Initialize block->page_zip properly so that all
assertions in page0zip can be enabled.
page_zip_decompress(): Note that corrupt data should not lead to assertions.
page_zip_dir_set(): Correct the interface. Fix off-by-one error.
page_zip_dir_get(): Fix off-by-one error.
page0zip.c: Replace n_heap with n_dense and add comments about
the infimum and supremum records whenever we subtract 2 from heap_no.
Fix some programming errors.
buf0flu.c: Allocate the temporary buffer from buf_frame_alloc().
page_zip_simple_validate(): Do not assert page_zip->m_start >= PAGE_DATA.
page_zip_compress(): Replace some assertions with page_zip_simple_validate(),
and do not assert anything about page_zip->data contents.
page_zip_validate(): Do not compare the page trailer bytes.
page_zip_write(): Assert that the entire page headers match and
that page_zip->m_start >= PAGE_DATA.
page_dir_set_n_slots(): Add debug assertion that there will be enough space
available in the modification log of the compressed page.
page_trailer_get_len(): Simplify the formula. Make more use of the function.
page_set_max_trx_id(), page_update_max_trx_id(): Add parameter page_zip.
Check that page_zip_decompress() is never called after modifying the
header or trailer of the compressed page, i.e., that page_zip_decompress()
will restore the uncompressed page as it was before the failed operation.
page_header_reset_last_insert(): Add page_zip parameter.
rec_get_next_ptr(): Return a pointer to the same page, or NULL.
page_dir_delete_slot(): Fix off-by-one errors.
Add page_zip parameter to page_dir_slot_set_rec().
Improve the documentation of the page_zip parameters in page0page.
Add function page_trailer_get_len().
Implement page_zip_write_header().
Add page_zip parameter to btr_page_set_ functions.
Allow page_copy_rec_list_end() to be called with invalid new_page_zip
if the page has been newly created.
