for the purpose of comparing different compression algorithms.
PAGE_ZIP_COMPRESS_DBG: New preprocessor condition, to see if deflate()
is wrapped.
page_zip_compress_log: Log file counter. If set to nonzero, logging
is enabled.
page_zip_compress_deflate(): Add the parameter logfile.
FILE_LOGFILE, LOGFILE: Macros for declaring and passing the parameter logfile.
page_zip_compress(): Open and close the logfile if needed. Write the
uncompressed page and the size of the compressed data. The data passed
to deflate() is written by the wrapper page_zip_compress_deflate().
symbols. Use it for all definitions of non-static variables and functions.
lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static. It is
referenced from pars0grm.c.
Actually, according to
nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:
* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs
The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.
The class ha_innodb is renamed from class ha_innobase by a #define. Thus,
there will be no clash with the builtin InnoDB. However, there will be some
overhead for invoking virtual methods of class ha_innodb. Ideas for making
the vtable hidden are welcome. -fvisibility=hidden is not available in GCC 3.
just one parameter: UNIV_PAGE_SIZE_SHIFT.
UNIV_PAGE_SIZE, BUF_BUDDY_SIZES: Define in terms of UNIV_PAGE_SIZE_SHIFT.
BUF_BUDDY_LOW_SHIFT: New macro, to simplify the definition of BUF_BUDDY_LOW
and BUF_BUDDY_SIZES.
PAGE_ZIP_DIR_SLOT_MASK: Relax the compile-time check. This bitmask must be
one less than a power of two, and at least UNIV_PAGE_SIZE - 1.
is an overlap between BLOB pointers and the modification log or the
zlib stream.
page_zip_decompress_clust_ext(): Remove the improper check. The
d_stream->avail_in cannot be decremented here, because we do not know
at this point if the record is deleted. No space is reserved for the
BLOB pointers in deleted records.
page_zip_decompress_clust(): Check for the overlap here, right before
copying the BLOB pointers.
page_zip_decompress_clust(): Also check that the target column is long
enough, and return FALSE instead of ut_ad() failure.
some decompression functions.
page_zip_apply_log_ext(), page_zip_apply_log(): Call page_zip_fail()
with appropriate diagnostics before returning NULL.
page_zip_decompress_node_ptrs(), page_zip_decompress_sec(),
page_zip_decompress_clust(): When detecting that the zlib stream
followed by the modification log overlaps the trailer, do not
let an assertion fail, but invoke page_zip_fail() and return FALSE.
Corrupt data should never lead into assertion failures in decompression
functions.
buf_page_get_release_on_io(): Removed this unused function.
ibuf_build_entry_from_ibuf_rec(): Justify why it is not necessary to
add system columns to the dummy table pointed to by the dummy secondary index.
page_zip_rec_set_deleted(): Add a page_zip_validate() assertion.
the wrapper macro page_zip_fail() for displaying error messages.
When the error output is enabled (at compile-time), a breakpoint
may be set in page_zip_fail_func to easily debug all decompression
errors in the context where they occur.
in page_zip_decompress().
page_zip_decompress_clust(), page_zip_decompress_clust_ext(): Zero-fill
the columns DB_TRX_ID and DB_ROLL_PTR on the uncompressed page.
page_zip_get_trailer_len(), page_zip_write_header(): Correct the
UNIV_MEM_ASSERT_RW() assertions.
page_zip_validate(): Read the validity bits of page, page_zip, and
page_zip->data.
page_zip_decompress(): Assert that the uncompressed page is completely defined.
page_zip_validate(): Assert that the compressed and uncompressed pages are
completely defined. Fetch the "valid" bits, so that they can be examined
when run under valgrind --db-attach=yes.
initialized, although Valgrind believes that some bits in the 7th or 8th
bytes from the end are uninitialized. (They might be, but the decompressor
should not care about those bits after encountering the end-of-stream marker
in the compressed bit stream.)
btr_cur_optimistic_insert(): On compressed tablespaces, check that both
the compressed and the uncompressed page are completely initialized in
the beginning of the function.
page_zip_compress(): After successful compression, check that the compressed
page is completely initialized.
page_zip_write_rec(), page_zip_write_blob_ptr(), page_zip_write_node_ptr(),
page_zip_write_trx_id_and_roll_ptr(), page_zip_clear_rec(),
page_zip_rec_set_deleted(), page_zip_rec_set_owned(), page_zip_dir_insert(),
page_zip_dir_delete(), page_zip_dir_add_slot(), page_zip_reorganize(),
page_zip_copy(), page_zip_get_trailer_len(), page_zip_write_header():
Assert that the complete contents of the compressed page is defined.
page_zip_compress(): Assert that the contents of the uncompressed page
is entirely initialized.
page_zip_decompress(): Assert that the contents of the compressed page
is entirely initialized. Assert that the uncompressed page is entirely
writeable. Flag the uncompressed page uninitialized in the beginning.
page_cur_set_before_first(), page_cur_set_after_last(),
page_cur_position(): Add const qualifiers to buf_block_t and rec.
A better solution would be to define a const_page_cur_t and a
set of accessors, but it would lead to severe code duplication.
page_rec_get_n_recs_before(): Add const qualifiers.
page_dir_get_nth_slot(): Define as a const-preserving macro.
page_dir_slot_get_rec(), page_dir_slot_get_n_owned(),
page_dir_find_owner_slot(), page_check_dir(): Add const qualifiers.
page_rec_get_next_low(): Add const qualifiers.
page_rec_get_next_const(), page_rec_get_prev_const(): New functions,
based on the const-less page_rec_get_next() and page_rec_get_prev().
page_cur_get_page(), page_cur_get_block(), page_cur_get_page_zip(),
page_cur_get_rec(): Define as const-preserving macros.
page_cur_try_search_shortcut(), page_cur_search_with_match():
Add const qualifiers.
buf_page_get_mutex(): Add a const qualifier to buf_page_t*.
rec_get_next_ptr_const(): Const variant of rec_get_next_ptr().
dtuple_validate(): Detect uninitialized data.
page_cur_insert_rec_low(), page_cur_insert_rec_zip(): Assert that the
record being inserted is valid before and after insertion.
For some reason, GCC 4.2.1 ignores casts (for removing constness)
in calls to inline functions.
page_align(), ut_align_down(): Make the parameter const void*, but still
return a non-const pointer. This is ugly, but these functions cannot be
replaced with a const-preserving macro in a portable way, given that
the pointer argument is not always pointing to bytes.
buf_block_get_page_zip(): Implement as a const-preserving macro.
buf_frame_get_page_zip(), buf_block_align(): Add const qualifiers.
lock_rec_get_prev(): Silence GCC 4.2.1 warnings.
mlog_write_initial_log_record(), mlog_write_initial_log_record_fast(),
mtr_memo_contains(): Add const qualifier to the pointer.
page_header_get_ptr(): Rewrite as page_header_get_offs(), and
implement as a macro that calls this function.
offsets_[] arrays, as suggested by Vasil.
rec_offs_set_n_alloc(): Declare as a public function. Assert that
n_alloc > REC_OFFS_HEADER_SIZE.
rec_offs_get_n_alloc(): Assert that n_alloc > REC_OFFS_HEADER_SIZE.
mem_heap_zalloc() and mem_zalloc(), because calloc() in the C runtime
library takes two size parameters, not one.
mem_heap_zalloc(): Add debug assertions. Document that the return value
is never NULL.
Some things still fail in innodb-index.test, and there seems to be
a race condition (data dictionary lock wait) when running with --valgrind.
dfield_t: Add an "external storage" flag, dfield->ext.
dfield_is_null(), dfield_is_ext(), dfield_set_ext(), dfield_set_null():
New functions.
dfield_copy(), dfield_copy_data(): Add const qualifiers, fix in/out comments.
data_write_sql_null(): Use memset().
big_rec_field_t: Replace byte* data with const void* data.
ut_ulint_sort(): Remove.
upd_field_t: Remove extern_storage.
upd_node_t: Replace ext_vec, n_ext_vec with n_ext.
row_merge_copy_blobs(): New function.
row_ins_index_entry(): Add the parameter "ibool foreign" for suppressing
foreign key checks during fast index creation or when inserting into
secondary indexes.
btr_page_insert_fits(): Add const qualifiers.
btr_cur_add_ext(), upd_ext_vec_contains(): Remove.
dfield_print_also_hex(), dfield_print(): Replace if...else if with switch.
Observe dfield_is_ext().
now that all of InnoDB code is built from a single Makefile and it should
not be possible to build the modules with mutually incompatible options.
#define INSIDE_HA_INNOBASE_CC: Remove.
srv_sizeof_trx_t_in_ha_innodb_cc: Remove.
dict_table_get_low_noninlined(): Remove. This function was unused.
Remove all _noninline functions. Remove the _noninline suffix from
all function calls in ha_innodb.cc.
of UT_SORT_FUNCTION_BODY is best done by defining SORT_FUN and CMP_FUN as
macros when needed. The solution of r1523 allows for only one extra parameter.
the insert buffer bitmap.
ibuf_set_free_bits_func(): Never disable redo logging.
ibuf_update_free_bits_zip(): Remove.
btr_page_reorganize_low(), page_zip_reorganize(): Do not update the insert
buffer bitmap. Instead, document that callers will have to take care of it,
and adapt the callers.
btr_compress(): On error, reset the insert buffer free bits.
btr_cur_insert_if_possible(): Do not modify the insert buffer bitmap.
btr_compress(), btr_cur_optimistic_insert(): On compressed pages,
reset the insert buffer bitmap. Document why.
btr_cur_update_alloc_zip(): Document why it is necessary and sufficient
to reset the insert buffer free bits.
btr_cur_update_in_place(), btr_cur_optimistic_update(),
btr_cur_pessimistic_update(): Update the free bits in the same
mini-transaction. Document that the mini-transaction must be
committed before latching any further pages. Verify that this
is the case in all execution paths.
row_ins_sec_index_entry_by_modify(), row_ins_clust_index_entry_by_modify(),
row_undo_mod_clust_low(): Because these functions call
btr_cur_update_in_place(), btr_cur_optimistic_update(), or
btr_cur_pessimistic_update(), document that the mini-transaction must be
committed before latching any further pages. Verify that this is the case
in all execution paths.
page_cur_tuple_insert(), page_cur_rec_insert(): Improve the documentation.
Note that the physical position of page_cursor may change.
page_cur_rec_insert(), page_cur_insert_rec_low(), page_cur_insert_rec_zip():
Add a const qualifier to the parameter rec.
crash recovery is in progress. This avoids a hang when
btr_parse_page_reorganize(), called from an I/O handler thread,
attempts to acquire log_sys->mutex while it is being held by
the main thread (the one that runs innobase_init()). This change
was committed accidentally. It may be unsafe to clear
mtr.modifications, because buf_page_release() at mtr_commit() may
forget to put modified pages to the flush list.
Cleanup: Remove the "type" parameter from many ibuf functions.
Let the caller check that !dict_index_is_clust(). This should avoid
function calls and register spilling.
ibuf_set_free_bits_func(), ibuf_set_free_bits(): Remove the parameter "type".
ibuf_reset_free_bits_with_type(): Rename to ibuf_reset_free_bits().
Remove the parameter "type".
ibuf_update_free_bits_if_full(), ibuf_update_free_bits_zip(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low():
Remove the parameter "index".
ha_innodb.cc: Add the columns COMPRESSED, COMPRESSED_OK, DECOMPRESSED
to INFORMATION_SCHEMA.INNODB_BUDDY.
page_zip_compress_count[], page_zip_compress_ok[]: New statistic counters,
incremented in page_zip_compress().
page_zip_decompress_count[]: New statistic counter,
incremented in page_zip_decompress().
failing insert. Reorganization will have been attempted in
page_cur_tuple_insert() or page_cur_rec_insert().
page_zip_reorganize(): Recompute the insert buffer free bits for
leaf pages of secondary indexes.
ibuf_data_enough_free_for_insert(): Simplify.
were introduced when reducing the memory footprint of the data dictionary
cache (Bug #20877), around r834. This function is only compiled if
PAGE_CUR_LE_OR_EXTENDS is defined in page0cur.h.
record does not contain externally stored columns.
page_zip_decompress_clust_ext(): New function for decompressing records
containing externally stored columns.
externally stored columns.
page_zip_compress_clust_ext(), page_zip_apply_log_ext():
New functions.
page_zip_compress_clust(), page_zip_apply_log(): Check rec_offs_any_extern()
and avoid invoking the costly loop in most cases.
Similar optimizations can be made in page_zip_decompress_clust() and
page_zip_write_rec().
by adding the REC_OFFS_EXTERNAL flag to rec_offs_base(offsets)[0].
This reduces the processor usage of page_zip_write_rec() by about 40%
in one test case. The code could be sped up further by testing
rec_offs_any_extern() outside of loops that check rec_offs_nth_extern().
The vast majority of records does not contain any externally stored columns.
