that Valgrind will not complain about freed data structures that are
reachable via pointers. This addresses Bug #45992 and Bug #46656.
This patch is mostly based on changes copied from branches/embedded-1.0,
mainly c5432, c3439, c3134, c2994, c2978, but also some other code was
copied. Some added cleanup code is specific to MySQL/InnoDB.
rb://199 approved by Sunny Bains
/*********************************
comments to Doxygen /** style like this:
/*****************************//**
This conversion was performed by the following command:
perl -i -e 'while(<ARGV>){if (m|^/\*{30}\**$|) {
s|\*{4}$|//**| if ++$com>1; $_ .= "\@file $ARGV\n" if $com==2}
print; if(eof){$.=0;undef $com}}' */*[ch] include/univ.i
This patch was created by running the following commands:
for i in */*[ch]; do doxygenify.pl $i; done
perl -i -pe 's#\*{3} \*/$#****/#' */*[ch]
where doxygenify.pl is
https://svn.innodb.com/svn/misc/trunk/tools/doxygenify.pl r510
Verified the consistency as follows:
(0) not too many /* in: */ or /* out: */ comments left in the code:
grep -l '/\*\s*\(in\|out\)[,:/]' */*[ch]
(1) no difference when ignoring blank lines, after stripping all
C90-style /* comments */, including multi-line ones, before and after
applying this patch:
perl -i -e 'undef $/;while(<ARGV>){s#/\*(.*?)\*/##gs;print}' */*[ch]
diff -I'^\s*$' --exclude .svn -ru TREE1 TREE2
(2) after stripping @return comments and !<, generated a diff and omitted
the hunks where /* out: */ function return comments were removed:
perl -i -e'undef $/;while(<ARGV>){s#!<##g;s#\n\@return\t.*?\*/# \*/#gs;print}'\
*/*[ch]
svn diff|
perl -e 'undef $/;$_=<>;s#\n-\s*/\* out[:,]([^\n]*?)(\n-[^\n]*?)*\*/##gs;print'
Some unintended changes were left. These will be removed in a
subsequent patch.
transactions that are started before the rollback of incomplete
transactions has finished may have an inconsistent view of the
secondary indexes.
dict_index_is_sec_or_ibuf(): Auxiliary function for controlling
updates and checks of PAGE_MAX_TRX_ID: check whether an index is a
secondary index or the insert buffer tree.
page_set_max_trx_id(), page_update_max_trx_id(),
lock_rec_insert_check_and_lock(),
lock_sec_rec_modify_check_and_lock(), btr_cur_ins_lock_and_undo(),
btr_cur_upd_lock_and_undo(): Add the parameter mtr.
page_set_max_trx_id(): Allow mtr to be NULL. When mtr==NULL, do not
attempt to write to the redo log. This only occurs when creating a
page or reorganizing a compressed page. In these cases, the
PAGE_MAX_TRX_ID will be set correctly during the application of redo
log records, even though there is no explicit log record about it.
btr_discard_only_page_on_level(): Preserve PAGE_MAX_TRX_ID. This
function should be unreachable, though.
btr_cur_pessimistic_update(): Update PAGE_MAX_TRX_ID.
Add some assertions for checking that PAGE_MAX_TRX_ID is set on all
secondary index leaf pages.
rb://115 tested by Michael, fixes Issue #211
and undo_no_t. Each type is still defined as dulint.
This is an initial step towards replacing dulint with a 64-bit data type.
Because modern compilers have no trouble supporting 64-bit arithmetics
even on 32-bit targets, the dulint struct is a relic that should go.
The last remaining major use of dulint is dictionary IDs
(table, index, and row ids).
rb://114 approved by Sunny Bains
* Remove old Innobase copyright lines from C source files
* Add a reference to the GPLv2 license as recommended by the lawyers
at Oracle Legal
[Step 4/28]
a gcc warning about an assertion that trivially holds.
The warning was introduced in r4061, in the merge of
branches/innodb+ -r4053.
ibuf_insert(): Let an assertion fail if ibuf_use is unknown.
Implement the global variable innodb_change_buffering, with the
following values:
none - buffer nothing
inserts - buffer inserts (the default)
Approved by Ken Jacobs.
------------------------------------------------------------------------
r3588 | inaam | 2008-12-18 14:26:54 +0200 (Thu, 18 Dec 2008) | 8 lines
branches/5.1
It is a bug in unused code. If we don't calculate the hash value when
calculating the mutex number then two pages which map to same hash
value can get two different mutex numbers.
Approved by: Marko
------------------------------------------------------------------------
r3590 | marko | 2008-12-18 15:33:36 +0200 (Thu, 18 Dec 2008) | 11 lines
branches/5.1: When converting a record to MySQL format, copy the default
column values for columns that are SQL NULL. This addresses failures in
row-based replication (Bug #39648).
row_prebuilt_t: Add default_rec, for the default values of the columns in
MySQL format.
row_sel_store_mysql_rec(): Use prebuilt->default_rec instead of
padding columns.
rb://64 approved by Heikki Tuuri
------------------------------------------------------------------------
r3598 | marko | 2008-12-22 15:28:03 +0200 (Mon, 22 Dec 2008) | 6 lines
branches/5.1: ibuf_delete_rec(): When the record cannot be found and
the tablespace has been dropped, commit the mini-transaction, so that
InnoDB will not hold the insert buffer tree latch in exclusive mode,
causing a potential deadlock. This bug was introduced in the fix of
Bug #27276 in r2924.
------------------------------------------------------------------------
Originally, there were provisions in InnoDB for multiple insert buffer
B-trees, apparently one for each tablespace.
When Heikki implemented innodb_file_per_table (multiple InnoDB
tablespaces) in MySQL 4.1, he made the insert buffer live only in the
system tablespace (space 0) but left the provisions in the code.
When Osku Salerma implemented delete buffering, he also cleaned up the
insert buffer subsystem so that only one insert buffer B-tree exists.
This patch applies the clean-up to the InnoDB Plugin.
Having a separate patch of the insert buffer clean-up should help us
better compare the essential changes of the InnoDB Plugin and InnoDB+
and to track down bugs that are specific to InnoDB+.
IBUF_SPACE_ID: New constant, defined as 0.
ibuf_data_t: Remove.
ibuf_t: Add the applicable fields from ibuf_data_t. There is only one
insert buffer tree from now on.
ibuf_page_low(), ibuf_page(): Merge to a single function ibuf_page().
fil_space_t: Remove ibuf_data.
fil_space_get_ibuf_data(): Remove. There is only one ibuf_data, for
space IBUF_SPACE_ID.
fil_ibuf_init_at_db_start(): Remove.
ibuf_init_at_db_start(): Fuse with ibuf_data_init_for_space().
ibuf_validate_low(): Remove. There is only one ibuf tree.
ibuf_free_excess_pages(), ibuf_header_page_get(),
ibuf_free_excess_pages(): Remove the parameter space, which was always
0.
ibuf_tree_root_get(): Remove the parameters space and data. There is
only one ibuf tree, for space IBUF_SPACE_ID.
ibuf_data_sizes_update(): Rename to ibuf_size_update(), and remove the
parameter data. There is only one ibuf data struct.
ibuf_build_entry_pre_4_1_x(): New function, refactored from
ibuf_build_entry_from_ibuf_rec().
ibuf_data_enough_free_for_insert(), ibuf_data_too_much_free(): Remove
the parameter data. There is only one insert buffer tree.
ibuf_add_free_page(), ibuf_remove_free_page(): Remove the parameters
space and data. There is only one insert buffer tree.
ibuf_get_merge_page_nos(): Add parenthesis, to reduce diffs to
branches/innodb+.
ibuf_contract_ext(): Do not pick an insert buffer tree at random.
There is only one.
ibuf_print(): Print the single insert buffer tree.
rb://19 approved by Heikki on IM
redundant checks for ibuf_fixed_addr_page() || fsp_descr_page() that
were removed in r3432, and add a comment explaining why.
Thanks to Michael for reporting this bug.
ibuf_get_volume_buffered(): Declare with static linkage.
This function is private to ibuf0ibuf.c.
btr_cur_pessimistic_delete(): Use the cached result of
btr_cur_get_index(cursor).
buf_block_dbg_add_level(block, level): Define as an empty macro when
UNIV_SYNC_DEBUG is not defined. Remove #ifdef UNIV_SYNC_DEBUG around
all invocations.
the maximum record size will never exceed the B-tree page size limit.
For uncompressed tables, there should always be enough space for two
records in an empty B-tree page. For compressed tables, there should
be enough space for storing two node pointer records or one data
record in an empty page in uncompressed format.
dict_build_table_def_step(): Remove the inaccurate check for table row
size.
dict_index_too_big_for_tree(): New function: check if the index
records would be too big for a B-tree page.
dict_index_add_to_cache(): Add the parameter "strict". Invoke
dict_index_too_big_for_tree() if it is set.
trx_is_strict(), thd_is_strict(): New functions, for determining if
innodb_strict_mode is enabled for the current transaction.
dict_create_index_step(): Pass the new parameter strict of
dict_index_add_to_cache() as trx_is_strict(trx). All other callers
pass it as FALSE.
innodb.test: Enable innodb_strict_mode before attempting to create a
table with a too big record size.
innodb-zip.test: Remove the test of inserting random data. Add tests
for checking that the maximum record lengths are enforced at table
creation time.
recovery, tolerate clustered index records whose externally stored
columns have not been written. This should remove the assertion failures
that were reported as Mantis issue#58, issue#62, issue#64.
trx_is_recv(): New function: TRUE if this transaction is rolling back
an incomplete transaction in crash recovery.
enum trx_rbmode: Rollback modes: no rollback, normal rollback, crash recovery.
btr_cur_pessimistic_delete(), btr_free_externally_stored_field(),
btr_rec_free_externally_stored_fields():
Replace the ibool parameter with enum trx_rbmode.
btr_free_externally_stored_field(): If field_ref is zero, return
but assert ut_a(rbmode == RB_RECOVERY). Unless InnoDB has crashed
while inserting a clustered index record, field_ref should not be zero.
btr_rec_free_updated_extern_fields(): Add the parameter enum trx_rbmode.
btr_cur_pessimistic_update(): Pass the rbmode parameter to
btr_rec_free_updated_extern_fields().
row_undo_ins(), row_undo_mod_upd_del_sec(): If row_build_index_entry()
fails, assert trx_is_recv() and skip this secondary index.
row_undo_mod_upd_del_sec(): Empty the heap at the end of each loop
iteration in order to conserve memory and to reduce the number of
low-level memory allocations.
variable innodb_file_format. Implement file format version stamping of
*.ibd files and SYS_TABLES.TYPE.
This change breaks introduces an incompatible change for for
compressed tables. We can do this, as we have not released yet.
innodb-zip.test: Add tests for stricter KEY_BLOCK_SIZE and ROW_FORMAT
checks.
DICT_TF_COMPRESSED_MASK, DICT_TF_COMPRESSED_SHIFT: Replace with
DICT_TF_ZSSIZE_MASK, DICT_TF_ZSSIZE_SHIFT.
DICT_TF_FORMAT_MASK, DICT_TF_FORMAT_SHIFT, DICT_TF_FORMAT_51,
DICT_TF_FORMAT_ZIP: File format version, stored in table->flags,
in the .ibd file header, and in SYS_TABLES.TYPE.
dict_create_sys_tables_tuple(): Write the table flags to SYS_TABLES.TYPE
if the format is at least DICT_TF_FORMAT_ZIP. For old formats
(DICT_TF_FORMAT_51), write DICT_TABLE_ORDINARY as the table type.
DB_TABLE_ZIP_NO_IBD: Remove the error code. The error handling is done
in ha_innodb.cc; as a failsafe measure, dict_build_table_def_step() will
silently clear the compression and format flags instead of returning this
error.
dict_mem_table_create(): Assert that no extra bits are set in the flags.
dict_sys_tables_get_zip_size(): Rename to dict_sys_tables_get_flags().
Check all flag bits, and return ULINT_UNDEFINED if the combination is
unsupported.
dict_boot(): Document the SYS_TABLES columns N_COLS and TYPE.
dict_table_get_format(), dict_table_set_format(),
dict_table_flags_to_zip_size(): New accessors to table->flags.
dtuple_convert_big_rec(): Introduce the auxiliary variables
local_len, local_prefix_len. Store a 768-byte prefix locally
if the file format is less than DICT_TF_FORMAT_ZIP.
dtuple_convert_back_big_rec(): Restore the columns.
srv_file_format: New variable: innodb_file_format.
fil_create_new_single_table_tablespace(): Replace the parameter zip_size
with table->flags.
fil_open_single_table_tablespace(): Replace the parameter zip_size_in_k
with table->flags. Check the flags.
fil_space_struct, fil_space_create(), fil_op_write_log():
Replace zip_size with flags.
fil_node_open_file(): Note a TODO item for InnoDB Hot Backup.
Check that the tablespace flags match.
fil_space_get_zip_size(): Rename to fil_space_get_flags(). Add a
wrapper for fil_space_get_zip_size().
fsp_header_get_flags(): New function.
fsp_header_init_fields(): Replace zip_size with flags.
FSP_SPACE_FLAGS: New name for the tablespace flags. This field used
to be called FSP_PAGE_ZIP_SIZE, or FSP_LOWEST_NO_WRITE. It has always
been written as 0 in MySQL/InnoDB versions 4.1 to 5.1.
MLOG_ZIP_FILE_CREATE: Rename to MLOG_FILE_CREATE2. Add a 32-bit
parameter for the tablespace flags.
ha_innobase::create(): Check the table attributes ROW_FORMAT and
KEY_BLOCK_SIZE. Issue errors if they are inappropriate, or warnings
if the inherited attributes (in ALTER TABLE) will be ignored.
PAGE_ZIP_MIN_SIZE_SHIFT: New constant: the 2-logarithm of PAGE_ZIP_MIN_SIZE.
symbols. Use it for all definitions of non-static variables and functions.
lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static. It is
referenced from pars0grm.c.
Actually, according to
nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:
* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs
The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.
The class ha_innodb is renamed from class ha_innobase by a #define. Thus,
there will be no clash with the builtin InnoDB. However, there will be some
overhead for invoking virtual methods of class ha_innodb. Ideas for making
the vtable hidden are welcome. -fvisibility=hidden is not available in GCC 3.
innodb-index.test: Add a test with a large number of externally stored
columns. Check that there may not be prefix indexes on too many columns.
dict_index_too_big_for_undo(): New function: Check if the undo log may
overflow.
dict_index_add_to_cache(): Return DB_SUCCESS or DB_TOO_BIG_RECORD.
Postpone the creation and linking of some data structures, so that
when dict_index_too_big_for_undo() holds, it will be easier to clean up.
Check the return status in all callers.
contents end up with conflicting versions of a record's state. The zipped
page record was not being marked as "(un)deleted" because we were not
passing the zipped page contents to the (un)delete function, which first
(un)delete marks the uncompressed version and then based on whether
page_zip is NULL or not (un)delete marks the record in the compressed page.
buf_page_get_release_on_io(): Removed this unused function.
ibuf_build_entry_from_ibuf_rec(): Justify why it is not necessary to
add system columns to the dummy table pointed to by the dummy secondary index.
page_zip_rec_set_deleted(): Add a page_zip_validate() assertion.
Do not assume anything about the contents of the bitmap.
ibuf_update_free_bits_low(): Use this function only for uncompressed pages.
Remove the parameter zip_size. This function avoids latching the bitmap page
and updating the bitmap when the bits do not change.
ibuf_update_free_bits_zip(): New function based on ibuf_update_free_bits_low(),
for use on compressed pages. Remove the parameter max_insert_size that
was used for computing the before image of the free bits. Always update the
bitmap.
ibuf_index_page_calc_free_zip(): New function, factored out from
ibuf_index_page_calc_free().
ibuf_update_free_bits_if_full(): Document that this function must only be
invoked on uncompressed pages. Add a debug assertion about this.
Some things still fail in innodb-index.test, and there seems to be
a race condition (data dictionary lock wait) when running with --valgrind.
dfield_t: Add an "external storage" flag, dfield->ext.
dfield_is_null(), dfield_is_ext(), dfield_set_ext(), dfield_set_null():
New functions.
dfield_copy(), dfield_copy_data(): Add const qualifiers, fix in/out comments.
data_write_sql_null(): Use memset().
big_rec_field_t: Replace byte* data with const void* data.
ut_ulint_sort(): Remove.
upd_field_t: Remove extern_storage.
upd_node_t: Replace ext_vec, n_ext_vec with n_ext.
row_merge_copy_blobs(): New function.
row_ins_index_entry(): Add the parameter "ibool foreign" for suppressing
foreign key checks during fast index creation or when inserting into
secondary indexes.
btr_page_insert_fits(): Add const qualifiers.
btr_cur_add_ext(), upd_ext_vec_contains(): Remove.
dfield_print_also_hex(), dfield_print(): Replace if...else if with switch.
Observe dfield_is_ext().
the insert buffer bitmap.
ibuf_set_free_bits_func(): Never disable redo logging.
ibuf_update_free_bits_zip(): Remove.
btr_page_reorganize_low(), page_zip_reorganize(): Do not update the insert
buffer bitmap. Instead, document that callers will have to take care of it,
and adapt the callers.
btr_compress(): On error, reset the insert buffer free bits.
btr_cur_insert_if_possible(): Do not modify the insert buffer bitmap.
btr_compress(), btr_cur_optimistic_insert(): On compressed pages,
reset the insert buffer bitmap. Document why.
btr_cur_update_alloc_zip(): Document why it is necessary and sufficient
to reset the insert buffer free bits.
btr_cur_update_in_place(), btr_cur_optimistic_update(),
btr_cur_pessimistic_update(): Update the free bits in the same
mini-transaction. Document that the mini-transaction must be
committed before latching any further pages. Verify that this
is the case in all execution paths.
row_ins_sec_index_entry_by_modify(), row_ins_clust_index_entry_by_modify(),
row_undo_mod_clust_low(): Because these functions call
btr_cur_update_in_place(), btr_cur_optimistic_update(), or
btr_cur_pessimistic_update(), document that the mini-transaction must be
committed before latching any further pages. Verify that this is the case
in all execution paths.
