This addresses Mantis Issue #23 in InnoDB Hot Backup and some
of MySQL Bug #41609.
In MLOG_FILE_CREATE, we need to distinguish temporary tables, so that
InnoDB Hot Backup can work correctly. It turns out that we can do this
easily, by using a bit of the previously unused parameter for page number.
(The page number parameter of MLOG_FILE_CREATE has been written as 0
ever since MySQL 4.1, which introduced MLOG_FILE_CREATE.)
MLOG_FILE_FLAG_TEMP: A flag for indicating a temporary table in
the page number parameter of MLOG_FILE_ operations.
fil_op_write_log(): Add the parameter log_flags.
fil_op_log_parse_or_replay(): Add the parameter log_flags.
Do not replay MLOG_FILE_CREATE when MLOG_FILE_FLAG_TEMP is set in log_flags.
This only affects ibbackup --apply-log. InnoDB itself never replays file
operations.
rb://117 approved by Heikki Tuuri
* Remove old Innobase copyright lines from C source files
* Add a reference to the GPLv2 license as recommended by the lawyers
at Oracle Legal
[Step 2/28]
satisfy some conditions when UNIV_DEBUG is defined.
HASH_SEARCH(): New parameter: ASSERTION. All users will pass an appropriate
ut_ad() or nothing.
dict_table_add_to_columns(): Assert that the table being added to the data
dictionary cache is not already being pointed to by the name_hash and
id_hash tables.
HASH_SEARCH_ALL(): New macro, for use in dict_table_add_to_columns().
dict_mem_table_free(): Set ut_d(table->cached = FALSE), so that we can
check ut_ad(table->cached) when traversing the hash tables, as in
HASH_SEARCH(name_hash, dict_sys->table_hash, ...) and
HASH_SEARCH(id_hash, dict_sys->table_id_hash, ...).
dict_table_get_low(), dict_table_get_on_id_low(): Assert
ut_ad(!table || table->cached).
fil_space_get_by_id(): Check ut_ad(space->magic_n == FIL_SPACE_MAGIC_N)
in HASH_SEARCH(hash, fil_system->spaces, ...).
fil_space_get_by_name(): Check ut_ad(space->magic_n == FIL_SPACE_MAGIC_N)
in HASH_SEARCH(name_hash, fil_system->name_hash, ...).
buf_buddy_block_free(): Check that the blocks are in valid state in
HASH_SEARCH(hash, buf_pool->zip_hash, ...).
buf_page_hash_get(): Check that the blocks are in valid state in
HASH_SEARCH(hash, buf_pool->page_hash, ...).
get_share(), free_share(): Check ut_ad(share->use_count > 0) in
HASH_SEARCH(table_name_hash, innobase_open_tables, ...).
This was posted as rb://75 for tracking down errors similar to Issue #153.
Originally, there were provisions in InnoDB for multiple insert buffer
B-trees, apparently one for each tablespace.
When Heikki implemented innodb_file_per_table (multiple InnoDB
tablespaces) in MySQL 4.1, he made the insert buffer live only in the
system tablespace (space 0) but left the provisions in the code.
When Osku Salerma implemented delete buffering, he also cleaned up the
insert buffer subsystem so that only one insert buffer B-tree exists.
This patch applies the clean-up to the InnoDB Plugin.
Having a separate patch of the insert buffer clean-up should help us
better compare the essential changes of the InnoDB Plugin and InnoDB+
and to track down bugs that are specific to InnoDB+.
IBUF_SPACE_ID: New constant, defined as 0.
ibuf_data_t: Remove.
ibuf_t: Add the applicable fields from ibuf_data_t. There is only one
insert buffer tree from now on.
ibuf_page_low(), ibuf_page(): Merge to a single function ibuf_page().
fil_space_t: Remove ibuf_data.
fil_space_get_ibuf_data(): Remove. There is only one ibuf_data, for
space IBUF_SPACE_ID.
fil_ibuf_init_at_db_start(): Remove.
ibuf_init_at_db_start(): Fuse with ibuf_data_init_for_space().
ibuf_validate_low(): Remove. There is only one ibuf tree.
ibuf_free_excess_pages(), ibuf_header_page_get(),
ibuf_free_excess_pages(): Remove the parameter space, which was always
0.
ibuf_tree_root_get(): Remove the parameters space and data. There is
only one ibuf tree, for space IBUF_SPACE_ID.
ibuf_data_sizes_update(): Rename to ibuf_size_update(), and remove the
parameter data. There is only one ibuf data struct.
ibuf_build_entry_pre_4_1_x(): New function, refactored from
ibuf_build_entry_from_ibuf_rec().
ibuf_data_enough_free_for_insert(), ibuf_data_too_much_free(): Remove
the parameter data. There is only one insert buffer tree.
ibuf_add_free_page(), ibuf_remove_free_page(): Remove the parameters
space and data. There is only one insert buffer tree.
ibuf_get_merge_page_nos(): Add parenthesis, to reduce diffs to
branches/innodb+.
ibuf_contract_ext(): Do not pick an insert buffer tree at random.
There is only one.
ibuf_print(): Print the single insert buffer tree.
rb://19 approved by Heikki on IM
single-table tablespaces. This bug was reported by Sunny as Mantis issue #26.
fil_space_create(), fil_create_new_single_table_tablespace(),
fil_open_single_table_tablespace(), fsp_header_init_fields():
Add ut_a(flags != DICT_TF_COMPACT).
dict_build_table_def_step(), row_import_tablespace_for_mysql(),
row_truncate_table_for_mysql(): Pass correct flags to
fil_create_new_single_table_tablespace() or fil_open_single_table_tablespace().
variable innodb_file_format. Implement file format version stamping of
*.ibd files and SYS_TABLES.TYPE.
This change breaks introduces an incompatible change for for
compressed tables. We can do this, as we have not released yet.
innodb-zip.test: Add tests for stricter KEY_BLOCK_SIZE and ROW_FORMAT
checks.
DICT_TF_COMPRESSED_MASK, DICT_TF_COMPRESSED_SHIFT: Replace with
DICT_TF_ZSSIZE_MASK, DICT_TF_ZSSIZE_SHIFT.
DICT_TF_FORMAT_MASK, DICT_TF_FORMAT_SHIFT, DICT_TF_FORMAT_51,
DICT_TF_FORMAT_ZIP: File format version, stored in table->flags,
in the .ibd file header, and in SYS_TABLES.TYPE.
dict_create_sys_tables_tuple(): Write the table flags to SYS_TABLES.TYPE
if the format is at least DICT_TF_FORMAT_ZIP. For old formats
(DICT_TF_FORMAT_51), write DICT_TABLE_ORDINARY as the table type.
DB_TABLE_ZIP_NO_IBD: Remove the error code. The error handling is done
in ha_innodb.cc; as a failsafe measure, dict_build_table_def_step() will
silently clear the compression and format flags instead of returning this
error.
dict_mem_table_create(): Assert that no extra bits are set in the flags.
dict_sys_tables_get_zip_size(): Rename to dict_sys_tables_get_flags().
Check all flag bits, and return ULINT_UNDEFINED if the combination is
unsupported.
dict_boot(): Document the SYS_TABLES columns N_COLS and TYPE.
dict_table_get_format(), dict_table_set_format(),
dict_table_flags_to_zip_size(): New accessors to table->flags.
dtuple_convert_big_rec(): Introduce the auxiliary variables
local_len, local_prefix_len. Store a 768-byte prefix locally
if the file format is less than DICT_TF_FORMAT_ZIP.
dtuple_convert_back_big_rec(): Restore the columns.
srv_file_format: New variable: innodb_file_format.
fil_create_new_single_table_tablespace(): Replace the parameter zip_size
with table->flags.
fil_open_single_table_tablespace(): Replace the parameter zip_size_in_k
with table->flags. Check the flags.
fil_space_struct, fil_space_create(), fil_op_write_log():
Replace zip_size with flags.
fil_node_open_file(): Note a TODO item for InnoDB Hot Backup.
Check that the tablespace flags match.
fil_space_get_zip_size(): Rename to fil_space_get_flags(). Add a
wrapper for fil_space_get_zip_size().
fsp_header_get_flags(): New function.
fsp_header_init_fields(): Replace zip_size with flags.
FSP_SPACE_FLAGS: New name for the tablespace flags. This field used
to be called FSP_PAGE_ZIP_SIZE, or FSP_LOWEST_NO_WRITE. It has always
been written as 0 in MySQL/InnoDB versions 4.1 to 5.1.
MLOG_ZIP_FILE_CREATE: Rename to MLOG_FILE_CREATE2. Add a 32-bit
parameter for the tablespace flags.
ha_innobase::create(): Check the table attributes ROW_FORMAT and
KEY_BLOCK_SIZE. Issue errors if they are inappropriate, or warnings
if the inherited attributes (in ALTER TABLE) will be ignored.
PAGE_ZIP_MIN_SIZE_SHIFT: New constant: the 2-logarithm of PAGE_ZIP_MIN_SIZE.
symbols. Use it for all definitions of non-static variables and functions.
lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static. It is
referenced from pars0grm.c.
Actually, according to
nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:
* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs
The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.
The class ha_innodb is renamed from class ha_innobase by a #define. Thus,
there will be no clash with the builtin InnoDB. However, there will be some
overhead for invoking virtual methods of class ha_innodb. Ideas for making
the vtable hidden are welcome. -fvisibility=hidden is not available in GCC 3.
trx_t: Remove dict_undo_list and dict_redo_list.
innobase_create_temporary_tablename(): Replace TEMP_TABLE_PREFIX with
a table name suffix "#1" or "#2". In this way, the user can restore
precious data, should anything go wrong. It is possible to reach an
inconsistent state, because the creation, deletion and renaming of
single-table tablespaces are not transactional.
ut_print_namel(), fil_make_ibd_name(), innobase_rename_table(): Remove
the special treatment of TEMP_TABLE_PREFIX.
Introduce TEMP_INDEX_PREFIX == 0xff for temporary indexes. This byte
cannot occur in index names since MySQL 4.1. However, it might have
been possible to use this byte in MySQL 4.0.
recv_recovery_from_checkpoint_finish(): Call the new function
row_merge_drop_temp_indexes(), to drop all indexes whose name starts
with the byte 0xff.
row_merge_rename_indexes(): Renamed from row_merge_rename_index().
Remove the parameter "index".
row_drop_table_for_mysql(): Unconditionally call trx_commit_for_mysql().
row_drop_table_for_mysql_no_commit(): Correct the function commit,
based on the corrected comment of row_drop_table_for_mysql(). Rely on
table->to_be_dropped instead of TEMP_TABLE_PREFIX.
ha_innobase::add_index(): Simplify the control flow.
C++ requires explicit type casts when converting from void*, and *.ic files
are now inlined in ha_innodb.cc.
fil_space_get_by_id(), fil_space_get_by_name(): New inline functions, to
replace occurrences of the HASH_SEARCH() macro in fil0fil.c. This should
improve readability.
temporary tablespaces in fast index creation. The tablespaces must
reside in the same directory as the persistent tablespaces, so that
they can be renamed without copying. Namespace collisions are avoided
by replacing the ".ibd" file name suffix with ".ib1" or ".ib2".
fil_make_ibd_name(): Treat names starting with TEMP_TABLE_PREFIX
specially. Document the dependence on
innobase_create_temporary_tablename().
dict_truncate_index_tree(): Add the parameter space for specifying the
new tablespace identifier of a single-table tablespace that has been
recreated. When SYS_INDEXES.PAGE_NO == FIL_NULL, do not abort but
create the index tree.
fil_discard_tablespace(): Pass the return code from fil_delete_tablespace().
row_truncate_table_for_mysql(): Attempt to discard and recreate
single-table tablespaces. Reassign the tablespace identifier both in
the data dictionary (SYS_TABLES and SYS_INDEXES) and in the data
dictionary cache.
buf_page_get_gen(). This saves one mutex operation per block request.
buf_page_get_gen(), various macros and functions: Add parameter zip_size.
btr_node_ptr_get_child(): Add parameter index.
fil_space_get_latch(): Add optional output parameter zip_size.
fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.
fsp_header_init(): Remove the parameter zip_size.
ibuf_free_excess_pages(): Remove the parameter zip_size.
trx_rseg_t, trx_undo_t: Add field zip_size.
xdes_lst_get_next(): Remove, unused.
Add m_nonempty for facilitating the test in page_zip_alloc(). This
reduces the combined size of the bit-fields to 32 bits. Thus,
sizeof(page_zip_des_t) == 2 machine words on 32-bit and wider systems.
decompression from page_zip_des_t to buf_page_t, because the fields
needed in compression are modified without holding the block mutex.
All writes to bit-fields sharing a machine word must be protected
by the same mutex or rw-lock.
storage size from 16 to 3 bits.
page_zip_get_size(), page_zip_set_size(): New functions.
Replace direct references to page_zip_des_t:size with calls to
buf_block_get_zip_size(), page_zip_get_size(), and page_zip_set_size().
and uncompressed buffer pool pages.
buf_block_t: Replace page_zip, space, and offset with buf_page_t page.
Replace some integers with bit-fields.
enum buf_block_state: Rename to buf_page_state. Add BUF_BLOCK_ZIP_PAGE.
page_zip_des_t: Add the field "state". Make the integer fields bit-fields.
page_zip_copy(): Document which fields are copied.
buf_pool_t: Remove n_frames, max_size, and blocks_of_frames.
The current buffer pool size is in curr_size.
buf_pool_init(): Remove parameter max_size.
buf_pool_get_max_size(), buf_pool_is_block(): Remove.
buf_block_align(): Do not assume that the buffer pool is allocated
in one chunk. Replace dependency on buf_pool->blocks_of_frames
with a call to buf_page_hash_get().
rec_get_nth_field_offs_old(): Split from rec_get_nth_field_old().
rec_get_nth_field_old(): Reimplement as a constness-preserving macro.
dict_drop_index_tree(), dict_truncate_index_tree(): Note that rec is
an in/out parameter, as the B-tree root page number will be updated.
buf_flush_init_for_writing(): Remove parameters space, page_no.
fsp_init_file_page_low(): Wriet space_id and page_no to the page.
fil_create_new_single_table_tablespace(): Write space_id to the page.
dict_load_foreigns(): Enclose in #ifndef UNIV_HOTBACKUP.
fil_extend_tablespaces_to_stored_len(): Pass zip_size to fil_read().
buf_page_init_for_backup_restore(): Add parameter zip_size.
Enclose the declaration in buf0buf.h in #ifdef UNIV_HOTBACKUP.
recv_apply_log_recs_for_backup(): Replace the local variable "page"
with the local variable "block". Add local variable zip_size.
fil_read(), fil_write(): Make these inlined functions in fil0fil.c.
fil_write_lsn_and_arch_no_to_file(): Remove the parameter space_id and
note that this function is to be called on the system tablespace, which
is uncompressed.
mlog_parse_string(): Remove an off-by-one error in an assertion.
fil_extend_space_to_desired_size(): fsp_header_get_free_limit(),
fsp_try_extend_data_file(): Replace UNIV_PAGE_SIZE with the actual page size.
PAGE_ZIP_MIN_SIZE: New constant, to be used in assertions.
FSP_SEG_INODES_PER_PAGE: Add parameter zip_size.
fsp_seg_inode_page_get_nth_inode(), fsp_seg_inode_page_find_used(),
fsp_seg_inode_page_find_free(): Add parameter zip_size.
