INNODB_COMPRESSION_BUDDY and INNODB_COMPRESSION_BUDDY_RESET.
buf_buddy_stat_struct, buf_buddy_stat_t, buf_buddy_stat[]:
Statistics of the buddy system grouped by block size.
i_s_innodb_compression_buddy, i_s_innodb_compression_buddy_reset:
New INFORMATION_SCHEMA plugins.
i_s_compression_buddy_fields_info[]: Define the fields:
size, used, free, relocated, relocated_sec.
i_s_compression_buddy_fill_low(), i_s_compression_buddy_fill(),
i_s_compression_buddy_reset_fill(): Fill the fields.
i_s_compression_buddy_init(), i_s_compression_buddy_reset_init():
Initialize the tables.
INNODB_ZIP and INNODB_ZIP_RESET to
INNODB_COMPRESSION and INNODB_COMPRESSION_RESET,
and remove the statistics of the buddy system.
This change was discussed with Ken. It makes the tables shorter
and easier to understand. The removed data will be represented in
the tables INNODB_COMPRESSION_BUDDY and INNODB_COMPRESSION_BUDDY_RESET
that will be added later.
i_s_innodb_zip, i_s_innodb_zip_reset, i_s_zip_fields_info[],
i_s_zip_fill_low(), i_s_zip_fill(), i_s_zip_reset_fill(),
i_s_zip_init(), i_s_zip_reset_init(): Replace "zip" with "compression".
i_s_compression_fields_info[]: Remove "used", "free",
"relocated", "relocated_usec". In "compressed_usec" and "decompressed_usec",
replace microseconds with seconds ("usec" with "sec").
page_zip_decompress(): Correct a typo in the function comment.
PAGE_ZIP_SSIZE_BITS, PAGE_ZIP_NUM_SSIZE: New constants.
page_zip_stat_t, page_zip_stat: Statistics of the compression, grouped
by page size.
page_zip_simple_validate(): Assert that page_zip->ssize is reasonable.
Add the prototype of check_global_access() if MySQL version is less than
5.1.24 to make zip compile with 5.1.23. This was removed when MySQL added
the prototype in their code in 5.1.24 but we still need zip to compile with
older versions.
my_error(ER_TOO_BIG_ROWSIZE, ...). Otherwise, MySQL can report
"Got error 139 from storage engine" instead of the appropriate
error message.
ha_innobase::index_read(), ha_innobase::general_fetch():
Replace if-else if-else with switch-case.
Pass table->flags to convert_error_code_to_mysql().
innodb_check_for_record_too_big_error(). Remove. This code belongs to
convert_error_code_to_mysql().
convert_error_code_to_mysql(): Add the parameter "flags", for table flags.
Translate DB_TOO_BIG_RECORD into ER_TOO_BIG_ROWSIZE.
create_index(): Add the parameter "flags".
create_clustered_index_when_no_primary(): Replace the parameter "comp"
with "flags".
innobase_drop_database(): Remove the #ifdef'd-out call to
convert_error_code_to_mysql().
Throw warnings, not errors for wrong ROW_FORMAT or KEY_BLOCK_SIZE,
so that any table dump can be loaded.
As of this change, InnoDB supports the following table formats:
ROW_FORMAT=REDUNDANT
the only format before MySQL/InnoDB 5.0.3
ROW_FORMAT=COMPACT
the new default format of MySQL/InnoDB 5.0.3
ROW_FORMAT=DYNAMIC
uncompressed, no prefix in the clustered index record for BLOBs
ROW_FORMAT=COMPRESSED
like ROW_FORMAT=DYNAMIC, but zlib compressed B-trees and BLOBs;
the compressed page size is specified by KEY_BLOCK_SIZE in
kilobytes (1, 2, 4, 8, or 16; default 8)
KEY_BLOCK_SIZE=1, 2, 4, 8, or 16: implies ROW_FORMAT=COMPRESSED;
ignored if ROW_FORMAT is not COMPRESSED
KEY_BLOCK_SIZE=anything else: ignored
The InnoDB row format is displayed in the 4th column (Row_format) of
the output of SHOW TABLE STATUS. The Create_options column may show
ROW_FORMAT= and KEY_BLOCK_SIZE=, but they do not necessarily have
anything to do with InnoDB.
The table format can also be queried like this:
SELECT table_schema, table_name, row_format
FROM information_schema.tables
WHERE engine='innodb' and row_format in ('Compressed','Dynamic');
When Row_format='Compressed', KEY_BLOCK_SIZE should usually correspond
to the compressed page size. But the .frm file could be manipulated
to show any KEY_BLOCK_SIZE.
For some reason, INFORMATION_SCHEMA.TABLES.CREATE_OPTIONS does not
include KEY_BLOCK_SIZE. It does include row_format (spelled in
lowercase). This looks like a MySQL bug, because the table
INFORMATION_SCHEMA.TABLES probably tries to replace SHOW TABLE STATUS.
I reported this as Bug #35275 <http://bugs.mysql.com/35275>.
ha_innobase::get_row_type(): Add ROW_TYPE_COMPRESSED, ROW_TYPE_DYNAMIC.
ha_innobase::create(): Implement ROW_FORMAT=COMPRESSED and
ROW_FORMAT=DYNAMIC. Do not throw errors for wrong ROW_FORMAT or
KEY_BLOCK_SIZE, but issue warnings instead.
ha_innobase::check_if_incompatible_data(): Return COMPATIBLE_DATA_NO
if KEY_BLOCK_SIZE has been specified.
innodb.result: Adjust the result for the warning issued for ROW_FORMAT=FIXED.
innodb-zip.test: Add tests. Query INFORMATION_SCHEMA.TABLES for ROW_FORMAT.
variable innodb_file_format. Implement file format version stamping of
*.ibd files and SYS_TABLES.TYPE.
This change breaks introduces an incompatible change for for
compressed tables. We can do this, as we have not released yet.
innodb-zip.test: Add tests for stricter KEY_BLOCK_SIZE and ROW_FORMAT
checks.
DICT_TF_COMPRESSED_MASK, DICT_TF_COMPRESSED_SHIFT: Replace with
DICT_TF_ZSSIZE_MASK, DICT_TF_ZSSIZE_SHIFT.
DICT_TF_FORMAT_MASK, DICT_TF_FORMAT_SHIFT, DICT_TF_FORMAT_51,
DICT_TF_FORMAT_ZIP: File format version, stored in table->flags,
in the .ibd file header, and in SYS_TABLES.TYPE.
dict_create_sys_tables_tuple(): Write the table flags to SYS_TABLES.TYPE
if the format is at least DICT_TF_FORMAT_ZIP. For old formats
(DICT_TF_FORMAT_51), write DICT_TABLE_ORDINARY as the table type.
DB_TABLE_ZIP_NO_IBD: Remove the error code. The error handling is done
in ha_innodb.cc; as a failsafe measure, dict_build_table_def_step() will
silently clear the compression and format flags instead of returning this
error.
dict_mem_table_create(): Assert that no extra bits are set in the flags.
dict_sys_tables_get_zip_size(): Rename to dict_sys_tables_get_flags().
Check all flag bits, and return ULINT_UNDEFINED if the combination is
unsupported.
dict_boot(): Document the SYS_TABLES columns N_COLS and TYPE.
dict_table_get_format(), dict_table_set_format(),
dict_table_flags_to_zip_size(): New accessors to table->flags.
dtuple_convert_big_rec(): Introduce the auxiliary variables
local_len, local_prefix_len. Store a 768-byte prefix locally
if the file format is less than DICT_TF_FORMAT_ZIP.
dtuple_convert_back_big_rec(): Restore the columns.
srv_file_format: New variable: innodb_file_format.
fil_create_new_single_table_tablespace(): Replace the parameter zip_size
with table->flags.
fil_open_single_table_tablespace(): Replace the parameter zip_size_in_k
with table->flags. Check the flags.
fil_space_struct, fil_space_create(), fil_op_write_log():
Replace zip_size with flags.
fil_node_open_file(): Note a TODO item for InnoDB Hot Backup.
Check that the tablespace flags match.
fil_space_get_zip_size(): Rename to fil_space_get_flags(). Add a
wrapper for fil_space_get_zip_size().
fsp_header_get_flags(): New function.
fsp_header_init_fields(): Replace zip_size with flags.
FSP_SPACE_FLAGS: New name for the tablespace flags. This field used
to be called FSP_PAGE_ZIP_SIZE, or FSP_LOWEST_NO_WRITE. It has always
been written as 0 in MySQL/InnoDB versions 4.1 to 5.1.
MLOG_ZIP_FILE_CREATE: Rename to MLOG_FILE_CREATE2. Add a 32-bit
parameter for the tablespace flags.
ha_innobase::create(): Check the table attributes ROW_FORMAT and
KEY_BLOCK_SIZE. Issue errors if they are inappropriate, or warnings
if the inherited attributes (in ALTER TABLE) will be ignored.
PAGE_ZIP_MIN_SIZE_SHIFT: New constant: the 2-logarithm of PAGE_ZIP_MIN_SIZE.
There is one consideration: fil_init() chooses the tablespace hash size
based on the initial value of srv_file_per_table. However, this is nothing
new: InnoDB could be started with innodb_file_per_table=0 even though
*.ibd files exist.
srv_file_per_table: Declare as my_bool instead of ibool, because
MYSQL_SYSVAR_BOOL() expects a pointer to my_bool. Document the
variable also in srv0srv.h.
innobase_start_or_create_for_mysql(): Note why it is OK to temporarily
clear srv_file_per_table.
innobase_file_per_table: Remove.
innobase_raw_format(), move the definition from row0row.c to
ha_innodb.cc. After this change, row0row.c no longer references
system_charset_info (Mantis issue #17). Patch prepared by Vasil,
tested by Calvin, and reviewed by Marko.
Use innobase_strcasecmp() insteaed of strcasecmp() in i_s.cc and get rid
of strings.h (that file is not present on Windows).
Move the prototype of innobase_strcasecmp() from ha_innodb.cc and
dict0dict.c to ha_prototypes.h.
Approved by: Heikki
symbols. Use it for all definitions of non-static variables and functions.
lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static. It is
referenced from pars0grm.c.
Actually, according to
nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:
* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs
The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.
The class ha_innodb is renamed from class ha_innobase by a #define. Thus,
there will be no clash with the builtin InnoDB. However, there will be some
overhead for invoking virtual methods of class ha_innodb. Ideas for making
the vtable hidden are welcome. -fvisibility=hidden is not available in GCC 3.
to the undo log, also store the original length of the column, so that the
changes will be correctly undone in transaction rollback or when fetching
previous versions of the row.
innodb-zip.test: New file, for tests of the compression.
upd_field_t: Add orig_len, the original length of new_val.
btr_push_update_extern_fields(): Restore the original prefix of the column.
Add the parameter heap where memory will be allocated if necessary.
trx_undo_rec_get_col_val(): Add the output parameter orig_len.
trx_undo_page_report_modify_ext(): New function: Write an externally
stored column to the undo log. This is only called from
trx_undo_page_report_modify(), and this is the only caller of
trx_undo_page_fetch_ext().
trx_undo_update_rec_get_update(): Read the original length of the column
prefix to upd_field->orig_len.
Forward port of r2236
Introduce retry/sleep logic as a workaround for a transient bug
where ::open fails for partitioned tables randomly if we are using
one file per table. (Bug #33349)
Reviewed by: Heikki
kernel_mutex must be released before calling this function.
innobase_mysql_end_print_arbitrary_thd(),
innobase_mysql_prepare_print_arbitrary_thd(): Assert that the
kernel_mutex is not being held by the current thread.
Non-functional change:
Move the prototypes of
innobase_mysql_prepare_print_arbitrary_thd() and
innobase_mysql_end_print_arbitrary_thd() from lock0lock.c to
ha_prototypes.h
Suggested by: Marko
Approved by: Marko
innodb_information_schema.test. Add tests that display most columns from
INFORMATION_SCHEMA.INNODB_LOCKS. Test that quoting of table names works
and respects SQL_MODE='ANSI_QUOTES'.
innobase_print_identifier(): Remove.
innobase_convert_identifier(): New function,
based on innobase_print_identifier().
innobase_convert_name(): New function, similar to ut_print_namel(), but
using a memory buffer.
ut_print_namel(): Use innobase_convert_name().
fill_innodb_locks_from_cache(): Convert lock_table and lock_index by
calling innobase_convert_name().
a compressed table in the system tablespace.
db0err.h: Introduce the error code DB_TABLE_ZIP_NO_IBD. Replace the
#define directives with an enum, to ease future code merges. These
error codes are never written out to files or displayed to the user.
Thus they need not remain constant.
dict_build_table_def_step(): Return DB_TABLE_ZIP_NO_IBD instead of DB_ERROR.
create_table_def(): Report ER_ILLEGAL_HA_CREATE_OPTION "KEY_BLOCK_SIZE"
when the table creation fails with DB_TABLE_ZIP_NO_IBD.
redefined so that the dynamic plugin can replace the builtin InnoDB
in MySQL 5.1.
ha_innodb.cc, handler0alter.cc: #include "univ.i" before any other InnoDB
header files or before defining any symbols
innodb_redefine.h: New file, to contain a mapping of symbols. The idea
is that this file will be replaced in the build process; because this
is a large file that can be generated automatically, it does not make sense
to keep it under version control.
univ.i: #include "innodb_redefine.h" and #define ha_innobase ha_innodb
Makefile.am (ha_innodb_la_CXXFLAGS): Remove -Dha_innobase=ha_innodb
NOTE: there are still some issues in the source code. One known issue is
the #undef mutex_free in sync0sync.h, which will cause the plugin to call the
function mutex_free in the builtin InnoDB. The preprocessor symbols defined
in innodb_redefine.h must not be undefined or redefined anywhere in the code.
plugin "InnoDB", not "InnoDBzip".
We can disable the builtin InnoDB by mysqld --skip-innodb. If the
builtin InnoDB is not disabled, installing the InnoDB plugin by the same
name will not work.
innodb_plugin_init(): Ignore differences in the PLUGIN_VAR_READONLY flag.
ha_innobase::write_row(): The printf format %p expects const void*.
STRUCT_FLD: Do not use the GCC extension when __STRICT_ANSI__ is defined.
row_merge_read_clustered_index(): Compound initializers must not contain
variables. Assign to struct fields instead.
to replace a built-in instance of InnoDB in mysqld. This is work in
progress, with several limitations:
* Other plugins defined in the builtin InnoDB are not disabled.
However, InnoDB in MySQL 5.1 only defines the storage engine plugin,
no INFORMATION_SCHEMA plugins.
* The global C symbols in ha_innodb.so except innodb_hton_ptr and
builtin_innobase_plugin will have to be redefined, e.g., by objcopy.
* The storage engine cannot be called "InnoDB" to avoid a conflict with
the builtin name. Here we call it InnoDBzip.
innobase_hton_name[]: Rename to "InnoDBzip" when building a dynamic plugin.
innodb_plugin_init(): New function for the dynamic plugin, to copy
and redirect configuration parameters from the builtin InnoDB.
innodb_dynamic: New configuration parameter. This has to be added
to the builtin InnoDB in MySQL 5.1. Also, innodb_hton_ptr must be made
global there.
innobase_init(): Interpret the parameter innodb_dynamic.
Makefile.am: Redefine class ha_innobase to ha_innodb by a preprocessor
define. Apparently, C++ classes cannot be easily renamed by objcopy.
INNODB_ZIP and INNODB_ZIP_RESET from ha_innodb.cc to i_s.cc.
plugin_author, END_OF_ST_FIELD_INFO, i_s_info: New common constants.
trx_i_s_common_deinit(): Renamed to i_s_common_deinit().
ha_innobase::update_thd(void): New function, to call the inline function
ha_innobase::update_thd(THD*).
check_trx_exists(): Make static. handler0alter.cc does not need to call
this function.
dtuple_create_for_mysql(), dtuple_free_for_mysql(): Remove.
ha_innobase::records_in_range(): Use mem_heap_create(), mem_heap_free(),
and dtuple_create() instead of the removed functions above. Since r1587,
InnoDB C++ functions can invoke inlined C functions.
innodb_lock_waits. See
https://svn.innodb.com/innobase/InformationSchema/TransactionsAndLocks
for design notes.
Things that need to be resolved before this goes live:
* MySQL must add thd_get_thread_id() function to their code
http://bugs.mysql.com/30930
* Allocate memory from mem_heap instead of using mem_alloc()
* Copy table name and index name into the cache because they may be
freed later which will result in referencing freed memory
Approved by: Marko
existed before the fast-index-creation code was merged, in r1414.
Do not call row_prebuilt_table_obsolete(innodb_table) until after
the table has been renamed to a temporary name. Otherwise, the
following could happen when creating a clustered index on table T:
(a1) copy T to T#1
(a2) mark T obsolete
(b1) add the name of T to the background drop list
(a3) rename T to T#2 and T#1 to T
(b2) drop the table on the background
These steps are executed by two threads, a and b.
If b1 is executed after a3, all is fine: the name T#2 will be added
to the background drop list. If b1 is executed before a3, the name
T will be added, and T#2 will survive.
fast index creation.
innobase_convert_tablename(): New function, to remove the filename
encoding of a database and table name.
ha_innobase::add_index(): Invoke innobase_convert_tablename() before
calling my_error().
innodb-index.result: Update the error messages.
trx_t: Remove dict_undo_list and dict_redo_list.
innobase_create_temporary_tablename(): Replace TEMP_TABLE_PREFIX with
a table name suffix "#1" or "#2". In this way, the user can restore
precious data, should anything go wrong. It is possible to reach an
inconsistent state, because the creation, deletion and renaming of
single-table tablespaces are not transactional.
ut_print_namel(), fil_make_ibd_name(), innobase_rename_table(): Remove
the special treatment of TEMP_TABLE_PREFIX.
Introduce TEMP_INDEX_PREFIX == 0xff for temporary indexes. This byte
cannot occur in index names since MySQL 4.1. However, it might have
been possible to use this byte in MySQL 4.0.
recv_recovery_from_checkpoint_finish(): Call the new function
row_merge_drop_temp_indexes(), to drop all indexes whose name starts
with the byte 0xff.
row_merge_rename_indexes(): Renamed from row_merge_rename_index().
Remove the parameter "index".
row_drop_table_for_mysql(): Unconditionally call trx_commit_for_mysql().
row_drop_table_for_mysql_no_commit(): Correct the function commit,
based on the corrected comment of row_drop_table_for_mysql(). Rely on
table->to_be_dropped instead of TEMP_TABLE_PREFIX.
ha_innobase::add_index(): Simplify the control flow.
