Commit graph

35 commits

Author SHA1 Message Date
marko
86361e032b branches/zip: Implement the configuration parameter and settable global
variable innodb_file_format.  Implement file format version stamping of
*.ibd files and SYS_TABLES.TYPE.

This change breaks introduces an incompatible change for for
compressed tables.  We can do this, as we have not released yet.

innodb-zip.test: Add tests for stricter KEY_BLOCK_SIZE and ROW_FORMAT
checks.

DICT_TF_COMPRESSED_MASK, DICT_TF_COMPRESSED_SHIFT: Replace with
DICT_TF_ZSSIZE_MASK, DICT_TF_ZSSIZE_SHIFT.

DICT_TF_FORMAT_MASK, DICT_TF_FORMAT_SHIFT, DICT_TF_FORMAT_51,
DICT_TF_FORMAT_ZIP: File format version, stored in table->flags,
in the .ibd file header, and in SYS_TABLES.TYPE.

dict_create_sys_tables_tuple(): Write the table flags to SYS_TABLES.TYPE
if the format is at least DICT_TF_FORMAT_ZIP.  For old formats
(DICT_TF_FORMAT_51), write DICT_TABLE_ORDINARY as the table type.

DB_TABLE_ZIP_NO_IBD: Remove the error code.  The error handling is done
in ha_innodb.cc; as a failsafe measure, dict_build_table_def_step() will
silently clear the compression and format flags instead of returning this
error.

dict_mem_table_create(): Assert that no extra bits are set in the flags.

dict_sys_tables_get_zip_size(): Rename to dict_sys_tables_get_flags().
Check all flag bits, and return ULINT_UNDEFINED if the combination is
unsupported.

dict_boot(): Document the SYS_TABLES columns N_COLS and TYPE.

dict_table_get_format(), dict_table_set_format(),
dict_table_flags_to_zip_size(): New accessors to table->flags.

dtuple_convert_big_rec(): Introduce the auxiliary variables
local_len, local_prefix_len.  Store a 768-byte prefix locally
if the file format is less than DICT_TF_FORMAT_ZIP.

dtuple_convert_back_big_rec(): Restore the columns.

srv_file_format: New variable: innodb_file_format.

fil_create_new_single_table_tablespace(): Replace the parameter zip_size
with table->flags.

fil_open_single_table_tablespace(): Replace the parameter zip_size_in_k
with table->flags.  Check the flags.

fil_space_struct, fil_space_create(), fil_op_write_log():
Replace zip_size with flags.

fil_node_open_file(): Note a TODO item for InnoDB Hot Backup.
Check that the tablespace flags match.

fil_space_get_zip_size(): Rename to fil_space_get_flags().  Add a
wrapper for fil_space_get_zip_size().

fsp_header_get_flags(): New function.

fsp_header_init_fields(): Replace zip_size with flags.

FSP_SPACE_FLAGS: New name for the tablespace flags.  This field used
to be called FSP_PAGE_ZIP_SIZE, or FSP_LOWEST_NO_WRITE.  It has always
been written as 0 in MySQL/InnoDB versions 4.1 to 5.1.

MLOG_ZIP_FILE_CREATE: Rename to MLOG_FILE_CREATE2.  Add a 32-bit
parameter for the tablespace flags.

ha_innobase::create(): Check the table attributes ROW_FORMAT and
KEY_BLOCK_SIZE.  Issue errors if they are inappropriate, or warnings
if the inherited attributes (in ALTER TABLE) will be ignored.

PAGE_ZIP_MIN_SIZE_SHIFT: New constant: the 2-logarithm of PAGE_ZIP_MIN_SIZE.
2008-03-10 11:05:32 +00:00
marko
c853494ea2 branches/zip: Make innodb_file_per_table a settable global variable.
There is one consideration: fil_init() chooses the tablespace hash size
based on the initial value of srv_file_per_table.  However, this is nothing
new: InnoDB could be started with innodb_file_per_table=0 even though
*.ibd files exist.

srv_file_per_table: Declare as my_bool instead of ibool, because
MYSQL_SYSVAR_BOOL() expects a pointer to my_bool.  Document the
variable also in srv0srv.h.

innobase_start_or_create_for_mysql(): Note why it is OK to temporarily
clear srv_file_per_table.

innobase_file_per_table: Remove.
2008-03-07 12:56:52 +00:00
marko
9d58e3ff99 branches/zip: Improve the LRU algorithm with a separate unzip_LRU list of
blocks that contains uncompressed and compressed frames.  This patch was
designed by Heikki and Inaam, implemented by Inaam, and refined and reviewed
by Marko and Sunny.

buf_buddy_n_frames, buf_buddy_min_n_frames, buf_buddy_max_n_frames: Remove.

buf_page_belongs_to_unzip_LRU(): New predicate:
bpage->zip.data && buf_page_get_state(bpage) == BUF_BLOCK_FILE_PAGE.

buf_pool_t, buf_block_t: Add the linked list unzip_LRU.  A block in the
regular LRU list is in unzip_LRU iff buf_page_belongs_to_unzip_LRU() holds.

buf_LRU_free_block(): Add a third return value to refine the case
"cannot free the block".

buf_LRU_search_and_free_block(): Update the documentation to reflect the
implementation.

buf_LRU_stat_t, buf_LRU_stat_cur, buf_LRU_stat_sum, buf_LRU_stat_arr[]:
Statistics for the unzip_LRU algorithm.

buf_LRU_stat_update(): New function: Update the statistics.  Called once
per second by srv_error_monitor_thread().

buf_LRU_validate(): Validate the unzip_LRU list as well.

buf_LRU_evict_from_unzip_LRU(): New predicate: Use the unzip_LRU before
falling back to the regular LRU?

buf_LRU_free_from_unzip_LRU_list(), buf_LRU_free_from_common_LRU_list():
Subfunctions of buf_LRU_search_and_free_block().

buf_LRU_search_and_free_block(): Reimplement.  Try to evict an uncompressed
page from the unzip_LRU list before falling back to evicting an entire block
from the common LRU list.

buf_unzip_LRU_remove_block_if_needed(): New function.

buf_unzip_LRU_add_block(): New function: Add a block to the unzip_LRU list.
2008-03-03 12:48:38 +00:00
marko
cdba733fb5 branches/zip: Merge 2263:2295 from branches/5.1. 2008-02-18 20:09:03 +00:00
inaam
c087bec896 branches/zip:
srv_log_file_size is misspelled as srv_log_filen_size in r2276

reviewed by: non-functional change.
2008-02-07 20:12:11 +00:00
marko
2c2b06ad75 branches/zip: Introduce UNIV_INTERN, a linkage specifier for InnoDB-global
symbols.  Use it for all definitions of non-static variables and functions.

lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static.  It is
referenced from pars0grm.c.

Actually, according to
	nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:

* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs

The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.

The class ha_innodb is renamed from class ha_innobase by a #define.  Thus,
there will be no clash with the builtin InnoDB.  However, there will be some
overhead for invoking virtual methods of class ha_innodb.  Ideas for making
the vtable hidden are welcome.  -fvisibility=hidden is not available in GCC 3.
2008-02-06 14:17:36 +00:00
marko
ddcf4f8d9c branches/zip: Remove some unnecessary memory references in the master thread.
srv_print_thread_releases, srv_print_lock_waits, srv_print_buf_io,
srv_print_log_io, srv_print_latch_waits: Define these variables as
constants (FALSE), unless UNIV_DEBUG is defined.  These variables
are never assigned to, and they are initialized to FALSE.  It could
be useful to set them when debugging InnoDB.

srv_slot_t: Fuse the fields type, in_use, suspended to a single machine word.

srv_meter[], srv_meter_low_water[], srv_meter_high_water[],
srv_meter_high_water2[], srv_meter_foreground[]: Enclose these arrays
inside #if 0.  The arrays are essentially constants that do not affect
the control flow.

enum srv_thread_type: New enum, to replace the #defines SRV_COM, ...
Enclose the unused values SRV_BUFFER, SRV_RECOVERY, SRV_INSERT
inside #if 0, so that some arrays and loops can be reduced.
2007-10-24 07:12:11 +00:00
marko
e6f452c7cd branches/zip: srv_suspend_mysql_thread(): Allow the transaction to
hold an X-latch to the data dictionary, to avoid an assertion failure
when a lock wait occurs in row_merge_lock_table().
2007-10-19 10:30:34 +00:00
vasil
b9982ac9ec Implement INFORMATION_SCHEMA tables innodb_trx, innodb_locks,
innodb_lock_waits. See
https://svn.innodb.com/innobase/InformationSchema/TransactionsAndLocks
for design notes.

Things that need to be resolved before this goes live:
* MySQL must add thd_get_thread_id() function to their code
  http://bugs.mysql.com/30930
* Allocate memory from mem_heap instead of using mem_alloc()
* Copy table name and index name into the cache because they may be
  freed later which will result in referencing freed memory

Approved by:	Marko
2007-09-20 11:23:44 +00:00
marko
f5e4a4c3ed branches/zip: Merge 1830:1862 from trunk. 2007-09-20 08:36:22 +00:00
marko
7fbc04fcdc branches/zip: Merge 1533:1553 from trunk. 2007-06-06 11:07:37 +00:00
marko
d75e64f98a branches/zip: Merge 1493:1533 from trunk.
Make use of UT_BITS_IN_BYTES().
2007-05-29 08:48:16 +00:00
marko
e7284367dc branches/zip: Merge revisions 1402:1493 from trunk.
Remove mysql.patch, because the configuration parameter interface has changed.
2007-05-14 09:07:15 +00:00
marko
33c1125d55 branches/zip: Merge revisions 1322:1402 from trunk. 2007-04-02 05:39:41 +00:00
marko
16ffd00dc1 branches/zip: Merge revisions 1206:1271 from trunk. 2007-02-02 11:31:29 +00:00
marko
0178dba161 branches/zip: Avoid memory fragmentation when adding column definitions
to tables.

dict_mem_table_add_col(): Add the parameter "heap" for temporary memory
allocation.  Allow it and "name" to be NULL.  These parameters are NULL
when creating dummy indexes.

dict_add_col_name(): Remove calls to ut_malloc() and ut_free().

dict_table_get_col_name(): Allow table->col_names to be NULL.

dict_table_add_system_columns(), dict_table_add_to_cache():
Add the parameter "heap".
2007-01-30 09:24:18 +00:00
marko
d8536b0de8 branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG. 2007-01-18 18:29:12 +00:00
marko
5e9d893508 branches/zip: Replace ib_ulonglong with ib_uint64_t, since we want exactly
64 bits.  "long long" might be longer than 64 bits on some systems.
2006-11-29 14:52:16 +00:00
marko
109570b94a branches/zip: Introduce the unsigned 64-bit data type ib_ulonglong
for log sequence numbers, to replace dulint.
2006-11-24 13:05:01 +00:00
marko
879b1fd73d branches/zip: Implement resizeable buffer pool.
mysql.patch: Patch to change or add variables to MySQL

innodb.patch: Patch to make the master thread poll requests to resize
the buffer pool.

Replace srv_pool_size and innobase_buffer_pool_size
with srv_buf_pool_size, srv_buf_pool_old_size,
and srv_buf_pool_curr_size.

Add buf_chunk_t, a collection of buf_block_t.
2006-11-14 14:27:26 +00:00
marko
91b16c3c56 branches/zip: Prepare for buffer pool allocation in several chunks.
buf_pool_t: Remove n_frames, max_size, and blocks_of_frames.
The current buffer pool size is in curr_size.

buf_pool_init(): Remove parameter max_size.

buf_pool_get_max_size(), buf_pool_is_block(): Remove.

buf_block_align(): Do not assume that the buffer pool is allocated
in one chunk.  Replace dependency on buf_pool->blocks_of_frames
with a call to buf_page_hash_get().
2006-10-30 15:15:19 +00:00
marko
69fc883be2 branches/zip: Remove support for AWE (Intel PAE on Win32).
Removal of the variable innobase_buffer_pool_awe_mem_mb requires
coordination with MySQL AB.
2006-10-30 13:48:08 +00:00
marko
e8a779e73e branches/zip: Remove the casts introduced in r920 and r925. Replace the
accessors returning pointers with macros that preserve const qualifiers.
In UNIV_DEBUG builds, retain the accessors and cast away constness there.

dfield_get_type(), dfield_get_data(), dtuple_get_nth_field(),
dict_table_get_nth_col(), dict_table_get_sys_col(): Implement as macro
unless #ifdef UNIV_DEBUG.

rec_get_nth_field(): Replace with rec_get_nth_field_offs() that does not
do pointer arithmetics.  Implement rec_get_nth_field() as a macro.
2006-10-19 07:27:26 +00:00
marko
8c6526d6d2 branches/zip: srv_master_thread(): Remove unnecessary assignments to
n_pages_purged.
2006-09-22 06:56:50 +00:00
marko
9a16aac783 branches/zip: Merge revisions 804:838 from trunk. 2006-09-19 10:14:07 +00:00
marko
78d1dd326c branches/zip: Merge revisions 767:789 from trunk. 2006-09-04 19:47:47 +00:00
marko
64f3cee67f branches/zip: Merge revisions 720:765 from trunk and reindent the code
as per revisions r763:765.
2006-08-29 09:30:31 +00:00
marko
36ed778816 branches/zip: Merge revisions 634:713 from trunk. 2006-08-11 11:49:38 +00:00
marko
645e56909f branches/zip: Merge revisions 583:634 from trunk. 2006-06-13 20:23:26 +00:00
marko
cfc945a954 branches/zip: Merge revisions 465:532 from trunk. 2006-05-08 06:18:59 +00:00
marko
450bbd7c3b branches/zip: Merge revisions 265:459 from trunk. 2006-04-12 09:32:17 +00:00
marko
82d38b63e3 branches/zip: Merge revisions 236:247 from trunk. 2006-02-27 09:33:26 +00:00
marko
f40121415d branches/zip: Merge revisions 200:236 from trunk. 2006-02-23 19:25:29 +00:00
marko
6ce8427cb8 branches/zip: Merge revisions 79:200 from trunk. 2006-02-17 14:19:39 +00:00
osku
c307820962 Import 5.0 code. 2005-10-27 07:29:40 +00:00