Commit graph

53 commits

Author SHA1 Message Date
marko
2696aa6d60 branches/zip: Remove buf_block_t:free_list. Rename buf_page_t:flush_list
to free_or_flush_list.  The list node pointers can be shared, because a
block can never be belong to both lists at the same time.
2006-12-04 12:44:06 +00:00
marko
6c0a4186af branches/zip: Allow some functions to work on compressed-only pages
(state == BUF_BLOCK_ZIP_PAGE).  Make use of buf_page_in_file()
and buf_page_get_mutex().

buf_block_get_newest_modification(): Rename to
buf_page_get_newest_modification().
2006-11-30 15:09:33 +00:00
marko
aedce8cde3 branches/zip: Move the fields related to the least-recently-used algorithm
of the buffer pool from buf_block_t to buf_page_t.  Replace some buf_block_t*
parameters with buf_page_t*.  Add accessor functions.
2006-11-30 12:27:49 +00:00
marko
329a29c2fb branches/zip: Replace ib_ulonglong with ib_uint64_t, since we want exactly
64 bits.  "long long" might be longer than 64 bits on some systems.
2006-11-29 14:52:16 +00:00
marko
519ba6f47c branches/zip: Make buf_pool->page_hash point to buf_page_t instead of
buf_block_t.  Move the fields "hash" and "file_page_was_freed" from
buf_block_t to buf_page_t.

buf_page_in_file(): New function, for checking block state in assertions.
2006-11-29 13:23:28 +00:00
marko
17842af300 branches/zip: Allow the code to compile with debugging enabled. This
was broken in r1066.
2006-11-28 11:08:33 +00:00
marko
b03460d90d branches/zip: Move the page flushing fields from buf_block_t to
buf_page_t and page_zip_des_t.
2006-11-27 14:12:09 +00:00
marko
0d7111c9cd branches/zip: page_zip_des_t: Rename "size" to "ssize" and reduce the
storage size from 16 to 3 bits.

page_zip_get_size(), page_zip_set_size(): New functions.

Replace direct references to page_zip_des_t:size with calls to
buf_block_get_zip_size(), page_zip_get_size(), and page_zip_set_size().
2006-11-27 13:44:32 +00:00
marko
7593a465db branches/zip: Introduce the unsigned 64-bit data type ib_ulonglong
for log sequence numbers, to replace dulint.
2006-11-24 13:05:01 +00:00
marko
53853ee573 branches/zip: Replace all references to block->space and block->offset,
which should have been replaced in r1057.
2006-11-24 12:49:59 +00:00
marko
8332c45ce4 branches/zip: Introduce buf_page_t, a common structure for compressed-only
and uncompressed buffer pool pages.

buf_block_t: Replace page_zip, space, and offset with buf_page_t page.
Replace some integers with bit-fields.

enum buf_block_state: Rename to buf_page_state.  Add BUF_BLOCK_ZIP_PAGE.

page_zip_des_t: Add the field "state".  Make the integer fields bit-fields.

page_zip_copy(): Document which fields are copied.
2006-11-24 08:32:18 +00:00
marko
6c8dfab1dc branches/zip: Replace most references to block->state, block->offset,
and block->space with buf_block_get_state(block), buf_block_get_page_no(block),
and buf_block_get_space(block).

enum buf_block_state: Replaces the #define'd buf_block_t.state values.

buf_block_get_state(): New function.

buf_block_get_frame(): Add __attribute__((const)).
2006-11-23 13:26:01 +00:00
marko
03bdaaf509 branches/zip: Introduce a new preprocessor symbol, UNIV_BUF_DEBUG, and
enclose buf_validate() and friends in it.
2006-11-16 09:00:30 +00:00
marko
274f2bc6a1 branches/zip: buf_LRU_free_block(): New function, split from
buf_LRU_search_and_free_block().
2006-11-15 21:41:27 +00:00
marko
19420d4621 branches/zip: Merge revisions 968:1009 from trunk. 2006-11-10 11:15:59 +00:00
marko
839ef125e1 branches/zip: Enclose some more debug code in #ifdef UNIV_SYNC_DEBUG
to allow the code to be built with UNIV_DEBUG but without UNIV_SYNC_DEBUG.
2006-10-31 16:51:16 +00:00
marko
89bacf3402 branches/zip: Remove support for AWE (Intel PAE on Win32).
Removal of the variable innobase_buffer_pool_awe_mem_mb requires
coordination with MySQL AB.
2006-10-30 13:48:08 +00:00
marko
5669dc3d2a branches/zip: Reduce the number of buf_block_align() calls.
btr_block_get(): New function to return buf_block_t.

btr_page_alloc(), buf_page_get_release_on_io(), buf_page_get_gen(),
buf_page_create(), fseg_create(), fseg_create_general(): Return buf_block_t.

buf_page_get_known_nowait(): Expect buf_block_t instead of buf_frame_t.

buf_frame_get_newest_modification(): Replace with
buf_block_get_newest_modification().

buf_page_dbg_add_level(): Replace with buf_block_dbg_add_level().

buf_block_get_zip_size(): New function.

buf_block_get_page_zip(): Reintroduce.

recv_recover_page(): Replace page, space, page_no with block.

ibuf_bitmap_page_init(): Replace page, zip_size with block.

ibuf_parse_bitmap_init(): Remove the parameter zip_size.

btr_search_drop_page_hash_index(): Replace page with block.
2006-10-12 11:05:22 +00:00
marko
6241b552aa branches/zip: Stamp (space_id, page_no) on buffer pool pages early on.
buf_flush_init_for_writing(): Remove parameters space, page_no.

fsp_init_file_page_low(): Wriet space_id and page_no to the page.

fil_create_new_single_table_tablespace(): Write space_id to the page.
2006-10-10 12:26:37 +00:00
marko
d374183221 branches/zip: buf_block_get_page_zip(): Replace with buf_frame_get_page_zip(). 2006-10-09 15:37:19 +00:00
marko
71e8ea627c branches/zip: Replace FIL_PAGE_ZBLOB_SPACE_ID and FIL_PAGE_ZBLOB_DATA
with FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID and FIL_PAGE_DATA.  The doublewrite
buffer needs to read the space_id in order to determine the type of the page.
Because FIL_PAGE_TYPE could contain garbage in MySQL/InnoDB 5.0 and earlier
versions, we cannot trust fil_page_get_type().  Instead, we have to always
store the space_id at the same location.  This modification wastes 12 bytes
per compressed BLOB page (1.2% on 1-kilobyte pages).
2006-09-27 10:51:05 +00:00
marko
68a65c142c branches/zip: Merge revisions 804:838 from trunk. 2006-09-19 10:14:07 +00:00
marko
38e09b2fea branches/zip: Merge revisions 720:765 from trunk and reindent the code
as per revisions r763:765.
2006-08-29 09:30:31 +00:00
marko
fbce3ef82e branches/zip: buf_flush_init_for_writing(): Invoke page_zip_calc_checksum()
on page_zip->data, so that the freshly written fields will be included.

page_cur_delete_rec(): Add a page_zip_validate() assertion to the end.
2006-08-01 12:05:04 +00:00
marko
6e2fa8bc85 branches/zip: Stamp some header fields to compressed BLOB pages.
buf_flush_init_for_writing(): On FIL_PAGE_TYPE_ZBLOB, write to
page_zip->data instead of page.

page_zip_write_rec(), page_zip_write_blob_ptr(), page_zip_write_node_ptr():
Add ut_ad(page_simple_validate_new()).
2006-07-31 18:27:17 +00:00
marko
97c781308a branches/zip: Add parameter zip_size to fil_io().
fil_read(), fil_write(): Make these inlined functions in fil0fil.c.

fil_write_lsn_and_arch_no_to_file(): Remove the parameter space_id and
note that this function is to be called on the system tablespace, which
is uncompressed.
2006-06-15 07:27:13 +00:00
marko
1f2483ee1d branches/zip: buf_flush_init_for_writing(): Determine zip_size from
page_zip instead of calling fil_space_get_zip_size().  In
fil_create_new_single_table_tablespace(), the table space has not yet
been created.  Handle also FIL_PAGE_TYPE_ALLOCATED.
2006-06-14 07:47:32 +00:00
marko
1dd4d56863 branches/zip: Write compressed pages to disk.
os_aio_simulated_handle(): Temporarily disable os_file_check_page_trailers(),
which cannot be invoked on compressed pages.

dict_table_add_system_columns(): New function, split from
dict_table_add_to_cache().

mlog_parse_index(): Add system columns to the dummy table and identify
DB_TRX_ID and DB_ROLL_PTR in the dummy index.

buf_LRU_get_free_block(): Note that page_zip->data should be allocated from
an aligned memory pool.

buf_flush_buffered_writes(): Write compressed pages to disk.

buf_flush_post_to_doublewrite_buf(): Copy compressed pages to the
doublewrite buffer.  Zero fill any excess space.

buf_flush_init_for_writing(): Treat all compressed pages the same.

buf_read_page_low(): Read compressed pages from disk.

buf_page_io_complete(): Process compressed pages.

trx_sys_doublewrite_init_or_restore_page(): Process compressed pages.

mlog_write_initial_log_record_fast(): Enable a debug printout
#ifdef UNIV_LOG_DEBUG.

fsp_header_init(), fsp_fill_free_list(): Pass the compressed page size
to buf_page_create().

page_zip_compress_write_log(): Flatten the if-else if-else logic.

page_zip_parse_write_blob_ptr(): Do not test page_zip if page==NULL.

page_zip_parse_write_node_ptr(): Do not test page_zip if page==NULL.
Invoke mlog_close() correctly.

row_sel_store_row_id_to_prebuilt(): Add UNIV_UNLIKELY hint to an
assertion-like test.
2006-05-30 09:04:57 +00:00
marko
662318ba2c branches/zip: buf_flush_buffered_writes(): Disable the checks on compressed
pages.

buf_flush_post_to_doublewrite_buf(): Add a TODO comment about compressed pages.
2006-05-24 14:35:22 +00:00
marko
927ed2fd56 branches/zip: buf_flush_buffered_writes(): Replace four occurrences of
mach_read_from_4(...) != mach_read_from_4(...) with memcmp(..., ..., 4).
2006-05-22 09:23:30 +00:00
marko
b15d26d6e2 branches/zip: Write the compressed page size to SYS_TABLES.TYPE and adjust
a few places accordingly.

os_aio_simulated_handle(): Add TODO comments about skipping the write
checks for compressed pages.

dict_create_sys_tables_tuple(): Write the compressed page size to
the TYPE column.

open_or_create_data_files(): Simplify the fil_node_create() call.

fil_node_create(): Do not touch space->zip_size.  It was already initialized
by fil_space_create().

fil_reset_too_high_lsns(), buf_flush_buffered_writes(): Add TODO comment
about compressed pages.

buf_flush_init_for_writing(): Handle pages of type FIL_PAGE_INODE,
FIL_PAGE_IBUF_BITMAP, and FIL_PAGE_TYPE_FSP_HDR as uncompressed ones.
2006-05-17 12:03:32 +00:00
marko
3547f43d22 branches/zip: Fix bugs.
btr_root_raise_and_insert(): When copying root to new_page byte for byte,
restore the page number of new_page afterwards.

buf_flush_init_for_writing(): For FIL_PAGE_INDEX, write the page number
and space id also to the uncompressed page.
2006-05-10 11:28:14 +00:00
marko
d30e44bc32 branches/zip: Merge revisions 465:532 from trunk. 2006-05-08 06:18:59 +00:00
marko
a9ee619a9b branches/zip: Minor improvements.
Introduce FIL_PAGE_ZBLOB_DATA as a synonym for FIL_PAGE_FILE_FLUSH_LSN.

btr_store_big_rec_extern_fields(): Make the assertion about
dict_table_zip_size() more accurate.

buf_LRU_get_free_block(), buf_block_alloc(): Add parameter zip_size.

buf_calc_zblob_page_checksum(): Remove. Replace with page_zip_calc_checksum().

buf_page_init(): Remove parameter zip_size.

buf_page_io_complete(): Add a placeholder for handling compressed pages.

trx_doublewrite_page_inside(): Remove redundant function.

page_zip_write_rec(): Relax an overly tight assertion about blob_no.
2006-05-04 11:44:49 +00:00
marko
f118a9c19f branches/zip: Prepare for writing compressed and reading pages.
buf_page_print(): Print also compressed pages.  Add parameter zip_size.

buf_flush_init_for_writing(): Stamp the fields on a compressed B-tree index
page.

Add the header field FIL_PAGE_ZBLOB_SPACE_ID as an alias of FIL_PAGE_PREV.

page_zip_calc_checksum(): New function.

page_zip_compress(): Avoid copying the fields that are written in
buf_flush_init_for_writing().

page_zip_header_cmp(): New function for comparing those fields of the
page header that will not be written in buf_flush_init_for_writing().
2006-05-02 11:44:39 +00:00
marko
bb6387683a branches/zip: Minor improvements.
buf_flush_init_for_writing(): Calculate the checksum with the actual zip_size.

buf_calc_zblob_page_checksum(): Skip the field FIL_PAGE_SPACE_OR_CHKSUM.

trx_sys_doublewrite_init_or_restore_page(): Use the actual zip_size.

page_cur_insert_rec_low(): If page_zip_alloc() fails, try compressing the
whole page afterwards.
2006-04-28 09:30:40 +00:00
marko
6dd1d8c4fe branches/zip: Write the compressed page size to SYS_TABLES.TYPE
and to the file space header (FSP_PAGE_ZIP_SIZE, renamed from
FSP_LOWEST_NO_WRITE).

fil_space_struct: Add zip_size.

dict_table_struct: Embed zip_size in flags.

dict_table_zip_size(): Infer zip_size from table->flags.

dict_sys_tables_get_zip_size(): Read zip_size from SYS_TABLES.TYPE.

fil_space_get_zip_size(): Read zip_size from the file space header.

Add the redo log entry type MLOG_ZIP_FILE_CREATE.
2006-04-25 07:12:32 +00:00
marko
0c4dfda3be branches/zip: Merge revisions 265:459 from trunk. 2006-04-12 09:32:17 +00:00
marko
e20dfbc0ce branches/zip: Disable accidentally committed debug code that would break
crash recovery of uncompressed tables.
2006-04-11 08:12:39 +00:00
marko
a1661efb8d branches/zip: Initial steps towards disk-based storage of compressed pages.
dict_mem_table_create(): Account for DICT_TF_COMPRESSED in a debug assertion.

btr_store_big_rec_extern_fields(), btr_free_externally_stored_field(),
btr_copy_externally_stored_field(): Implement the disk format for
compressed BLOB pages.

btr_copy_externally_stored_field(): Improve error reporting and handling
when decompressing BLOB pages.

buf_flush_init_for_writing(), buf_page_is_corrupted(), buf_page_print():
Account for compressed BLOB pages (FIL_PAGE_TYPE_ZBLOB).

buf_calc_zblob_page_checksum(): New function.
2006-04-05 13:41:12 +00:00
marko
4a09767df0 branches/zip: Minor bug fix
buf_flush_init_for_writing(): Copy FIL_PAGE_LSN to the compressed page.

page_cur_insert_rec_low(): Add debug assertion.
2006-03-07 13:47:33 +00:00
marko
7f2d91cba9 branches/zip: Fix some bugs in incremental compression.
btr_create(): page_zip_compress() returns FALSE on failure.

page_zip_write_header(): Write to page_zip->data[] instead of page_zip[].

buf_flush_init_for_writing(): Add parameter page_zip and set the fields
also in the header of the compressed page.

btr_cur_search_to_nth_level(): Add ut_ad() on page_zip_validate().
2006-03-02 14:05:32 +00:00
marko
747c8fad44 branches/zip: Merge revisions 200:236 from trunk. 2006-02-23 19:25:29 +00:00
marko
d5f33dafd8 branches/zip: Prepare for upcoming implementation of page_zip_compress()
that will require complete index information.

dict_create_index_step(): invoke dict_index_add_to_cache() before btr_create()

dict_index_remove_from_cache(): make public

dict_index_get_if_in_cache_low(): new function, for holding dict_sys->mutex

buf_flush_init_for_writing(): remove the temporary hook to page_zip_compress()

page_create(): add temporary hook to page_zip_compress()
2006-02-03 14:44:29 +00:00
marko
3a55efc44c branches/zip:
buf_flush_init_for_writing(): The reported dense page directory size was
4 bytes too much.  Subtract 2 (infimum and supremum) from n_heap.

page_zip_decompress(): When decompressing the last user record, only set
heap_no and the status bits if there is data to decode, i.e., there
are user records on the page.
2005-11-30 23:09:36 +00:00
marko
82c74a7535 branches/zip: Minor improvements for debugging and testing purposes.
buf_flush_init_for_writing(): Improve the diagnostics and make the
condition for skipping pages accurate.

univ.i: Introduce UNIV_ZIP_DEBUG for enabling some page_zip_validate() tests.

page0zip.h, page0zip.c: Define and use page_zip_validate() in
page_zip_compress() and page_zip_write() if UNIV_ZIP_DEBUG or UNIV_DEBUG
is defined.
2005-11-30 13:57:54 +00:00
marko
541fcfcd45 branches/zip: More fixes to problems found in speedc test runs.
Before the speedc test was interrupted, 121,765 B-tree pages were written.

buf_flush_init_for_writing(): Do not compress other than B-tree pages
outside the system tablespace.  Report non-B-tree pages.

page_zip_decompress(): Clear the unused heap space on the uncompressed page,
so that the whole buffer for the uncompressed page will be initialized and
page_zip_validate() will always succeed.
2005-11-29 10:24:45 +00:00
marko
d1f9c8ecea branches/zip: Minor fixes to get a page with one record compress/decompress.
A page with multiple records or deleted records still does not compress
or decompress properly.

buf_flush_init_for_writing(): Initialize block->page_zip properly so that all
assertions in page0zip can be enabled.

page_zip_decompress(): Note that corrupt data should not lead to assertions.

page_zip_dir_set(): Correct the interface.  Fix off-by-one error.
page_zip_dir_get(): Fix off-by-one error.

page0zip.c: Replace n_heap with n_dense and add comments about
the infimum and supremum records whenever we subtract 2 from heap_no.
Fix some programming errors.
2005-11-25 12:34:38 +00:00
marko
13002bdffc branches/zip: Minor fixes. Compression/decompression still fails.
buf0flu.c: Allocate the temporary buffer from buf_frame_alloc().

page_zip_simple_validate(): Do not assert page_zip->m_start >= PAGE_DATA.

page_zip_compress(): Replace some assertions with page_zip_simple_validate(),
and do not assert anything about page_zip->data contents.

page_zip_validate(): Do not compare the page trailer bytes.

page_zip_write(): Assert that the entire page headers match and
that page_zip->m_start >= PAGE_DATA.
2005-11-25 07:51:28 +00:00
marko
48a1ffa5f4 branches/zip: Implement a more compact page format.
Add a hook to buf0flu.c for testing compression and decompression.
TODO: adapt page_zip_write() calls.
2005-11-24 14:13:10 +00:00