Commit graph

123 commits

Author SHA1 Message Date
marko
aa845361c3 branches/zip: buf_pool_resize(): When shrinking the buffer pool, do not
assume that non-file pages are free.  After trying to free or flush
file pages, do not proceed to buf_chunk_free(), because the calls will
temporarily release buf_pool->mutex.  Do not flush if there are non-free
blocks, because it would not achieve anything.
2006-11-16 11:23:20 +00:00
marko
303d8f474f branches/zip: Introduce a new preprocessor symbol, UNIV_BUF_DEBUG, and
enclose buf_validate() and friends in it.
2006-11-16 09:00:30 +00:00
marko
21bf2206ec branches/zip: Add #define UNIV_DEBUG_FILE_ACCESSES to univ.i and
enclose all related debug code in #ifdef UNIV_DEBUG_FILE_ACCESSES.
This should have no effect on the behaviour, as the symbol is
not defined by default.  It only reduces the size of buf_block_t
and removes some assignments and debug functions.
2006-11-16 08:48:28 +00:00
marko
f339090d97 branches/zip: Improve the shrinking of the buffer pool.
buf_LRU_block_free_non_file_page(): Deallocate block->page_zip.data
to avoid ut_a(!block->page_zip.data) in buf_chunk_free().

buf_chunk_free(): Add the assertion ut_a(!block->in_LRU_list).

buf_pool_resize(): When shrinking the buffer pool and there are
non-free blocks in the candidate chunk, free the clean blocks
and move the dirty blocks to the end of the LRU list and request a flush.
Proceed if the chunk becomes free, and retry otherwise.
2006-11-15 21:49:14 +00:00
marko
c161077922 branches/zip: buf_LRU_free_block(): New function, split from
buf_LRU_search_and_free_block().
2006-11-15 21:41:27 +00:00
marko
879b1fd73d branches/zip: Implement resizeable buffer pool.
mysql.patch: Patch to change or add variables to MySQL

innodb.patch: Patch to make the master thread poll requests to resize
the buffer pool.

Replace srv_pool_size and innobase_buffer_pool_size
with srv_buf_pool_size, srv_buf_pool_old_size,
and srv_buf_pool_curr_size.

Add buf_chunk_t, a collection of buf_block_t.
2006-11-14 14:27:26 +00:00
marko
f03f262d38 branches/zip: buf_LRU_block_free_non_file_page(),
buf_LRU_block_remove_hashed_page(): Overwrite (space_id,page_no)
when freeing a buffer block.  This will help catching non-file
pages being passed to buf_block_align().
2006-11-13 09:55:31 +00:00
marko
4574bdda81 branches/zip: Merge revisions 968:1009 from trunk. 2006-11-10 11:15:59 +00:00
marko
c2fb9e5966 branches/zip: os_mem_alloc_large(), os_mem_free_large():
On POSIX, use mmap() and munmap().  On Windows, use VirtualAlloc()
and VirtualFree().  Only on Netware, use ut_malloc_low() and ut_free().
The lower-level functions on POSIX and Windows allow InnoDB to return
memory to the operating system when the buffer pool is shrunk.
2006-11-01 08:13:58 +00:00
marko
8c292a263a branches/zip: Enclose some more debug code in #ifdef UNIV_SYNC_DEBUG
to allow the code to be built with UNIV_DEBUG but without UNIV_SYNC_DEBUG.
2006-10-31 16:51:16 +00:00
marko
85e6beafeb branches/zip: Remove buf_pool->frame_zero and buf_pool->high_end. 2006-10-31 09:59:15 +00:00
marko
91b16c3c56 branches/zip: Prepare for buffer pool allocation in several chunks.
buf_pool_t: Remove n_frames, max_size, and blocks_of_frames.
The current buffer pool size is in curr_size.

buf_pool_init(): Remove parameter max_size.

buf_pool_get_max_size(), buf_pool_is_block(): Remove.

buf_block_align(): Do not assume that the buffer pool is allocated
in one chunk.  Replace dependency on buf_pool->blocks_of_frames
with a call to buf_page_hash_get().
2006-10-30 15:15:19 +00:00
marko
69fc883be2 branches/zip: Remove support for AWE (Intel PAE on Win32).
Removal of the variable innobase_buffer_pool_awe_mem_mb requires
coordination with MySQL AB.
2006-10-30 13:48:08 +00:00
marko
497a2ef429 branches/zip: Minor cleanup.
btr_pcur_restore_position(): Add const qualifiers.

buf_LRU_block_remove_hashed_page(): Reduce the amount
of buf_page_hash_get() calls and add a UNIV_UNLIKELY hint
to an assertion-like test.
2006-10-30 09:19:41 +00:00
marko
5d64128b54 branches/zip: Clean up things found in an inlined build.
buf0lru.c: Always #include "srv0srv.h"

buf_block_get_lock_mutex(), buf_frame_get_lock_mutex(): Remove.

mtr0log.ic: Remove #include "page0page.h" and replace the page_
functions with lower-level ones to break an #include cycle.

dict0dict.ic: Remove unnecessary #include "trx0undo.h" and "trx0sys.h"
that would create an #include cycle.
2006-10-25 11:03:27 +00:00
marko
7e8c129370 branches/zip: buf_page_get_release_on_io(), buf_page_get_gen():
Replace buf_frame_t* guess with buf_block_t* guess in order to avoid
a buf_block_align() call.

trx_undo_t: Replace page_t* guess_page with buf_block_t* guess_block.

btr_search_t: Replace page_t* root_guess with buf_block_t* root_guess.
2006-10-23 19:34:45 +00:00
marko
866f51741d branches/zip: Minor cleanup.
buf_calc_page_new_checksum(), buf_calc_page_old_checksum(),
buf_page_is_corrupted(), buf_page_print():
Add const qualifier to byte* parameter.

page_parse_create_zip(): Remove this declaration of non-existing function.

page_rec_check(), page_rec_print(), row_vers_impl_x_locked_off_kernel():
Add const qualifier to rec_t* parameter.

row_vers_old_has_index_entry(): Add const qualifier to dtuple_t* parameter.

row_sel_sec_rec_is_for_blob(): Add const qualifiers.

Replace some occurrences of mtr_memo_contains_page() with mtr_memo_contains().
2006-10-23 19:14:36 +00:00
marko
91f8a3dde5 branches/zip: Remove many fil_space_get_zip_size() calls.
ibuf_page(), ibuf_page_low(), ibuf_free_excess_pages(), ibuf_insert(),
buf_read_page(), buf_read_ahead_linear(), buf_read_recv_pages():
Add parameter zip_size.
2006-10-19 11:07:50 +00:00
marko
a3860f7126 branches/zip: Remove some more buf_block_align() calls.
btr_search_build_page_hash_index(), fsp_parse_init_file_page(),
ibuf_parse_bitmap_init(): Replace page_t* with buf_block_t*.

buf_read_ibuf_merge_pages(): Add const qualifiers.  Retrieve zip_size.

ibuf_merge_or_delete_for_page(): Replace page_t* with buf_block_t*.
Add parameter zip_size.
2006-10-19 08:27:34 +00:00
marko
f52a757e24 branches/zip: ha_create(): Remove parameter in_btr_search, which was
passed as TRUE.

Enclose hash_table_t::adaptive and buf_block_t::n_pointers in
#ifdef UNIV_DEBUG.

btr_search_drop_page_hash_index(): Enclose the corruption check
(which depends on buf_block_t::n_pointers) in #ifdef UNIV_DEBUG.
2006-10-18 18:52:04 +00:00
marko
e961e3c2cd branches/zip: Allow compilation with UNIV_DEBUG without defining
the symbol UNIV_DEBUG_PRINT, which was introduced in r729.

buf_LRU_print(), buf_print(): Replace #ifdef UNIV_DEBUG_PRINT
with #if defined UNIV_DEBUG || defined UNIV_DEBUG_PRINT.
2006-10-17 12:13:15 +00:00
marko
138ee61049 branches/zip: Remove more references to buf_block_align().
ibuf_reset_free_bits(): Remove, as there already is a similar function
ibuf_reset_free_bits_with_type().

ibuf_reset_free_bits_with_type(), ibuf_set_free_bits(),
ibuf_update_free_bits_if_full(), btr_leaf_page_release(),
buf_page_make_young(): Replace page_t with buf_block_t.

btr_compress(): Replace btr_page_get() with btr_block_get().
2006-10-13 07:45:52 +00:00
marko
c755e88727 branches/zip: Reduce the number of buf_block_align() calls.
btr_block_get(): New function to return buf_block_t.

btr_page_alloc(), buf_page_get_release_on_io(), buf_page_get_gen(),
buf_page_create(), fseg_create(), fseg_create_general(): Return buf_block_t.

buf_page_get_known_nowait(): Expect buf_block_t instead of buf_frame_t.

buf_frame_get_newest_modification(): Replace with
buf_block_get_newest_modification().

buf_page_dbg_add_level(): Replace with buf_block_dbg_add_level().

buf_block_get_zip_size(): New function.

buf_block_get_page_zip(): Reintroduce.

recv_recover_page(): Replace page, space, page_no with block.

ibuf_bitmap_page_init(): Replace page, zip_size with block.

ibuf_parse_bitmap_init(): Remove the parameter zip_size.

btr_search_drop_page_hash_index(): Replace page with block.
2006-10-12 11:05:22 +00:00
marko
d61d296ccd branches/zip: Replace buf_frame_get_page_no() and buf_frame_get_space_id()
with page_get_page_no() and page_get_space_id().  We want to avoid
buf_block_align() calls, and the page_no and space_id are now stamped
on the pages early on.
2006-10-12 07:02:36 +00:00
marko
606bd38428 branches/zip: Stamp (space_id, page_no) on buffer pool pages early on.
buf_flush_init_for_writing(): Remove parameters space, page_no.

fsp_init_file_page_low(): Wriet space_id and page_no to the page.

fil_create_new_single_table_tablespace(): Write space_id to the page.
2006-10-10 12:26:37 +00:00
marko
11cd834bb1 branches/zip: buf_block_get_page_zip(): Replace with buf_frame_get_page_zip(). 2006-10-09 15:37:19 +00:00
marko
152fbb78a2 branches/zip: Remove compilation errors with -DUNIV_HOTBACKUP.
dict_load_foreigns(): Enclose in #ifndef UNIV_HOTBACKUP.

fil_extend_tablespaces_to_stored_len(): Pass zip_size to fil_read().

buf_page_init_for_backup_restore(): Add parameter zip_size.
Enclose the declaration in buf0buf.h in #ifdef UNIV_HOTBACKUP.

recv_apply_log_recs_for_backup(): Replace the local variable "page"
with the local variable "block".  Add local variable zip_size.
2006-10-06 11:23:19 +00:00
marko
9d2146e431 branches/zip: Replace FIL_PAGE_ZBLOB_SPACE_ID and FIL_PAGE_ZBLOB_DATA
with FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID and FIL_PAGE_DATA.  The doublewrite
buffer needs to read the space_id in order to determine the type of the page.
Because FIL_PAGE_TYPE could contain garbage in MySQL/InnoDB 5.0 and earlier
versions, we cannot trust fil_page_get_type().  Instead, we have to always
store the space_id at the same location.  This modification wastes 12 bytes
per compressed BLOB page (1.2% on 1-kilobyte pages).
2006-09-27 10:51:05 +00:00
marko
c12e237a09 branches/zip: Instrument the buffer pool allocator and deallocator
for more accurate Valgrind debugging.

univ.i: Introduce UNIV_DEBUG_VALGRIND, UNIV_MEM_VALID, and UNIV_MEM_INVALID.

buf_LRU_block_free_non_file_page(): Invalidate the buffer frame
with UNIV_MEM_INVALID().

buf_LRU_get_free_block(): Declare the buffer frame valid
with UNIV_MEM_VALID().

Other memory is allocated and deallocated via malloc() and free(),
which are already overridden by Valgrind.  Without the added
instrumentation, accesses to free pages in the buffer pool cannot
be caught.

The diagnostics could probably be improved further by declaring all
non-latched buffer frames invalid.
2006-09-20 12:42:08 +00:00
marko
9a16aac783 branches/zip: Merge revisions 804:838 from trunk. 2006-09-19 10:14:07 +00:00
marko
64f3cee67f branches/zip: Merge revisions 720:765 from trunk and reindent the code
as per revisions r763:765.
2006-08-29 09:30:31 +00:00
marko
ea4572cecf branches/zip: btr_cur_pessimistic_insert(): Prior to splitting, ensure that
the record will fit on an empty compressed page.

buf_block_alloc(), buf_block_free(), buf_frame_alloc(), buf_frame_free():
Make inlined.
2006-08-17 11:57:51 +00:00
marko
6942bfe031 branches/zip: buf_LRU_print(): Terminate every displayed entry with a newline. 2006-08-15 14:27:19 +00:00
marko
37f03d23d4 branches/zip: Introduce the symbol UNIV_DEBUG_PRINT for enabling debug
print functions.

buf_LRU_print(), buf_print(): Replace #ifdef UNIV_DEBUG
with #ifdef UNIV_DEBUG_PRINT.
2006-08-15 11:54:20 +00:00
marko
36ed778816 branches/zip: Merge revisions 634:713 from trunk. 2006-08-11 11:49:38 +00:00
marko
be44b82273 branches/zip: Improve diagnostics.
buf_LRU_get_free_block(): When zip_size changes, initialize all fields
of page_zip.  This avoids an assertion failure in page_create_zip() when
a block with an originally larger zip_size is reallocated.

fsp_get_space_header(): Assert that the stored space id matches.

xdes_get_state(): Assert that the state is valid.
2006-08-09 08:55:00 +00:00
marko
441c29e94f branches/zip: Fix some crash recovery bugs.
dict_load_table(): Initialize table->flags with zip_size.

mlog_parse_nbytes(), mlog_parse_string(): Add parameter page_zip and
write the changes also to the compressed page if one is specified.
Assert that these functions are not called on FIL_PAGE_INDEX pages.

buf_page_io_complete(): Replace block->frame with frame where appropriate.

recv_parse_or_apply_log_rec_body(): Add ut_a(!page_zip) where appropriate.

page_parse_delete_rec_list(): Add parameter page_zip.
2006-08-02 05:49:15 +00:00
marko
e4077d5651 branches/zip: buf_flush_init_for_writing(): Invoke page_zip_calc_checksum()
on page_zip->data, so that the freshly written fields will be included.

page_cur_delete_rec(): Add a page_zip_validate() assertion to the end.
2006-08-01 12:05:04 +00:00
marko
8f7bbe15ca branches/zip: Bug fixes.
buf_page_io_complete(): On FIL_PAGE_TYPE_ZBLOB (compressed BLOB pages),
read the space_id from a different location.

page_zip_compress(), page_zip_write_rec(), page_zip_write_blob_ptr():
Replace page_simple_validate_new() with page_validate().

page_zip_clear_rec(): When running out of log space, do not attempt to
recompress the page, because the directory slots might be unbalanced and
the page_validate() assertion in page_zip_compress() would fail.
Instead, clear the BLOB pointers of the deleted record on the
uncompressed page, so that page_zip_validate() will succeed.

page_zip_validate(): Remove the comment about page_zip_clear_rec().
A mismatch always indicates a serious inconsistency.
2006-08-01 07:58:38 +00:00
marko
8c55373635 branches/zip: Stamp some header fields to compressed BLOB pages.
buf_flush_init_for_writing(): On FIL_PAGE_TYPE_ZBLOB, write to
page_zip->data instead of page.

page_zip_write_rec(), page_zip_write_blob_ptr(), page_zip_write_node_ptr():
Add ut_ad(page_simple_validate_new()).
2006-07-31 18:27:17 +00:00
marko
05c16ac6d2 branches/zip: buf_page_io_complete(): Copy also uninitialized pages to
uncompressed storage, to avoid bogus warnings about page number mismatch.
2006-06-21 12:38:42 +00:00
marko
be7add3b56 branches/zip: Add parameter zip_size to fil_io().
fil_read(), fil_write(): Make these inlined functions in fil0fil.c.

fil_write_lsn_and_arch_no_to_file(): Remove the parameter space_id and
note that this function is to be called on the system tablespace, which
is uncompressed.
2006-06-15 07:27:13 +00:00
marko
fa63e363c2 branches/zip: buf_flush_init_for_writing(): Determine zip_size from
page_zip instead of calling fil_space_get_zip_size().  In
fil_create_new_single_table_tablespace(), the table space has not yet
been created.  Handle also FIL_PAGE_TYPE_ALLOCATED.
2006-06-14 07:47:32 +00:00
marko
fe5474851a branches/zip: When identifying insert buffer bitmap pages, note that the
pages occur every zip_size pages in compressed tablespaces, instead of
UNIV_PAGESIZE (or XDES_DESCRIBED_PER_PAGE).

ibuf_bitmap_page(): Add parameter zip_size.

ibuf_fixed_addr_page(): Add parameter space.
2006-06-05 09:04:19 +00:00
marko
831916878d branches/zip: Write compressed pages to disk.
os_aio_simulated_handle(): Temporarily disable os_file_check_page_trailers(),
which cannot be invoked on compressed pages.

dict_table_add_system_columns(): New function, split from
dict_table_add_to_cache().

mlog_parse_index(): Add system columns to the dummy table and identify
DB_TRX_ID and DB_ROLL_PTR in the dummy index.

buf_LRU_get_free_block(): Note that page_zip->data should be allocated from
an aligned memory pool.

buf_flush_buffered_writes(): Write compressed pages to disk.

buf_flush_post_to_doublewrite_buf(): Copy compressed pages to the
doublewrite buffer.  Zero fill any excess space.

buf_flush_init_for_writing(): Treat all compressed pages the same.

buf_read_page_low(): Read compressed pages from disk.

buf_page_io_complete(): Process compressed pages.

trx_sys_doublewrite_init_or_restore_page(): Process compressed pages.

mlog_write_initial_log_record_fast(): Enable a debug printout
#ifdef UNIV_LOG_DEBUG.

fsp_header_init(), fsp_fill_free_list(): Pass the compressed page size
to buf_page_create().

page_zip_compress_write_log(): Flatten the if-else if-else logic.

page_zip_parse_write_blob_ptr(): Do not test page_zip if page==NULL.

page_zip_parse_write_node_ptr(): Do not test page_zip if page==NULL.
Invoke mlog_close() correctly.

row_sel_store_row_id_to_prebuilt(): Add UNIV_UNLIKELY hint to an
assertion-like test.
2006-05-30 09:04:57 +00:00
marko
b560b0ea20 branches/zip: buf_flush_buffered_writes(): Disable the checks on compressed
pages.

buf_flush_post_to_doublewrite_buf(): Add a TODO comment about compressed pages.
2006-05-24 14:35:22 +00:00
marko
3979344d88 branches/zip: buf_flush_buffered_writes(): Replace four occurrences of
mach_read_from_4(...) != mach_read_from_4(...) with memcmp(..., ..., 4).
2006-05-22 09:23:30 +00:00
marko
93ea672ae3 branches/zip: Write the compressed page size to SYS_TABLES.TYPE and adjust
a few places accordingly.

os_aio_simulated_handle(): Add TODO comments about skipping the write
checks for compressed pages.

dict_create_sys_tables_tuple(): Write the compressed page size to
the TYPE column.

open_or_create_data_files(): Simplify the fil_node_create() call.

fil_node_create(): Do not touch space->zip_size.  It was already initialized
by fil_space_create().

fil_reset_too_high_lsns(), buf_flush_buffered_writes(): Add TODO comment
about compressed pages.

buf_flush_init_for_writing(): Handle pages of type FIL_PAGE_INODE,
FIL_PAGE_IBUF_BITMAP, and FIL_PAGE_TYPE_FSP_HDR as uncompressed ones.
2006-05-17 12:03:32 +00:00
marko
6748a82b84 branches/zip: Remove GCC 4.0.4 warnings from
make AM_CFLAGS='-std=c89 -Wall -W -pedantic'
2006-05-10 13:58:35 +00:00
marko
8f167973c2 branches/zip: Fix bugs.
btr_root_raise_and_insert(): When copying root to new_page byte for byte,
restore the page number of new_page afterwards.

buf_flush_init_for_writing(): For FIL_PAGE_INDEX, write the page number
and space id also to the uncompressed page.
2006-05-10 11:28:14 +00:00