Commit graph

77 commits

Author SHA1 Message Date
marko
eec9380475 branches/zip: When reading pages for merging buffered inserts, request that
the pages be decompressed.

buf_read_page_low(), buf_page_init_for_read(): Add parameter "unzip".
2007-05-06 13:21:59 +00:00
marko
76fa1cfbdd branches/zip: buf_page_init_for_read(): Do not allocate an uncompressed page
frame when reading compressed pages, unless crash recovery is in progress.

buf_page_read_low(): Adapt for buf_page_init_for_read() returning
buf_page_t* instead of buf_block_t*.
2007-02-08 12:03:03 +00:00
marko
bedc6a976b branches/zip: buf_page_init_for_read(), buf_page_create(): Note in the
function comments that buf_page_init_for_read() does not perform the
state transition to FILE_PAGE.  This should have been done in r1286.
2007-02-08 09:08:11 +00:00
marko
1f0b739f4f branches/zip: Do not decompress blocks in buf_page_init_for_read(),
but in buf_page_get_gen().  Adjust comments.  This prevents
unnecessary decompression in read-ahead when the compressed block
exists in the buffer pool.

If the block does not exist in the buffer pool, read-ahead will still
allocate an uncompressed page and decompress the block.

Move buf_block_init_low() and buf_zip_decompress() earlier in the file,
because some compilers are unable to inline functions that are defined
after the invocation.

buf_page_get_gen(): Decompress the block if needed.

buf_page_init_for_read(): Do not decompress.
2007-02-07 13:59:43 +00:00
marko
ae0ecb11aa branches/zip: buf_page_t: Add the in_page_hash and in_zip_hash
for extra validation in UNIV_DEBUG builds.
2007-01-24 10:36:05 +00:00
marko
adbd5c3a85 branches/zip: When retrieving compressed BLOBs, avoid allocating
uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().
2007-01-18 23:10:49 +00:00
marko
f116afa724 branches/zip: Remove the fil_space_get_zip_size() call from
buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.
2007-01-18 09:59:00 +00:00
marko
225e82c112 branches/zip: buf_page_io_complete(): Change the parameter type from
buf_block_t* to buf_page_t*.  Handle compressed-only blocks.
2007-01-10 14:36:39 +00:00
marko
d3917bb5c3 branches/zip: buf_page_t: Introduce the field in_flush_list #ifdef UNIV_DEBUG.
buf_flush_batch(): Remove the local variable "found".
2007-01-09 12:35:42 +00:00
marko
51d0b62947 branches/zip: buf_page_peek(): Return TRUE also if the page exists in the
buffer pool in compressed form only.

buf_page_peek_block(): Remove.

mutex_enter_fast_func: Remove.
2007-01-05 11:19:56 +00:00
marko
e2db2a873f branches/zip: buf_pool_page_hash_rebuild(): Rebuild also buf_pool->zip_hash
and add compressed-only pages to buf_pool->page_hash.

BUF_POOL_ZIP_FOLD(), BUF_POOL_ZIP_FOLD_PTR(), BUF_POOL_ZIP_FOLD_BPAGE():
Macros for computing buf_pool->zip_hash fold values.  Make use of these
in buf0buddy.c.

buf0buddy.c: Replace some ptr == ut_align_down(ptr, ...) in assertions with
!ut_align_offset(ptr, ...).
2007-01-05 10:49:56 +00:00
marko
91c87cd343 branches/zip: Allow dirty blocks to be relocated. Allow a transition
to BUF_BLOCK_ZIP_DIRTY (discarding the uncompressed page corresponding
to a modified compressed page that has not been flushed to disk).

buf_page_can_relocate(): New function, modelled after
buf_flush_ready_for_replace().

buf_LRU_free_block(): Allow the transition to BUF_BLOCK_ZIP_DIRTY.

buf_flush_insert_into_flush_list(): Make the prototype public.

buf_buddy_relocate(): Remove an unnecessary switch statement.
Use buf_page_can_relocate() instead of buf_flush_ready_for_replace().

buf_page_peek(): Made UNIV_INLINE.

Document UNIV_ZIP_DEBUG.
2007-01-04 21:40:10 +00:00
marko
0b89bb7905 branches/zip: Add __attribute__((const)) or __attribute__((pure))
to all fold functions.
2007-01-03 12:34:54 +00:00
marko
9244a3a503 branches/zip: Allow the buddy allocator to work for 16-kilobyte pages,
twice the maximum block size of the buddy system.

buf_page_t: Note that state may change from BUF_BLOCK_READY_FOR_USE
to BUF_BLOCK_MEMORY without buf_page_get_mutex() protection
[only buf_pool->mutex].

buf_buddy_get_slot(): Extend the output to BUF_BUDDY_SIZES.

buf_buddy_alloc(), buf_buddy_free(): Allow sizes up to UNIV_PAGE_SIZE.

buf_buddy_alloc_low(), buf_buddy_free_low(), buf_buddy_alloc_clean():
Allow i==BUF_BUDDY_SIZES.
2006-12-20 15:56:58 +00:00
marko
3b2806f48f branches/zip: Add the debug function buf_pool_contains_zip() #ifdef UNIV_DEBUG. 2006-12-19 22:12:44 +00:00
marko
483cf9bee9 branches/zip: Correct some bugs in the buddy allocator and add debug code.
buf_buddy_alloc_zip() [UNIV_DEBUG]: Byte-fill the allocated blocks with ~i.

buf_buddy_block_free() [UNIV_DEBUG]: Clear the frame before releasing it
to the buffer pool.

buf_buddy_alloc_from() [UNIV_DEBUG]: Byte-fill the free blocks with j.

buf_buddy_relocate(): Allow the buf_page_hash_get() lookup to fail,
and note the reason.

buf_buddy_free_low(): Correctly compute the address of the combined free block.

buf_page_struct: Clarify the mutex protection rules.  Note that
the pointer zip.data is also protected by buf_pool->mutex.
2006-12-19 12:38:18 +00:00
marko
797822636d branches/zip: Refactor some code for relocating buffer control blocks.
buf_relocate(): New function, split from buf_buddy_relocate().
2006-12-14 15:15:51 +00:00
marko
cb530ba3e7 branches/zip: Make use of the previously unused state BUF_BLOCK_MEMORY.
buf_LRU_get_free_block(): Document that the block is in the state
BUF_BLOCK_READY_FOR_USE.

buf_block_alloc(): Change the state of the block to BUF_BLOCK_MEMORY
and document it.

Replace those invocations of buf_block_alloc() with
buf_LRU_get_free_block() where the allocated block is used for buffer
pool input and output.  However, temporary copies of B-tree pages
during reorganization are not used for file I/O, and such blocks are
still allocated with buf_block_alloc().
2006-12-13 14:58:54 +00:00
marko
d387f148b4 branches/zip: Add buf_pool->zip_hash for keeping track on pages allocated
to the buddy system for allocating compressed pages and their descriptors.

buf_buddy_free_block(): New function: Deallocate the buffer frame.

buf_buddy_free(), buf_buddy_free_low(): Return void instead of a pointer
to a freed buffer frame.
2006-12-11 09:54:13 +00:00
marko
ba1fb95598 branches/zip: Try to adhere to ISO 9899:1989 where possible. (The recently
introduced ib_longlong and %ll length specifier are not C89.)

row_ext_t: Change char* buf to byte* buf to avoid signedness warnings.

Change the type of all bit fields to unsigned.

rec_get_next_ptr(): Add (byte*) casts to avoid arithmetics on void*.

page_zip_decompress_clust(): Do not mix declarations and code.
2006-12-08 15:55:59 +00:00
marko
12682d42ff branches/zip: Introduce the constant BUF_BUDDY_HIGH. 2006-12-07 15:20:53 +00:00
marko
2a25f556e7 branches/zip: Add the first bits of the binary buddy system for allocating
compressed pages from the buffer pool.

Makefile.am: Add buf0buddy.h, buf0buddy.ic.

buf/Makefile.am: Add buf0buddy.c.

Introduce the constants BUF_BUDDY_LOW and BUF_BUDDY_SIZES.

buf_pool_t: Add zip_mutex and the lists zip_clean and zip_free[].

buf_page_get_mutex(): Return &buf_pool->zip_mutex instead of NULL.

buf_buddy_get_offset(), buf_buddy_get(), buf_buddy_get_slot(),
buf_buddy_alloc_free(), buf_buddy_alloc_free_low(): New functions.
2006-12-07 13:29:04 +00:00
marko
ca02e14530 branches/zip: Fix some comments in buf0buf.h. 2006-12-07 10:02:23 +00:00
marko
a9e16776e3 branches/zip: buf_page_t: Rename free_or_flush_list to list
and document the possible list memberships.
2006-12-05 12:31:38 +00:00
marko
f81ecc5ad2 branches/zip: enum buf_page_state: Add BUF_BLOCK_ZIP_FREE, BUF_BLOCK_ZIP_DIRTY. 2006-12-05 12:10:30 +00:00
marko
2696aa6d60 branches/zip: Remove buf_block_t:free_list. Rename buf_page_t:flush_list
to free_or_flush_list.  The list node pointers can be shared, because a
block can never be belong to both lists at the same time.
2006-12-04 12:44:06 +00:00
marko
6c0a4186af branches/zip: Allow some functions to work on compressed-only pages
(state == BUF_BLOCK_ZIP_PAGE).  Make use of buf_page_in_file()
and buf_page_get_mutex().

buf_block_get_newest_modification(): Rename to
buf_page_get_newest_modification().
2006-11-30 15:09:33 +00:00
marko
ea16688c66 branches/zip: buf_page_t: Move "old" to the same machine word with
LRU_position and possibly freed_page_clock.  All these fields are
protected by buf_pool->mutex rather than buf_page_get_mutex().

buf_page_set_accessed(): Add a mutex_own() assertion.
2006-11-30 13:27:34 +00:00
marko
aedce8cde3 branches/zip: Move the fields related to the least-recently-used algorithm
of the buffer pool from buf_block_t to buf_page_t.  Replace some buf_block_t*
parameters with buf_page_t*.  Add accessor functions.
2006-11-30 12:27:49 +00:00
marko
69f87fb058 branches/zip: Move all fields that are not needed in compression or
decompression from page_zip_des_t to buf_page_t, because the fields
needed in compression are modified without holding the block mutex.
All writes to bit-fields sharing a machine word must be protected
by the same mutex or rw-lock.
2006-11-30 10:33:35 +00:00
marko
329a29c2fb branches/zip: Replace ib_ulonglong with ib_uint64_t, since we want exactly
64 bits.  "long long" might be longer than 64 bits on some systems.
2006-11-29 14:52:16 +00:00
marko
519ba6f47c branches/zip: Make buf_pool->page_hash point to buf_page_t instead of
buf_block_t.  Move the fields "hash" and "file_page_was_freed" from
buf_block_t to buf_page_t.

buf_page_in_file(): New function, for checking block state in assertions.
2006-11-29 13:23:28 +00:00
marko
17842af300 branches/zip: Allow the code to compile with debugging enabled. This
was broken in r1066.
2006-11-28 11:08:33 +00:00
marko
8e77f38e21 branches/zip: Make buf_block_t:old a bit-field. Document the mutex protection
of the fields "freed_page_clock" and "old".
2006-11-28 10:02:03 +00:00
marko
620df99df0 branches/zip: Enclose buf_block_t:in_LRU_list in #ifdef UNIV_DEBUG 2006-11-28 09:37:02 +00:00
marko
b03460d90d branches/zip: Move the page flushing fields from buf_block_t to
buf_page_t and page_zip_des_t.
2006-11-27 14:12:09 +00:00
marko
7593a465db branches/zip: Introduce the unsigned 64-bit data type ib_ulonglong
for log sequence numbers, to replace dulint.
2006-11-24 13:05:01 +00:00
marko
8332c45ce4 branches/zip: Introduce buf_page_t, a common structure for compressed-only
and uncompressed buffer pool pages.

buf_block_t: Replace page_zip, space, and offset with buf_page_t page.
Replace some integers with bit-fields.

enum buf_block_state: Rename to buf_page_state.  Add BUF_BLOCK_ZIP_PAGE.

page_zip_des_t: Add the field "state".  Make the integer fields bit-fields.

page_zip_copy(): Document which fields are copied.
2006-11-24 08:32:18 +00:00
marko
c476d38e07 branches/zip: Remove buf_block_t:magic_n.
buf_block_state_valid(): New macro for replacing the magic_n check
2006-11-23 15:14:45 +00:00
marko
c5be47e7f6 branches/zip: Remove most direct references to block->state.
buf_block_set_state(): New function.  Also checks the state transitions.

buf_block_set_file_page(): New function.
2006-11-23 14:12:58 +00:00
marko
6c8dfab1dc branches/zip: Replace most references to block->state, block->offset,
and block->space with buf_block_get_state(block), buf_block_get_page_no(block),
and buf_block_get_space(block).

enum buf_block_state: Replaces the #define'd buf_block_t.state values.

buf_block_get_state(): New function.

buf_block_get_frame(): Add __attribute__((const)).
2006-11-23 13:26:01 +00:00
marko
03bdaaf509 branches/zip: Introduce a new preprocessor symbol, UNIV_BUF_DEBUG, and
enclose buf_validate() and friends in it.
2006-11-16 09:00:30 +00:00
marko
97b49ef91f branches/zip: Add #define UNIV_DEBUG_FILE_ACCESSES to univ.i and
enclose all related debug code in #ifdef UNIV_DEBUG_FILE_ACCESSES.
This should have no effect on the behaviour, as the symbol is
not defined by default.  It only reduces the size of buf_block_t
and removes some assignments and debug functions.
2006-11-16 08:48:28 +00:00
marko
2018ce35b3 branches/zip: Implement resizeable buffer pool.
mysql.patch: Patch to change or add variables to MySQL

innodb.patch: Patch to make the master thread poll requests to resize
the buffer pool.

Replace srv_pool_size and innobase_buffer_pool_size
with srv_buf_pool_size, srv_buf_pool_old_size,
and srv_buf_pool_curr_size.

Add buf_chunk_t, a collection of buf_block_t.
2006-11-14 14:27:26 +00:00
marko
19420d4621 branches/zip: Merge revisions 968:1009 from trunk. 2006-11-10 11:15:59 +00:00
marko
ac05c89b24 branches/zip: buf_block_align(): Remove from non-debug builds.
btr_search_guess_on_hash(), btr_search_validate(): Instead of
buf_block_align(), call buf_page_hash_get(), and allow it to return NULL.
2006-11-02 09:06:49 +00:00
marko
42bd3289a3 branches/zip: os_mem_alloc_large(), os_mem_free_large():
On POSIX, use mmap() and munmap().  On Windows, use VirtualAlloc()
and VirtualFree().  Only on Netware, use ut_malloc_low() and ut_free().
The lower-level functions on POSIX and Windows allow InnoDB to return
memory to the operating system when the buffer pool is shrunk.
2006-11-01 08:13:58 +00:00
marko
fc2c6729b4 branches/zip: Remove buf_pool->frame_zero and buf_pool->high_end. 2006-10-31 09:59:15 +00:00
marko
d266262af6 branches/zip: Prepare for buffer pool allocation in several chunks.
buf_pool_t: Remove n_frames, max_size, and blocks_of_frames.
The current buffer pool size is in curr_size.

buf_pool_init(): Remove parameter max_size.

buf_pool_get_max_size(), buf_pool_is_block(): Remove.

buf_block_align(): Do not assume that the buffer pool is allocated
in one chunk.  Replace dependency on buf_pool->blocks_of_frames
with a call to buf_page_hash_get().
2006-10-30 15:15:19 +00:00
marko
89bacf3402 branches/zip: Remove support for AWE (Intel PAE on Win32).
Removal of the variable innobase_buffer_pool_awe_mem_mb requires
coordination with MySQL AB.
2006-10-30 13:48:08 +00:00