Commit graph

114 commits

Author SHA1 Message Date
marko
cb530ba3e7 branches/zip: Make use of the previously unused state BUF_BLOCK_MEMORY.
buf_LRU_get_free_block(): Document that the block is in the state
BUF_BLOCK_READY_FOR_USE.

buf_block_alloc(): Change the state of the block to BUF_BLOCK_MEMORY
and document it.

Replace those invocations of buf_block_alloc() with
buf_LRU_get_free_block() where the allocated block is used for buffer
pool input and output.  However, temporary copies of B-tree pages
during reorganization are not used for file I/O, and such blocks are
still allocated with buf_block_alloc().
2006-12-13 14:58:54 +00:00
marko
310b625ef5 branches/zip: Make the buddy allocator reclaim blocks from the end of
the LRU list regardless of their size.

buf_buddy_alloc_clean_zip(): Rename to buf_buddy_alloc_clean(),
now that also uncompressed blocks can be reclaimed.
Use buf_buddy_alloc_zip() instead of reusing the freed block
directly, since buf_buddy_free() may combine blocks.
2006-12-12 16:28:22 +00:00
marko
ff5f343b2d branches/zip: Minor improvements to the buddy allocator.
buf_buddy_alloc(), buf_buddy_alloc_low(): Add parameter "lru" for
enabling allocation from the list of least-recently-used blocks.

buf_buddy_alloc_low(): Release buf_pool->mutex while calling
buf_LRU_get_free_block().
2006-12-12 12:14:58 +00:00
marko
91aae17842 branches/zip: Make the code compile with all debugging enabled
buf_buddy_relocate(): Replace ut_align_down() with ut_align_offset()
in assertions.

buf0buddy.h: Include buf0buf.h and sync0sync.h.
2006-12-11 15:48:37 +00:00
marko
f9a10f4cef branches/zip: Implement the whole allocation algorithm of compressed pages.
buf_LRU_free_block(): Free compressed-only pages and their
descriptors with buf_buddy_free().

buf_LRU_get_free_only(): New function for returning a block from
buf_pool->free if one is available.  Split from buf_LRU_get_free_block().

buf_buddy_alloc_zip(): Rename from buf_buddy_alloc_low() and make static.
Remove parameter "split".  Always try to split.

buf_buddy_free_block(): Rename to buf_buddy_block_free().

buf_buddy_block_register(): New function for registering buf_block_t
objects reserved by the allocator.

buf_buddy_alloc_from(): New function for allocating an object from a
bigger object, and putting the unused parts on the free list.

buf_buddy_alloc_clean_zip(): New function for allocating memory by
replacing an unmodified compressed page.

buf_buddy_alloc_low(): New function for allocating memory, either from
the free lists of compressed pages, from the global free list, or from
unmodified pages in the buffer pool.
2006-12-11 14:27:43 +00:00
marko
9bf07dae11 branches/zip: buf_buddy_relocate(): Split some lines differently on
popular demand.
2006-12-11 09:59:06 +00:00
marko
d387f148b4 branches/zip: Add buf_pool->zip_hash for keeping track on pages allocated
to the buddy system for allocating compressed pages and their descriptors.

buf_buddy_free_block(): New function: Deallocate the buffer frame.

buf_buddy_free(), buf_buddy_free_low(): Return void instead of a pointer
to a freed buffer frame.
2006-12-11 09:54:13 +00:00
marko
e2a384eb7f branches/zip: buf_buddy_relocate(): Relocate buf_page_t objects that represent
clean compressed pages (BUF_BLOCK_ZIP_PAGE).
2006-12-11 08:43:55 +00:00
marko
0f8523319d branches/zip: buf_buddy_free_low(): Attempt to relocate the buddy of a
free block in order to create two adjacent free blocks that can be
recombined.  Add some debug assertions.

buf_buddy_relocate(): New function: Try to relocate a free block.
2006-12-08 14:44:28 +00:00
marko
449de0f4f9 branches/zip: Add buf_buddy_free() and buf_buddy_free_low(). 2006-12-07 15:22:44 +00:00
marko
b5fb7cd1f2 branches/zip: Rename buf_buddy_alloc_free() to buf_buddy_alloc() and
buf_buddy_alloc_free_low() to buf_buddy_alloc_low().
2006-12-07 15:05:06 +00:00
marko
0d5f0ab7e7 branches/zip: buf_buddy_alloc_free(), buf_buddy_alloc_free_low():
Add parameter "split" for enabling splits.
2006-12-07 14:58:58 +00:00
marko
2a25f556e7 branches/zip: Add the first bits of the binary buddy system for allocating
compressed pages from the buffer pool.

Makefile.am: Add buf0buddy.h, buf0buddy.ic.

buf/Makefile.am: Add buf0buddy.c.

Introduce the constants BUF_BUDDY_LOW and BUF_BUDDY_SIZES.

buf_pool_t: Add zip_mutex and the lists zip_clean and zip_free[].

buf_page_get_mutex(): Return &buf_pool->zip_mutex instead of NULL.

buf_buddy_get_offset(), buf_buddy_get(), buf_buddy_get_slot(),
buf_buddy_alloc_free(), buf_buddy_alloc_free_low(): New functions.
2006-12-07 13:29:04 +00:00
marko
a9e16776e3 branches/zip: buf_page_t: Rename free_or_flush_list to list
and document the possible list memberships.
2006-12-05 12:31:38 +00:00
marko
f81ecc5ad2 branches/zip: enum buf_page_state: Add BUF_BLOCK_ZIP_FREE, BUF_BLOCK_ZIP_DIRTY. 2006-12-05 12:10:30 +00:00
marko
0b88392a78 branches/zip: buf_LRU_free_block(): Note that
buf_LRU_block_remove_hashed_page() changes the state of the block.
This error was introduced in r1090.
2006-12-04 13:12:07 +00:00
marko
2696aa6d60 branches/zip: Remove buf_block_t:free_list. Rename buf_page_t:flush_list
to free_or_flush_list.  The list node pointers can be shared, because a
block can never be belong to both lists at the same time.
2006-12-04 12:44:06 +00:00
marko
229d901921 branches/zip: Fix compilation errors when UNIV_HOTBACKUP is defined. 2006-11-30 15:25:52 +00:00
marko
6c0a4186af branches/zip: Allow some functions to work on compressed-only pages
(state == BUF_BLOCK_ZIP_PAGE).  Make use of buf_page_in_file()
and buf_page_get_mutex().

buf_block_get_newest_modification(): Rename to
buf_page_get_newest_modification().
2006-11-30 15:09:33 +00:00
marko
7b32620f7d branches/zip: page_zip_des_t: Enclose m_start in #ifdef UNIV_DEBUG.
Add m_nonempty for facilitating the test in page_zip_alloc().  This
reduces the combined size of the bit-fields to 32 bits.  Thus,
sizeof(page_zip_des_t) == 2 machine words on 32-bit and wider systems.
2006-11-30 14:09:29 +00:00
marko
db71ab1cd6 branches/zip: buf_page_peek_block(): Do not invoke buf_block_get_state()
on a NULL pointer.  This error was introduced in r1082.
2006-11-30 12:40:33 +00:00
marko
aedce8cde3 branches/zip: Move the fields related to the least-recently-used algorithm
of the buffer pool from buf_block_t to buf_page_t.  Replace some buf_block_t*
parameters with buf_page_t*.  Add accessor functions.
2006-11-30 12:27:49 +00:00
marko
69f87fb058 branches/zip: Move all fields that are not needed in compression or
decompression from page_zip_des_t to buf_page_t, because the fields
needed in compression are modified without holding the block mutex.
All writes to bit-fields sharing a machine word must be protected
by the same mutex or rw-lock.
2006-11-30 10:33:35 +00:00
marko
17e108986f branches/zip: Make buf_block_buf_fix_inc() a macro wrapper for
buf_block_buf_fix_inc_func() to eliminate an #ifdef in every invocation.
2006-11-30 09:24:57 +00:00
marko
329a29c2fb branches/zip: Replace ib_ulonglong with ib_uint64_t, since we want exactly
64 bits.  "long long" might be longer than 64 bits on some systems.
2006-11-29 14:52:16 +00:00
marko
519ba6f47c branches/zip: Make buf_pool->page_hash point to buf_page_t instead of
buf_block_t.  Move the fields "hash" and "file_page_was_freed" from
buf_block_t to buf_page_t.

buf_page_in_file(): New function, for checking block state in assertions.
2006-11-29 13:23:28 +00:00
marko
17842af300 branches/zip: Allow the code to compile with debugging enabled. This
was broken in r1066.
2006-11-28 11:08:33 +00:00
marko
8e77f38e21 branches/zip: Make buf_block_t:old a bit-field. Document the mutex protection
of the fields "freed_page_clock" and "old".
2006-11-28 10:02:03 +00:00
marko
620df99df0 branches/zip: Enclose buf_block_t:in_LRU_list in #ifdef UNIV_DEBUG 2006-11-28 09:37:02 +00:00
marko
b03460d90d branches/zip: Move the page flushing fields from buf_block_t to
buf_page_t and page_zip_des_t.
2006-11-27 14:12:09 +00:00
marko
0d7111c9cd branches/zip: page_zip_des_t: Rename "size" to "ssize" and reduce the
storage size from 16 to 3 bits.

page_zip_get_size(), page_zip_set_size(): New functions.

Replace direct references to page_zip_des_t:size with calls to
buf_block_get_zip_size(), page_zip_get_size(), and page_zip_set_size().
2006-11-27 13:44:32 +00:00
marko
7593a465db branches/zip: Introduce the unsigned 64-bit data type ib_ulonglong
for log sequence numbers, to replace dulint.
2006-11-24 13:05:01 +00:00
marko
f8da258f4e branches/zip: Since r1057, page_zip_des_init() affects
buf_block_get_state().

buf_block_init(), buf_page_init_for_backup_restore(): Assign
block->page.zip.state after invoking page_zip_des_init(&block->page).
2006-11-24 13:02:31 +00:00
marko
53853ee573 branches/zip: Replace all references to block->space and block->offset,
which should have been replaced in r1057.
2006-11-24 12:49:59 +00:00
marko
9580b5f26f branches/zip: buf_read_ahead_random(): Stop searching upon exceeding the
threshold.
2006-11-24 08:39:53 +00:00
marko
8332c45ce4 branches/zip: Introduce buf_page_t, a common structure for compressed-only
and uncompressed buffer pool pages.

buf_block_t: Replace page_zip, space, and offset with buf_page_t page.
Replace some integers with bit-fields.

enum buf_block_state: Rename to buf_page_state.  Add BUF_BLOCK_ZIP_PAGE.

page_zip_des_t: Add the field "state".  Make the integer fields bit-fields.

page_zip_copy(): Document which fields are copied.
2006-11-24 08:32:18 +00:00
marko
c476d38e07 branches/zip: Remove buf_block_t:magic_n.
buf_block_state_valid(): New macro for replacing the magic_n check
2006-11-23 15:14:45 +00:00
marko
c5be47e7f6 branches/zip: Remove most direct references to block->state.
buf_block_set_state(): New function.  Also checks the state transitions.

buf_block_set_file_page(): New function.
2006-11-23 14:12:58 +00:00
marko
6c8dfab1dc branches/zip: Replace most references to block->state, block->offset,
and block->space with buf_block_get_state(block), buf_block_get_page_no(block),
and buf_block_get_space(block).

enum buf_block_state: Replaces the #define'd buf_block_t.state values.

buf_block_get_state(): New function.

buf_block_get_frame(): Add __attribute__((const)).
2006-11-23 13:26:01 +00:00
marko
95cd37e28e branches/zip: Rebuild buf_pool->page_hash after resizing the buffer pool.
buf_pool_page_hash_rebuild(): New function.
2006-11-21 15:38:18 +00:00
marko
97c15d9fe5 branches/zip: When shrinking the buffer pool, disable and purge the adaptive
hash index, because it might occupy the chunk we would like to free.

TODO: In btr_search_check_free_space_in_heap(), release the block if
btr_search_latch is not immediately available.

buf_pool_shrink(): Split from buf_pool_resize().

btr_search_disabled: New variable, similar to srv_use_adaptive_hash_indexes
that was removed earlier.

btr_search_disable(): New function: disable and purge the adaptive hash index.

btr_search_enable(): New function: enable the adaptive hash index.

ha_clear(): New function: Empty a hash table and free the memory heaps.
2006-11-21 14:40:14 +00:00
marko
add0bdeb07 branches/zip: buf_pool_resize(): When shrinking the buffer pool, do not
assume that non-file pages are free.  After trying to free or flush
file pages, do not proceed to buf_chunk_free(), because the calls will
temporarily release buf_pool->mutex.  Do not flush if there are non-free
blocks, because it would not achieve anything.
2006-11-16 11:23:20 +00:00
marko
03bdaaf509 branches/zip: Introduce a new preprocessor symbol, UNIV_BUF_DEBUG, and
enclose buf_validate() and friends in it.
2006-11-16 09:00:30 +00:00
marko
97b49ef91f branches/zip: Add #define UNIV_DEBUG_FILE_ACCESSES to univ.i and
enclose all related debug code in #ifdef UNIV_DEBUG_FILE_ACCESSES.
This should have no effect on the behaviour, as the symbol is
not defined by default.  It only reduces the size of buf_block_t
and removes some assignments and debug functions.
2006-11-16 08:48:28 +00:00
marko
e2c305f32b branches/zip: Improve the shrinking of the buffer pool.
buf_LRU_block_free_non_file_page(): Deallocate block->page_zip.data
to avoid ut_a(!block->page_zip.data) in buf_chunk_free().

buf_chunk_free(): Add the assertion ut_a(!block->in_LRU_list).

buf_pool_resize(): When shrinking the buffer pool and there are
non-free blocks in the candidate chunk, free the clean blocks
and move the dirty blocks to the end of the LRU list and request a flush.
Proceed if the chunk becomes free, and retry otherwise.
2006-11-15 21:49:14 +00:00
marko
274f2bc6a1 branches/zip: buf_LRU_free_block(): New function, split from
buf_LRU_search_and_free_block().
2006-11-15 21:41:27 +00:00
marko
2018ce35b3 branches/zip: Implement resizeable buffer pool.
mysql.patch: Patch to change or add variables to MySQL

innodb.patch: Patch to make the master thread poll requests to resize
the buffer pool.

Replace srv_pool_size and innobase_buffer_pool_size
with srv_buf_pool_size, srv_buf_pool_old_size,
and srv_buf_pool_curr_size.

Add buf_chunk_t, a collection of buf_block_t.
2006-11-14 14:27:26 +00:00
marko
1bb75bc187 branches/zip: buf_LRU_block_free_non_file_page(),
buf_LRU_block_remove_hashed_page(): Overwrite (space_id,page_no)
when freeing a buffer block.  This will help catching non-file
pages being passed to buf_block_align().
2006-11-13 09:55:31 +00:00
marko
19420d4621 branches/zip: Merge revisions 968:1009 from trunk. 2006-11-10 11:15:59 +00:00
marko
42bd3289a3 branches/zip: os_mem_alloc_large(), os_mem_free_large():
On POSIX, use mmap() and munmap().  On Windows, use VirtualAlloc()
and VirtualFree().  Only on Netware, use ut_malloc_low() and ut_free().
The lower-level functions on POSIX and Windows allow InnoDB to return
memory to the operating system when the buffer pool is shrunk.
2006-11-01 08:13:58 +00:00