Commit graph

260 commits

Author SHA1 Message Date
marko
106f9f7eb4 branches/zip: Apply some white-space changes from branches/innodb+. 2008-09-18 06:10:12 +00:00
marko
3bfc4ef89f branches/zip: buf_LRU_get_free_block(): Add missing spaces to and
correct the grammar of the warning message.
2008-08-22 17:30:42 +00:00
marko
0d8f5237dd branches/zip: Introduce UNIV_LRU_DEBUG for debugging the LRU buffer pool
cache, especially buf_pool->LRU_old and bpage->old.

buf_LRU_old_adjust_len(), buf_LRU_remove_block(): Check that blocks in
buf_pool->LRU_old have the "old" flag set and the blocks preceding
buf_pool->LRU_old have the "old" flag clear.

buf_LRU_add_block_low(), buf_relocate(): Check that buf_pool->LRU_old
is the first block in the LRU list whose "old" flag is set.

buf_LRU_free_block(): When replacing a control block in the LRU list
with a control block for a compressed page, assert that the "old"
flags in the neighboring LRU list entries grow monotonically.

buf_page_set_old(): Assert that the "old" flags in the neighboring LRU
list entries grow monotonically.
2008-08-18 18:28:20 +00:00
marko
aebc0edf5a branches/zip: buf_page_set_old(): Add the assertion ut_ad(bpage->in_LRU_list).
Adjust the callers where necessary:
buf_LRU_add_block_to_end_low() and buf_LRU_add_block_low().
2008-08-18 18:06:33 +00:00
marko
7d69d45534 branches/zip: buf_LRU_free_block(): When buf_page_is_old(b), we incremented
buf_pool->LRU_old_len.  However, we forgot to check if buf_pool->LRU_old
happens to point to b's successor in the LRU list.  If it does, we must
assign buf_pool->LRU_old = b.  The following invariants hold:

In the LRU list, the "old" flag should grow monotonically, i.e., it is 0
for the first few items and 1 from thereafter.

If buf_pool->LRU_old != NULL, it must point to the first item with old=1
in the LRU list, and there must be buf_pool->LRU_old_len old items in the list.

This should fix Mantis issue#50 and issue#68.
2008-08-08 13:01:02 +00:00
marko
9c0d868a8d branches/zip: Correct some typos in comments. 2008-08-06 12:45:35 +00:00
marko
6cddd9c551 branches/zip: Improve the diagnostics for tracking down issue#63 and issue#65.
buf_flush_init_for_writing(), buf_LRU_block_remove_hashed_page():
Dump the page frame featuring the incorrect FIL_PAGE_TYPE along with the
page_zip->data that might contain an earlier version of the page.
2008-08-06 08:44:17 +00:00
marko
cd124b7524 branches/zip: Always display a newline after ut_print_buf(stderr, ...). 2008-08-06 08:10:21 +00:00
marko
2b20246451 branches/zip: Dump the compressed page when fil_page_get_type() returns
an incorrect value.  This is to track down Mantis issue#63 and issue#65.

buf_LRU_block_remove_hashed_page(),
buf_flush_init_for_writing(): dump the compressed page before ut_error.
2008-08-04 08:52:09 +00:00
sunny
33c5bfc756 branches/zip: Cover the diagnostic print statement with a suitable lock. It
caused an assertion failure during testing.
2008-06-17 19:15:37 +00:00
vasil
b21c94dc01 branches/zip: Merge 2437:2485 from branches/5.1:
(r2478 was skipped for the obvious reason)

  ------------------------------------------------------------------------
  r2464 | vasil | 2008-05-19 17:59:42 +0300 (Mon, 19 May 2008) | 9 lines
  
  branches/5.1:
  
  Fix Bug#36600 SHOW STATUS takes a lot of CPU in buf_get_latched_pages_number
  
  by removing the Innodb_buffer_pool_pages_latched variable from
  SHOW STATUS output in non-UNIV_DEBUG compilation.
   
  Approved by:	Heikki
  
  ------------------------------------------------------------------------
  r2466 | calvin | 2008-05-20 01:37:14 +0300 (Tue, 20 May 2008) | 12 lines
  
  branches/5.1:
  
  Fix Bug#11894 innodb_file_per_table crashes w/ Windows .sym symbolic
  link hack
  
  The crash was due to un-handled error 3 (path not found). In the case
  of file per table, change the call to os_file_handle_error_no_exit()
  from os_file_handle_error(). Also, checks for full path pattern during
  table create (Windows only), which is used in symbolic link and temp
  table creation.
  
  Approved by:	Heikki
  ------------------------------------------------------------------------
  r2478 | sunny | 2008-05-23 08:29:08 +0300 (Fri, 23 May 2008) | 3 lines
  
  branches/5.1: Fix for bug# 36793. This is a back port from branches/zip. This
  code has been tested on a big-endian machine too.
  
  ------------------------------------------------------------------------
  r2480 | vasil | 2008-05-27 11:40:07 +0300 (Tue, 27 May 2008) | 11 lines
  
  branches/5.1:
  
  Fix Bug#36819 ut_usectime does not handle errors from gettimeofday
  
  by retrying gettimeofday() several times if it fails in ut_usectime().
  If it fails on all calls then return error to the caller to be handled
  at higher level.
  
  Update the variable innodb_row_lock_time_max in SHOW STATUS output only
  if ut_usectime() was successful.
  
  ------------------------------------------------------------------------
  r2482 | sunny | 2008-05-28 12:18:35 +0300 (Wed, 28 May 2008) | 5 lines
  
  branches/5.1: Fix for Bug#35602, "Failed to read auto-increment value from
  storage engine". The test for REPLACE was an error of ommission since it's
  classified as a simple INSERT. For REPLACE statements we don't acquire
  the special AUTOINC lock for AUTOINC_NEW_STYLE_LOCKING with this fix.
  
  ------------------------------------------------------------------------
  r2485 | vasil | 2008-05-28 16:01:14 +0300 (Wed, 28 May 2008) | 9 lines
  
  branches/5.1:
  
  Fix Bug#36149 Read buffer overflow in srv0start.c found during "make test"
  
  Use strncmp(3) instead of memcmp(3) to avoid reading past end of the string
  if it is empty (*str == '\0'). This bug is _not_ a buffer overflow.
  
  Discussed with:	Sunny (via IM)
  
  ------------------------------------------------------------------------
2008-06-06 15:24:58 +00:00
sunny
fd41c6a315 branches/zip: The block can't be in io fix state WRITE and we check for
that in the fix but the earlier check fails because it tries to be too
strict and assumes that if the block is on the clean list then it can
only be in state io fix NONE.
2008-04-12 10:04:28 +00:00
marko
eb04f0f8ef branches/zip: Implement the INFORMATION_SCHEMA tables
INNODB_COMPRESSION_BUDDY and INNODB_COMPRESSION_BUDDY_RESET.

buf_buddy_stat_struct, buf_buddy_stat_t, buf_buddy_stat[]:
Statistics of the buddy system grouped by block size.

i_s_innodb_compression_buddy, i_s_innodb_compression_buddy_reset:
New INFORMATION_SCHEMA plugins.

i_s_compression_buddy_fields_info[]: Define the fields:
size, used, free, relocated, relocated_sec.

i_s_compression_buddy_fill_low(), i_s_compression_buddy_fill(),
i_s_compression_buddy_reset_fill(): Fill the fields.

i_s_compression_buddy_init(), i_s_compression_buddy_reset_init():
Initialize the tables.
2008-03-28 10:03:58 +00:00
marko
77d8c98735 branches/zip: Rename ib_ulonglong to ib_uint64_t and ib_longlong to ib_int64_t. 2008-03-17 14:19:04 +00:00
marko
6227278c20 branches/zip: buf_read_ahead_random(), buf_read_ahead_linear():
Compute BUF_READ_AHEAD_RANDOM_AREA and BUF_READ_AHEAD_LINEAR_AREA
only once.  The definition of BUF_READ_AHEAD_AREA depends on
buf_pool->curr_size, which could change while this code is running.
2008-03-05 09:16:40 +00:00
marko
e32628b41c branches/zip: buf_flush_try_neighbors(): Compute buf_flush_area only
once and assign it to an auxiliary variable.
2008-03-05 08:36:16 +00:00
marko
9d58e3ff99 branches/zip: Improve the LRU algorithm with a separate unzip_LRU list of
blocks that contains uncompressed and compressed frames.  This patch was
designed by Heikki and Inaam, implemented by Inaam, and refined and reviewed
by Marko and Sunny.

buf_buddy_n_frames, buf_buddy_min_n_frames, buf_buddy_max_n_frames: Remove.

buf_page_belongs_to_unzip_LRU(): New predicate:
bpage->zip.data && buf_page_get_state(bpage) == BUF_BLOCK_FILE_PAGE.

buf_pool_t, buf_block_t: Add the linked list unzip_LRU.  A block in the
regular LRU list is in unzip_LRU iff buf_page_belongs_to_unzip_LRU() holds.

buf_LRU_free_block(): Add a third return value to refine the case
"cannot free the block".

buf_LRU_search_and_free_block(): Update the documentation to reflect the
implementation.

buf_LRU_stat_t, buf_LRU_stat_cur, buf_LRU_stat_sum, buf_LRU_stat_arr[]:
Statistics for the unzip_LRU algorithm.

buf_LRU_stat_update(): New function: Update the statistics.  Called once
per second by srv_error_monitor_thread().

buf_LRU_validate(): Validate the unzip_LRU list as well.

buf_LRU_evict_from_unzip_LRU(): New predicate: Use the unzip_LRU before
falling back to the regular LRU?

buf_LRU_free_from_unzip_LRU_list(), buf_LRU_free_from_common_LRU_list():
Subfunctions of buf_LRU_search_and_free_block().

buf_LRU_search_and_free_block(): Reimplement.  Try to evict an uncompressed
page from the unzip_LRU list before falling back to evicting an entire block
from the common LRU list.

buf_unzip_LRU_remove_block_if_needed(): New function.

buf_unzip_LRU_add_block(): New function: Add a block to the unzip_LRU list.
2008-03-03 12:48:38 +00:00
marko
d8759fd52e branches/zip: buf_buddy_alloc_clean(): Remove, as it violates the LRU policy. 2008-02-29 08:32:14 +00:00
marko
f7595d4278 branches/zip: buf_relocate(): Document and assert that bpage must be
in one of the states BUF_BLOCK_ZIP_DIRTY or BUF_BLOCK_ZIP_PAGE.
Note that the caller will have to relocate bpage->list.
2008-02-29 08:20:17 +00:00
marko
5377a7dd94 branches/zip: Add duration statistics to INFORMATION_SCHEMA.INNODB_ZIP.
buf_buddy_relocated_duration[],
page_zip_compress_duration[]
page_zip_decompress_duration[]: Record the total duration of the operations.

buf_buddy_relocate(), page_zip_compress(), page_zip_decompress():
Add ut_time_us() instrumentation.

i_s_zip_fields_info[], i_s_zip_fill_low(): Move the columns containing
cumulated statistics last.  Add relocated_usec, compressed_usec, and
decompressed_usec.
2008-02-27 14:34:44 +00:00
marko
aca7dcbeaa branches/zip: Minor fixes.
buf_LRU_old_adjust_len(): Replace a constant ut_ad() with a preprocessor check.

buf_LRU_free_block(): Remove the check for the unlikely case
buf_pool->LRU_old == prev_b in order to simplify the function.
The check was implemented as part of r2306.
2008-02-18 15:43:16 +00:00
marko
d917b27aa8 branches/zip: buf_LRU_free_block(): When freeing the uncompressed page
corresponding to a compressed page, do not flag the block as recently
used, but maintain the position of the control block on the LRU list.
2008-02-16 10:33:15 +00:00
marko
9b9dad7bca branches/zip: buf0lru.c: Minor cleanup.
Use ut_d() in assignments to bpage->in_LRU_list instead of #ifdef UNIV_DEBUG.

buf_LRU_remove_block(): Move an assertion to a more appropriate place.
2008-02-15 11:45:37 +00:00
marko
ab8b2a144a branches/zip: buf_buddy_alloc_from(): Relax a debug assertion that fails
on i==j==BUF_BUDDY_SIZES.
2008-02-15 10:07:42 +00:00
marko
2c2b06ad75 branches/zip: Introduce UNIV_INTERN, a linkage specifier for InnoDB-global
symbols.  Use it for all definitions of non-static variables and functions.

lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static.  It is
referenced from pars0grm.c.

Actually, according to
	nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:

* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs

The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.

The class ha_innodb is renamed from class ha_innobase by a #define.  Thus,
there will be no clash with the builtin InnoDB.  However, there will be some
overhead for invoking virtual methods of class ha_innodb.  Ideas for making
the vtable hidden are welcome.  -fvisibility=hidden is not available in GCC 3.
2008-02-06 14:17:36 +00:00
marko
473d7c16a9 branches/zip: Introduce the page type code FIL_PAGE_TYPE_ZBLOB2 for
continuation pages containing compressed BLOBs.  The first compressed
BLOB page will be of type FIL_PAGE_TYPE_ZBLOB.
2008-01-24 08:12:02 +00:00
marko
66791e5a1b branches/zip: Free the buffer pool at shutdown.
buf_pool_free(): New function: Free all chunks of the buffer pool.

innobase_shutdown_for_mysql(): Call buf_pool_free() right before
ut_free_all_mem().
2008-01-16 12:44:44 +00:00
marko
7d18bba5e7 branches/zip: Add comments about the lock and latch protection of externally
stored columns (BLOBs).

btr_copy_blob_prefix(), btr_copy_zblob_prefix(),
btr_copy_externally_stored_field_prefix_low(),
btr_copy_externally_stored_field_prefix(),
btr_copy_externally_stored_field(),
btr_rec_copy_externally_stored_field():
Note that the page containing the clustered index record that points to
the BLOB must be latched.

btr_copy_zblob_prefix(): Note that there is no latch on the page, and thus
all accesses to a given page via this function must be covered by the same
set of locks or latches.

btr_copy_zblob_prefix(): Note that the block acquired by
buf_page_get_zip() is protected by an exclusive table lock or
or by a latch on the clustered index record.
2008-01-16 10:45:14 +00:00
marko
5c95ec4035 branches/zip: buf_pool_init(): Initialize buf_pool_zip_mutex before
acquiring buf_pool_mutex.  This avoids triggering the debug assertion
that was added in r2227.
2008-01-15 09:43:14 +00:00
marko
e8574ccb03 branches/zip: buf_LRU_search_and_free_block(): Do not forbid the release
of the buffer pool mutex.  Apparently, it is temporarily released also
in older versions of MySQL/InnoDB for the duration of the
btr_search_drop_page_hash_index() call [in buf_LRU_free_block()].
2008-01-10 12:34:25 +00:00
marko
523a8a0e8d branches/zip: Add instrumentation for prohibiting the release of
the buffer pool mutex.  The instrumentation can be activated by
defining UNIV_DEBUG or UNIV_BUF_DEBUG at compilation time.

buf_pool_mutex_exit_forbidden: New variable.  When this is nonzero,
an assertion will fail in buf_pool_mutex_exit().

buf_pool_mutex_exit_forbid(): Macro for declaring that the buffer pool
mutex must not be released.  Calls may be nested.

buf_pool_mutex_exit_allow(): Macro for declaring that the buffer pool
mutex may be released.  Calls may be nested.

buf_LRU_search_and_free_block(): Prohibit buf_pool_mutex_exit() in the
scope of the function.

buf_LRU_free_block(): Prohibit buf_pool_mutex_exit() in buf_buddy_alloc()
and buf_buddy_free().

buf_LRU_block_remove_hashed_page(): Prohibit buf_pool_mutex_exit()
in buf_buddy_free().
2008-01-10 09:51:57 +00:00
marko
3bb405841f branches/zip: Implement wrappers for all operations on the buffer pool mutex.
buf_pool->mutex: Rename to buf_pool_mutex, so that the wrappers will have
to be used when changes are merged from other source trees.

buf_pool->zip_mutex: Rename to buf_pool_zip_mutex.

buf_pool_mutex_own(), buf_pool_mutex_enter(), buf_pool_mutex_exit():
Wrappers for buf_pool_mutex.
2008-01-10 09:37:13 +00:00
marko
051ee49116 branches/zip: Split the source-only configuration parameter
buf_buddy_min_n_frames into two, also buf_buddy_max_n_frames.
Set the default values in such a way that a strict LRU policy will apply
for replacing compressed or uncompressed page frames in the buffer pool.

These parameters have not yet been exposed to the MySQL layer.
2008-01-04 14:08:41 +00:00
vasil
815bb2ccf4 branches/zip:
Non-functional change: add "out:" comment for the return value.
2007-12-17 10:03:15 +00:00
marko
a563fbc3d6 branches/zip: Clarify that buf_buddy_alloc() should only be used for
allocating compressed page frames or their control blocks.  Also note
that if buf_buddy_alloc() is used for allocating a control block,
it must be initialized before releasing buf_pool->mutex.

buf_page_init_for_read(): When the page hash check fails after
buf_buddy_alloc(), free the uninitialized control block before freeing
the compressed page frame.  This fixes a potential error in
buf_buddy_relocate_block().
2007-12-12 14:12:52 +00:00
marko
902615e374 branches/zip: Document how the data structures of the buddy allocator
are interfaced with the buffer pool.
2007-12-12 13:42:03 +00:00
marko
40dce922cc branches/zip: buf_page_get_gen(): Check the return status of
buf_zip_decompress() and return NULL on decompression failure.
2007-12-10 12:54:53 +00:00
marko
6f306fad26 branches/zip: buf_buddy_alloc(): Assign *lru = TRUE whenever the buffer pool
mutex is temporarily released.

buf_LRU_free_block(), buf_buddy_alloc_clean(): Add an output parameter that
will be assigned TRUE when the buffer pool mutex is released.

This bug was spotted by and fix provided by Sunny.
2007-12-10 09:48:28 +00:00
marko
7a1d5d91b7 branches/zip: Fix a bug that was introduced in r2123.
buf_block_is_uncompressed(): Check that the pointer is aligned.  Use the
C modulus operator % instead of ut_align_offset(), because sizeof(buf_block_t)
is not guaranteed to be a power of 2.
2007-11-28 13:45:22 +00:00
marko
f6a5cc1d47 branches/zip: buf_page_get_gen(): Note that the guessed block may also
point to a buffer pool chunk that has been released when resizing the
buffer pool.

buf_block_is_uncompressed(): Check that the pointer is aligned.  Thanks
to this check, it is safe to pass an arbitrary pointer as a guess
to buf_page_get_gen().
2007-11-28 11:31:12 +00:00
marko
7aedcf308e branches/zip: buf_page_get_gen(): Ignore "guess" if it does not point to
buf_pool->chunks[]->blocks[].

buf_block_is_uncompressed(): New function, to detect if a block points to
buf_pool->chunks[]->blocks[].
2007-11-05 10:31:41 +00:00
marko
bcd672cae7 branches/zip: buf_page_init_low(): Initialize also flush_type. 2007-10-31 22:02:23 +00:00
marko
e00f417abb branches/zip: buf_LRU_block_remove_hashed_page(): Add some Valgrind
instrumentation.
2007-10-31 10:40:41 +00:00
marko
d321ea1141 branches/zip: buf_buddy_relocate(): The src block may be partially freed.
Remove the Valgrind check about it.
2007-10-31 10:40:09 +00:00
marko
2d77ed4f40 branches/zip: Improve Valgrind instrumentation.
buf_buddy_relocate(): Allow the source block to contain uninitialized data.

buf0buddy.c: Replace the remaining VALGRIND_CHECK_ macros with the wrappers
defined in univ.i.
2007-10-31 09:00:08 +00:00
marko
7319444c08 branches/zip: Improve Valgrind instrumentation.
buf_LRU_free_block(): Check that the block descriptor contains valid data.

buf_buddy_relocate(): Check that the source block contains valid data.

buf_page_get_gen(): Do not dereference bpage after calling buf_relocate().
This avoids a bogus Valgrind warning; the memory itself was valid.

buf_page_hash_get(): Check that the returned block descriptor contains
valid data.
2007-10-30 09:27:09 +00:00
marko
ada2f71ab7 branches/zip: Reduce WAIT_FOR_READ from 20 to 5 milliseconds, to correspond
to the time in which modern disks can serve a random disk read.
2007-10-26 12:34:53 +00:00
marko
a4b26c4ffb branches/zip: buf_pool_t: Add n_pend_unzip. Display it in buf_print(). 2007-10-26 08:53:36 +00:00
marko
1890ec59c7 branches/zip: buf_page_get_gen(): Improve the comments about
wait_until_unfixed.
2007-10-26 08:48:01 +00:00
marko
aa8c8329f2 branches/zip: buf_pool_init(): Allocate buf_pool with mem_zalloc(). 2007-10-25 12:54:18 +00:00