Commit graph

285 commits

Author SHA1 Message Date
marko
b90b4f59b4 branches/zip: Make use of the function dict_index_is_unique(). 2007-04-16 09:21:39 +00:00
marko
758858549c branches/zip: Clean up fast index creation.
dict_index_is_unique(): New function.

row_merge_rec_fits_to_block(), row_merge_store_rec_to_block():
Add const qualifiers to rec, offsets.

row_merge_select(): Rename to row_merge_cmp(), simplify the interface.

row_merge_sort_linked_list(): Change the return type to ibool.
Replace sec_offs1,sec_offs2 with offsets1,offsets2.  Replace offset_heap
with heap.  Define a func_exit label.

row_merge_block_merge(): Reduce the scope of rec_offsets.
2007-04-16 09:13:44 +00:00
marko
13817f1d52 branches/zip: Minor cleanup of merge_file_t and merge_block_header_t.
Replace byte offsets of type dulint with block offsets of type ulint.
Avoid references to MERGE_BLOCK_SIZE.

Improve the language of some comments.

row_merge_insert_index_tuples(), row_merge_sort_linked_list_in_disk():
Make the offset a ulint.

row_merge_read(), row_merge_write(): Helper functions.  Return the
status of os_file_read() and os_file_write().  TO DO: check the status
in the callers.
2007-04-12 10:32:16 +00:00
marko
96e9017471 branches/zip: More cleanup of fast index creation.
innobase_create_temporary_tablename(): Add static qualifier.  Allocate
the memory from heap.  Return a fixed identifier.

ha_innobase::add_index(): Allocate all memory from a single heap.
Simplify error handling.  Use a fixed prefix for temporary table names,
because only one add_index() can be active for a given table.

mem_heap_empty_noninline(): New function, to be called from ha_innodb.cc.

row_build_index_for_mysql(): Remove the parameter new_primary.
2007-04-12 05:14:41 +00:00
marko
4b1e9a91db branches/zip: Some more cleanup of fast index creation.
Move more definitions from row0mysql.h to row0mysql.c.  Remove the
unused definition of merge_thread.

merge_index_def_t: Replace merge_index_field_t** with merge_index_field_t*.
Use mem_heap_strdup() when copying strings.

ha_innobase::add_index(): Avoid excessive use of current_thd.
2007-04-11 12:17:55 +00:00
marko
fcfed19818 branches/zip: Initial clean up of the merge sort subsystem.
Define the merge_rec_ structures in row0merge.c instead of
row0merge.h.  They are not used in the interface defined in row0merge.h.

row_merge_create_file_structure(): Rename to row_merge_file_create().
Input a pointer to a pre-allocated block of memory.

row_merge_read_clustered_index(): Replace merge_file_t** with merge_file_t*.

row_merge_sort_and_insert_thread(): Remove the declaration.
This function was never defined or called.

row_merge_is_index_usable(): Correct the formatting.

row_build_index_for_mysql(): Allocate an array of merge_file_t instead
of an array of pointers to merge_file_t.
2007-04-11 08:16:14 +00:00
marko
38c02b75df branches/zip: Add statistics on page compression and decompression counts.
ha_innodb.cc: Add the columns COMPRESSED, COMPRESSED_OK, DECOMPRESSED
to INFORMATION_SCHEMA.INNODB_BUDDY.

page_zip_compress_count[], page_zip_compress_ok[]: New statistic counters,
incremented in page_zip_compress().

page_zip_decompress_count[]: New statistic counter,
incremented in page_zip_decompress().
2007-04-10 08:11:49 +00:00
marko
a47b9e64e6 branches/zip: Merge branches/fast-index-creation -r1413.
Fix some bugs.  The tests innodb and innodb-index fail, but that might
be due to an old MySQL source tree being used.
2007-04-04 11:05:33 +00:00
marko
33c1125d55 branches/zip: Merge revisions 1322:1402 from trunk. 2007-04-02 05:39:41 +00:00
marko
3423833053 branches/zip: Make TRUNCATE TABLE recreate single-table tablespaces.
dict_truncate_index_tree(): Add the parameter space for specifying the
new tablespace identifier of a single-table tablespace that has been
recreated.  When SYS_INDEXES.PAGE_NO == FIL_NULL, do not abort but
create the index tree.

fil_discard_tablespace(): Pass the return code from fil_delete_tablespace().

row_truncate_table_for_mysql(): Attempt to discard and recreate
single-table tablespaces.  Reassign the tablespace identifier both in
the data dictionary (SYS_TABLES and SYS_INDEXES) and in the data
dictionary cache.
2007-03-29 08:51:13 +00:00
marko
750e130ce9 branches/zip: btr_cur_pessimistic_update(): Add parameter heap.
Previously, when big_rec was returned, the fields would point to
freed memory.  The memory heap was allocated locally, and the data tuple
was allocated from the heap, and the big_rec would point to some fields
in the data tuple.

row_ins_clust_index_entry_by_modify(): Add parameter heap,
for the same reason.
2007-03-28 19:35:52 +00:00
marko
e4d8b192fe branches/zip: Make dtuple_big_rec_free() an inline function. 2007-03-28 18:31:16 +00:00
marko
3a23bff39b branches/zip: Add Valgrind instrumentation to the InnoDB memory management
functions.

ut_malloc_low(): Flag the block with UNIV_MEM_ALLOC().  Do not flag the
block with UNIV_MEM_FREE() in ut_free(), because it would cause bogus
Valgrind warnings in the underlying memory allocator.

mem_pool_create(): Flag the data area with UNIV_MEM_FREE().

mem_pool_fill_free_list(): Flag the area header with UNIV_MEM_ALLOC().

mem_area_alloc(): Flag the data area with UNIV_MEM_ALLOC().

mem_area_free(): Flag the data area with UNIV_MEM_FREE().

mem_heap_alloc(): Flag the buffer with UNIV_MEM_ALLOC().

mem_heap_block_free(): Flag the block with UNIV_MEM_FREE().

mem_heap_free_top(): Flag the block with UNIV_MEM_FREE().
2007-03-28 09:02:55 +00:00
marko
e4910d4775 branches/zip: Add the pseudo table INFORMATION_SCHEMA.INNODB_BUDDY
for displaying buddy allocator statistics.

buf_buddy_used[], buf_buddy_relocated[]: Declare as global symbols.

mysql_declare_plugin(innobase): Add MYSQL_INFORMATION_SCHEMA_PLUGIN
for innobase_is_buddy ("INNODB_BUDDY").
2007-03-12 12:01:45 +00:00
marko
8332c1f717 branches/zip: Merge revisions 1271:1322 from trunk. 2007-03-01 13:53:57 +00:00
marko
8ff43bd3db branches/zip: ibuf_set_free_bits(): Rename to ibuf_set_free_bits_func(),
and remove the max_val parameter unless #ifdef UNIV_IBUF_DEBUG
2007-03-01 11:33:26 +00:00
marko
9311391ec2 branches/zip: ibuf_update_free_bits_if_full(): Remove the zip_size
parameter.  Add ibuf_update_free_bits_zip() for compressed pages.
2007-03-01 11:28:30 +00:00
marko
17f87488fb branches/zip: dict_table_copy_types(), dict_index_copy_types(): Add const
qualifiers.  Correct the documentation of the parameters.
2007-02-28 16:04:19 +00:00
marko
2b060715ef branches/zip: Split UNIV_IBUF_COUNT_DEBUG from UNIV_IBUF_DEBUG. 2007-02-27 11:56:38 +00:00
marko
58c45a0695 branches/zip: Prepare for a negative return from page_zip_max_ins_size(). 2007-02-27 10:21:44 +00:00
marko
22ad99fb0c branches/zip: Update the insert buffer free bits when recompressing
a page during update-in-place.

page_zip_alloc(): Rename to btr_cur_update_alloc_zip().
Invoke ibuf_update_free_bits_if_full().
2007-02-26 09:35:02 +00:00
marko
37b0f7bf1c branches/zip: Enable the insert buffer on compressed tablespaces.
page_zip_max_ins_size(): New function.

btr_cur_optimistic_insert(), btr_cur_optimistic_delete(),
btr_page_split_and_insert(), btr_compress(): Do not update the
ibuf free bits for non-leaf pages or pages belonging to a clustered index.
The insert buffer only covers operations on leaf pages of secondary indexes.
For pages covered by the insert buffer, limit the max_ins_size to
page_zip_max_ins_size().

buf_page_get_gen(): Merge the insert buffer after decompressing the page.

buf_page_io_complete(): Relax the assertion about ibuf_count.  For
compressed-only pages, the insert buffer merge takes place
in buf_page_get_gen().

ibuf_index_page_calc_free_bits(), ibuf_index_page_calc_free_from_bits(),
ibuf_index_page_calc_free(), ibuf_update_free_bits_if_full(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(),
ibuf_set_free_bits_low(): Add the parameter zip_size.  Limit the maximum
insert size to page_zip_max_ins_size().
2007-02-19 20:32:06 +00:00
marko
fed27de38b branches/zip: page_zip_get_trailer_len(), page_zip_available():
Replace the parameter "dict_index_t* index" with "ibool is_clust".

dict_index_is_clust(): Add __attribute__((pure)).
2007-02-19 14:51:44 +00:00
marko
d56453be8e branches/zip: buf_LRU_search_and_free_block(): Avoid freeing compressed-only
pages when they do not occupy too much of the buffer pool.

buf0buddy.c, buf0buddy.h: Export buf_buddy_n_frames and buf_buddy_min_n_frames.
2007-02-16 09:22:50 +00:00
marko
eb03297439 branches/zip: buf_page_init_for_read(): Defer the costly calls to
buf_buddy_alloc() until after the block has been found not to exist
in the buffer pool.

buf_buddy_alloc(), buf_buddy_alloc_low(): Make lru an output parameter.
Improve documentation.

buf_page_init_for_read(): Simplify the code.  Check buf_page_hash_get()
again if buf_buddy_alloc() released buf_pool->mutex.
2007-02-08 14:19:55 +00:00
marko
5ac6f3217d branches/zip: buf_page_init_for_read(): Do not allocate an uncompressed page
frame when reading compressed pages, unless crash recovery is in progress.

buf_page_read_low(): Adapt for buf_page_init_for_read() returning
buf_page_t* instead of buf_block_t*.
2007-02-08 12:03:03 +00:00
marko
7315b06291 branches/zip: buf_page_init_for_read(), buf_page_create(): Note in the
function comments that buf_page_init_for_read() does not perform the
state transition to FILE_PAGE.  This should have been done in r1286.
2007-02-08 09:08:11 +00:00
marko
329c469e70 branches/zip: Do not decompress blocks in buf_page_init_for_read(),
but in buf_page_get_gen().  Adjust comments.  This prevents
unnecessary decompression in read-ahead when the compressed block
exists in the buffer pool.

If the block does not exist in the buffer pool, read-ahead will still
allocate an uncompressed page and decompress the block.

Move buf_block_init_low() and buf_zip_decompress() earlier in the file,
because some compilers are unable to inline functions that are defined
after the invocation.

buf_page_get_gen(): Decompress the block if needed.

buf_page_init_for_read(): Do not decompress.
2007-02-07 13:59:43 +00:00
marko
e45f4cdaeb branches/zip: Speed up rec_offs_any_extern() and rec_offs_n_extern()
by adding the REC_OFFS_EXTERNAL flag to rec_offs_base(offsets)[0].

This reduces the processor usage of page_zip_write_rec() by about 40%
in one test case.  The code could be sped up further by testing
rec_offs_any_extern() outside of loops that check rec_offs_nth_extern().
The vast majority of records does not contain any externally stored columns.
2007-02-02 16:20:48 +00:00
marko
16ffd00dc1 branches/zip: Merge revisions 1206:1271 from trunk. 2007-02-02 11:31:29 +00:00
marko
35f83441a0 branches/zip: Simplify lock_mode_stronger_or_eq() and lock_mode_compatible()
and introduce enum lock_mode.

lock_mode_stronger_or_eq(), lock_mode_compatible(): Replace if-then-else
chain with a bitwise and against a constant.
2007-02-01 20:56:23 +00:00
marko
f05b0766d9 branches/zip: Split the function page_cur_insert_rec_low().
page_cur_insert_rec_zip_reorg(): New function: Recompress or
reorganize a compressed page.

page_cur_insert_rec_zip(): New function: insert a record to
a compressed page.

page_cur_insert_rec_low(): Only handle inserts to uncompressed pages.
2007-02-01 09:53:26 +00:00
marko
10bb43856c branches/zip: Minor optimization.
trx_undo_page_report_modify(), trx_undo_report_row_operation():
Add const qualifier to the parameter rec.  Remove some local variables.

trx_undo_report_row_operation(): Invoke rec_get_offsets() only once.
2007-01-31 15:11:28 +00:00
marko
0178dba161 branches/zip: Avoid memory fragmentation when adding column definitions
to tables.

dict_mem_table_add_col(): Add the parameter "heap" for temporary memory
allocation.  Allow it and "name" to be NULL.  These parameters are NULL
when creating dummy indexes.

dict_add_col_name(): Remove calls to ut_malloc() and ut_free().

dict_table_get_col_name(): Allow table->col_names to be NULL.

dict_table_add_system_columns(), dict_table_add_to_cache():
Add the parameter "heap".
2007-01-30 09:24:18 +00:00
marko
4b07b77f5a branches/zip: Make zlib use InnoDB memory heaps when processing BLOBs.
page_zip_set_alloc(): New function, to configure zlib to use a memory heap.

btr_store_big_rec_extern_fields(): Reduce memLevel, so that a heap of
256 kilobytes will suffice.  Use page_zip_set_alloc().

btr_copy_externally_stored_field_prefix_low(): Use page_zip_set_alloc().

page_zip_compress(), page_zip_decompress(): Use page_zip_set_alloc().
2007-01-29 08:51:20 +00:00
marko
13efcc13fc branches/zip: Add UNIV_HASH_DEBUG for detecting multiple invocations
of HASH_DELETE().

HASH_ASSERT_VALID(), HASH_INVALIDATE(): New macros, used in HASH_DELETE()
and HASH_SEARCH().

buf_page_init_low(): Add HASH_INVALIDATE(bpage, hash);

buf_page_init(): Call buf_page_init_low() before HASH_INSERT(), so that
bpage->hash will not be invalidated.
2007-01-24 11:41:19 +00:00
marko
3b5c872916 branches/zip: buf_page_t: Add the in_page_hash and in_zip_hash
for extra validation in UNIV_DEBUG builds.
2007-01-24 10:36:05 +00:00
marko
88efbdb335 branches/zip: When retrieving compressed BLOBs, avoid allocating
uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().
2007-01-18 23:10:49 +00:00
marko
d8536b0de8 branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG. 2007-01-18 18:29:12 +00:00
marko
c712838401 branches/zip: Merge revisions 1165:1206 from trunk. 2007-01-18 12:58:39 +00:00
marko
fff329cd3d branches/zip: Remove the fil_space_get_zip_size() call from
buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.
2007-01-18 09:59:00 +00:00
marko
b76c8a5a8d branches/zip: buf_flush_write_block_low(): Avoid recomputing the compressed
page checksum of compressed-only blocks.  Pass the compressed page frame
to fil_io() when needed.

page_zip_calc_checksum(): Skip also FIL_PAGE_LSN and FIL_PAGE_FILE_FLUSH_LSN.

buf_flush_init_for_writing(): Expect page to be non-NULL.
2007-01-16 21:51:40 +00:00
marko
361fe1a5b9 branches/zip: Allow dirty compressed-only blocks to exist in the buffer pool
and to be flushed to disk.

buf_LRU_free_block(): Enable the freeing of uncompressed pages of
compressed tablespaces.

trx_doublewrite->buf_block_arr[]: Change the type from buf_block_t*
to buf_page_t*.

buf_flush_ready_for_flush(): Add debug assertion.

buf_flush_buffered_writes(), buf_flush_try_page(): Support blocks of type
BUF_BLOCK_ZIP_DIRTY.

buf_flush_post_to_doublewrite_buf(), buf_flush_write_block_low():
Change the type of the parameter from buf_block_t* to buf_page_t*.

buf_flush_init_for_writing(): Allow page to be NULL if page_zip_ is non-NULL.
2007-01-16 18:02:49 +00:00
marko
5637695ad2 branches/zip: Improve Valgrind instrumentation of allocated memory.
UNIV_MEM_FREE(): Declare a memory area free.
UNIV_MEM_ALLOC(): Declare a memory area allocated (but uninitialized).
UNIV_MEM_DESC(): Associate a memory area with a control block.
UNIV_MEM_UNDESC(): Unassociate a control block.

trx_sys_create_doublewrite_buf(): Clear the buffer with memset().

buf_page_init(): Add a bogus UNIV_MEM_VALID(block->frame) to silence
valid warnings about InnoDB data pages containing uninitialized data.

buf_LRU_get_free_only(): Add UNIV_MEM_ALLOC(block->frame).

buf_LRU_get_free_block(): Add UNIV_MEM_DESC(block->page.zip.data, block).

buf_LRU_free_block(): Add UNIV_MEM_DESC(b->zip.data, b) when allocating
a compressed-only control block for a compressed page.

buf_LRU_block_free_non_file_page(): Replace UNIV_MEM_INVALID() with
UNIV_MEM_FREE().

buf_LRU_block_remove_hashed_page(): Add UNIV_MEM_UNDESC(bpage) when
deallocating a compressed-only control block.  Add
UNIV_MEM_INVALID(block->frame).  (The frame should be flagged free
by buf_LRU_block_free_non_file_page() moments later.)

buf0buddy.c: Disable some extra checks in Valgrind-instrumented builds.
Add UNIV_MEM_VALID(), UNIV_MEM_INVALID(), UNIV_MEM_FREE(), UNIV_MEM_ALLOC()
as necessary.
2007-01-12 12:36:40 +00:00
marko
0c19de98c0 branches/zip: buf_page_io_complete(): Change the parameter type from
buf_block_t* to buf_page_t*.  Handle compressed-only blocks.
2007-01-10 14:36:39 +00:00
marko
10c92bd7ac branches/zip: buf_buddy_alloc_clean(): Restart the scan after a
successful buf_LRU_free_block().

buf_LRU_free_block(): Note that buf_pool->mutex may be temporarily released.
2007-01-09 12:39:50 +00:00
marko
8af159117e branches/zip: buf_page_t: Introduce the field in_flush_list #ifdef UNIV_DEBUG.
buf_flush_batch(): Remove the local variable "found".
2007-01-09 12:35:42 +00:00
marko
e24afbc387 branches/zip: mem_heap_create_block(), mem_heap_create_func():
Remove the unused parameter init_block.

mem_heap_fast_create(): Remove.

struct mem_block_info_struct: Remove the field init_block.
2007-01-05 20:59:40 +00:00
marko
62391d4280 branches/zip: buf_page_peek(): Return TRUE also if the page exists in the
buffer pool in compressed form only.

buf_page_peek_block(): Remove.

mutex_enter_fast_func: Remove.
2007-01-05 11:19:56 +00:00
marko
b75e9777dd branches/zip: buf_pool_page_hash_rebuild(): Rebuild also buf_pool->zip_hash
and add compressed-only pages to buf_pool->page_hash.

BUF_POOL_ZIP_FOLD(), BUF_POOL_ZIP_FOLD_PTR(), BUF_POOL_ZIP_FOLD_BPAGE():
Macros for computing buf_pool->zip_hash fold values.  Make use of these
in buf0buddy.c.

buf0buddy.c: Replace some ptr == ut_align_down(ptr, ...) in assertions with
!ut_align_offset(ptr, ...).
2007-01-05 10:49:56 +00:00