Commit graph

694 commits

Author SHA1 Message Date
marko
0954bbedb8 branches/zip: Fix bugs in merge sort.
row_merge_buf_add(): Correctly calculate buf->total_size.

row_merge_buf_write(): Add an assertion about buf->total_size.

row_merge_heap_create(), row_merge_read_rec(), row_merge_insert_index_tuples():
offsets needs to be 1 + REC_OFFS_HEADER_SIZE + n_fields.  Previously, we
allocated one element too little.

row_merge_blocks(): Use goto instead of break in the AT_END argument
of ROW_MERGE_WRITE_GET_NEXT.
2007-06-08 20:30:26 +00:00
marko
ffe5998e4a branches/zip: row_merge_build_indexes(): Allocate all buffers with a
single os_mem_alloc_large() call.  The function mem_alloc() that was
used previously allocates unaligned memory.

row_merge_blocks(): Replace block1, block2, block3 with a single array.
Replace buf1, buf2, buf3 with buf[3].  Replace b1..b3 with b0..b2.
Replace mrec1,2 with mrec0,1.  Replace offsets1,2 with offsets0,1.

row_merge(), row_merge_sort(): Replace block1, block2, block3 with a
single array.
2007-06-08 09:49:50 +00:00
marko
fd91f7bb76 branches/zip: Reimplement merge sort in fast index creation.
The creation of the primary key does not work.  We will have to flag
externally stored columns and copy the externally stored part from
the old table.

row_build_index_for_mysql(): Rename to row_merge_build_indexes().
Move from row0mysql.c to row0merge.c.

Remove private declarations from row0merge.h.  Make many functions static
in row0merge.c.

cmp_rec_rec_simple(): A new comparison function.

dict_index_get_min_size(): New function.

OS_FILE_FROM_FD(fd): A macro for converting from int to os_file_t.

rec_convert_dtuple_to_rec_comp(): Make the interface lower-level.

rec_get_converted_size_comp(): Return also extra_size.

UT_SORT_FUNCTION_BODY(): Remove reference to an obsolete test program.

row_rec_to_index_entry_low(): New function.

row0merge.c: Implement merge sort based on file streams instead of
fixed-size blocks.  Sort the small blocks as arrays of dfield_t*,
because it is faster than invoking rec_get_offsets() for every
comparison.
2007-06-08 07:37:07 +00:00
marko
2d90ae68e7 branches/zip: Merge 1553:1556 from trunk. 2007-06-08 07:05:17 +00:00
marko
d1bdef34ac branches/zip: innodb-index.test: Check the length of the long columns. 2007-06-07 19:57:34 +00:00
marko
07d004c46e branches/zip: Reduce memory allocation and pointer indirection
in fast index creation.

innobase_create_index_def(), innobase_copy_index_def(): Copy to pre-allocated
space.

innobase_create_key_def(): Change the return type from merge_index_def_t**
to merge_index_def_t*.
2007-06-07 18:29:57 +00:00
marko
f0c4bdbcea branches/zip: Minor cleanup.
innobase_create_index_def(): Add parameter new_primary.

innobase_copy_index_def(): Simplify the documented algorithm,
and try to implement it properly.

innodb-index.test: Replace CHECKSUM TABLE with something more stable and
useful.  The test passes on an older BitKeeper snapshot:

ChangeSet@1.2475.18.9, 2007-05-08 11:16:41+02:00, jbruehe@mysql.com +1 -0
  Raise version number after cloning 5.1.18-beta

But it fails on a newer one where the statement
	alter table t1 add primary key (a), add key (b(20));
results in fast index creation:

ChangeSet@1.2500.1.40, 2007-06-01 20:06:13+04:00, kostja@bodhi.(none) +2 -0
  Merge bodhi.(none):/opt/local/work/mysql-5.0-runtime
  into  bodhi.(none):/opt/local/work/mysql-5.1-runtime
  MERGE: 1.1810.2984.14
2007-06-07 12:23:26 +00:00
marko
c09c8a27d1 branches/zip: Document dict_rename_indexes() better. 2007-06-07 09:47:59 +00:00
marko
e337caa161 branches/zip: row_merge_read_clustered_index(): Correctly skip the page
infimum and supremum.  This was broken in r1515.
2007-06-07 08:14:42 +00:00
marko
0bcef79fb1 branches/zip: Clean up fast index creation API.
row_merge_create_temporary_table(), row_merge_create_index():
Return the dictionary object.  On error, set trx->error_state.
2007-06-06 20:25:35 +00:00
marko
7fbc04fcdc branches/zip: Merge 1533:1553 from trunk. 2007-06-06 11:07:37 +00:00
marko
1d2284c336 branches/zip: Minor cleanup.
row_merge_mark_prebuilt_obsolete(): Rename to row_prebuilt_table_obsolete()
and move from row0merge.c to row0mysql.c.  Remove the parameter trx.

row_merge_print_table(): Remove.
2007-06-04 07:51:10 +00:00
marko
d1bf0b7dab branches/zip: Minor cleanup.
row_remove_indexes_for_mysql(): Rename to row_merge_drop_indexes(),
move from row0mysql.c to row0merge.c and make the return type void.

row_merge_remove_index(): Rename to row_merge_drop_index() and make
the return type void.
2007-06-04 07:49:31 +00:00
marko
d75e64f98a branches/zip: Merge 1493:1533 from trunk.
Make use of UT_BITS_IN_BYTES().
2007-05-29 08:48:16 +00:00
marko
2379a7e3e9 branches/zip: Revert r1523. Passing extra parameters to SORT_FUN and CMP_FUN
of UT_SORT_FUNCTION_BODY is best done by defining SORT_FUN and CMP_FUN as
macros when needed.  The solution of r1523 allows for only one extra parameter.
2007-05-28 07:42:28 +00:00
marko
6eb0d1ad53 branches/zip: UT_SORT_FUNCTION_BODY(): Add the parameter CTX, which will be
needed in row0merge.c for merge sorting the small blocks in main memory.
Pass CTX also to SORT_FUN.  Adjust all users.
2007-05-24 13:14:57 +00:00
marko
f1706a1355 branches/zip: rec_get_offsets_reverse(): Fix the type of the parameter
node_ptr.
2007-05-22 11:55:07 +00:00
marko
2a77ce2769 branches/zip: Refactor some code for handling records in ROW_FORMAT=COMPACT.
rec_get_converted_size_new(): Rename to rec_get_converted_size_comp().
Add const qualifier to dtuple.  Remove the assertion
dict_table_is_comp(index->table), because this function will be invoked
when populating the temporary files in fast index creation.

rec_convert_dtuple_to_rec_new(): Rename to rec_convert_dtuple_to_rec_comp().
Add parameter "extra".

rec_init_offsets_comp_ordinary(): New function.  Used in rec_init_offsets().
2007-05-22 11:12:35 +00:00
marko
ac6fb591bb branches/zip: Rename dtuple_datas_are_ordering_equal() to dtuple_coll_cmp(),
and change the return type from ibool to int (-1, 0, 1).  This comparison
function will be needed in fast index creation.
2007-05-22 09:34:25 +00:00
marko
3f7aa60c81 branches/zip: Correct the terminology: s/cluster index/clustered index/.
row_merge_read_clustered_index(): Simplify the loop that scans the
clustered index.
2007-05-22 06:48:53 +00:00
marko
da1b4f9c39 branches/zip: Implement INFORMATION_SCHEMA.INNODB_ZIP_RESET, which is
otherwise like INNODB_ZIP, but resets the cumulated counts.

Rename innodb_buddy to innodb_zip in some comments.

innobase_stat_zip_fill(): Add parameter "reset".  Document the race condition
with the compression statistics counters.
2007-05-21 09:35:13 +00:00
marko
3cbaab0b2a branches/zip: Minor cleanup.
trx_undo_report_row_operation(), trx_undo_report_dict_operation():
Reduce the scope of some variables.  Move the return(DB_SUCCESS)
case inside the for loop.
2007-05-18 13:00:53 +00:00
marko
3970397157 branches/zip: row_merge_block_merge(): Add an assertion against buffer
overflow.
2007-05-18 11:37:05 +00:00
marko
a8b6524b14 branches/zip: Use ut_dulint_zero and ut_dulint_is_zero whenever possible. 2007-05-18 11:01:58 +00:00
marko
f88827d806 branches/zip: Fix some minor things in row0uins.c after comparing to trunk.
Replace row0merge.h with row0mysql.h.

Remove redundant test trx->dict_operation_lock_mode == 0.
Fix typos in comments.

row_undo_ins(): Make the function more similar to the one in trunk.
2007-05-18 10:53:29 +00:00
marko
6613d53767 branches/zip: btr_cur_optimistic_insert(): Set the insert buffer free bits
exactly.  Rename the local variable "ulint level" to "ibool leaf".
Document that if the function returns DB_SUCCESS on a compressed page that
is covered by the insert buffer, the mini-transaction must be committed
before latching any further pages.  Verify that this is the case on all
execution paths.
2007-05-16 12:01:31 +00:00
marko
36e875f7a5 branches/zip: Document and obey the rules for modifying the free bits in
the insert buffer bitmap.

ibuf_set_free_bits_func(): Never disable redo logging.

ibuf_update_free_bits_zip(): Remove.

btr_page_reorganize_low(), page_zip_reorganize(): Do not update the insert
buffer bitmap.  Instead, document that callers will have to take care of it,
and adapt the callers.

btr_compress(): On error, reset the insert buffer free bits.

btr_cur_insert_if_possible(): Do not modify the insert buffer bitmap.

btr_compress(), btr_cur_optimistic_insert(): On compressed pages,
reset the insert buffer bitmap.  Document why.

btr_cur_update_alloc_zip(): Document why it is necessary and sufficient
to reset the insert buffer free bits.

btr_cur_update_in_place(), btr_cur_optimistic_update(),
btr_cur_pessimistic_update(): Update the free bits in the same
mini-transaction.  Document that the mini-transaction must be
committed before latching any further pages.  Verify that this
is the case in all execution paths.

row_ins_sec_index_entry_by_modify(), row_ins_clust_index_entry_by_modify(),
row_undo_mod_clust_low(): Because these functions call
btr_cur_update_in_place(), btr_cur_optimistic_update(), or
btr_cur_pessimistic_update(), document that the mini-transaction must be
committed before latching any further pages.  Verify that this is the case
in all execution paths.
2007-05-16 09:23:53 +00:00
marko
19569a8b4a branches/zip: Minor cleanup.
page_cur_tuple_insert(), page_cur_rec_insert(): Improve the documentation.
Note that the physical position of page_cursor may change.

page_cur_rec_insert(), page_cur_insert_rec_low(), page_cur_insert_rec_zip():
Add a const qualifier to the parameter rec.
2007-05-15 09:31:41 +00:00
marko
e7284367dc branches/zip: Merge revisions 1402:1493 from trunk.
Remove mysql.patch, because the configuration parameter interface has changed.
2007-05-14 09:07:15 +00:00
marko
fbc6f37cb1 branches/zip: Fix two bugs.
mtr_commit(): Do not acquire the log mutex if no log records are to be written.

ibuf_set_free_bits_func(): Do not clear mtr.modifications, because that would
prevent the dirty insert buffer bitmap block from being moved to the flush
list.  Instead, assume that mtr_commit() will not acquire log_sys->mutex
because redo logging is disabled.

buf_page_init_for_read(): I/O-fix the block before buf_buddy_alloc(), because
the newly allocated block could otherwise be reused by the compressed page.
2007-05-07 05:53:42 +00:00
marko
dd529ee402 branches/zip: When reading pages for merging buffered inserts, request that
the pages be decompressed.

buf_read_page_low(), buf_page_init_for_read(): Add parameter "unzip".
2007-05-06 13:21:59 +00:00
marko
2e73922495 branches/zip: ibuf_set_free_bits_func(): Disable redo logging when
crash recovery is in progress.  This avoids a hang when
btr_parse_page_reorganize(), called from an I/O handler thread,
attempts to acquire log_sys->mutex while it is being held by
the main thread (the one that runs innobase_init()).  This change
was committed accidentally.  It may be unsafe to clear
mtr.modifications, because buf_page_release() at mtr_commit() may
forget to put modified pages to the flush list.

Cleanup: Remove the "type" parameter from many ibuf functions.
Let the caller check that !dict_index_is_clust().  This should avoid
function calls and register spilling.

ibuf_set_free_bits_func(), ibuf_set_free_bits(): Remove the parameter "type".

ibuf_reset_free_bits_with_type(): Rename to ibuf_reset_free_bits().
Remove the parameter "type".

ibuf_update_free_bits_if_full(), ibuf_update_free_bits_zip(),
ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low():
Remove the parameter "index".
2007-05-06 12:39:46 +00:00
marko
87e42d059a branches/zip: buf_page_io_query(): Shorten the code by using a local variable. 2007-05-06 12:30:10 +00:00
marko
fdb017ff69 branches/zip: Minor cleanup.
log_write_up_to(): Replace if-else if-else with a switch block.

log_archive_do(): Remove an excess mutex_enter().  Replace if-else if-else
with a switch block.  (This function is not included unless UNIV_LOG_ARCHIVE
is defined.)
2007-05-06 12:28:05 +00:00
marko
08582d6d94 branches/zip: Improve the documentation of UNIV_DEBUG. 2007-05-06 12:05:21 +00:00
marko
cbdfb4785a branches/zip: Correct the comment of row_merge_cmp(). 2007-05-03 07:42:54 +00:00
marko
7e4f4db6bb branches/zip: Replace the table attribute AVG_ROW_LENGTH (ugly hack to
pass the compressed page size) with KEY_BLOCK_SIZE.
2007-05-02 11:58:45 +00:00
marko
dfcb2c48ee branches/zip: row_merge_sort_linked_list(): Add a default: ut_error; branch
to the switch (row_merge_cmp(...)) to eliminate bogus warning about
uninitialized variable tmp.
2007-04-30 07:02:28 +00:00
marko
2ec8ec2418 branches/zip: Apply a fix from Sunny. The function row_merge_block_merge()
was broken in one of the earlier cleanups.

row_merge_block_merge(): Remove the variable "selected".  Clear the record
count of new_block1 and new_block2.  Anticipate the right return values
from row_merge_cmp().
2007-04-30 06:25:47 +00:00
marko
e0cb06f84f branches/zip: Minor cleanup.
Add __attribute__((const)) or __attribute__((pure)) to various
mach_read functions.

dtype_new_read_for_order_and_null_size(): Remove a statement
without effect: mach_read_from_2(buf + 4).
2007-04-17 12:25:51 +00:00
marko
a55e2191dc branches/zip: Reintroduce the local variables "sorted" that were removed
in r1445.  They are needed after all, but apparently the tests do not
actually verify that the records are sorted.  (Also, a buffer overflow
will occur in index_merge_innodb.test if MERGE_BLOCK_SIZE is made smaller.)
2007-04-17 09:44:29 +00:00
marko
93558a3ba7 branches/zip: Clean up fast index creation.
Correct the language of some comments.  Eliminate the local variables
"num_of_merges" (renamed to "sorted" in one function).
2007-04-17 08:53:47 +00:00
marko
65a34315bf branches/zip: Clean up.
cmp_rec_rec_with_match(): Initialize ret=0.  Remove assignments ret=0.

rec_init_offsets(): Remove bogus comment.
2007-04-17 08:21:28 +00:00
marko
b90b4f59b4 branches/zip: Make use of the function dict_index_is_unique(). 2007-04-16 09:21:39 +00:00
marko
758858549c branches/zip: Clean up fast index creation.
dict_index_is_unique(): New function.

row_merge_rec_fits_to_block(), row_merge_store_rec_to_block():
Add const qualifiers to rec, offsets.

row_merge_select(): Rename to row_merge_cmp(), simplify the interface.

row_merge_sort_linked_list(): Change the return type to ibool.
Replace sec_offs1,sec_offs2 with offsets1,offsets2.  Replace offset_heap
with heap.  Define a func_exit label.

row_merge_block_merge(): Reduce the scope of rec_offsets.
2007-04-16 09:13:44 +00:00
marko
f1ac868e15 branches/zip: Clean up fast index creation.
merge_rec_list_t: Define n_records only #ifdef UNIV_DEBUG.
2007-04-16 07:11:00 +00:00
marko
f05204f9fd branches/zip: Clean up fast index creation.
row_merge_dict_table_get_index(): Add function comment.
Use inlined versions of mem_alloc() and mem_free().
2007-04-16 07:10:00 +00:00
marko
a05c36c24b branches/zip: fast index creation: Check the status of os_file_read()
and os_file_write(), and return DB_CORRUPTION when they fail.  Minor cleanup.

row_mege_insert_index_tuples(): Use a shared heap for dtuple and rec.
Use fewer gotos.

row_merge_sort_linked_list_in_disk(): Free the merge blocks at error exit.
2007-04-13 12:11:43 +00:00
marko
4e14254e3c branches/zip: Clean up fast index creation.
row_merge_read(), row_merge_write(): Add UNIV_LIKELY hints.

row_merge_sort_linked_list_in_disk(): Reduce the scope of many variables.
Eliminate the variable "selected".
2007-04-12 12:58:09 +00:00
marko
13817f1d52 branches/zip: Minor cleanup of merge_file_t and merge_block_header_t.
Replace byte offsets of type dulint with block offsets of type ulint.
Avoid references to MERGE_BLOCK_SIZE.

Improve the language of some comments.

row_merge_insert_index_tuples(), row_merge_sort_linked_list_in_disk():
Make the offset a ulint.

row_merge_read(), row_merge_write(): Helper functions.  Return the
status of os_file_read() and os_file_write().  TO DO: check the status
in the callers.
2007-04-12 10:32:16 +00:00