Commit graph

692 commits

Author SHA1 Message Date
marko
47da8abc14 branches/zip: Try to synchronize the updates of uncompressed and
compressed pages.

btr_root_raise_and_insert(): Distinguish root_page_zip and new_page_zip.

btr_cur_set_ownership_of_extern_field(): Do not log the write on the
uncompressed page if it will be logged for page_zip.

lock_rec_insert_check_and_lock(), lock_sec_rec_modify_check_and_lock():
Update the max_trx_id field also on the compressed page.

mlog_write_ulint(): Add UNIV_UNLIKELY hints.  Remove trailing white space.

mlog_log_string(): Remove trailing white space.

rec_set_field_extern_bits(): Remove parameter mtr, as the write will either
occur in the heap, or it will be logged at a higher level.

recv_parse_or_apply_log_rec_body(),
page_zip_write_header(): Add log record type MLOG_ZIP_WRITE_HEADER.

page_header_set_field(): Pass mtr=NULL to page_zip_write_header().

page_header_reset_last_insert(): Pass mtr to page_zip_write_header().

btr_page_set_index_id(), btr_page_set_level(),
btr_page_set_next(), btr_page_set_prev(): Pass mtr to page_zip_write_header().

row_upd_rec_sys_fields(): Pass mtr=NULL to page_zip_write_trx_id() and
page_zip_write_roll_ptr(), since the write will be logged at a higher level.

page_zip_write_header(): Add parameter mtr.
page_zip_write_header_log(): New function.

Remove rec_set_nth_field_extern_bit().
Make rec_set_nth_field_extern_bit_old() static.
Rename rec_set_nth_field_extern_bit_new()
to rec_set_field_extern_bits_new() and make it static.

row_ins_index_entry_low(): Remove bogus TODO comment.
2006-02-22 13:02:40 +00:00
marko
03a6c35b09 branches/zip: Remove temporary hooks for testing BLOB compression.
dict_mem_table_create(): Initialize table->zip.

btr0cur.c: Distinguish compressed BLOBs based on page_zip or table->zip.
2006-02-21 14:43:23 +00:00
marko
9d4fa728f4 branches/zip:
Fix the way how btr_free_externally_stored_field() is called in purge.

btr_free_externally_stored_field(): Add parameter field_ref that points
directly to the BLOB reference.  Use rec, offsets, page_zip, and i
only for the page_zip_write_blob_ptr() call.

row_purge_upd_exist_or_extern(): Do not assume that the undo log contains
the entire record.  Only pass the BLOB reference to
btr_free_externally_stored_field().
2006-02-21 14:15:11 +00:00
marko
0d4b8a3123 branches/zip: Fix bugs in the handling of externally stored columns.
btr_cur_pessimistic_insert(): Pass the "extern storage" flags to
btr_root_raise_and_insert() or btr_page_split_and_insert().

btr_copy_externally_stored_field(): Include local_len in the returned length.
2006-02-21 10:24:32 +00:00
marko
dcfc19c47c branches/zip: Merge revisions 79:200 from trunk. 2006-02-17 14:19:39 +00:00
marko
faa9d8b149 branches/zip:
btr_cur_optimistic_insert(): Replace page_cur_insert_rec_low()
with page_cur_tuple_insert().

btr_cur_pessimistic_update(): Add the columns of big_rec_vec to ext_vect,
so that the "extern storage" bits will be set correctly in
btr_cur_insert_if_possible().
2006-02-17 14:14:07 +00:00
marko
2e7817bfd8 lock_rec_move(): Compute receiver_heap_no correctly. 2006-02-17 13:44:16 +00:00
marko
44d80eaee6 branches/zip: row_purge_upd_exist_or_extern(): use the offsets correctly 2006-02-16 13:47:20 +00:00
marko
1b7710af93 branches/zip: Implement the compression of BLOB columns.
This has not been extensively tested yet, because some other part of the
code breaks in "ibtestblob".

btr_free_page_low(): Add parameters "space" and "page_no", because they
are omitted from compressed BLOB pages.

btr0cur.c: Implement the compression and decompression of BLOB columns,
enabled at compile-time (#define ZIP_BLOB TRUE) for now.

btr_rec_free_externally_stored_fields(),
btr_copy_externally_stored_field(): Made static

mlog_log_string(): New function, split from mlog_write_string(), allows
to avoid a dummy memcpy() of compressed BLOB pages.
2006-02-16 12:58:18 +00:00
marko
4b3be43c2a branches/zip: Remove in-place updates of the "external storage"
flag of records.  The flags may only be updated in heap-allocated
copies of records.

btr_root_raise_and_insert(),
btr_page_split_and_insert(),
btr_cur_insert_if_possible(),
btr_cur_optimistic_insert(),
btr_cur_pessimistic_insert(),
page_cur_tuple_insert(),
page_cur_insert_rec_low(): Add parameters "ext" and "n_ext".

dtuple_convert_big_rec(): Make parameter "ext" const.
2006-02-13 14:28:00 +00:00
marko
12050c5c1c branches/zip: Prepare for in-place updates of B-tree node pointers,
BLOB pointers, trx_id, and roll_ptr.

btr_empty(), btr_create(), page_create(): Add parameter "index", as some
index information will be encoded on the compressed page.

Define REC_NODE_PTR_SIZE as 4.

Allow btr_page_reorganize() and btr_page_reorganize_low() to fail.

Define the error code DB_ZIP_OVERFLOW.

Make row_ins_index_entry_low() static.

page0zip: Encode the index, log reorganized records, and store uncompressed
fields separately from the compressed data stream.
2006-02-10 15:06:17 +00:00
marko
d5f33dafd8 branches/zip: Prepare for upcoming implementation of page_zip_compress()
that will require complete index information.

dict_create_index_step(): invoke dict_index_add_to_cache() before btr_create()

dict_index_remove_from_cache(): make public

dict_index_get_if_in_cache_low(): new function, for holding dict_sys->mutex

buf_flush_init_for_writing(): remove the temporary hook to page_zip_compress()

page_create(): add temporary hook to page_zip_compress()
2006-02-03 14:44:29 +00:00
marko
3a55efc44c branches/zip:
buf_flush_init_for_writing(): The reported dense page directory size was
4 bytes too much.  Subtract 2 (infimum and supremum) from n_heap.

page_zip_decompress(): When decompressing the last user record, only set
heap_no and the status bits if there is data to decode, i.e., there
are user records on the page.
2005-11-30 23:09:36 +00:00
marko
82c74a7535 branches/zip: Minor improvements for debugging and testing purposes.
buf_flush_init_for_writing(): Improve the diagnostics and make the
condition for skipping pages accurate.

univ.i: Introduce UNIV_ZIP_DEBUG for enabling some page_zip_validate() tests.

page0zip.h, page0zip.c: Define and use page_zip_validate() in
page_zip_compress() and page_zip_write() if UNIV_ZIP_DEBUG or UNIV_DEBUG
is defined.
2005-11-30 13:57:54 +00:00
marko
fe9e164f49 branches/zip: Merge revisions 72:79 from trunk. 2005-11-30 13:29:31 +00:00
marko
0bf7f4bd1d branches/zip: Merge revisions 9:72 from trunk. 2005-11-29 12:30:46 +00:00
marko
541fcfcd45 branches/zip: More fixes to problems found in speedc test runs.
Before the speedc test was interrupted, 121,765 B-tree pages were written.

buf_flush_init_for_writing(): Do not compress other than B-tree pages
outside the system tablespace.  Report non-B-tree pages.

page_zip_decompress(): Clear the unused heap space on the uncompressed page,
so that the whole buffer for the uncompressed page will be initialized and
page_zip_validate() will always succeed.
2005-11-29 10:24:45 +00:00
marko
de60fbdfe3 branches/zip: Fixes to allow the speedc test to run without assertion failures.
page_zip_dir_decode(): Correct the handling of the free list.

page_zip_set_extra_bytes(): Remove off-by-one error in the first loop.

page_zip_apply_log(): Replace parameter end with size, and let end=data+size.

page_zip_decompress(): Properly handle pages where the first user record has
more than REC_N_NEW_EXTRA_BYTES extra bytes.

page_delete_rec_list_end(): Remove page_zip_temp.  This operation will be
done completely in-place on page_zip.  If page_zip is specified, clear the
data bytes and the info and status bits of deleted records and clear the
removed directory slots.

page_dir_delete_slot(): Clear the last directory slot, which will be removed.
2005-11-28 11:32:52 +00:00
marko
818d12292a branches/zip: Minor fixes. Now one non-trivial page compresses and
decompresses properly.

page_mem_free(): Adjust the temporary condition for zeroing data.

page_zip_compress(): Do not accept Z_STREAM_END from non-final deflate().
Tighten the assertion on c_stream.avail_in.

page_zip_dir_sort(): Note that the high bound is not inclusive.

page_zip_dir_decode(): Remove off-by-one error in page_zip_dir_sort() call.

page_zip_set_extra_bytes(): Add the termination condition that was forgotten
from the second for loop.  Simplify the code a little.

page_zip_decompress(): Do not accept Z_STREAM_END from non-final inflate().
Tighten the assertion on d_stream.avail_out.  Increment heap_status correctly.
2005-11-25 14:17:40 +00:00
marko
d1f9c8ecea branches/zip: Minor fixes to get a page with one record compress/decompress.
A page with multiple records or deleted records still does not compress
or decompress properly.

buf_flush_init_for_writing(): Initialize block->page_zip properly so that all
assertions in page0zip can be enabled.

page_zip_decompress(): Note that corrupt data should not lead to assertions.

page_zip_dir_set(): Correct the interface.  Fix off-by-one error.
page_zip_dir_get(): Fix off-by-one error.

page0zip.c: Replace n_heap with n_dense and add comments about
the infimum and supremum records whenever we subtract 2 from heap_no.
Fix some programming errors.
2005-11-25 12:34:38 +00:00
marko
13002bdffc branches/zip: Minor fixes. Compression/decompression still fails.
buf0flu.c: Allocate the temporary buffer from buf_frame_alloc().

page_zip_simple_validate(): Do not assert page_zip->m_start >= PAGE_DATA.

page_zip_compress(): Replace some assertions with page_zip_simple_validate(),
and do not assert anything about page_zip->data contents.

page_zip_validate(): Do not compare the page trailer bytes.

page_zip_write(): Assert that the entire page headers match and
that page_zip->m_start >= PAGE_DATA.
2005-11-25 07:51:28 +00:00
marko
48a1ffa5f4 branches/zip: Implement a more compact page format.
Add a hook to buf0flu.c for testing compression and decompression.
TODO: adapt page_zip_write() calls.
2005-11-24 14:13:10 +00:00
marko
b0c0c2b393 branches/zip: Split page_zip_apply_log() from page_zip_decompress(). 2005-11-18 07:42:02 +00:00
marko
10fffd0dc5 branches/zip: Remember to update page_zip when updating "extern" flags.
Add function parameters and TODO comments regarding that.
2005-11-18 07:40:34 +00:00
marko
ab61377fd1 branches/zip: Attach TODO comments to added btr_page_reorganize() calls 2005-11-15 11:00:37 +00:00
marko
b0b4936341 branches/zip: Minor improvements.
page_dir_slot_set_n_owned(): Check page_zip_available().
page_dir_add_slots(): Add parameter page_zip.
btr_attach_half_pages(): Add parameter page_zip.
btr_node_ptr_set_child_page_no(): Add parameter page_zip.
Make comments about page_zip modification log usage more accurate.
2005-11-14 15:46:12 +00:00
marko
2f435d29f9 branches/zip: Minor improvements.
page_dir_set_n_slots(): Add debug assertion that there will be enough space
available in the modification log of the compressed page.
page_trailer_get_len(): Simplify the formula.  Make more use of the function.
2005-11-14 10:40:02 +00:00
marko
d8f576e734 branches/zip: Minor improvements.
page_set_max_trx_id(), page_update_max_trx_id(): Add parameter page_zip.

Check that page_zip_decompress() is never called after modifying the
header or trailer of the compressed page, i.e., that page_zip_decompress()
will restore the uncompressed page as it was before the failed operation.
2005-11-11 21:18:42 +00:00
marko
83539afab4 branches/zip: Restore left b-tree splits.
page_move_rec_list_start(): Restored.  Reorganize old page if compressed.
btr_page_split_and_insert(): Do left page splits.
2005-11-11 13:59:40 +00:00
marko
3bf12fb47e branches/zip:
buf_flush_buffered_writes(): Remove bogus assertion failure.
2005-11-10 09:31:10 +00:00
marko
61f211489a branches/zip: Bug fixes and improvements.
page_header_reset_last_insert(): Add page_zip parameter.
rec_get_next_ptr(): Return a pointer to the same page, or NULL.
page_dir_delete_slot(): Fix off-by-one errors.
2005-11-09 15:43:36 +00:00
marko
18f8313aa7 branches/zip:
Fix some (not all) bugs found by testing.
2005-11-07 14:24:37 +00:00
marko
5ec88a30dc branches/zip:
Fix bugs found during testing.
2005-11-04 12:49:39 +00:00
marko
dbfea3e935 branches/zip:
Add page_zip parameter to page_dir_slot_set_rec().
Improve the documentation of the page_zip parameters in page0page.
Add function page_trailer_get_len().
2005-11-04 08:50:25 +00:00
marko
9e21c7fecd branches/zip:
Slightly improve error handling.
2005-11-02 07:51:47 +00:00
marko
1d5d3f2ccd branches/zip:
Implement page_zip_write_header().
Add page_zip parameter to btr_page_set_ functions.
Allow page_copy_rec_list_end() to be called with invalid new_page_zip
if the page has been newly created.
2005-10-28 10:45:20 +00:00
marko
b34b37b502 branches/zip:
Initial import of the zip-like compression of B-tree pages.
2005-10-27 11:48:10 +00:00
marko
4979fcaefe Create zip branch (LZ77 based compression of B-tree index pages). 2005-10-27 11:33:12 +00:00
osku
000bff7df4 include/Makefile.am: Remove reference to removed makefilewin.i. 2005-10-27 09:37:17 +00:00
osku
e653f84f3f Add 5.0 -> 5.1 changes. 2005-10-27 07:51:34 +00:00
osku
2903c72469 Import 5.0 code. 2005-10-27 07:29:40 +00:00
osku
3de9a53e2f Create directory structure. 2005-10-27 07:02:45 +00:00