Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
maria_read_log used to always print a warning message at startup
to say it is unsafe if ALTER TABLE was used. Now it prints it only
if the log does show the problem (=ALTER TABLE or CREATE SELECT, which
both disable logging of REDO_INSERT*).
For that, when ha_maria::external_lock() disables transactionality
it writes a LOGREC_INCOMPLETE_LOG to the log, which "maria_read_log -a"
picks up to write a warning.
REPAIR TABLE also disables those REDO_INSERT* but as maria_read_log
executes LOGREC_REDO_REPAIR no warning is needed.
storage/maria/ha_maria.cc:
as we now log a record when disabling transactionility, we need the
TRN to be set up first
storage/maria/ma_blockrec.c:
comment
storage/maria/ma_loghandler.c:
new type of log record
storage/maria/ma_loghandler.h:
new type of log record
storage/maria/ma_recovery.c:
* maria_apply_log() now returns a count of warnings. What currently
produces warnings is:
- skipping applying UNDOs though there are some (=> inconsistent table)
- replaying log (in maria_read_log) though the log contains some
ALTER TABLE or CREATE SELECT (log misses REDO_INSERT* for those
and is so incomplete).
Count of warnings affects the final message of maria_read_log and
recovery (though in recovery none of the two conditions above should
happen).
* maria_read_log used to always print a warning message at startup
to say it is unsafe if ALTER TABLE was used. Now it prints it only
if the log does show the problem, i.e. ALTER TABLE or CREATE SELECT
was used (both disable logging of REDO_INSERT* as those records are
not needed for recovery; those missing records in turn make
recreation-from-scratch, via maria_read_log, impossible). For that,
when ha_maria::external_lock() disables transactionality,
_ma_tmp_disable_logging_for_table() writes a LOGREC_INCOMPLETE_LOG to
the log, which maria_apply_log() picks up to write a warning.
storage/maria/ma_recovery.h:
maria_apply_log() returns a count of warnings
storage/maria/maria_def.h:
_ma_tmp_disable_logging_for_table() grows so becomes a function
storage/maria/maria_read_log.c:
maria_apply_log can now return a count of warnings, to temper the
"SUCCESS" message printed in the end by maria_read_log.
Advise users to make a backup first.
storage/maria/ma_loghandler.c:
Infinite loop in case of given address more then last LSN fixed
(now it means just flush all log).
Removed unneeded ASSERT.
storage/maria/unittest/ma_test_loghandler-t.c:
The test case of flushing all log added.
into mysql.com:/home/my/mysql-maria
include/my_sys.h:
Auto merged
mysql-test/r/maria.result:
Auto merged
mysql-test/t/maria.test:
Auto merged
sql/handler.h:
Auto merged
sql/mysqld.cc:
Auto merged
storage/maria/ha_maria.cc:
Auto merged
storage/maria/ma_bitmap.c:
Auto merged
storage/maria/ma_blockrec.c:
Auto merged
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/ma_pagecache.c:
Auto merged
storage/maria/ma_test1.c:
Auto merged
storage/maria/ma_test_recovery.expected:
Auto merged
storage/maria/ma_test_recovery:
Auto merged
sql/mysql_priv.h:
manual merge
storage/maria/ma_recovery.c:
manual merge
storage/maria/ma_test2.c:
manual merge
Changed format for REDO_INSERT_ROWS_BLOBS
Fixed several bugs in handling of big blobs
Added redo_free_head_or_tail() & redo_insert_row_blobs()
Added uuid to control file
maria_checks now verifies that not used part of bitmap is 0
REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
Added REDO_FREE_HEAD_OR_TAIL
Fixes problem when trying to read block outside of file during REDO
include/my_global.h:
STACK_DIRECTION is already set by configure
mysql-test/r/maria.result:
Updated results
mysql-test/t/maria.test:
Test shrinking of VARCHAR
mysys/my_realloc.c:
Fixed indentation
mysys/safemalloc.c:
Fixed indentation
sql/filesort.cc:
Removed some casts
sql/mysqld.cc:
Added missing setting of myisam_stats_method_str
sql/uniques.cc:
Removed some casts
storage/maria/ma_bitmap.c:
Added printing of bitmap (for debugging)
Renamed _ma_print_bitmap() -> _ma_print_bitmap_changes()
Added _ma_set_full_page_bits()
Fixed bug in ma_bitmap_find_new_place() (affecting updates) when using big files
storage/maria/ma_blockrec.c:
Changed format for REDO_INSERT_ROWS_BLOBS
Fixed several bugs in handling of big blobs
Added code to fix some cases where redo when using blobs didn't produce idenital .MAD files as normal usage
REDO_FREE_ROW_BLOCKS doesn't anymore change pages; We only mark things free in bitmap
Remove TAIL and filler extents from REDO_FREE_BLOCKS log entry. (Fixed some asserts)
REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
Delete tails in update. (Fixed bug when doing update that shrinks blob/varchar length)
Fixed bug when doing insert in block outside of file size.
Added redo_free_head_or_tail() & redo_insert_row_blobs()
Added pagecache_unlock_by_link() when read fails.
Much more comments, DBUG and ASSERT entries
storage/maria/ma_blockrec.h:
Prototypes of new functions
Define of SUB_RANGE_SIZE & BLOCK_FILLER_SIZE
storage/maria/ma_check.c:
Verify that not used part of bitmap is 0
storage/maria/ma_control_file.c:
Added uuid to control file
storage/maria/ma_loghandler.c:
REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_loghandler.h:
REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_pagecache.c:
If we write full block, remove error flag for block.
(Fixes problem when trying to read block outside of file)
storage/maria/ma_recovery.c:
REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_test1.c:
Allow option after 'b' to be compatible with ma_test2
(This is just to simplify test scripts like ma_test_recovery)
storage/maria/ma_test2.c:
Default size of blob is now 1000 instead of 1
storage/maria/ma_test_all.sh:
Added test for bigger blobs
storage/maria/ma_test_recovery.expected:
Updated results
storage/maria/ma_test_recovery:
Added test for bigger blobs
on the loghandler start.
Variable definition moved because it is C programm.
storage/maria/ma_loghandler.c:
Checking that the very last record is fully written
on the loghandler start.
storage/maria/ma_recovery.c:
Variable definition moved because it is C programm.
- fixes (in recovery, checkpoint, log handler) of bugs found
during testing.
- new option --check for maria_read_log: with --only-display (which only
reads the header) it reads the full record, for debugging.
storage/maria/ma_loghandler.c:
importing patch from Sanja for bug of translog_next_LSN() found
during recovery
storage/maria/ma_loghandler_lsn.h:
better types (0L is 4 bytes on some platforms, it causes problems
when used into lsn_store(): right shift >= width of type.
storage/maria/ma_pagecache.c:
work around infamous "PAGECACHE_PLAIN_PAGE used for transactional
tables in specialm case"; REDO phase disables logging and this causes
pages to be PAGECACHE_PLAIN_PAGE, thus ignored wrongly by the
checkpoint taken at the end of the REDO phase.
storage/maria/ma_recovery.c:
- a #ifdef which broke maria_read_log in non-debug builds (no output!)
- support for maria_read_log --check
- detect record corruption before opening the table
- updating is_of_horizon requires writing the state
- fix for wrong parsing of checkpoint record by recovery
storage/maria/ma_recovery.h:
support for maria_read_log --check
storage/maria/maria_read_log.c:
Option --check: --only-display only looks at the header;
adding --check tries a translog_read_record() to see if record can
be fully read (this is to find bugs).
Added ability top change empty space filler of the loghandler.
Fixed end of log reaction.
Fixed memory corruprion bug caused by reading non-filled hage header.
Added debug output.
storage/maria/ma_bitmap.c:
Compiler warnings removed.
storage/maria/ma_blockrec.c:
Compiler warnings removed.
storage/maria/ma_loghandler.c:
Added ability top change empty space filler of the loghandler.
Fixed end of log reaction.
Fixed memory corruprion bug caused by reading non-filled hage header.
Added debug output.
storage/maria/ma_loghandler_lsn.h:
Compiler warnings removed.
moved debugging defines.
Fixed buffer flushing (we copied last page
content before it was actually written, now
we abvance pointer in new buffer and unlock
it while waiting for filling old buffer)
Misc changes:
- fix for benign Valgrind error, compiler warnings
- fix for a segfault in execution of maria_delete_all_rows() and one
when taking multiple checkpoints
- fix for too paranoid assertion
- adding ability to take checkpoints at the end of the REDO phase
and at the end of recovery.
- other minor changes
storage/maria/ha_maria.cc:
The checkpoint done after Recovery is finished, is moved to
maria_recover().
storage/maria/ma_bitmap.c:
fix for Valgrind error: the "shadow debug copy" of the bitmap page
started unitialized and so ma_print_bitmap() would use it uninitialized
storage/maria/ma_checkpoint.c:
* reset pointers to NULL after freeing them, or we segfault at
next checkpoint in my_realloc().
* fix for compiler warnings.
storage/maria/ma_delete_all.c:
info->trn is NULL for non-transactional tables
storage/maria/ma_locking.c:
correct assertion (it fired wrongly in execution of REDO_DROP_TABLE
due to the maria_extra(HA_PREPARE_FOR_DROP)->_ma_decrement_open_count()
->maria_lock_database(F_UNLCK); another solution would have been to
not call _ma_decrement_open_count() (it's ok to have a wrong open
count in a table which we are dropping), but the same problem
would still exist for REDO_RENAME_TABLE.
storage/maria/ma_loghandler.c:
fail early if UNRECOVERABLE_ERROR
storage/maria/ma_recovery.c:
* new argument to maria_apply_log(): should it take checkpoints
(at end of REDO phase and at the very end) or no.
* moving the call to translog_next_LSN() into
parse_checkpoint_record() ("hide the details").
* Refining an error detection for something which could happen
if there is a checkpoint record in the log.
* Using close_one_table() instead of maria_extra(HA_EXTRA_PREPARE_FOR_DROP|RENAME),
as it looks safer, and also changing how close_one_table() works:
it now limits itself to scanning all_tables[], thus having one loopp
instead of two, which should be faster (as a result, it does not
close tables not registered in this array, which is ok as there
should not be any).
storage/maria/ma_recovery.h:
new parameter
storage/maria/maria_read_log.c:
update to new prototype
Debug output information fixed.
Fixed direct page referencing in write mode.
storage/maria/ma_loghandler.c:
TODO added.
storage/maria/ma_pagecache.c:
Debug output information fixed.
Fixed direct page referencing in write mode.
* Recovery of the table's live checksum (CREATE TABLE ... CHECKSUM=1)
is achieved in this patch. The table's live checksum
(info->s->state.state.checksum) is updated in inwrite_rec_hook's
under the log mutex when writing UNDO_ROW_INSERT|UPDATE|DELETE
and REDO_DELETE_ALL. The checksum variation caused by the operation
is stored in these UNDOs, so that the REDO phase, when it sees such
UNDOs, can update the live checksum if it is older (state.is_of_lsn is
lower) than the record. It is also used, as a nice add-on with no
cost, to do less row checksum computation during the UNDO phase
(as we have it in the record already).
Doing this work, it became pressing to move in-write hooks
(write_hook_for_redo() et al) to ma_blockrec.c.
The 'parts' argument of inwrite_rec_hook is unpredictable (it comes
mangled at this stage, for example by LSN compression) so it is
replaced by a 'void* hook_arg', which is used to pass down information,
currently only to write_hook_for_clr_end() (previous undo_lsn and
type of undone record).
* If from ha_maria, we print to stderr how many seconds (with one
fractional digit) the REDO phase took, same for UNDO phase and for
final table close. Just to give an indication for debugging and maybe
also for Support.
storage/maria/ha_maria.cc:
question for Monty
storage/maria/ma_blockrec.c:
* log in-write hooks (write_hook_for_redo() etc) move from
ma_loghandler.c to here; this is natural: the hooks are coupled
to their callers (functions in ma_blockrec.c).
* translog_write_record() now has a new argument "hook_arg";
using it to pass down to write_hook_for_clr_end() the transaction's
previous_undo_lsn and the type of the being undone record, and also
to pass down to all UNDOs the live checksum variation caused by the
operation.
* If table has live checksum, store in UNDO_ROW_INSERT|UPDATE|DELETE
and in CLR_END the checksum variation ("delta") caused by the
operation. For example if a DELETE caused the table's live checksum
to change from 123 to 456, we store in the UNDO_ROW_DELETE, in 4 bytes,
the value 333 (456-123).
* Instead of hard-coded "1" as length of the place where we store
the undone record's type in CLR_END, use a symbol CLR_TYPE_STORE_SIZE;
use macros clr_type_store and clr_type_korr.
* write_block_record() has a new parameter 'old_record_checksum'
which is the pre-computed checksum of old_record; that value is used
to update the table's live checksum when writing UNDO_ROW_UPDATE|CLR_END.
* In allocate_write_block_record(), if we are executing UNDO_ROW_DELETE
the row's checksum is already computed.
* _ma_update_block_record2() now expect the new row's checksum into
cur_row.checksum (was already true) and the old row's checksum into
new_row.checksum (that's new). Its two callers, maria_update() and
_ma_apply_undo_row_update(), honour this.
* When executing an UNDO_ROW_INSERT|UPDATE|DELETE in UNDO phase, pick
up the checksum delta from the log record. It is then used to update
the table's live checksum when writing CLR_END, and saves us a
computation of record.
storage/maria/ma_blockrec.h:
in-write hooks move from ma_loghandler.c
storage/maria/ma_check.c:
more straightforward size of buffer
storage/maria/ma_checkpoint.c:
<= is enough
storage/maria/ma_commit.c:
new prototype of translog_write_record()
storage/maria/ma_create.c:
new prototype of translog_write_record()
storage/maria/ma_delete.c:
The row's checksum must be computed before calling(*delete_record)(),
not after, because it must be known inside _ma_delete_block_record()
(to update the table's live checksum when writing UNDO_ROW_DELETE).
If deleting from a transactional table, live checksum was already updated
when writing UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
@todo is now done (in ma_loghandler.c)
storage/maria/ma_delete_table.c:
new prototype of translog_write_record()
storage/maria/ma_loghandler.c:
* in-write hooks move to ma_blockrec.c.
* translog_write_record() gets a new argument 'hook_arg' which is
passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
for those hooks, because when those hooks are called, 'parts' has
possibly been mangled (like with LSN compression) and is so
unpredictable.
* fix for compiler warning (unused buffer_start when compiling without
debug support)
* Because checksum delta is stored into UNDO_ROW_INSERT|UPDATE|DELETE
and CLR_END, but only if the table has live checksum, these records
are not PSEUDOFIXEDLENGTH anymore, they are now VARIABLE_LENGTH (their
length is X if no live checksum and X+4 otherwise).
* add an inwrite_rec_hook for UNDO_ROW_UPDATE, which updates the
table's live checksum. Update it also in hooks of UNDO_ROW_INSERT|
DELETE and REDO_DELETE_ALL and CLR_END.
* Bugfix: when reading a record in translog_read_record(), it happened
that "length" became negative, because the function assumed that
the record extended beyond the page's end, whereas it may be shorter.
storage/maria/ma_loghandler.h:
* Instead of hard-coded "1" and "4", use symbols and macros
to store/retrieve the type of record which the CLR_END corresponds
to, and the checksum variation caused by the operation which logs the
record
* translog_write_record() gets a new argument 'hook_arg' which is
passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
for those hooks, because when those hooks are called, 'parts' has
possibly been mangled (like with LSN compression) and is so
unpredictable.
storage/maria/ma_open.c:
fix for "empty body in if() statement" (when compiling without safemutex)
storage/maria/ma_pagecache.c:
<= is enough
storage/maria/ma_recovery.c:
* print the time that each recovery phase (REDO/UNDO/flush) took;
this is enabled only when recovering from ha_maria. Is it printed
n seconds with a fractional part of one digit (like 123.4 seconds).
* In the REDO phase, update the table's live checksum by using
the checksum delta stored in UNDO_ROW_INSERT|DELETE|UPDATE and CLR_END.
Update it too when seeing REDO_DELETE_ALL.
* In the UNDO phase, when executing UNDO_ROW_INSERT, if the table does
not have live checksum then reading the record's header (as done by
the master loop of run_undo_phase()) is enough; otherwise we
do a translog_read_record() to have the checksum delta ready
for _ma_apply_undo_row_insert().
* When at the end of the REDO phase we notice that there is an unfinished
group of REDOs, don't assert in debug binaries, as I verified that it
can happen in real life (with kill -9)
* removing ' in #error as it confuses gcc3
storage/maria/ma_rename.c:
new prototype of translog_write_record()
storage/maria/ma_test_recovery.expected:
Change in output of ma_test_recovery: now all live checksums of
original tables equal those of tables recreated by the REDO phase
and those of tables fixed by the UNDO phase. I.e. recovery of
the live checksum looks like working (which was after all the only
goal of this changeset).
I checked by hand that it's not just all live checksums which are
now 0 and that's why they match. They are the old values like
3757530372. maria.test has hard-coded checksum values in its result
file so checks this too.
storage/maria/ma_update.c:
* It's useless to put up HA_STATE_CHANGED in 'key_changed',
as we put up HA_STATE_CHANGED in info->update anyway.
* We need to compute the old and new rows' checksum before calling
(*update_record)(), as checksum delta must be known when logging
UNDO_ROW_UPDATE which is done by _ma_update_block_record(). Note that
some functions change the 'newrec' record (at least _ma_check_unique()
does) so we cannot move the checksum computation too early in the
function.
storage/maria/ma_write.c:
If inserting into a transactional table, live's checksum was
already updated when writing UNDO_ROW_INSERT. The multiplication
is a trick to save an if().
storage/maria/unittest/ma_test_loghandler-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
new prototype of translog_write_record()
storage/myisam/sort.c:
fix for compiler warnings in pushbuild (write_merge_key* functions
didn't have their declaration match MARIA_HA::write_key).
gprof build for amd64.
storage/maria/ma_loghandler.c:
fater way to detect last page address for the last log file.
BUILD/compile-amd64-gprof-no-ndb:
New BitKeeper file ``BUILD/compile-amd64-gprof-no-ndb''
REDO optimization (Bascily avoid moving blocks from/to pagecache)
More command line arguments to maria_read_log
Fixed recovery bug when recreating table
sql/opt_range.cc:
Remove SAFE_MODE for opt_range as it disables UPDATE to use keys
storage/maria/ma_blockrec.c:
REDO optimization
Use new interface for pagecache_reads to avoid copying page buffers
storage/maria/ma_loghandler.c:
Patch from Sanja:
- Added new parameter to translog_get_page to use direct links to pagecache
- Changed scanner to be able to use direct links
This avoids a lot of calls to bmove512() in page cache.
storage/maria/ma_loghandler.h:
Added direct link to pagecache objects
storage/maria/ma_open.c:
Added const to parameter
Added missing braces
storage/maria/ma_pagecache.c:
From Sanja:
- Added direct links to pagecache (from pagecache_read())
Dirrect link means that on pagecache_read we get back a pointer to the pagecache buffer
From Monty:
- Fixed arguments to init_page_cache to handle big page caches
- Fixed compiler warnings
- Replaced PAGECACHE_PAGE_LINK with PAGECACHE_BLOCK_LINK * to catch errors
storage/maria/ma_pagecache.h:
Changed block numbers from int to long to be able to handle big page caches
Changed some PAGECACHE_PAGE_LINK to PAGECACHE_BLOCK_LINK
storage/maria/ma_recovery.c:
Fixed recovery bug when recreating table (table was kept open)
Moved some variables to function start (portability)
Added space to some print messages
storage/maria/maria_chk.c:
key_buffer_size -> page_buffer_size
storage/maria/maria_def.h:
Changed default page_buffer_size to 10M
storage/maria/maria_read_log.c:
Added more startup options:
--version
--undo (apply undo)
--page_cache_size (to run with big cache sizes)
--silent (to not get any output from --apply)
storage/maria/unittest/ma_control_file-t.c:
Fixed compiler warning
storage/maria/unittest/ma_test_loghandler-t.c:
Added new argument to translog_init_scanner()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Added new argument to translog_init_scanner()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Added new argument to translog_init_scanner()
record (for example in ma_test_loghandler-t). Reason was
wrongly matched () in DBUG.
storage/maria/ma_loghandler.c:
Wrongly matched parenthesis:
DBUG_PRINT(keyword, argslist) expands to roughly
_db_doprnt arglist;
So DBUG_PRINT("enter",(a);
b;
c);
expands to roughly
_db_doprnt(a);b;c;
which is valid code. Except that in non-debug builds, DBUG_PRINT(
expands to nothing so the wrongly "included" code is thrown away,
leading to some members of "scanner" to not be initialized.
horizon addres.
storage/maria/ma_loghandler.c:
New function to get correct LSN from chunk address.
storage/maria/ma_loghandler.h:
New function to get correct LSN from chunk address.
storage/maria/ma_checkpoint.c:
The new macro for easier printing LSN added.
storage/maria/ma_loghandler.c:
The assertion returned.
The new macro for easier printing LSN added.
storage/maria/ma_loghandler_lsn.h:
The new macro for easier printing LSN added.
storage/maria/ma_pagecache.c:
The new macro for easier printing LSN added.
storage/maria/ma_recovery.c:
Recovery checks empty log state.
RECHEADER_READ_ERROR means some real error.
storage/maria/maria_read_log.c:
Read log starts from real beggining of the log and precess
error and empty log states.
The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler-t.c:
The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
The new macro for easier printing LSN added.
Finally this is the real checkpoint code.
It however exhibits unstabilities when a checkpoint runs concurrently
with data-modifying clients (table corruption, transaction log's
assertions) so for now a checkpoint is taken only at startup after
recovery and at shutdown, i.e. not in concurrent situations. Later
we will let it run periodically, as well as flush dirty pages
periodically (almost all needed code is there already, only pagecache
code is written but not committed).
WL#3072 Maria recovery
* replacing UNDO_ROW_PURGE with CLR_END; testing of those CLR_END via
ma_test2 which has INSERTs failing with duplicate keys.
* replaying of REDO_RENAME_TABLE
Now, off to test Recovery in ha_maria :)
BitKeeper/deleted/.del-ma_least_recently_dirtied.c:
Delete: storage/maria/ma_least_recently_dirtied.c
BitKeeper/deleted/.del-ma_least_recently_dirtied.h:
Delete: storage/maria/ma_least_recently_dirtied.h
storage/maria/Makefile.am:
compile Checkpoint module
storage/maria/ha_maria.cc:
When ha_maria starts, do a recovery from last checkpoint.
Take a checkpoint when that recovery has ended and when ha_maria
shuts down cleanly.
storage/maria/ma_blockrec.c:
* even if my_sync() fails we have to my_close() (otherwise we leak
a descriptor)
* UNDO_ROW_PURGE is replaced by a simple CLR_END for UNDO_ROW_INSERT,
as promised in the old comment; it gives us skipping during the
UNDO phase.
storage/maria/ma_check.c:
All REDOs before create_rename_lsn are ignored by Recovery. So
create_rename_lsn must be set only after all data/index has been
flushed and forced to disk. We thus move write_log_record_for_repair()
to after _ma_flush_tables_files_after_repair().
storage/maria/ma_checkpoint.c:
Checkpoint module.
storage/maria/ma_checkpoint.h:
optional argument if caller wants a thread to periodically take
checkpoints and flush dirty pages.
storage/maria/ma_create.c:
* no need to init some vars as the initial bzero(share) takes care of this.
* update to new function's name
* even if we fail in my_sync() we have to my_close()
storage/maria/ma_extra.c:
Checkpoint reads share->last_version under intern_lock, so we make
maria_extra() update it under intern_lock. THR_LOCK_maria still needed
because of _ma_test_if_reopen().
storage/maria/ma_init.c:
destroy checkpoint module when Maria shuts down.
storage/maria/ma_loghandler.c:
* UNDO_ROW_PURGE gone (see ma_blockrec.c)
* we need to remember the LSN of the LOGREC_FILE_ID for a share,
because this LSN is needed into the checkpoint record (Recovery wants
to know the validity domain of an id->name mapping)
* translog_get_horizon_no_lock() needed for Checkpoint
* comment about failing assertion (Sanja knows)
* translog_init_reader_data() thought that translog_read_record_header_scan()
returns 0 in case of error, but 0 just means "0-length header".
* translog_assign_id_to_share() now needs the MARIA_HA because
LOGREC_FILE_ID uses a log-write hook.
* Verify that (de)assignment of share->id happens only under intern_lock,
as Checkpoint reads this id with intern_lock.
* translog_purge() can accept TRANSLOG_ADDRESS, not necessarily
a real LSN.
storage/maria/ma_loghandler.h:
prototype updates
storage/maria/ma_open.c:
no need to initialize "res"
storage/maria/ma_pagecache.c:
When taking a checkpoint, we don't need to know the maximum rec_lsn
of dirty pages; this LSN was intended to be used in the two-checkpoint
rule, but last_checkpoint_lsn is as good.
4 bytes for stored_list_size is enough as PAGECACHE::blocks (number
of blocks which the pagecache can contain) is int.
storage/maria/ma_pagecache.h:
new prototype
storage/maria/ma_recovery.c:
* added replaying of REDO_RENAME_TABLE
* UNDO_ROW_PURGE gone (see ma_blockrec.c), replaced by CLR_END
* Recovery from the last checkpoint record now possible
* In new_table() we skip the table if the id->name mapping is older than
create_rename_lsn (mapping dates from lsn_of_file_id).
* in get_MARIA_HA_from_REDO_record() we skip the record
if the id->name mapping is newer than the record (can happen if processing
a record which is before the checkpoint record).
* parse_checkpoint_record() has to return a LSN, that's what caller expects
storage/maria/ma_rename.c:
new function's name; log end zeroes of tables' names (ease recovery)
storage/maria/ma_test2.c:
* equivalent of ma_test1's --test-undo added (named -u here).
* -t=1 now stops right after creating the table, so that
we can test undoing of INSERTs with duplicate keys (which tests the
CLR_END logged by _ma_write_abort_block_record()).
storage/maria/ma_test_recovery.expected:
Result of testing undoing of INSERTs with duplicate keys; there are
some differences in maria_chk -dvv but they are normal (removing
records does not shrink data/index file, does not put back the
"analyzed, optimized keys"(etc) index state.
storage/maria/ma_test_recovery:
Test undoing of INSERTs with duplicate keys, using ma_test2;
when such INSERT happens, it logs REDO_INSERT, UNDO_INSERT, REDO_DELETE,
CLR_END; we abort after that, and test that CLR_END causes recovery
to jump over UNDO_INSERT.
storage/maria/ma_write.c:
comment
storage/maria/maria_chk.c:
comment
storage/maria/maria_def.h:
* a new bit in MARIA_SHARE::in_checkpoint, used to build a list
of unique shares during Checkpoint.
* MARIA_SHARE::lsn_of_file_id added: the LSN of the last LOGREC_FILE_ID
for this share; needed to know to which LSN domain the mappings
found in the Checkpoint record apply (new mappings should not apply
to old REDOs).
storage/maria/trnman.c:
* small changes to how trnman_collect_transactions() fills its buffer;
it also uses a non-dummy lsn_read_non_atomic() found in ma_checkpoint.h
* testing of execution of UNDO_ROW_UPDATE
* when executing an UNDO_ROW_UPDATE, store "UNDO_ROW_UPDATE" as
"type of undone record" into the CLR_END record.
storage/maria/ma_blockrec.c:
When logging a CLR_END in write_block_record(), it can be for
a DELETE or for an UPDATE (now that Monty has coded execution of
UNDO_UPDATE)
storage/maria/ma_loghandler.c:
UNDO_ROW_UPDATE's execution coded, so no crash
storage/maria/ma_recovery.c:
UNDO_ROW_UPDATE's execution now coded, so no crash
storage/maria/ma_test1.c:
upper case letter
storage/maria/ma_test_recovery.expected:
output of testing execution of UNDO_ROW_UPDATE. Table's checksum
not recovered (known issue not specific to UPDATE).
storage/maria/ma_test_recovery:
Test execution of UNDO_ROW_UPDATE: first we stop ma_test1 after
deletes and commit, then we stop ma_test1 after updates and abort;
we verify that updates are rolled back by comparing tables
into mysql.com:/home/my/mysql-maria
storage/maria/ma_check.c:
Auto merged
storage/maria/ma_locking.c:
Auto merged
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/ma_open.c:
Auto merged
storage/maria/ma_recovery.c:
Auto merged
storage/maria/maria_def.h:
Auto merged
storage/maria/maria_read_log.c:
Auto merged
storage/maria/ma_blockrec.c:
Manual merge
storage/maria/ma_test1.c:
Manual merge (using Guilhems code)
Fixed bug in duplicate key handling for block records during repair
All read-row methods now return error number in case of error
Don't calculate checksum for null fields
Fixed bug when running maria_read_log with -o
BUILD/SETUP.sh:
Added STACK_DIRECTION
BUILD/compile-pentium-debug-max:
Moved STACK_DIRECTION to SETUP
include/myisam.h:
Added extra parameter to write_key
storage/maria/ma_blockrec.c:
Added applying of undo for updates
Fixed indentation
Removed some not needed casts
Fixed wrong logging of CLR record
Split ma_update_block_record to two functions to be able to reuse it from undo-applying
Simplify filling of packed fields
ma_record_block_record) now returns error number on failure
Sligtly changed log record information for undo-update
storage/maria/ma_check.c:
Fixed bug in duplicate key handling for block records during repair
storage/maria/ma_checksum.c:
Don't calculate checksum for null fields
storage/maria/ma_dynrec.c:
_ma_read_dynamic_reocrd() now returns error number on error
Rest of the changes are code simplification and indentation fixes
storage/maria/ma_locking.c:
Added comment
storage/maria/ma_loghandler.c:
More debugging
Removed printing of total_record_length as this was always same as record_length
storage/maria/ma_open.c:
Allocate bitmap for changed fields
storage/maria/ma_packrec.c:
read_record now returns error number on error
storage/maria/ma_recovery.c:
Fixed wrong arguments to undo_row_update
storage/maria/ma_statrec.c:
read_record now returns error number on error (not 1)
Code simplification
storage/maria/ma_test1.c:
Added exit possibility after update phase (to test undo of updates)
storage/maria/maria_def.h:
Include bitmap header file
storage/maria/maria_read_log.c:
Fixed bug when running with -o
end of translog_flush() when datadir was in /dev/shm.
storage/maria/ma_loghandler.c:
directory syncing can fail on shared memory devices (/dev/shm on Linux
in this case); see my_sync_dir().
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
misc fixes of execution of UNDOs in the UNDO phase:
- into the CLR_END, store the LSN of the _previous_ UNDO (we debated
what was best, so far we're going with "previous"; later we can change
to "current" if needed), and store the type of record which is being
undone (needed to know how to update state.records when we see the
CLR_END during the REDO phase).
- declaring all UNDOs and CLR_END as "compressed"
- when executing an UNDO in the UNDO phase, state.records is updated
as a hook when writing CLR_END (needed for "recovery of the state"),
and so is trn->undo_lsn (needed for when we have checkpoints).
- bugfix (execution of UNDO_ROW_DELETE didn't store the correct checksum
into the re-inserted row, maria_chk -r thus threw the row away).
- modifications of ma_test1: where to stop is now driven by --testflag;
--test-undo just tells how to stop (flush data, flush log, nothing).
- ma_test_recovery: testing of the UNDO phase, more testing of the
REDO phase, identification of a bug.
storage/maria/ma_blockrec.c:
- bugfix: execution of UNDO_ROW_DELETE didn't store the correct
checksum into the row (leading to "maria_chk -r" eliminating the
re-inserted row, net effect was that rollback appeared to have
rolled back no deletion). Reason was that write_block_record() used
info->cur_row.checksum, while "row" can be != &info->cur_row
(case of UNDO_ROW_DELETE). After fixing this, problems with
_ma_update_block_record() appeared; indeed checksum was computed
by allocate_and_write_block_record() while _ma_update_block_record()
directly calls write_block_record(). Solution is to compute checksum
in write_block_record() instead.
- when executing an UNDO, we now pass the LSN of the _previous_ UNDO
to block_format functions. This LSN can be 0 (if the being-executed UNDO
was the transaction's first UNDO), so "undo_lsn==0" cannot work
anymore to indicate "this is not UNDO work". Using undo_lsn==LSN_ERROR
instead (this is an impossible LSN).
- store into CLR_END the type of log record which was undone
(INSERT/UPDATE/DELETE); needed for Recovery to know if/how it has
to update state.records if it sees this CLR_END in the REDO phase.
- when writing the CLR_END in _ma_apply_undo_row_insert(),
the place to store file's id is log_data+LSN_STORE_SIZE.
- in _ma_apply_undo_row_insert(), the records-- is moved
to a hook when writing the CLR_END (this way it is under log's mutex
which is needed for "recovery of the state")
storage/maria/ma_loghandler.c:
- all UNDOs, and CLR_END, start with the LSN of another UNDO; so
we can declare them "compressed".
- write_hook_for_clr_end() to set trn->undo_lsn (to the previous
UNDO's LSN) under log's lock (like UNDOs set trn->undo_lsn under log's
lock), and also update, if appropriate, state.records.
- reset share->id to 0 when deassigning; not useful for now but
sounds logical.
storage/maria/ma_recovery.c:
- if no table is found for a REDO, it's not an error; for an UNDO, it is
- in the REDO phase, when we see a CLR_END we must update trn->undo_lsn
and sometimes state.records.
- in the UNDO phase, when we execute an UNDO_ROW_INSERT:
* update trn->undo_lsn only after executing the record
* store the _previous_ undo_lsn into the CLR_END
- at the end of the REDO phase, when we recreate TRN objects, they
have already their long id in the log (either via a
LOGREC_LONG_TRANSACTION_ID, or in a checkpoint record), don't write
a new, useless LOGREC_LONG_TRANSACTION_ID for them.
storage/maria/ma_test1.c:
* where to stop execution is now driven by --testflag and not --test-undo
(ma_test2 already has --testflag for the same purpose). This allows
us to do a clean stop (with commit) at any point.
* --test-undo=# tells how to abort (flush all pages (which implies
flushing log) or only log or nothing); all such "ways of crashing"
are tested in ma_test_recovery
storage/maria/ma_test_recovery:
* Testing execution of UNDOs, with and without BLOBs.
* Testing idempotency of REDOs.
* See @todo for a probable bug with BLOBs.
* maria_chk -rq instead of -r, as with -q it nicely stops on any
problem in the data file (like the checksum bug see comment of
ma_blockrec.c).
* Testing if log was written by UNDO phase (often expected),
not written by REDO phase (always expected).
* Less output on the screen, compares with expected output in the end.
* some shell thingies like "set --" and $# are courtesy of
Danny and Pekka.
storage/maria/maria_read_log.c:
when only displaying the records, don't do an UNDO phase
storage/maria/ma_test_recovery.expected:
This is the expected output of a great part of ma_test_recovery.
ma_test_recovery compares its output to the expected output
and tells if different.
If we look at this file it mentions differences in checksum
(normal, it's not recovered yet) and in records count
(getting a correct records' count when recovery starts on an
already existing table, like when testing rollback,
is coded but not yet pushed).
Small fixes made.
storage/maria/ma_loghandler.c:
Check of transaction log descriptor table consistance added.\
Incorrect record description fixed.
Compiler warning fixed.
storage/maria/ma_loghandler.h:
fixed ident.
storage/maria/unittest/ma_test_loghandler-t.c:
Suppressing of automatic record writing
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Suppressing of automatic record writing
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Suppressing of automatic record writing
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Suppressing of automatic record writing
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Suppressing of automatic record writing
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Suppressing of automatic record writing
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Suppressing of automatic record writing
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Suppressing of automatic record writing
storage/maria/ma_blockrec.c:
Added UNDO handling of insert during recovery
To do this, I also had to add write locking of tail pages during undo phase (As we need to access the same page twice if extents are split over two pages)
Another way to handle the undo of insert would be to store the extent information as part of the UNDO_INSERT block.
storage/maria/ma_blockrec.h:
Added new prototype
storage/maria/ma_loghandler.c:
Changed type of CLR_END (to avoid crash in log handler)
Removed not used variable
storage/maria/ma_loghandler.h:
Added TRN argument to record_execute_in_undo_phase()
storage/maria/ma_pagecache.c:
Hack for undo phase of recovery. During REDO we work with PLAIN pages, but UNDO works with LSN pages, which caused an abort when trying to access a cached page.
storage/maria/ma_recovery.c:
Added execution of UNDO_ROW_INSERT
storage/maria/ma_test1.c:
Added option --test-undo for testing recovery with undo
storage/maria/maria_read_log.c:
Added processing of undos
into desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-purge
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/ma_loghandler.h:
Auto merged
into gbichot4.local:/home/mysql_src/mysql-maria-for-undo-phase
storage/maria/ha_maria.cc:
Auto merged
storage/maria/ma_blockrec.c:
Auto merged
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/ma_loghandler.h:
Auto merged
storage/maria/ma_loghandler_lsn.h:
Auto merged
storage/maria/maria_chk.c:
Auto merged
storage/maria/maria_read_log.c:
Auto merged
* create page cache before initializing engine and not after, because
Maria's recovery needs a page cache
* make the creation of a bitmap page more crash-resistent
* bugfix (see ma_blockrec.c)
* back to old way: create an 8k bitmap page when creating table
* preparations for the UNDO phase: recreate TRNs
* preparations for Checkpoint: list of dirty pages, testing
of rec_lsn to know if page should be skipped during Recovery
(unused in this patch as no Checkpoint module pushed yet)
* maria_chk tags repaired table with a special LSN
* reworking all around in ma_recovery.c (less duplication)
mysys/my_realloc.c:
noted an issue in my_realloc()
sql/mysqld.cc:
page cache needs to be created before engines are initialized,
because Maria's initialization may do a recovery which needs
the page cache.
storage/maria/ha_maria.cc:
update to new prototype
storage/maria/ma_bitmap.c:
when creating the first bitmap page we used chsize to 8192 bytes then
pwrite (overwrite) the last 2 bytes (8191-8192). If crash between
the two operations, this leaves a bitmap page full without its end
marker. A later recovery may try to read this page and find it
exists and misses a marker and conclude it's corrupted and fail.
Changing the chsize to only 8190 bytes: recovery will then find
the page is too short and recreate it entirely.
storage/maria/ma_blockrec.c:
Fix for a bug: when executing a REDO, if the data page is created,
data_file_length was increased before _ma_bitmap_set():
_ma_bitmap_set() called _ma_read_bitmap_page() which, due to the
increased data_file_length, expected to find a bitmap page on disk
with a correct end marker; if the bitmap page didn't exist already
in fact, this failed. Fixed by increasing data_file_length only after
_ma_read_bitmap_page() has created the new bitmap page correctly.
This bug could happen every time a REDO is about creating a new
bitmap page.
storage/maria/ma_check.c:
empty data file has a bitmap page
storage/maria/ma_control_file.c:
useless parameter to ma_control_file_create_or_open(), just
test if this is recovery.
storage/maria/ma_control_file.h:
new prototype
storage/maria/ma_create.c:
Back to how it was before: maria_create() creates an 8k bitmap page.
Thus (bugfix) data_file_length needs to reflect this instead of being 0.
storage/maria/ma_loghandler.c:
as ma_test1 and ma_test2 now use real transactions and not
dummy_transaction_object, REDO for INSERT/UPDATE/DELETE are always
about real transactions, can assert this.
A function for Recovery to assign a short id to a table.
storage/maria/ma_loghandler.h:
new function
storage/maria/ma_loghandler_lsn.h:
maria_chk tags repaired tables with this LSN
storage/maria/ma_open.c:
* enforce that DMLs on transactional tables use real transactions
and not dummy_transaction_object.
* test if table was repaired with maria_chk (which has to been
seen as an import of an external table into the server), test
validity of create_rename_lsn (header corruption detection)
* comments.
storage/maria/ma_recovery.c:
* preparations for the UNDO phase: recreate TRNs
* preparations for Checkpoint: list of dirty pages, testing
of rec_lsn to know if page should be skipped during Recovery
(unused in this patch as no Checkpoint module pushed yet)
* reworking all around (less duplication)
storage/maria/ma_recovery.h:
a parameter to say if the UNDO phase should be skipped
storage/maria/maria_chk.c:
tag repaired tables with a special LSN
storage/maria/maria_read_log.c:
* update to new prototype
* no UNDO phase in maria_read_log for now
storage/maria/trnman.c:
* a function for Recovery to create a transaction (TRN), needed
in the UNDO phase
* a function for Recovery to grab an existing transaction, needed
in the UNDO phase (rollback all existing transactions)
storage/maria/trnman_public.h:
new functions
Now ma_test1 -M -T and ma_test2 -M -T produces readable, applyable logs
Note: The .MAD file is not binary identical after applying redo compare to a an original file.
(This is becasue we don't have full information which function called PURGE_REDO_BLOCKS).
To verify if a file was correctly applied, we now instead compare row checksums
BitKeeper/etc/ignore:
added storage/maria/tmp/*
include/maria.h:
Added maria_commit() and maria_begin() to be used with external tests
storage/maria/ha_maria.cc:
Ensure maria_def. is read in C mode
storage/maria/ma_blockrec.c:
Fixed redo handling.
_ma_apply_redo_purge_blocks() updated to handle any number of purged blocks
Removed code to make data file idenitcal after redo (can't easily be done). See changeset comments
Now ma_test1 -M -T and ma_test2 -M -T produces readable, applyable logs
storage/maria/ma_commit.c:
More DBUG statements
Moved variable declaration to start of function (portability fix)
Added helper functions 'maria_commit()' and 'maria_begin()'
storage/maria/ma_loghandler.c:
Fixed wrong REDO_PURGE_BLOCKS initialization
storage/maria/ma_recovery.c:
Added UNDO_ROW_UPDATE
Removed wrong setting of lsn (there was no lsn at the used position)
Fixed REDO_PURGE_BLOCKS to handle any number of blocks
storage/maria/ma_test1.c:
Added transaction support (via maria_begin() & maria_commit()) to get a log that can be applied with maria_read_log
storage/maria/ma_test2.c:
Added transaction support (via maria_begin() & maria_commit()) to get a log that can be applied with maria_read_log
storage/maria/ma_test_recovery:
Create temporary files in maria/tmp
Verify files with checksums instead of byte comparisons
storage/maria/maria_chk.c:
When using with -dss we only get filename, records and checksum.
This is useful to do a quick comparision if a files is identical to another one.
storage/maria/maria_def.h:
Added ma_commit()
storage/maria/maria_read_log.c:
Added --help
File number types fixed.
storage/maria/unittest/Makefile.am:
Test of purger added.
storage/maria/unittest/ma_test_loghandler_purge-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_purge-t.c''
in the file to the file header.
storage/maria/ma_loghandler.h:
Getting maximum LSN of the record which parts written
in the file to the file header.
storage/maria/unittest/Makefile.am:
Test suite for getting max LSN added.
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Spelling fixed.
Cleanup fixed.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Cleanup fixed.
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_max_lsn-t.c''
storage/maria/ma_checkpoint.c:
Definitions of LSN should be collected in
the one file (ma_loghandler_lsn.h)
storage/maria/ma_loghandler.c:
New calls to get first theoretical and first real LSN.
storage/maria/ma_loghandler.h:
New calls to get first theoretical and first real LSN.
storage/maria/ma_loghandler_lsn.h:
Defined yet another impossible LSN to indicate error.
storage/maria/ma_recovery.c:
The first LSN call changed.
storage/maria/maria_read_log.c:
The first LSN call changed.
storage/maria/unittest/Makefile.am:
New unittest added.
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_first_lsn-t.c''
into desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria
storage/maria/ma_pagecache.c:
Auto merged
storage/maria/unittest/ma_test_loghandler-t.c:
Auto merged
storage/maria/ma_loghandler.c:
merge
storage/maria/ma_pagecache.h:
merge
Already written pages injection to the cache fixed.
storage/maria/ma_loghandler.c:
Ability to read unflushed data added.
storage/maria/ma_page.c:
Parameters added
storage/maria/ma_pagecache.c:
Already written pages injection to the cache fixed.
Validator for case of page content injection added.
storage/maria/ma_pagecache.h:
Validator for case of page content injection added.
storage/maria/unittest/Makefile.am:
Test of reading unflushed data
storage/maria/unittest/ma_test_loghandler-t.c:
Define fixed.
Restart of the log removed.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_noflush-t.c''
Preparation: for the two-checkpoint rule we will flush all pages
which have block->rec_lsn <= last_checkpoint_lsn. Pages with
rec_lsn not yet set (i.e. ==0) were dirtied very recently and so should
not be flushed (otherwise it's not efficient: we should try to let
a page be modified several times in memory before we flush it to disk).
To make this easy, "block->rec_lsn not yet set" is now expressed
with block->rec_lsn==LSN_MAX, not block->rec_lsn==0 anymore.
It is easier this way because LSN_MAX>last_checkpoint_lsn whereas
0<=last_checkpoint_lsn.
storage/maria/ma_blockrec.c:
typo
storage/maria/ma_loghandler.c:
typo
storage/maria/ma_loghandler_lsn.h:
LSN_MAX
storage/maria/ma_pagecache.c:
"block->rec_lsn not yet set" is now expressed by block->rec_lsn==LSN_MAX,
not block_rec_lsn==0.
preparation to enable recoverability of the table's state, more exactly
info->state->checksum. This will require that info->state->checksum
be updated in a inwrite_hook when writing an UNDO record, thus
info->cur_row.checksum needs to be accessible to inwrite_hook, so we
make translog_write_record() accept a MARIA_HA* (info) instead of
MARIA_SHARE* (info->s); with this, we will be able to access
info->cur_row.checksum. Old code which needed the MARIA_SHARE
can derive it from MARIA_HA. Fix for typos and compiler warnings.
storage/maria/ma_blockrec.c:
fix for new loghandler API. Removing strange lines (how could gcc
accept that?)
storage/maria/ma_check.c:
fix for new loghandler API
storage/maria/ma_delete_all.c:
fix for new loghandler API
storage/maria/ma_loghandler.c:
functions now take a MARIA_HA in argument, this is more powerful
than a MARIA_SHARE (MARIA_SHARE can be derived from MARIA_HA, not
the other way around). MARIA_HA will be needed to allow recoverability
of the table's state.
Fixing wrong DBUG_PRINT ('i' is not the id).
When writing the LOGREC_FILE_ID, we don't have a MARIA_HA around,
so we cannot ask translog_write_record() to store the id for us;
we thus store the file's id by ourselves. Alternative would have been
to pass MARIA_HA to translog_assign_id_to_share() but I didn't like it.
storage/maria/ma_loghandler.h:
new loghandler API
storage/maria/tablockman.c:
fix for compiler warning (intptr is int on my machine)
errors when reading a log record which has a 0-length header
(like LOGREC_REDO_DROP_TABLE).
storage/maria/ma_loghandler.c:
Functions reading record's header now don't use 0 to indicate error,
as some valid records have a 0-length header (like REDO_DROP_TABLE).
Instead, negative values are used for EOF and error.
storage/maria/ma_loghandler.h:
functions to read record's header now return an int
(either the length of this header (>=0) or some negative values
for EOF or error).
storage/maria/ma_recovery.c:
update to the new log handler's behaviour. Note the @todo.
storage/maria/maria_read_log.c:
inform when program failed
storage/maria/unittest/ma_test_loghandler-t.c:
update to new log handler's API
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update to new log handler's API
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update to new log handler's API
Jul 7th). "maria.test" and "ps_maria.test" still fail;
"ma_test_all" starts failing (MyISAM has the same issue see BUG#30094).
include/maria.h:
merging MyISAM changes into Maria
mysys/mf_keycache.c:
mi_test_all showed "floating point exception", this was already
fixed in the latest 5.1, importing fix.
sql/item_xmlfunc.cc:
compiler warning (already fixed in latest 5.1)
storage/maria/ha_maria.cc:
merging MyISAM changes into Maria. See #ifdef ASK_MONTY.
storage/maria/ha_maria.h:
merging MyISAM changes into Maria
storage/maria/ma_cache.c:
merging MyISAM changes into Maria
storage/maria/ma_check.c:
merging MyISAM changes into Maria
storage/maria/ma_create.c:
merging MyISAM changes into Maria
storage/maria/ma_dynrec.c:
merging MyISAM changes into Maria
storage/maria/ma_extra.c:
merging MyISAM changes into Maria
storage/maria/ma_ft_boolean_search.c:
merging MyISAM changes into Maria
storage/maria/ma_ft_nlq_search.c:
merging MyISAM changes into Maria
storage/maria/ma_info.c:
merging MyISAM changes into Maria
storage/maria/ma_key.c:
merging MyISAM changes into Maria
storage/maria/ma_loghandler.c:
compiler warning (part->length is size_t)
storage/maria/ma_open.c:
merging MyISAM changes into Maria
storage/maria/ma_preload.c:
merging MyISAM changes into Maria
storage/maria/ma_range.c:
merging MyISAM changes into Maria
storage/maria/ma_rkey.c:
merging MyISAM changes into Maria
storage/maria/ma_rt_index.c:
merging MyISAM changes into Maria
storage/maria/ma_rt_key.c:
merging MyISAM changes into Maria
storage/maria/ma_rt_split.c:
merging MyISAM changes into Maria
storage/maria/ma_search.c:
merging MyISAM changes into Maria
storage/maria/ma_sort.c:
merging MyISAM changes into Maria
storage/maria/maria_def.h:
merging MyISAM changes into Maria
found in pushbuild on sapsrv1. Some not fixed as not repeatable
on my machine (32/64 bit issue?).
Fixes for some test failures:
- "maria-connect" now passes;
- "maria": after fixing the obvious reasons for failures, the test
went further and hit a more complex issues: difference in the output
of EXPLAIN output; not fixed;
- "ps_maria" still crashes in assertion
mysqld: ha_maria.cc:1627: virtual int ha_maria::index_read(uchar*, const uchar*, uint, ha_rkey_function): Ass
ertion `inited == INDEX' failed, as already observable in pushbuild.
All this might just be due to an incomplete merge of MyISAM changes
into Maria when 5.1 was last merged to mysql-maria.
include/my_global.h:
temporary fix until next merge of 5.1; without this it does not build
mysql-test/r/maria-connect.result:
position changed
mysql-test/t/maria-connect.test:
If one wants to use the binlog it has to ask for it.
1582 is not used for dup entry error anymore (it was in older 5.1).
Size of first event in binlog was increased by 4 (when the new type
of event "gap" was added).
mysql-test/t/maria.test:
1582 not used anymore in this case
storage/maria/ha_maria.cc:
engine now has to say what binlogging it supports
storage/maria/ma_blockrec.c:
fix for compiler warnings ("comparison is always true" or "always
false")
storage/maria/ma_loghandler.c:
fix for compiler warnings (comparing char* to uchar*)
storage/maria/ma_packrec.c:
fix for compiler warning (fix simply merged from MyISAM)
storage/maria/ma_pagecache.c:
info_check_pin() was not used so gave a compiler warning.
storage/maria/ma_pagecache.h:
fixing typo from the last 5.1->maria merge.
storage/maria/ma_recovery.c:
my_free() has a void* argument, so why cast. byte->uchar.
storage/maria/ma_search.c:
fix for compiler warning (fix simply merged from MyISAM)
storage/maria/maria_read_log.c:
gptr->uchar*
storage/maria/trnman.c:
probable fix for warning found in pushbuild (but not on my machine):
storage/maria/trnman.c: 142
passing argument 6 of \u2018lf_hash_init\u2019 from incompatible pointer type
on sapsrv1.
sql/handler.cc:
Auto merged
storage/maria/ha_maria.cc:
Auto merged
storage/maria/ma_blockrec.c:
Auto merged
storage/maria/ma_close.c:
Auto merged
storage/maria/ma_create.c:
Auto merged
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/ma_loghandler.h:
Auto merged
storage/maria/ma_test2.c:
Auto merged
storage/maria/maria_read_log.c:
will merge by hand
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.
sql/handler.cc:
typo
storage/maria/Makefile.am:
Adding ma_test_recovery (which ma_test_all invokes, and which can
also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
comments
storage/maria/ma_bitmap.c:
fixing comments. 2 -> sizeof(maria_bitmap_marker).
Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
Now putting the "bm" signature when creating the first bitmap page
(it used to happen only at next open, but that
caused an annoying difference when testing Recovery if the original
run didn't open the table, and it looks more
logical like this: it goes to disk only with its signature correct);
see the "QQ" comment towards the _ma_initialize_data_file() call
in ma_create.c for more).
When reading a bitmap page, verify its signature (happens when normally
using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
* no need to sync the data file if table is not transactional
* Comments, code cleanup (log-related data moved to log-related code
block, int5store->page_store).
* Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
do for other records (though this record will soon be replaced
with a CLR).
* If "page" is 1 it means the page which extends from byte
page*block_size+1 to (page+1)*block_size (byte number 1 being
the first byte of the file). The last byte of the file is
data_file_length (same convention).
A new page needs to be created if the last byte of the page is
beyond the last byte of the file, i.e.
(page+1)*block_size+1 > data_file_length, so we correct the test
(bug found when testing log applying for ma_test1 -M -T --skip-update).
* update the page's LSN when removing a row from it during
execution of a REDO_PURGE_ROW record (bug found when testing log
applying for ma_test1 -M -T --skip-update).
* applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
we can always flush the table's state when closing the last instance
of the table. And it is needed for maria_read_log (as it does
not use maria_lock_database()).
storage/maria/ma_control_file.c:
when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
double-inclusion safe
storage/maria/ma_create.c:
during recovery, don't log records. Comments.
Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
during recovery, don't log records. Log the end-zero of the dropped
table's name, so that recovery can use the string in place without
extending it to fit an end zero.
storage/maria/ma_loghandler.c:
* inwrite_rec_hook also needs access to the MARIA_SHARE, like
prewrite_rec_hook. This will be needed to update
share->records_diff (in the upcoming patch "recovery of the state").
* LOG_DESC::record_ends_group changed to an enum.
* LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
corrected
* Sanja please see the @todo LOG BUG
* avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
- log write hooks called while the log's lock is held (inwrite_rec_hook)
now need the MARIA_SHARE, like prewrite_rec_hook already had
- instead of a bool saying if this record's type ends groups or not,
we refine: it may not end a group, it may end a group, or it may
be a group in itself. Imagine that we had a physical write failure
to a table before we log the UNDO, we still end up in
external_lock(F_UNLCK) and then we log a COMMIT: we don't want
to consider this COMMIT as ending the group of REDOs (don't want
to execute those REDOs during Recovery), that's why we say "COMMIT
is a group in itself, it aborts any previous group". This also
gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
New Recovery code, replacing the old pseudocode.
Most of maria_read_log moved here.
Call-able from ha_maria, but not enabled yet.
Compared to the previous version of maria_read_log, some bugs have
been fixed, debugging output can go to stdout or a disk file (for now
it's useful for me, later it can be changed), execution of
REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
has been factored into functions. We abort an unfinished group
of records if we see a record which is a group in itself (like COMMIT).
No need for maria_panic() after a bug (which caused tables to not
be closed) was fixed; if there is yet another bug I prefer to see it.
When opening a table for Recovery, set data_file_length
and key_file_length to their real physical value (these are the
easiest state members to restore :). Warn us if the last page
was truncated (but Recovery handles it).
MARIA_SHARE::state::state::records is now partly recovered (not
idempotent, but works if recreating tables from scracth).
When applying a REDO to a page, stamp it with the UNDO's LSN
(current_group_end_lsn), not with the REDO's LSN; it makes
the table more identical to the original table (easier to compare
the two tables in the end).
Big thing missing: some types of REDOs are not handled,
and the UNDO phase does not exist (missing functions to execute UNDOs
to actually rollback). So for now tests are only inserting/deleting
a few 100 rows, closing the table and seeing if the log is applied ok;
it works. UPDATE not handled.
storage/maria/ma_recovery.h:
new functions: ma_recover() for recovery from inside ha_maria;
_ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
don't write log records during recovery
storage/maria/ma_test2.c:
- fail if maria_info() or other subtests find some wrong information
- new option -g to skip updates.
- init the translog before creating the table, so that log applying
can work.
- in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
cleanup files. Test log applying.
storage/maria/maria_read_log.c:
most of the logic moves to ma_recovery.c to be shared between
maria_read_log and recovery-from-inside-mysqld.
See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
unit test for Recovery. Tests insert and delete,
REDO_UPDATE not yet coded.
Script is called from ma_test_all. Can run standalone.
into labbari.dsl.inet.fi:/home/my/bk/mysql-maria.prod
mysys/thr_lock.c:
Auto merged
storage/csv/ha_tina.cc:
Auto merged
storage/csv/ha_tina.h:
Auto merged
storage/maria/ma_blockrec.c:
Auto merged
storage/maria/ma_check.c:
Auto merged
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/trnman.c:
Auto merged
storage/myisam/mi_locking.c:
Auto merged
sql/lock.cc:
SCCS merged
mode) so ma_test_all works again; more error detection in ma_test_all;
maria_control renamed to maria_log_control (Monty's suggestion,
so that a "rm maria_log*" removes all log-related files). Disabling
failing wrong assertion.
storage/maria/ma_blockrec.c:
disabling assertion which fails because cur_block is a
local variable not initialized
storage/maria/ma_check.c:
comment
storage/maria/ma_control_file.h:
control file renamed
storage/maria/ma_loghandler.c:
assertions needed only in multi-threaded mode (ma_test1 and ma_test2
are single-threaded, it's ok for them to use dummy_transaction_object
with transactional tables: trn->rec_lsn can be set without interfering
with other threads).
storage/maria/ma_test_all.sh:
got caught by failures in some ma_test1 runs, which I didn't see
because ma_test_all returned 0 and I didn't scroll up in the window;
now using "set -e" to avoid that. Also testing that
we get the errors and warnings we expect.
storage/maria/unittest/Makefile.am:
maria_control renamed
into a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-maria.prod
BitKeeper/etc/ignore:
auto-union
include/maria.h:
Auto merged
include/my_base.h:
Auto merged
mysql-test/include/ps_conv.inc:
Auto merged
mysql-test/r/ps_2myisam.result:
Auto merged
mysql-test/r/ps_3innodb.result:
Auto merged
mysql-test/r/ps_4heap.result:
Auto merged
mysql-test/r/ps_5merge.result:
Auto merged
mysql-test/r/ps_7ndb.result:
Auto merged
mysys/Makefile.am:
Auto merged
mysys/mf_iocache.c:
Auto merged
mysys/mf_tempfile.c:
Auto merged
mysys/my_error.c:
Auto merged
mysys/my_init.c:
Auto merged
mysys/my_open.c:
Auto merged
sql/handler.cc:
Auto merged
sql/handler.h:
Auto merged
sql/lex.h:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.cc:
Auto merged
sql/table.h:
Auto merged
storage/maria/ha_maria.cc:
Auto merged
storage/maria/ha_maria.h:
Auto merged
storage/maria/ma_bitmap.c:
Auto merged
storage/maria/ma_blockrec.h:
Auto merged
storage/maria/ma_create.c:
Auto merged
storage/maria/ma_dynrec.c:
Auto merged
storage/maria/ma_loghandler.h:
Auto merged
storage/maria/ma_loghandler_lsn.h:
Auto merged
storage/maria/ma_open.c:
Auto merged
storage/maria/ma_pagecache.c:
Auto merged
storage/maria/ma_pagecache.h:
Auto merged
storage/maria/ma_test1.c:
Auto merged
storage/maria/ma_test2.c:
Auto merged
storage/maria/ma_update.c:
Auto merged
storage/maria/maria_chk.c:
Auto merged
storage/maria/unittest/ma_test_loghandler-t.c:
Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Auto merged
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Auto merged
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Auto merged
storage/myisam/ft_stopwords.c:
Auto merged
storage/myisam/mi_close.c:
Auto merged
support-files/compiler_warnings.supp:
Auto merged
mysys/mf_keycache.c:
Pulled from mysql-maria tree, manual merge.
mysys/my_handler.c:
Pulled from mysql-maria tree, manual merge.
mysys/my_symlink2.c:
Pulled from mysql-maria tree, manual merge.
sql/mysqld.cc:
Pulled from mysql-maria tree, manual merge.
sql/sql_table.cc:
Pulled from mysql-maria tree, manual merge.
storage/maria/ma_blockrec.c:
Pulled from mysql-maria tree, manual merge.
storage/maria/ma_check.c:
Pulled from mysql-maria tree, manual merge.
storage/maria/ma_close.c:
Pulled from mysql-maria tree, manual merge.
storage/maria/ma_loghandler.c:
Pulled from mysql-maria tree, manual merge.
storage/maria/maria_def.h:
Pulled from mysql-maria tree, manual merge.
* Don't modify share->base.born_transactional; now it is a value carved
in stone at creation time. share->now_transactional is what can be
modified: it starts at born_transactional, can become false during
ALTER TABLE (when we want no logging), and restored later.
* Not resetting create_rename_lsn to 0 during delete_all or repair.
* when we temporarily disable transactionality, we also change
the page type to PAGECACHE_PLAIN_PAGE: it bypasses some work in the
page cache (optimization), and avoids assertions related to LSNs.
* Disable INSERT DELAYED for transactional tables, because
durability could not be guaranteed (insertion may even not happen)
mysys/mf_keycache.c:
comment
storage/maria/ha_maria.cc:
* a transactional table cannot do INSERT DELAYED
* ha_maria::save_transactional not needed anymore, as now instead
we don't modify MARIA_SHARE::MARIA_BASE_INFO::born_transactional
(born_transactional plays the role of save_transactional), and modify
MARIA_SHARE::now_transactional.
* REPAIR_TABLE log record is now logged by maria_repair()
* comment why we rely on born_transactional to know if we should
skipping a transaction.
* putting together two if()s which test for F_UNLCK
storage/maria/ha_maria.h:
ha_maria::save_transactional not needed anymore (moved to the C layer)
storage/maria/ma_blockrec.c:
* For the block record's code (writing/updating/deleting records),
all that counts is now_transactional, not born_transactional.
* As we now set the page type to PAGECACHE_PLAIN_PAGE for tables
which have now_transactional==FALSE, pagecache will not expect
a meaningful LSN for them in pagecache_unlock_by_link(), so
we can pass it LSN_IMPOSSIBLE.
storage/maria/ma_check.c:
* writing LOGREC_REPAIR_TABLE moves from ha_maria::repair()
to maria_repair(), sounds cleaner (less functions to export).
* when opening a table during REPAIR, don't use the realpath-ed name,
as this may fail if the table has symlinked files (maria_open()
would try to find the data and index file in the directory
of unique_file_name, it would fail if data and index files are in
different dirs); use the unresolved name, open_file_name, which is
the argument which was passed to the maria_open() which created 'info'.
storage/maria/ma_close.c:
assert that when a statement is done with a table, it cleans up
storage/maria/ma_create.c:
new name
storage/maria/ma_delete_all.c:
* using now_transactional
* no reason to reset create_rename_lsn during delete_all (a bug);
also no reason to do it during repair: it was put there because
a positive create_rename_lsn caused a call to check_and_set_lsn()
which asserted in DBUG_ASSERT(block->type == PAGECACHE_LSN_PAGE);
first solution was to use LSN_IMPOSSIBLE in _ma_unpin_all_pages() if
not transactional; but then in the case of ALTER TABLE, with
transactionality temporarily disabled, it asserted in
DBUG_ASSERT(LSN_VALID(lsn)) in pagecache_fwrite() (PAGECACHE_LSN_PAGE
page with zero LSN - bad). The additional solution is to use
PAGECACHE_PLAIN_PAGE when we disable transactionality temporarily: this
avoids checks on the LSN, and also bypasses (optimization) the "flush
log up to LSN" call when the pagecache flushes our page (in other
words, no WAL needed).
storage/maria/ma_delete_table.c:
use now_transactional
storage/maria/ma_locking.c:
assert that when a statement is done with a table, it cleans up.
storage/maria/ma_loghandler.c:
* now_transactional should be used to test if we want a log record.
* Assertions to make sure dummy_transaction_object is not spoilt
by its many users.
storage/maria/ma_open.c:
base.transactional -> base.born_transactional
storage/maria/ma_pagecache.c:
missing name for page's type. Comment for future.
storage/maria/ma_rename.c:
use now_transactional
storage/maria/maria_chk.c:
use born_transactional
storage/maria/maria_def.h:
MARIA_BASE_INFO::transactional renamed to born_transactional.
MARIA_SHARE::now_transactional introduced.
_ma_repair_write_log_record() is made local to ma_check.c.
Macros to temporarily disable, and re-enable, transactionality for a
table.
storage/maria/maria_read_log.c:
assertions and using the new macros. Adding a forgotten resetting
when we finally close all tables.
into mysql.com:/home/my/mysql-maria
include/maria.h:
Auto merged
sql/handler.h:
Auto merged
storage/maria/ha_maria.cc:
Auto merged
storage/maria/ma_close.c:
Auto merged
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/ma_open.c:
Auto merged
storage/maria/ma_pagecache.c:
Auto merged
storage/maria/ma_pagecache.h:
Auto merged
storage/maria/maria_chk.c:
Auto merged
storage/maria/ma_blockrec.c:
Manual merge (No changes)
storage/maria/ma_check.c:
Manual merge
storage/maria/ma_create.c:
Manual merge
storage/maria/ma_delete_all.c:
Manual merge
storage/maria/ma_init.c:
Manual merge (no changes)
storage/maria/ma_test_all.sh:
Manual merge
storage/maria/maria_def.h:
Manual merge
Fixed maria_chk to repair BLOCK-ROW tables.
Added CREATE options ROW_FORMAT=PAGE & TRANSACTIONAL= 0|1
More DBUG information in a lot of functions
Some minor code cleanups
Enable handler errors earlier for better clear text error messages at handler startup / standalone usage.
Don't print NULL strings in my_create_with_symlink(); Fixes core dump when used with --debug
include/maria.h:
Added extra variables needed for REPAIR with BLOCK records
include/my_base.h:
Added argument for opening copy of maria table without a shared object
include/my_handler.h:
Prototypes for my_handler_error_register() & my_handler_error_unregister()
include/pagecache.h:
Added PAGECACHE_READ_UNKNOWN_PAGE
mysql-test/include/ps_conv.inc:
Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/maria.result:
Moved some things to maria-connect.test
Updared results as REPAIR now works
Added tests for creation option TRANSACTIONAL
mysql-test/r/ps_2myisam.result:
Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_3innodb.result:
Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_4heap.result:
Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_5merge.result:
Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_7ndb.result:
Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_maria.result:
Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/t/maria.test:
Moved some things to maria-connect.test
Updared results as REPAIR now works
Added tests for creation option TRANSACTIONAL
mysys/mf_iocache.c:
More debugging
mysys/mf_tempfile.c:
Added missing close()
mysys/my_error.c:
init_glob_errs() is now done in my_init()
mysys/my_handler.c:
Added functions to initialize handler error messages
mysys/my_init.c:
Moevd init_glob_errs() here.
mysys/my_open.c:
More comments
More debugging
Code cleanup (join multiple code paths) and indentation fixes. No change in logic.
mysys/my_symlink2.c:
Don't print NULL strings
sql/handler.cc:
Added printing of PAGE row type
Moved out initializing of handler errors to allow handler to give better error messages at startup
sql/handler.h:
ROW_TYPE_PAGES -> ROW_TYPE_PAGE
sql/lex.h:
Added 'PAGE' and 'TRANSACTIONAL'
sql/mysqld.cc:
Initialize handler error messages early to get better error messages from handler startup
sql/sql_show.cc:
ROW_TYPE_PAGES -> ROW_TYPE_PAGE
sql/sql_table.cc:
Removed not needed initializer
sql/sql_yacc.yy:
Added CREATE options ROW_FORMAT=PAGE and TRANSACTIONAL=[0|1]
sql/table.cc:
Store transactional flag in .frm
More comments
sql-bench/example:
Better example
sql/table.h:
Added transactional table option
storage/maria/ha_maria.cc:
More debug information
Enable REPAIR
Detect usage of TRANSACTIONAL table option
storage/maria/ma_bitmap.c:
More comments (from Guilhem)
storage/maria/ma_blockrec.c:
SANITY_CHECK -> SANITY_CHECKS (fixed typo)
Write out pages on delete even if there is no rows. (Fixed problem with REPAIR)
Removed some ASSERTS to runtime checks (for better REPAIR)
Fixed bug when scanning rows
More DBUG information
storage/maria/ma_check.c:
Partial rewrite to allow REPAIR of BLOCK/PAGE format.
Repair of BLOCK format rows is for now only done with 'maria_repair()' (= repair through key cache)
The new logic to repair rows with BLOCK format is:
- Create new, unrelated MARIA_HA of the table
- Create new datafile and associate it with new handler
- Reset all statistic information in new handler
- Copy all data to new handler with normal write operations
- Move state of new handler to old handler
- Close new handler
- Close data file in old handler
- Rename old data file to new data file.
- Reopen data file in old handler
storage/maria/ma_close.c:
REmoved not needed block
storage/maria/ma_create.c:
Swap arguments to _ma_initialize_data_file()
storage/maria/ma_delete_all.c:
Split maria_delete_all_rows() to two functions to allow REPAIR to easily reset all status information.
storage/maria/ma_dynrec.c:
Added checksum argument to _ma_rec_check (multi-thread fix)
storage/maria/ma_info.c:
Indentation fix
storage/maria/ma_init.c:
Register error message to get better error message on init and when using as standalone module.
storage/maria/ma_loghandler.c:
Fixed typo that disabled some error detection by valgrind
storage/maria/ma_open.c:
Added 'calc_check_checksum()'
Don't log things during repair
Added option HA_OPEN_COPY to allow one to open a Maria table with an independent share (required by REPAIR)
storage/maria/ma_pagecache.c:
Fixed some compiler warnings
Added support for PAGECACHE_READ_UNKNOWN_PAGE (used for scanning file without knowing page types)
storage/maria/ma_test_all.sh:
More test of REPAIR
storage/maria/ma_update.c:
Optimized checksum code
storage/maria/maria_chk.c:
Use DBUG_SET_INITIAL() to get DBUG to work with --parallel-repair
Ensure we always use maria_repair() for BLOCK format (for now)
More DBUG information
storage/maria/maria_def.h:
For now, always run with more checkings (SANITY_CHECKS)
Added share->calc_check_checksum to be used with REPAIR / CHECK table.
Swaped arguments to _ma_initialize_data_file()
storage/myisam/ft_stopwords.c:
Added DBUG information
mysql-test/r/maria-connect.result:
New BitKeeper file ``mysql-test/r/maria-connect.result''
mysql-test/t/maria-connect.test:
New BitKeeper file ``mysql-test/t/maria-connect.test''
creating the data file, and sync this log, so that the table cannot be
used if log record didn't reach disk. The same way, we force the log
in DROP/RENAME TABLE. Also in REPAIR TABLE though logging in this
case is not polished.
Making DELETE FROM t <no WHERE> atomic: we log the record before
starting the operation, and will finish this op at Recovery if needed.
storage/maria/ma_check.c:
comment. Force the log record for the log to have a complete history.
storage/maria/ma_create.c:
better conformance to the text of WL#3239 "log CREATE TABLE in Maria":
write the log record before creating the data file. This ensures
that the log can be applied to an old backup in all circumstances.
errpos=2 was wrong.
storage/maria/ma_delete_all.c:
making DELETE FROM t <no WHERE> atomic: we log the record before
starting the operation, and will finish the operation at Recovery
if needed. Thus there is no need to force files to disk.
storage/maria/ma_delete_table.c:
forcing the log before dropping a table, so that the log has the
entire history.
storage/maria/ma_loghandler.c:
LOGREC_REDO_DELETE_ALL needs to set trn's rec_lsn so that the log's
low-water mark and Checkpoint retain this record until the
delete operation has finished.
storage/maria/ma_rename.c:
force the log before renaming a table, so that the log has a complete
history.
fixes for build failures; copyrights; small bugfixes and comments
mysys/Makefile.am:
missing .h breaks building from tarball
storage/maria/ma_loghandler.c:
applying Serg's bugfix of trnman_new_trid() to translog_assign_id_to_share()
storage/maria/ma_loghandler.h:
copyright
storage/maria/ma_loghandler_lsn.h:
copyright
storage/maria/maria_read_log.c:
fix for compiler warnings. Comments.
Close tables when program ends.
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
into desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-lsn
storage/maria/ma_loghandler.h:
Auto merged
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Auto merged
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Auto merged
storage/maria/ma_loghandler.c:
merge
storage/maria/unittest/ma_test_loghandler-t.c:
merge
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
merge
relative LSN independed on "distance".
Added support for test record descriptors to avoid
interfere woth real record descriptors.
Fixed descriptor of pseudofixed length record length,
now it is length of record passed from/to client of
the loghandler.
BitKeeper/etc/ignore:
Added storage/maria/unittest/ma_test_loghandler_long-t-big to the ignore list
storage/maria/ma_init.c:
Removed loghandler_init call because it is present in translog_init()
storage/maria/ma_loghandler.c:
Fixed LSN codding to allow code all
relative LSN independed on "distance".
Added support for test record descriptors to avoid
interfere woth real record descriptors.
Fixed length of LOGREC_REDO_INSERT_ROW_TAIL.
Fixed descriptor of pseudofixed length record length,
now it is length of record passed from/to client of
the loghandler.
storage/maria/ma_loghandler.h:
Added support for test record descriptors to avoid
interfere woth real record descriptors.
storage/maria/unittest/Makefile.am:
Made new test for log with reference over 63 files.
Layout fixed.
storage/maria/unittest/ma_test_loghandler-t.c:
Added support for test record descriptors to avoid
interfere woth real record descriptors.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Added support for test record descriptors to avoid
interfere woth real record descriptors.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Added support for test record descriptors to avoid
interfere woth real record descriptors.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Added support for test record descriptors to avoid
interfere woth real record descriptors.
specific
- adding TRN::first_undo_lsn, needed to know when a log can be deleted;
this variable must be set under log's mutex and that leads to setting
TRN::rec_lsn, TRN::undo_lsn and TRN::first_undo_lsn in a
inwrite_rec_hook; adding implementation of one hook for REDOs and one
for UNDOs. Thus translog_write_record() always uses TRN and so does
not need a short_id argument, can find it from TRN.
- Monty's patch for the last Valgrind error in the tree.
- Log handler's unit tests fail but Sanja says it's known
include/Makefile.am:
pagecache.h moved and renamed
include/maria.h:
pagecache.h moved and renamed
sql/handler.h:
pagecache.h moved and renamed
storage/maria/Makefile.am:
pagecache.h moved and renamed
storage/maria/ha_maria.cc:
adding an assertion which sounds logical
storage/maria/ma_blockrec.c:
trn->rec_lsn and trn->undo_lsn are now set via hooks inside the log
record's writing; this allows to also set trn->first_undo_lsn
needed to compute the log's low-water mark.
The PAGERANGE_STORE_SIZE -> PAGE_STORE_SIZE is Monty's fix to a
Valgrind error.
storage/maria/ma_loghandler.c:
"tcb" renamed to "trn". Log handler now knows what is a transaction,
and finds short_id from trn. trn's rec_lsn, undo_lsn, first_undo_lsn
are now set by some inwrite_rec_hookS (one for REDOs, one for UNDOs).
The HAVE_purify blocks are Monty's fix to a Valgrind error.
storage/maria/ma_loghandler.h:
Log handler functions use TRN, that needs a forward declaration
storage/maria/ma_pagecache.c:
pagecache.h was moved and renamed
storage/maria/ma_pagecache.h:
pagecache.h was moved and renamed
storage/maria/ma_pagecaches.c:
pagecache.h was moved and renamed
storage/maria/trnman.c:
initializing some members of TRN.
storage/maria/trnman.h:
TRN::first_undo_lsn needed for log's low-water mark calculation
(which will serve to know which logs can be deleted)
storage/maria/unittest/ma_test_loghandler-t.c:
translog_write_record() now needs a valid TRN
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
translog_write_record() now needs a valid TRN
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
translog_write_record() now needs a valid TRN
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
translog_write_record() now needs a valid TRN
storage/maria/unittest/test_file.h:
pagecache.h was moved and renamed
Fixed bug in ma_dbug.c that gave valgrind warning (only relevant when using --debug)
Fixed bug in blob logging (Fixes valgrind warning)
maria_getint() -> maria_data_on_page()
mysys/safemalloc.c:
Added debug function to print out where a piece of memory was allocated
sql/opt_range.cc:
Remove DBUG_PRINT of unitailized memory
storage/maria/ma_blockrec.c:
Fixed bug in blob logging
storage/maria/ma_check.c:
Fixed compiler warning
storage/maria/ma_dbug.c:
Added missed end++; Caused usage of unitialized memory for nullable keys that was not NULL
storage/maria/ma_delete.c:
maria_getint() -> maria_data_on_page()
storage/maria/ma_init.c:
Added header file to get rid of warning
storage/maria/ma_key.c:
More debugging
storage/maria/ma_loghandler.c:
Removed some wrong ';' to get rid of compiler errors when compiling without debugging
Indentation fixes
Removed not needed 'break's
Fixed some compiler warnings
Added code to detect logging of unitialized memory
storage/maria/ma_page.c:
maria_getint() -> maria_data_on_page()
Clear rest of index page before writing when used with valgrind
(Fixes warning of writing pages with unitialized data)
storage/maria/ma_range.c:
maria_getint() -> maria_data_on_page()
storage/maria/ma_rt_index.c:
maria_getint() -> maria_data_on_page()
storage/maria/ma_rt_index.h:
maria_getint() -> maria_data_on_page()
storage/maria/ma_rt_key.c:
maria_getint() -> maria_data_on_page()
storage/maria/ma_rt_split.c:
maria_getint() -> maria_data_on_page()
storage/maria/ma_search.c:
maria_getint() -> maria_data_on_page()
storage/maria/ma_test1.c:
Fixed compiler warning
storage/maria/ma_write.c:
maria_getint() -> maria_data_on_page()
storage/maria/maria_chk.c:
maria_getint() -> maria_data_on_page()
storage/maria/maria_def.h:
maria_getint() -> maria_data_on_page()
storage/maria/unittest/ma_pagecache_consist.c:
Fixed compiler warning
storage/maria/unittest/ma_pagecache_single.c:
Fixed compiler warning
storage/maria/unittest/ma_test_loghandler-t.c:
Fixed compiler warning
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Fixed compiler warning
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Fixed compiler warning
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Fixed compiler warning
storage/myisam/mi_dbug.c:
Added missed end++; Caused usage of unitialized memory for nullable keys that was not NULL
Added maria_clone(), needed by future REPAIR code
storage/maria/unittest/ma_pagecache_consist.c:
Change mode to -rw-rw-r--
storage/maria/unittest/lockman-t.c:
Change mode to -rw-rw-r--
storage/maria/unittest/lockman1-t.c:
Change mode to -rw-rw-r--
storage/maria/unittest/lockman2-t.c:
Change mode to -rw-rw-r--
storage/maria/unittest/trnman-t.c:
Change mode to -rw-rw-r--
include/maria.h:
Added prototype for maria_clone (for future)
storage/maria/ha_maria.cc:
Move filename to share structure
storage/maria/ma_blockrec.c:
rec_lsn (first REDO LSN( is now given to the page cache on unpinning
Removed impossible lock handling in get_head_or_tail_page()
Changed calls ot translog_write_record() to remember rec_lsn
Removed some logging in csse of not transactions
storage/maria/ma_delete.c:
info->filename -> info->s->open_file_name
storage/maria/ma_loghandler.c:
Indentation fixes
storage/maria/ma_open.c:
Added maria_clone(), needed by future REPAIR code
storage/maria/ma_packrec.c:
Fixed typo in comment
storage/maria/ma_pagecache.c:
Added comment.
Allow setting REC_LSN in case of read lock
storage/maria/ma_update.c:
info->filename -> info->s->open_file_name
storage/maria/ma_write.c:
info->filename -> info->s->open_file_name
storage/maria/maria_def.h:
info->filename -> info->s->open_file_name
Added have_rtree to simplify test in ma_clone()
storage/maria/maria_ftdump.c:
info->filename -> info->s->open_file_name
storage/maria/maria_pack.c:
info->filename -> info->s->open_file_name
storage/maria/trnman.h:
Added rec_lsn
Some fixes from Sanja
BitKeeper/etc/ignore:
added storage/maria/maria_log.*
include/pagecache.h:
Always have enum PAGECACHE_EMPTY_PAGE available
(Simpler code)
storage/maria/ma_bitmap.c:
Reset 'debugging' bitmap when creating new one (fixes valgrind warning)
storage/maria/ma_blockrec.c:
Removed duplicate (wrong) initialization
Reset not initialized variable
storage/maria/ma_check.c:
Use right page type
(Patch from Sanja)
storage/maria/ma_init.c:
Reset logging in maria_end()
(Fixes memory leak)
storage/maria/ma_loghandler.c:
Add missing copyright header
Added checking of duplicate calls or calls without init to translog_destroy()
Don't lock mutex before destroying them (not needed as you can't use a destroyed mutex anyway)
storage/maria/ma_pagecache.c:
Added extra page type text
Trivial indentation fixes
storage/maria/ma_test1.c:
Added transaction setup
(Patch from Sanja)
storage/maria/ma_test2.c:
Added transaction setup
(Patch from Sanja)
Added logging and pinning of pages to block format.
Integration of transaction manager, log handler.
Better page cache intergration
Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs.
Renaming of structures, more comments, more debugging etc.
Fixed problem with small head block + long varchar.
Added extra argument to delete_record() and update_record() (needed for UNDO logging)
Small changes to interface of pagecache and log handler.
Change initialization of log_record_type_descriptors to not be depending on enum order.
Use array of LEX_STRING's to send data to log handler
Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists.
include/lf.h:
Interface fixes
Rename of structures
(Patch from Sergei via Sanja)
include/my_atomic.h:
More comments
include/my_global.h:
Added MY_ERRPTR
include/pagecache.h:
Added undo LSN when unlocking pages
mysql-test/r/maria.result:
Updated results
mysql-test/t/maria.test:
Added autocommit around lock tables
(Patch from Sanja)
mysys/lf_alloc-pin.c:
Post-review fixes, simple optimizations
More comments
Struct slot renames
Check amount of memory on stack
(Patch from Sergei)
mysys/lf_dynarray.c:
More comments
mysys/lf_hash.c:
More comments
After review fixes
(Patch from Sergei)
storage/maria/ha_maria.cc:
Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program.
(Temporary fix to avoid bug in gcc)
Move out all deferencing of the transaction structure.
Transaction manager integrated (Patch from Sergei)
storage/maria/ha_maria.h:
Added prototype for start_stmt()
storage/maria/lockman.c:
Function call rename
storage/maria/ma_bitmap.c:
Mark deleted pages free from page cache
storage/maria/ma_blockrec.c:
Offset -> rownr
More debugging
Fixed problem with small head block + long varchar
Added logging of changed pages
Added logging of undo (Including only loggging of changed fields in case of update)
Added pinning/unpinning of all changed pages
More comments
Added free_full_pages() as the same code was used in several places.
fill_rows_parts() renamed as fill_insert_undo_parts()
offset -> rownr
Added some optimization of not transactional tables
_ma_update_block_record() has new parameter, as we need original row to do efficent undo for update
storage/maria/ma_blockrec.h:
Added ROW_EXTENTS_ON_STACK
Changed prototype for update and delete of row
storage/maria/ma_check.c:
Added original row to delete_record() call
storage/maria/ma_control_file.h:
Added ifdefs for C++
storage/maria/ma_delete.c:
Added original row to delete_record() call
(Needed for efficent undo logging)
storage/maria/ma_dynrec.c:
Added extra argument to delete_record() and update_record()
Removed not used variable
storage/maria/ma_init.c:
Initialize log handler
storage/maria/ma_loghandler.c:
Removed not used variable
Change initialization of log_record_type_descriptors to not be depending on enum order
Use array of LEX_STRING's to send data to log handler
storage/maria/ma_loghandler.h:
New defines
Use array of LEX_STRING's to send data to log handler
storage/maria/ma_open.c:
Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists.
Store in MARIA_SHARE->page_type if pages will have up to date LSN's
storage/maria/ma_pagecache.c:
Don't decrease number of readers when using pagecache_write()/pagecache_read()
In pagecache_write() decrement request count if page was left pinned
Added pagecache_delete_pages()
Removed some casts
Make trace output consistent with rest of code
Simplify calling of DBUG_ASSERT(0)
Only update LSN if the LSN is bigger than what's already on the page
Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock()
(Part of patch from Sanja)
storage/maria/ma_static.c:
Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists.
Added default page cache
storage/maria/ma_statrec.c:
Added extra argument to delete_record() and update_record()
storage/maria/ma_test1.c:
Added option -T for transactions
storage/maria/ma_test2.c:
Added option -T for transactions
storage/maria/ma_test_all.sh:
Test with transactions
storage/maria/ma_update.c:
Changed prototype for update of row
storage/maria/maria_def.h:
Changed prototype for update & delete of row as block records need to access the old row
Store in MARIA_SHARE->page_type if pages will have up to date LSN's
Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need.
Removed not used 'empty_bits_buffer'
Added pointer to transaction object
Added array for pinned pages
Added log_row_parts array for logging of field data.
Added MARIA_PINNED_PAGE to store pinned pages
storage/maria/trnman.c:
Added accessor functions to transaction object
Added missing DBUG_RETURN()
More debugging
More comments
Changed // comment of code to #ifdef NOT_USED
Transaction manager integrated.
Post review fixes
Part of patch originally from Sergei
storage/maria/trnman.h:
Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program.
(Temporary fix to avoid bug in gcc)
storage/maria/unittest/ma_pagecache_single.c:
Added missing argument
Added SKIP_BIG_TESTS
(Patch from Sanja)
storage/maria/unittest/ma_test_loghandler-t.c:
Test logging with new LEX_STRING parameter
(Patch from Sanja)
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Test logging with new LEX_STRING parameter
(Patch from Sanja)
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Test logging with new LEX_STRING parameter
(Patch from Sanja)
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Test logging with new LEX_STRING parameter
(Patch from Sanja)
storage/maria/unittest/trnman-t.c:
Stack overflow detection
(Patch from Sergei)
unittest/unit.pl:
Command-line options --big and --verbose
(Patch from Sergei)
unittest/mytap/tap.c:
Detect --big
(Patch from Sergei)
unittest/mytap/tap.h:
Skip_big_tests and SKIP_BIG_TESTS
(Patch from Sergei)
storage/maria/trnman_public.h:
New BitKeeper file ``storage/maria/trnman_public.h''
into desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-test
include/maria.h:
Auto merged
mysys/Makefile.am:
Auto merged
mysys/mf_keycaches.c:
Auto merged
sql/handler.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/set_var.h:
Auto merged
storage/maria/Makefile.am:
Auto merged
storage/maria/ha_maria.cc:
Auto merged
storage/maria/ma_bitmap.c:
Auto merged
storage/maria/ma_check.c:
Auto merged
storage/maria/ma_close.c:
Auto merged
storage/maria/ma_dynrec.c:
Auto merged
storage/maria/ma_extra.c:
Auto merged
storage/maria/ma_info.c:
Auto merged
storage/maria/ma_keycache.c:
Auto merged
storage/maria/ma_locking.c:
Auto merged
storage/maria/ma_loghandler.c:
Auto merged
storage/maria/ma_open.c:
Auto merged
storage/maria/ma_packrec.c:
Auto merged
storage/maria/ma_page.c:
Auto merged
storage/maria/ma_pagecache.c:
Auto merged
storage/maria/ma_panic.c:
Auto merged
storage/maria/ma_preload.c:
Auto merged
storage/maria/ma_static.c:
Auto merged
storage/maria/ma_test1.c:
Auto merged
storage/maria/ma_test2.c:
Auto merged
storage/maria/ma_test3.c:
Auto merged
storage/maria/ma_write.c:
Auto merged
storage/maria/maria_chk.c:
Auto merged
storage/maria/maria_def.h:
Auto merged
storage/maria/maria_ftdump.c:
Auto merged
storage/maria/maria_pack.c:
Auto merged
sql/handler.cc:
merge
storage/maria/ma_blockrec.c:
merge
storage/maria/ma_delete_all.c:
merge
Fixed test for row based replication
mysql-test/mysql-test-run.pl:
After merge fix
mysql-test/r/maria.result:
Fixed test for row based replication
mysql-test/t/maria.test:
Fixed test for row based replication
storage/maria/ha_maria.cc:
After merge fix
storage/maria/ma_blockrec.c:
Better to clear whole page, as 'length' may be bigger than what we need.
storage/maria/ma_loghandler.c:
Fix compiler warning
Removed access to not initialized memory
storage/maria/ma_open.c:
Remove wrong (not needed) test
Tests: "maria" and "ps_maria" fail like before merge (assertions),
"ma_test_all" fails like before merge (ma_test2 segfaults, I'll try
to find out why).
mysys/mf_pagecache.c:
using a more distinctive tag
storage/maria/ha_maria.cc:
merge from MyISAM into Maria
storage/maria/ma_check.c:
merge from MyISAM into Maria
storage/maria/ma_close.c:
TODO as a word
storage/maria/ma_create.c:
merge from MyISAM into Maria
storage/maria/ma_delete_all.c:
TODO as a word
storage/maria/ma_delete_table.c:
TODO as a word
storage/maria/ma_dynrec.c:
merge from MyISAM into Maria
storage/maria/ma_extra.c:
merge from MyISAM into Maria
storage/maria/ma_ft_boolean_search.c:
merge from MyISAM into Maria
storage/maria/ma_locking.c:
merge from MyISAM into Maria
storage/maria/ma_loghandler.c:
fix for compiler warning
storage/maria/ma_open.c:
merge from MyISAM into Maria.
I will ask Monty to check the ASKMONTY-marked piece of code.
storage/maria/ma_packrec.c:
merge from MyISAM into Maria
storage/maria/ma_range.c:
merge from MyISAM into Maria
storage/maria/ma_rename.c:
TODO as a word
storage/maria/ma_rt_index.c:
merge from MyISAM into Maria
storage/maria/ma_rt_split.c:
merge from MyISAM into Maria
storage/maria/ma_search.c:
merge from MyISAM into Maria
storage/maria/ma_sort.c:
merge from MyISAM into Maria
storage/maria/ma_update.c:
merge from MyISAM into Maria
storage/maria/ma_write.c:
merge from MyISAM into Maria
storage/maria/maria_chk.c:
merge from MyISAM into Maria
storage/maria/maria_def.h:
merge from MyISAM into Maria
storage/maria/maria_pack.c:
merge from MyISAM into Maria
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
fix for compiler warning
storage/myisam/ha_myisam.cc:
merge from MyISAM into Maria
storage/myisammrg/ha_myisammrg.cc:
merge from MyISAM into Maria
Fixed befaviour when loghandler flags changed from one
run to another one.
Description of maria transaction log and control file
added to the file commandÍs magic number file.
mysys/mf_pagecache.c:
postreview changes
storage/maria/ma_control_file.c:
Postreview changes.
storage/maria/ma_control_file.h:
Postreview changes.
storage/maria/ma_loghandler.c:
Postreview changes.
Fixed befaviour when loghandler flags changed from
one run to another one.
storage/maria/ma_loghandler.h:
Postreview changes.
Functions comment left only near the function body.
storage/maria/ma_loghandler_lsn.h:
Postreview changes.
storage/maria/unittest/ma_test_loghandler-t.c:
Postreview changes.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Postreview changes.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Postreview changes.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Postreview changes.
support-files/magic:
Description of maria transaction log and control file
added to the file commandÕs magic number file.
Some debug info and comments added
include/pagecache.h:
postmerge fix
mysys/mf_pagecache.c:
Postmerge fix (including changing type of LSN)
Additional DBUG_ASSERTs added
Comment about pinning mechanism added
storage/maria/ma_control_file.c:
Used the same LSN storing procedure everywhere
Postmerge fix (including changing type of LSN)
storage/maria/ma_control_file.h:
Postmerge fix (including changing type of LSN)
storage/maria/ma_loghandler.c:
Postmerge fix (including changing type of LSN)
storage/maria/ma_loghandler.h:
Postmerge fix (including changing type of LSN)
storage/maria/ma_loghandler_lsn.h:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/Makefile.am:
Postmerge fix
storage/maria/unittest/ma_control_file-t.c:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler-t.c:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/mf_pagecache_consist.c:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/mf_pagecache_single.c:
Postmerge fix (including changing type of LSN)
storage/maria/unittest/test_file.c:
Rename: unittest/mysys/test_file.c -> storage/maria/unittest/test_file.c
storage/maria/unittest/test_file.h:
Rename: unittest/mysys/test_file.h -> storage/maria/unittest/test_file.h
include/pagecache.h:
A waiting queue mechanism moved to separate file wqueue.*
Pointer name changed for compatibility
mysys/Makefile.am:
A waiting queue mechanism moved to separate file wqueue.*
mysys/mf_keycache.c:
fixed unsigned comparison
mysys/mf_pagecache.c:
A waiting queue mechanism moved to separate file wqueue.*
Fixed bug in unregistering block during write
storage/maria/Makefile.am:
The loghandler files added
storage/maria/ma_control_file.h:
Now we have loghandler and can compile control file
storage/maria/maria_def.h:
Including files need for compilation of maria
storage/maria/unittest/Makefile.am:
unit tests of loghandler
storage/maria/unittest/ma_control_file-t.c:
Used maria def
storage/maria/unittest/mf_pagecache_consist.c:
fixed memory overrun
storage/maria/unittest/mf_pagecache_single.c:
fixed used uninitialized memory
unittest/mysys/Makefile.am:
unittests of pagecache moved to maria becase pagecache need loghandler
include/wqueue.h:
New BitKeeper file ``include/wqueue.h''
mysys/wqueue.c:
New BitKeeper file ``mysys/wqueue.c''
storage/maria/ma_loghandler.c:
New BitKeeper file ``storage/maria/ma_loghandler.c''
storage/maria/ma_loghandler.h:
New BitKeeper file ``storage/maria/ma_loghandler.h''
storage/maria/ma_loghandler_lsn.h:
New BitKeeper file ``storage/maria/ma_loghandler_lsn.h''
storage/maria/unittest/ma_test_loghandler-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler-t.c''
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_multigroup-t.c''
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_multithread-t.c''
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_pagecache-t.c''
