2006-09-14 19:06:51 +02:00
|
|
|
/* Copyright (C) 2006 MySQL AB & MySQL Finland AB & TCX DataKonsult AB
|
|
|
|
|
|
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
|
|
|
it under the terms of the GNU General Public License as published by
|
2007-03-02 11:20:23 +01:00
|
|
|
the Free Software Foundation; version 2 of the License.
|
2006-09-14 19:06:51 +02:00
|
|
|
|
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
|
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
GNU General Public License for more details.
|
|
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
|
|
|
along with this program; if not, write to the Free Software
|
Update FSF address
This commit is based on the work of Michal Schorm, rebased on the
earliest MariaDB version.
Th command line used to generate this diff was:
find ./ -type f \
-exec sed -i -e 's/Foundation, Inc., 59 Temple Place, Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/Foundation, Inc. 59 Temple Place.* Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/MA.*.....-1307.*USA/MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/Foundation, Inc., 59 Temple/Foundation, Inc., 51 Franklin/g' {} \; \
-exec sed -i -e 's/Place, Suite 330, Boston, MA.*02111-1307.*USA/Street, Fifth Floor, Boston, MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/MA.*.....-1307/MA 02110-1335/g' {} \;
2019-05-10 19:49:46 +02:00
|
|
|
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA */
|
2006-09-14 19:06:51 +02:00
|
|
|
|
2006-09-01 17:53:10 +02:00
|
|
|
/*
|
|
|
|
|
WL#3234 Maria control file
|
|
|
|
|
First version written by Guilhem Bichot on 2006-04-27.
|
|
|
|
|
*/
|
|
|
|
|
|
WL#3072 - Maria recovery
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.
sql/handler.cc:
typo
storage/maria/Makefile.am:
Adding ma_test_recovery (which ma_test_all invokes, and which can
also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
comments
storage/maria/ma_bitmap.c:
fixing comments. 2 -> sizeof(maria_bitmap_marker).
Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
Now putting the "bm" signature when creating the first bitmap page
(it used to happen only at next open, but that
caused an annoying difference when testing Recovery if the original
run didn't open the table, and it looks more
logical like this: it goes to disk only with its signature correct);
see the "QQ" comment towards the _ma_initialize_data_file() call
in ma_create.c for more).
When reading a bitmap page, verify its signature (happens when normally
using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
* no need to sync the data file if table is not transactional
* Comments, code cleanup (log-related data moved to log-related code
block, int5store->page_store).
* Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
do for other records (though this record will soon be replaced
with a CLR).
* If "page" is 1 it means the page which extends from byte
page*block_size+1 to (page+1)*block_size (byte number 1 being
the first byte of the file). The last byte of the file is
data_file_length (same convention).
A new page needs to be created if the last byte of the page is
beyond the last byte of the file, i.e.
(page+1)*block_size+1 > data_file_length, so we correct the test
(bug found when testing log applying for ma_test1 -M -T --skip-update).
* update the page's LSN when removing a row from it during
execution of a REDO_PURGE_ROW record (bug found when testing log
applying for ma_test1 -M -T --skip-update).
* applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
we can always flush the table's state when closing the last instance
of the table. And it is needed for maria_read_log (as it does
not use maria_lock_database()).
storage/maria/ma_control_file.c:
when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
double-inclusion safe
storage/maria/ma_create.c:
during recovery, don't log records. Comments.
Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
during recovery, don't log records. Log the end-zero of the dropped
table's name, so that recovery can use the string in place without
extending it to fit an end zero.
storage/maria/ma_loghandler.c:
* inwrite_rec_hook also needs access to the MARIA_SHARE, like
prewrite_rec_hook. This will be needed to update
share->records_diff (in the upcoming patch "recovery of the state").
* LOG_DESC::record_ends_group changed to an enum.
* LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
corrected
* Sanja please see the @todo LOG BUG
* avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
- log write hooks called while the log's lock is held (inwrite_rec_hook)
now need the MARIA_SHARE, like prewrite_rec_hook already had
- instead of a bool saying if this record's type ends groups or not,
we refine: it may not end a group, it may end a group, or it may
be a group in itself. Imagine that we had a physical write failure
to a table before we log the UNDO, we still end up in
external_lock(F_UNLCK) and then we log a COMMIT: we don't want
to consider this COMMIT as ending the group of REDOs (don't want
to execute those REDOs during Recovery), that's why we say "COMMIT
is a group in itself, it aborts any previous group". This also
gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
New Recovery code, replacing the old pseudocode.
Most of maria_read_log moved here.
Call-able from ha_maria, but not enabled yet.
Compared to the previous version of maria_read_log, some bugs have
been fixed, debugging output can go to stdout or a disk file (for now
it's useful for me, later it can be changed), execution of
REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
has been factored into functions. We abort an unfinished group
of records if we see a record which is a group in itself (like COMMIT).
No need for maria_panic() after a bug (which caused tables to not
be closed) was fixed; if there is yet another bug I prefer to see it.
When opening a table for Recovery, set data_file_length
and key_file_length to their real physical value (these are the
easiest state members to restore :). Warn us if the last page
was truncated (but Recovery handles it).
MARIA_SHARE::state::state::records is now partly recovered (not
idempotent, but works if recreating tables from scracth).
When applying a REDO to a page, stamp it with the UNDO's LSN
(current_group_end_lsn), not with the REDO's LSN; it makes
the table more identical to the original table (easier to compare
the two tables in the end).
Big thing missing: some types of REDOs are not handled,
and the UNDO phase does not exist (missing functions to execute UNDOs
to actually rollback). So for now tests are only inserting/deleting
a few 100 rows, closing the table and seeing if the log is applied ok;
it works. UPDATE not handled.
storage/maria/ma_recovery.h:
new functions: ma_recover() for recovery from inside ha_maria;
_ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
don't write log records during recovery
storage/maria/ma_test2.c:
- fail if maria_info() or other subtests find some wrong information
- new option -g to skip updates.
- init the translog before creating the table, so that log applying
can work.
- in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
cleanup files. Test log applying.
storage/maria/maria_read_log.c:
most of the logic moves to ma_recovery.c to be shared between
maria_read_log and recovery-from-inside-mysqld.
See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
unit test for Recovery. Tests insert and delete,
REDO_UPDATE not yet coded.
Script is called from ma_test_all. Can run standalone.
2007-07-26 11:56:21 +02:00
|
|
|
#ifndef _ma_control_file_h
|
|
|
|
|
#define _ma_control_file_h
|
|
|
|
|
|
2010-09-12 18:40:01 +02:00
|
|
|
#define CONTROL_FILE_BASE_NAME "aria_log_control"
|
2007-11-20 16:42:16 +01:00
|
|
|
/*
|
|
|
|
|
Major version for control file. Should only be changed when doing
|
|
|
|
|
big changes that made the new control file incompatible with all
|
|
|
|
|
older versions of Maria.
|
|
|
|
|
*/
|
|
|
|
|
#define CONTROL_FILE_VERSION 1
|
2006-09-01 17:53:10 +02:00
|
|
|
|
|
|
|
|
/* Here is the interface of this module */
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
LSN of the last checkoint
|
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
2007-06-26 16:49:23 +02:00
|
|
|
(if last_checkpoint_lsn == LSN_IMPOSSIBLE then there was never a checkpoint)
|
2006-09-01 17:53:10 +02:00
|
|
|
*/
|
|
|
|
|
extern LSN last_checkpoint_lsn;
|
|
|
|
|
/*
|
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
2007-06-26 16:49:23 +02:00
|
|
|
Last log number (if last_logno == FILENO_IMPOSSIBLE then there is no log
|
|
|
|
|
file yet)
|
2006-09-01 17:53:10 +02:00
|
|
|
*/
|
|
|
|
|
extern uint32 last_logno;
|
|
|
|
|
|
2018-08-31 09:47:18 +02:00
|
|
|
extern TrID max_trid_in_control_file, max_long_trid;
|
Store maximum transaction id into control file at clean shutdown.
This can serve to maria_chk to check that trids found in rows and keys
are not too big. Also used by Recovery when logs are lost.
Options --require-control-file, --datadir, --log-dir (yes, the dashes are
inconsistent but I imitated mysqld --datadir and --maria-log-dir) for
maria_chk.
Lock control file _before_ reading its content.
storage/maria/ha_maria.cc:
new prototype
storage/maria/ma_check.c:
A function to find the max trid in the system (consults transaction
manager and control file), to check tables.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Store max trid into control file, in a backward-compatible way
(can still read old control files).
Parameter to ma_control_file_open(), to not create the log if it's
missing (maria_chk needs that).
Lock control file _before_ reading its content.
Fix for a segfault when reading an old control file (bzero() with a
negative second argument)
storage/maria/ma_control_file.h:
changes to the control file module's API
storage/maria/ma_init.c:
When Maria shuts down cleanly, store max trid into control file.
storage/maria/ma_loghandler.c:
new prototype
storage/maria/ma_recovery.c:
During recovery, consult max trid stored in control file, in case it is
bigger than what we found in log (case of logs manually removed by user).
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/maria_chk.c:
New option --require-control-file (abort if control file not found),
--datadir (path for control file (and for logs if --log-dir not specified)),
--log-dir (path for logs).
Try to open control file when maria_chk starts.
storage/maria/maria_read_log.c:
new prototype
storage/maria/trnman.c:
A new function to know max trid in transaction manager
storage/maria/trnman_public.h:
New function
storage/maria/unittest/ma_control_file-t.c:
new prototypes. Testing storing and retrieving the max trid to/from
control file
storage/maria/unittest/ma_test_loghandler-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_purge-t.c:
new prototype
2008-04-04 19:10:53 +02:00
|
|
|
|
WL#4374 "Maria - force start if Recovery fails multiple times"
http://forge.mysql.com/worklog/task.php?id=4374
new option --maria-force-start-after-recovery-failures=N; number of consecutive recovery failures (failures
of log reading or recovery processing, anything in [translog_init(),maria_recovery_from_log()])
is stored in the control file; if at a Maria start they are more than N, logs are removed. This is for automated
systems which have to run whatever happens. As tables risk staying corrupted, --maria-recover should also
be used on them: this revision makes maria-recover work (it was disabled).
Fixed bug in translog_is_log_files(). translog_init() now prints message to error log if failed.
Removed \0 in the output of SHOW ENGINE MARIA LOGS; removed hard-coded engine name there.
KNOWN_BUGS.txt:
As option --maria-force-start-after-recovery-failures is added, it corresponds to the wish "we should fix that if this happens etc".
LOAD INDEX is not ignored since a few weeks. Listed concurrency bugs have been fixed some time ago.
Recovery of fulltext and GIS indexes works since a few weeks.
mysql-test/include/maria_make_snapshot.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_make_snapshot_for_comparison.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_make_snapshot_for_feeding_recovery.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_verify_recovery.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/lib/mtr_report.pl:
new test maria-recover.test generates expected corruption warnings in the error log. maria-recovery.test's corrupted table is renamed to t_corrupted1 instead of t1.
mysql-test/r/maria-preload.result:
result update. maria_pagecache_read* values are similar to the previous version of this file, though a bit bigger
because using the information_schema and the join leads to some internal maria temp table being used, and thus some
blocks of it being read.
mysql-test/r/maria-purge.result:
engine's name in SHOW ENGINE MARIA LOGS changed.
mysql-test/r/maria-recover.result:
result for new test. We see corruption messages at first SELECT and then none at second SELECT, expected.
mysql-test/r/maria-recovery.result:
result update
mysql-test/r/maria.result:
new variables show up
mysql-test/t/disabled.def:
BUG#34911 is not fixed but the test had been made independent of the bug (workaround). A new bug (crash) has popped recently, so it has to stay
disabled (BUG#35107).
mysql-test/t/maria-preload.test:
Work around BUG#34911 "FLUSH STATUS doesn't flush what it should":
compute differences in status variables before and after relevant queries
mysql-test/t/maria-recover-master.opt:
test --maria-recover
mysql-test/t/maria-recover.test:
Test of the --maria-recover option (build a corrupted table and see if it is auto-repaired)
mysql-test/t/maria-recovery-big.test:
update for new API of include/maria*.inc
mysql-test/t/maria-recovery-bitmap.test:
update for new API of include/maria*.inc
mysql-test/t/maria-recovery.test:
update for new API of include/maria*.inc. Corrupted table t1 renamed to t_corrupted1, so that mtr_report.pl
does not blindly remove all corruption messages for t1 which is
a common name.
storage/maria/ha_maria.cc:
Enabling maria-recover.
Adding option and global variable --maria_force_start_after_recovery_failures: ha_maria_init()
calls mark_recovery_start() and mark_recovery_success() to keep track of failed consecutive recoveries
and remove logs if needed.
Removed \0 in the output of SHOW ENGINE MARIA LOGS; removed hard-coded engine name there.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Storing in one byte in the control file, the number of consecutive recovery failures.
storage/maria/ma_control_file.h:
new prototype
storage/maria/ma_init.c:
new prototype
storage/maria/ma_locking.c:
Need to update open_count on disk at first write and close for transactional tables, like we already did for
non-transactional tables, otherwise we cannot notice that the table is dubious.
storage/maria/ma_loghandler.c:
translog_is_log_files() is made more generic to serve either to search or to delete logs (the latter is
for --maria-force-start-after-recovery-failures). It also had a bug (always returned FALSE).
storage/maria/ma_loghandler.h:
export function because ha_maria::mark_recovery_start() needs it
storage/maria/ma_recovery.c:
changing name of maria_recover() to distinguish from the maria-recover option.
storage/maria/ma_recovery.h:
changing name of maria_recover() to distinguish from the maria-recover option.
storage/maria/ma_test_force_start.pl:
Test of --maria-force-start-after-recovery-failures (and also, to be realistic, of --maria-recover).
This is standalone because mysql-test-run does not support testing that multiple mysqld restarts expectedly failed.
I'll have to run it on my machine and also on a Windows machine.
storage/maria/unittest/ma_control_file-t.c:
adding recovery_failures to the test
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
fix for compiler warning (unused variable in non-debug build)
2008-06-02 22:53:25 +02:00
|
|
|
extern uint8 recovery_failures;
|
|
|
|
|
|
WL#3072 - Maria recovery
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.
sql/handler.cc:
typo
storage/maria/Makefile.am:
Adding ma_test_recovery (which ma_test_all invokes, and which can
also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
comments
storage/maria/ma_bitmap.c:
fixing comments. 2 -> sizeof(maria_bitmap_marker).
Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
Now putting the "bm" signature when creating the first bitmap page
(it used to happen only at next open, but that
caused an annoying difference when testing Recovery if the original
run didn't open the table, and it looks more
logical like this: it goes to disk only with its signature correct);
see the "QQ" comment towards the _ma_initialize_data_file() call
in ma_create.c for more).
When reading a bitmap page, verify its signature (happens when normally
using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
* no need to sync the data file if table is not transactional
* Comments, code cleanup (log-related data moved to log-related code
block, int5store->page_store).
* Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
do for other records (though this record will soon be replaced
with a CLR).
* If "page" is 1 it means the page which extends from byte
page*block_size+1 to (page+1)*block_size (byte number 1 being
the first byte of the file). The last byte of the file is
data_file_length (same convention).
A new page needs to be created if the last byte of the page is
beyond the last byte of the file, i.e.
(page+1)*block_size+1 > data_file_length, so we correct the test
(bug found when testing log applying for ma_test1 -M -T --skip-update).
* update the page's LSN when removing a row from it during
execution of a REDO_PURGE_ROW record (bug found when testing log
applying for ma_test1 -M -T --skip-update).
* applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
we can always flush the table's state when closing the last instance
of the table. And it is needed for maria_read_log (as it does
not use maria_lock_database()).
storage/maria/ma_control_file.c:
when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
double-inclusion safe
storage/maria/ma_create.c:
during recovery, don't log records. Comments.
Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
during recovery, don't log records. Log the end-zero of the dropped
table's name, so that recovery can use the string in place without
extending it to fit an end zero.
storage/maria/ma_loghandler.c:
* inwrite_rec_hook also needs access to the MARIA_SHARE, like
prewrite_rec_hook. This will be needed to update
share->records_diff (in the upcoming patch "recovery of the state").
* LOG_DESC::record_ends_group changed to an enum.
* LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
corrected
* Sanja please see the @todo LOG BUG
* avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
- log write hooks called while the log's lock is held (inwrite_rec_hook)
now need the MARIA_SHARE, like prewrite_rec_hook already had
- instead of a bool saying if this record's type ends groups or not,
we refine: it may not end a group, it may end a group, or it may
be a group in itself. Imagine that we had a physical write failure
to a table before we log the UNDO, we still end up in
external_lock(F_UNLCK) and then we log a COMMIT: we don't want
to consider this COMMIT as ending the group of REDOs (don't want
to execute those REDOs during Recovery), that's why we say "COMMIT
is a group in itself, it aborts any previous group". This also
gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
New Recovery code, replacing the old pseudocode.
Most of maria_read_log moved here.
Call-able from ha_maria, but not enabled yet.
Compared to the previous version of maria_read_log, some bugs have
been fixed, debugging output can go to stdout or a disk file (for now
it's useful for me, later it can be changed), execution of
REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
has been factored into functions. We abort an unfinished group
of records if we see a record which is a group in itself (like COMMIT).
No need for maria_panic() after a bug (which caused tables to not
be closed) was fixed; if there is yet another bug I prefer to see it.
When opening a table for Recovery, set data_file_length
and key_file_length to their real physical value (these are the
easiest state members to restore :). Warn us if the last page
was truncated (but Recovery handles it).
MARIA_SHARE::state::state::records is now partly recovered (not
idempotent, but works if recreating tables from scracth).
When applying a REDO to a page, stamp it with the UNDO's LSN
(current_group_end_lsn), not with the REDO's LSN; it makes
the table more identical to the original table (easier to compare
the two tables in the end).
Big thing missing: some types of REDOs are not handled,
and the UNDO phase does not exist (missing functions to execute UNDOs
to actually rollback). So for now tests are only inserting/deleting
a few 100 rows, closing the table and seeing if the log is applied ok;
it works. UPDATE not handled.
storage/maria/ma_recovery.h:
new functions: ma_recover() for recovery from inside ha_maria;
_ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
don't write log records during recovery
storage/maria/ma_test2.c:
- fail if maria_info() or other subtests find some wrong information
- new option -g to skip updates.
- init the translog before creating the table, so that log applying
can work.
- in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
cleanup files. Test log applying.
storage/maria/maria_read_log.c:
most of the logic moves to ma_recovery.c to be shared between
maria_read_log and recovery-from-inside-mysqld.
See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
unit test for Recovery. Tests insert and delete,
REDO_UPDATE not yet coded.
Script is called from ma_test_all. Can run standalone.
2007-07-26 11:56:21 +02:00
|
|
|
extern my_bool maria_multi_threaded, maria_in_recovery;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
|
WL#3234 Maria control file manager.
Fitting ma_control_file_test into the mytap unittest framework:
new directories:
- unittest/storage/ for unit tests of any storage engine
- unittest/storage/maria for ... Maria, containing ma_control_file-t.
Later, older tests like ma_test*, ma_test_all (but which is Unix
dependent in its current form) could move here too.
The plugins macro enable building of unittest/storage/X for any
enabled engine X which has such a directory.
If Falcon wants to have unit tests there too, I may have to merge
this patch into 5.x one day.
config/ac-macros/plugins.m4:
If a storage engine has a directory in unittest/storage, build this
directory.
configure.in:
build storage engines' unit tests.
storage/maria/Makefile.am:
ma_control_file_test moves to unittest/storage/maria
storage/maria/ma_control_file.c:
more error codes when opening the control file fails.
ma_control_file_end() may now return an error if my_close() failed.
storage/maria/ma_control_file.h:
more error codes when opening the control file fails.
unittest/Makefile.am:
adding unit tests for storage engines.
Note that unit.pl simply recurses into "storage", so if a unit test for
storage engine X has been built previously, and now you re-configure
(without making clean) to disable this engine, then the unit test of
X will not be rebuilt but will still be present in storage/X, so will
be run.
unittest/storage/maria/ma_control_file-t.c:
Making the test fit the mytap framework (return all the way up
the stack instead of assert(); use the mytap functions plan(), ok() etc).
Adding test of file too short/long.
unittest/storage/maria/Makefile.am:
a_control_file-t is added to the Maria unit tests.
Later, older tests (ma_test1 etc) could also move here.
unittest/storage/Makefile.am:
New BitKeeper file ``unittest/storage/Makefile.am''
2006-09-11 16:12:31 +02:00
|
|
|
typedef enum enum_control_file_error {
|
|
|
|
|
CONTROL_FILE_OK= 0,
|
|
|
|
|
CONTROL_FILE_TOO_SMALL,
|
|
|
|
|
CONTROL_FILE_TOO_BIG,
|
|
|
|
|
CONTROL_FILE_BAD_MAGIC_STRING,
|
2007-11-20 16:42:16 +01:00
|
|
|
CONTROL_FILE_BAD_VERSION,
|
WL#3234 Maria control file manager.
Fitting ma_control_file_test into the mytap unittest framework:
new directories:
- unittest/storage/ for unit tests of any storage engine
- unittest/storage/maria for ... Maria, containing ma_control_file-t.
Later, older tests like ma_test*, ma_test_all (but which is Unix
dependent in its current form) could move here too.
The plugins macro enable building of unittest/storage/X for any
enabled engine X which has such a directory.
If Falcon wants to have unit tests there too, I may have to merge
this patch into 5.x one day.
config/ac-macros/plugins.m4:
If a storage engine has a directory in unittest/storage, build this
directory.
configure.in:
build storage engines' unit tests.
storage/maria/Makefile.am:
ma_control_file_test moves to unittest/storage/maria
storage/maria/ma_control_file.c:
more error codes when opening the control file fails.
ma_control_file_end() may now return an error if my_close() failed.
storage/maria/ma_control_file.h:
more error codes when opening the control file fails.
unittest/Makefile.am:
adding unit tests for storage engines.
Note that unit.pl simply recurses into "storage", so if a unit test for
storage engine X has been built previously, and now you re-configure
(without making clean) to disable this engine, then the unit test of
X will not be rebuilt but will still be present in storage/X, so will
be run.
unittest/storage/maria/ma_control_file-t.c:
Making the test fit the mytap framework (return all the way up
the stack instead of assert(); use the mytap functions plan(), ok() etc).
Adding test of file too short/long.
unittest/storage/maria/Makefile.am:
a_control_file-t is added to the Maria unit tests.
Later, older tests (ma_test1 etc) could also move here.
unittest/storage/Makefile.am:
New BitKeeper file ``unittest/storage/Makefile.am''
2006-09-11 16:12:31 +02:00
|
|
|
CONTROL_FILE_BAD_CHECKSUM,
|
2007-11-20 16:42:16 +01:00
|
|
|
CONTROL_FILE_BAD_HEAD_CHECKSUM,
|
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
2007-06-26 16:49:23 +02:00
|
|
|
CONTROL_FILE_MISSING,
|
2007-11-20 16:42:16 +01:00
|
|
|
CONTROL_FILE_INCONSISTENT_INFORMATION,
|
|
|
|
|
CONTROL_FILE_WRONG_BLOCKSIZE,
|
WL#3234 Maria control file manager.
Fitting ma_control_file_test into the mytap unittest framework:
new directories:
- unittest/storage/ for unit tests of any storage engine
- unittest/storage/maria for ... Maria, containing ma_control_file-t.
Later, older tests like ma_test*, ma_test_all (but which is Unix
dependent in its current form) could move here too.
The plugins macro enable building of unittest/storage/X for any
enabled engine X which has such a directory.
If Falcon wants to have unit tests there too, I may have to merge
this patch into 5.x one day.
config/ac-macros/plugins.m4:
If a storage engine has a directory in unittest/storage, build this
directory.
configure.in:
build storage engines' unit tests.
storage/maria/Makefile.am:
ma_control_file_test moves to unittest/storage/maria
storage/maria/ma_control_file.c:
more error codes when opening the control file fails.
ma_control_file_end() may now return an error if my_close() failed.
storage/maria/ma_control_file.h:
more error codes when opening the control file fails.
unittest/Makefile.am:
adding unit tests for storage engines.
Note that unit.pl simply recurses into "storage", so if a unit test for
storage engine X has been built previously, and now you re-configure
(without making clean) to disable this engine, then the unit test of
X will not be rebuilt but will still be present in storage/X, so will
be run.
unittest/storage/maria/ma_control_file-t.c:
Making the test fit the mytap framework (return all the way up
the stack instead of assert(); use the mytap functions plan(), ok() etc).
Adding test of file too short/long.
unittest/storage/maria/Makefile.am:
a_control_file-t is added to the Maria unit tests.
Later, older tests (ma_test1 etc) could also move here.
unittest/storage/Makefile.am:
New BitKeeper file ``unittest/storage/Makefile.am''
2006-09-11 16:12:31 +02:00
|
|
|
CONTROL_FILE_UNKNOWN_ERROR /* any other error */
|
|
|
|
|
} CONTROL_FILE_ERROR;
|
2007-02-19 22:01:27 +01:00
|
|
|
|
Store maximum transaction id into control file at clean shutdown.
This can serve to maria_chk to check that trids found in rows and keys
are not too big. Also used by Recovery when logs are lost.
Options --require-control-file, --datadir, --log-dir (yes, the dashes are
inconsistent but I imitated mysqld --datadir and --maria-log-dir) for
maria_chk.
Lock control file _before_ reading its content.
storage/maria/ha_maria.cc:
new prototype
storage/maria/ma_check.c:
A function to find the max trid in the system (consults transaction
manager and control file), to check tables.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Store max trid into control file, in a backward-compatible way
(can still read old control files).
Parameter to ma_control_file_open(), to not create the log if it's
missing (maria_chk needs that).
Lock control file _before_ reading its content.
Fix for a segfault when reading an old control file (bzero() with a
negative second argument)
storage/maria/ma_control_file.h:
changes to the control file module's API
storage/maria/ma_init.c:
When Maria shuts down cleanly, store max trid into control file.
storage/maria/ma_loghandler.c:
new prototype
storage/maria/ma_recovery.c:
During recovery, consult max trid stored in control file, in case it is
bigger than what we found in log (case of logs manually removed by user).
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/maria_chk.c:
New option --require-control-file (abort if control file not found),
--datadir (path for control file (and for logs if --log-dir not specified)),
--log-dir (path for logs).
Try to open control file when maria_chk starts.
storage/maria/maria_read_log.c:
new prototype
storage/maria/trnman.c:
A new function to know max trid in transaction manager
storage/maria/trnman_public.h:
New function
storage/maria/unittest/ma_control_file-t.c:
new prototypes. Testing storing and retrieving the max trid to/from
control file
storage/maria/unittest/ma_test_loghandler-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_purge-t.c:
new prototype
2008-04-04 19:10:53 +02:00
|
|
|
C_MODE_START
|
Added versioning of row data
Will in future changeset (soon) av versioning of status variables (number of rows) and index
Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings
Removed some not needed variables (as noticed by Guilhem)
include/maria.h:
Added prototypes for maria_chk_init_for_check(), maria_versioning() and maria_ignore_trids()
include/my_base.h:
Add new error HA_ERR_ROW_NOT_VISIBLE
include/myisamchk.h:
Added variables for checking visibility of rows during maria_chk
include/thr_lock.h:
Changed argument type from int to my_bool for get_status
Added variable allow_multiple_concurrent_insert, to signal if table supports multiple concurrent inserts
mysql-test/r/maria-page-checksum.result:
Added missing drop table
mysql-test/t/maria-page-checksum.test:
Added missing drop table
mysys/my_handler.c:
Added new error messages
mysys/thr_lock.c:
Added support for multiple concurrent inserts, if table handler supports it
sql/sql_yacc.yy:
Added LOCK TABLE table_name WRITE CONCURRENT
This was added (temporarly?) to be able to check versioning with Maria
storage/csv/ha_tina.cc:
Updated parameter for get_status
storage/maria/ha_maria.cc:
Added calls to maria_chk_init_status()
Fixed call to ma_control_file_open()
storage/maria/ma_blockrec.c:
Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings
Changed back some 'header' parameters to const char*
Removed some casts
Added support for versioning:
- If info->row_flag & ROW_FLAG_TRANSID is set, store transaction id together with the row
- When reading rows, check if rows are visible. Give error if not
- When scanning table, ignore not visible rows
- Added function parameters to some functions, to be able to call _ma_compact_block_page() with different parameters depending of if the page is a HEAD or TAIL page
- _ma_compact_block_page() deletes transaction id's that are visible by all running transactions
- Added functions for thr_lock() to enable multiple concurrent inserts
- Added helper function 'mysql_versioning()' to enable/disable versioning
- Added helper function maria_ignore_trids(), used by maria_chk and maria_pack to see all rows.
storage/maria/ma_blockrec.h:
Updated parameters for some functions.
Added new functions to read/store state with thr_lock
storage/maria/ma_check.c:
Enable handling of transaction id's in rows
Give a readable error if a table contains a transation id that makes rows not visible
storage/maria/ma_control_file.c:
Added option to not give warning if control file doesn't exists.
storage/maria/ma_control_file.h:
Updated parameter lists for ma_control_file_open()
storage/maria/ma_delete.c:
Removed not used variable (suggestion by Guilhem)
storage/maria/ma_locking.c:
Changed type of argument from int -> my_bool
storage/maria/ma_open.c:
Removed not used variables 'key_write_undo_lsn' and 'key_delete_undo_lsn'
Added new thr_lock interface functions for BLOCK_RECORD to enable multiple concurrent insert
storage/maria/ma_test1.c:
Added option --versioning (-C) to check versioning
storage/maria/ma_test2.c:
Added option -C to check versioning
storage/maria/ma_test_recovery:
Forward argumetns to ma_test_recovery.pl
storage/maria/ma_write.c:
Removed not used variable key_write_undo_lsn
storage/maria/maria_chk.c:
Always read control file (if exist) at start
Initialize checking of tables by calling maria_chk_init_for_check()
In verbose mode and in case of error, print max found transaction id
storage/maria/maria_def.h:
Added Trid to MARIA_ROW to be able to check transaction id for found row
Moved 'base_length' from MARIA_ROW to MARIA_HA to be able to handle different base length (with and without TRANSID) without if's
Added default row_flag to MARIA_HA for the same reason
Changed LEX_STRING -> LEX_CUSTRING to avoid casts in ma_blockrec.c
Removed not needed variables key_write_undo_lsn and key_delete_undo_lsn
Added prototypes for new functions and fixed those that had changed
storage/maria/maria_pack.c:
Ensure we can read all rows from the file, independent of the used transaction id
storage/maria/maria_read_log.c:
Updated arguments to ma_control_file_open()
storage/maria/trnman.c:
If we have only one transaction, fixed that min_read_from contains current transaction
Fixed that trnman_can_read_from() returns that row is readable if it was written by current transaction
storage/maria/unittest/ma_control_file-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_all-t:
Added test of versioning
Removed printing of one extra space
storage/maria/unittest/ma_test_loghandler-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_recovery.expected:
Updated file with result from new tests
storage/maria/unittest/ma_test_recovery.pl:
Added options --abort-on-error and --verbose
In case of --verbose, print all excuted shell commands
Added test of versioning
storage/myisam/mi_locking.c:
Updated type of parameter
storage/myisam/myisamdef.h:
Updated type of parameter
mysql-test/r/maria-mvcc.result:
New BitKeeper file ``mysql-test/r/maria-mvcc.result''
mysql-test/t/maria-mvcc.test:
New BitKeeper file ``mysql-test/t/maria-mvcc.test''
2008-04-10 04:26:36 +02:00
|
|
|
CONTROL_FILE_ERROR ma_control_file_open(my_bool create_if_missing,
|
|
|
|
|
my_bool print_error);
|
WL#4374 "Maria - force start if Recovery fails multiple times"
http://forge.mysql.com/worklog/task.php?id=4374
new option --maria-force-start-after-recovery-failures=N; number of consecutive recovery failures (failures
of log reading or recovery processing, anything in [translog_init(),maria_recovery_from_log()])
is stored in the control file; if at a Maria start they are more than N, logs are removed. This is for automated
systems which have to run whatever happens. As tables risk staying corrupted, --maria-recover should also
be used on them: this revision makes maria-recover work (it was disabled).
Fixed bug in translog_is_log_files(). translog_init() now prints message to error log if failed.
Removed \0 in the output of SHOW ENGINE MARIA LOGS; removed hard-coded engine name there.
KNOWN_BUGS.txt:
As option --maria-force-start-after-recovery-failures is added, it corresponds to the wish "we should fix that if this happens etc".
LOAD INDEX is not ignored since a few weeks. Listed concurrency bugs have been fixed some time ago.
Recovery of fulltext and GIS indexes works since a few weeks.
mysql-test/include/maria_make_snapshot.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_make_snapshot_for_comparison.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_make_snapshot_for_feeding_recovery.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_verify_recovery.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/lib/mtr_report.pl:
new test maria-recover.test generates expected corruption warnings in the error log. maria-recovery.test's corrupted table is renamed to t_corrupted1 instead of t1.
mysql-test/r/maria-preload.result:
result update. maria_pagecache_read* values are similar to the previous version of this file, though a bit bigger
because using the information_schema and the join leads to some internal maria temp table being used, and thus some
blocks of it being read.
mysql-test/r/maria-purge.result:
engine's name in SHOW ENGINE MARIA LOGS changed.
mysql-test/r/maria-recover.result:
result for new test. We see corruption messages at first SELECT and then none at second SELECT, expected.
mysql-test/r/maria-recovery.result:
result update
mysql-test/r/maria.result:
new variables show up
mysql-test/t/disabled.def:
BUG#34911 is not fixed but the test had been made independent of the bug (workaround). A new bug (crash) has popped recently, so it has to stay
disabled (BUG#35107).
mysql-test/t/maria-preload.test:
Work around BUG#34911 "FLUSH STATUS doesn't flush what it should":
compute differences in status variables before and after relevant queries
mysql-test/t/maria-recover-master.opt:
test --maria-recover
mysql-test/t/maria-recover.test:
Test of the --maria-recover option (build a corrupted table and see if it is auto-repaired)
mysql-test/t/maria-recovery-big.test:
update for new API of include/maria*.inc
mysql-test/t/maria-recovery-bitmap.test:
update for new API of include/maria*.inc
mysql-test/t/maria-recovery.test:
update for new API of include/maria*.inc. Corrupted table t1 renamed to t_corrupted1, so that mtr_report.pl
does not blindly remove all corruption messages for t1 which is
a common name.
storage/maria/ha_maria.cc:
Enabling maria-recover.
Adding option and global variable --maria_force_start_after_recovery_failures: ha_maria_init()
calls mark_recovery_start() and mark_recovery_success() to keep track of failed consecutive recoveries
and remove logs if needed.
Removed \0 in the output of SHOW ENGINE MARIA LOGS; removed hard-coded engine name there.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Storing in one byte in the control file, the number of consecutive recovery failures.
storage/maria/ma_control_file.h:
new prototype
storage/maria/ma_init.c:
new prototype
storage/maria/ma_locking.c:
Need to update open_count on disk at first write and close for transactional tables, like we already did for
non-transactional tables, otherwise we cannot notice that the table is dubious.
storage/maria/ma_loghandler.c:
translog_is_log_files() is made more generic to serve either to search or to delete logs (the latter is
for --maria-force-start-after-recovery-failures). It also had a bug (always returned FALSE).
storage/maria/ma_loghandler.h:
export function because ha_maria::mark_recovery_start() needs it
storage/maria/ma_recovery.c:
changing name of maria_recover() to distinguish from the maria-recover option.
storage/maria/ma_recovery.h:
changing name of maria_recover() to distinguish from the maria-recover option.
storage/maria/ma_test_force_start.pl:
Test of --maria-force-start-after-recovery-failures (and also, to be realistic, of --maria-recover).
This is standalone because mysql-test-run does not support testing that multiple mysqld restarts expectedly failed.
I'll have to run it on my machine and also on a Windows machine.
storage/maria/unittest/ma_control_file-t.c:
adding recovery_failures to the test
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
fix for compiler warning (unused variable in non-debug build)
2008-06-02 22:53:25 +02:00
|
|
|
int ma_control_file_write_and_force(LSN last_checkpoint_lsn_arg,
|
|
|
|
|
uint32 last_logno_arg, TrID max_trid_arg,
|
|
|
|
|
uint8 recovery_failures_arg);
|
Store maximum transaction id into control file at clean shutdown.
This can serve to maria_chk to check that trids found in rows and keys
are not too big. Also used by Recovery when logs are lost.
Options --require-control-file, --datadir, --log-dir (yes, the dashes are
inconsistent but I imitated mysqld --datadir and --maria-log-dir) for
maria_chk.
Lock control file _before_ reading its content.
storage/maria/ha_maria.cc:
new prototype
storage/maria/ma_check.c:
A function to find the max trid in the system (consults transaction
manager and control file), to check tables.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Store max trid into control file, in a backward-compatible way
(can still read old control files).
Parameter to ma_control_file_open(), to not create the log if it's
missing (maria_chk needs that).
Lock control file _before_ reading its content.
Fix for a segfault when reading an old control file (bzero() with a
negative second argument)
storage/maria/ma_control_file.h:
changes to the control file module's API
storage/maria/ma_init.c:
When Maria shuts down cleanly, store max trid into control file.
storage/maria/ma_loghandler.c:
new prototype
storage/maria/ma_recovery.c:
During recovery, consult max trid stored in control file, in case it is
bigger than what we found in log (case of logs manually removed by user).
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/maria_chk.c:
New option --require-control-file (abort if control file not found),
--datadir (path for control file (and for logs if --log-dir not specified)),
--log-dir (path for logs).
Try to open control file when maria_chk starts.
storage/maria/maria_read_log.c:
new prototype
storage/maria/trnman.c:
A new function to know max trid in transaction manager
storage/maria/trnman_public.h:
New function
storage/maria/unittest/ma_control_file-t.c:
new prototypes. Testing storing and retrieving the max trid to/from
control file
storage/maria/unittest/ma_test_loghandler-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_purge-t.c:
new prototype
2008-04-04 19:10:53 +02:00
|
|
|
int ma_control_file_end(void);
|
|
|
|
|
my_bool ma_control_file_inited(void);
|
2018-11-16 15:50:12 +01:00
|
|
|
my_bool print_aria_log_control(void);
|
Store maximum transaction id into control file at clean shutdown.
This can serve to maria_chk to check that trids found in rows and keys
are not too big. Also used by Recovery when logs are lost.
Options --require-control-file, --datadir, --log-dir (yes, the dashes are
inconsistent but I imitated mysqld --datadir and --maria-log-dir) for
maria_chk.
Lock control file _before_ reading its content.
storage/maria/ha_maria.cc:
new prototype
storage/maria/ma_check.c:
A function to find the max trid in the system (consults transaction
manager and control file), to check tables.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Store max trid into control file, in a backward-compatible way
(can still read old control files).
Parameter to ma_control_file_open(), to not create the log if it's
missing (maria_chk needs that).
Lock control file _before_ reading its content.
Fix for a segfault when reading an old control file (bzero() with a
negative second argument)
storage/maria/ma_control_file.h:
changes to the control file module's API
storage/maria/ma_init.c:
When Maria shuts down cleanly, store max trid into control file.
storage/maria/ma_loghandler.c:
new prototype
storage/maria/ma_recovery.c:
During recovery, consult max trid stored in control file, in case it is
bigger than what we found in log (case of logs manually removed by user).
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/maria_chk.c:
New option --require-control-file (abort if control file not found),
--datadir (path for control file (and for logs if --log-dir not specified)),
--log-dir (path for logs).
Try to open control file when maria_chk starts.
storage/maria/maria_read_log.c:
new prototype
storage/maria/trnman.c:
A new function to know max trid in transaction manager
storage/maria/trnman_public.h:
New function
storage/maria/unittest/ma_control_file-t.c:
new prototypes. Testing storing and retrieving the max trid to/from
control file
storage/maria/unittest/ma_test_loghandler-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_purge-t.c:
new prototype
2008-04-04 19:10:53 +02:00
|
|
|
C_MODE_END
|
WL#3072 - Maria recovery
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.
sql/handler.cc:
typo
storage/maria/Makefile.am:
Adding ma_test_recovery (which ma_test_all invokes, and which can
also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
comments
storage/maria/ma_bitmap.c:
fixing comments. 2 -> sizeof(maria_bitmap_marker).
Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
Now putting the "bm" signature when creating the first bitmap page
(it used to happen only at next open, but that
caused an annoying difference when testing Recovery if the original
run didn't open the table, and it looks more
logical like this: it goes to disk only with its signature correct);
see the "QQ" comment towards the _ma_initialize_data_file() call
in ma_create.c for more).
When reading a bitmap page, verify its signature (happens when normally
using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
* no need to sync the data file if table is not transactional
* Comments, code cleanup (log-related data moved to log-related code
block, int5store->page_store).
* Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
do for other records (though this record will soon be replaced
with a CLR).
* If "page" is 1 it means the page which extends from byte
page*block_size+1 to (page+1)*block_size (byte number 1 being
the first byte of the file). The last byte of the file is
data_file_length (same convention).
A new page needs to be created if the last byte of the page is
beyond the last byte of the file, i.e.
(page+1)*block_size+1 > data_file_length, so we correct the test
(bug found when testing log applying for ma_test1 -M -T --skip-update).
* update the page's LSN when removing a row from it during
execution of a REDO_PURGE_ROW record (bug found when testing log
applying for ma_test1 -M -T --skip-update).
* applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
we can always flush the table's state when closing the last instance
of the table. And it is needed for maria_read_log (as it does
not use maria_lock_database()).
storage/maria/ma_control_file.c:
when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
double-inclusion safe
storage/maria/ma_create.c:
during recovery, don't log records. Comments.
Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
during recovery, don't log records. Log the end-zero of the dropped
table's name, so that recovery can use the string in place without
extending it to fit an end zero.
storage/maria/ma_loghandler.c:
* inwrite_rec_hook also needs access to the MARIA_SHARE, like
prewrite_rec_hook. This will be needed to update
share->records_diff (in the upcoming patch "recovery of the state").
* LOG_DESC::record_ends_group changed to an enum.
* LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
corrected
* Sanja please see the @todo LOG BUG
* avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
- log write hooks called while the log's lock is held (inwrite_rec_hook)
now need the MARIA_SHARE, like prewrite_rec_hook already had
- instead of a bool saying if this record's type ends groups or not,
we refine: it may not end a group, it may end a group, or it may
be a group in itself. Imagine that we had a physical write failure
to a table before we log the UNDO, we still end up in
external_lock(F_UNLCK) and then we log a COMMIT: we don't want
to consider this COMMIT as ending the group of REDOs (don't want
to execute those REDOs during Recovery), that's why we say "COMMIT
is a group in itself, it aborts any previous group". This also
gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
New Recovery code, replacing the old pseudocode.
Most of maria_read_log moved here.
Call-able from ha_maria, but not enabled yet.
Compared to the previous version of maria_read_log, some bugs have
been fixed, debugging output can go to stdout or a disk file (for now
it's useful for me, later it can be changed), execution of
REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
has been factored into functions. We abort an unfinished group
of records if we see a record which is a group in itself (like COMMIT).
No need for maria_panic() after a bug (which caused tables to not
be closed) was fixed; if there is yet another bug I prefer to see it.
When opening a table for Recovery, set data_file_length
and key_file_length to their real physical value (these are the
easiest state members to restore :). Warn us if the last page
was truncated (but Recovery handles it).
MARIA_SHARE::state::state::records is now partly recovered (not
idempotent, but works if recreating tables from scracth).
When applying a REDO to a page, stamp it with the UNDO's LSN
(current_group_end_lsn), not with the REDO's LSN; it makes
the table more identical to the original table (easier to compare
the two tables in the end).
Big thing missing: some types of REDOs are not handled,
and the UNDO phase does not exist (missing functions to execute UNDOs
to actually rollback). So for now tests are only inserting/deleting
a few 100 rows, closing the table and seeing if the log is applied ok;
it works. UPDATE not handled.
storage/maria/ma_recovery.h:
new functions: ma_recover() for recovery from inside ha_maria;
_ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
don't write log records during recovery
storage/maria/ma_test2.c:
- fail if maria_info() or other subtests find some wrong information
- new option -g to skip updates.
- init the translog before creating the table, so that log applying
can work.
- in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
cleanup files. Test log applying.
storage/maria/maria_read_log.c:
most of the logic moves to ma_recovery.c to be shared between
maria_read_log and recovery-from-inside-mysqld.
See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
unit test for Recovery. Tests insert and delete,
REDO_UPDATE not yet coded.
Script is called from ma_test_all. Can run standalone.
2007-07-26 11:56:21 +02:00
|
|
|
#endif
|