in release builds, use it to fine-tune physical file comparisons in
ma_test_recovery.pl.
storage/maria/CMakeLists.txt:
build maria_dump_log under Windows too
storage/maria/maria_chk.c:
make --zerofill-keep-lsn usable in release builds too, it makes sense.
This option needn't reset state's LSNs, and it's more useful if it
does not (see added comment) as it changes less the table.
storage/maria/unittest/ma_test_recovery.pl:
- getting rid of 'cmp'
- if physical comparison of two MAD or MAI files fails, it might
be in irrelevant portions (unushed index or data page pieces):
we use --zerofill-keep-lsn and retry the comparison. This fixes
a part of test failures under Windows.
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
-new option WITH_MARIA_STORAGE_ENGINE for config.js
-correct build errors
-build test executables
-downport changes for atomic functions from 5.2
-remove LOCK_uuid_generator from C++ files to avoid linker errors
-new function my_uuid2str()
BitKeeper/deleted/.del-x86-msvc.h:
Delete: include/atomic/x86-msvc.h
CMakeLists.txt:
Windows fixes:
-New option WITH_MARIA_STORAGE_ENGINE
-Add unit tests
include/Makefile.am:
replace x86-msvc.h with generic-msvc.h
include/config-win.h:
my_chmod() support
include/my_atomic.h:
Downport my_atomic from 5.2 tree
include/my_bit.h:
Correct unresolved symbol errors on Windows
include/my_pthread.h:
pthread_mutex_unlock now returns 0 (was void previously)
defined PTHREAD_STACK_MIN
include/my_sys.h:
New function my_uuid2str()
define MY_UUID_STRING_LENGTH
include/atomic/nolock.h:
Downport my_atomic from 5.2 tree
libmysqld/CMakeLists.txt:
New option WITH_MARIA_STORAGE_ENGINE
mysys/CMakeLists.txt:
Add missing files
mysys/lf_dynarray.c:
Fix compiler errors on Windows
mysys/my_getncpus.c:
Windows port
mysys/my_uuid.c:
Windows fixes: there is no random() on Windows, use ANSI rand()
New function my_uuid2str()
mysys/my_winthread.c:
Downport from 5.2 tree
-Call my_thread_end() before pthread_exit()
-Avoid crash if pthread_create is called with NULL attributes
sql/CMakeLists.txt:
Link mysqld with Maria storage engine
sql/item_func.cc:
Remove LOCK_uuid_generator from C++ to avoid linker errors.
Use dedicated mutex for short uuids
sql/item_strfunc.cc:
Use my_uuid() and my_uuid2str() functions from mysys.
sql/item_strfunc.h:
Define MY_UUID_STRING_LENGTH in my_sys.h
sql/mysql_priv.h:
LOCK_uuid_generator must be declared as extern "C"
sql/mysqld.cc:
Init and destroy LOCK_uuid_short mutex
storage/maria/CMakeLists.txt:
-Use the same source files as in Makefile.am
-Build test binaries
storage/maria/ha_maria.cc:
snprintf->my_snprintf
storage/maria/lockman.c:
Fix compiler error on Windows
storage/maria/ma_check.c:
Fix compiler error on Windows
storage/maria/ma_loghandler.c:
Fix compile errors
my_open()/my_sync() do not work for directories on Windows
storage/maria/ma_recovery.c:
Fix compile error on Windows
storage/maria/ma_test2.c:
Rename variable to avoid naming conflict with Microsoft C runtime
function
storage/maria/ma_test3.c:
Fix build errors on Windows
storage/maria/tablockman.c:
Fix build errors on Windows
storage/maria/unittest/Makefile.am:
Add CMakeLists.txt
storage/maria/unittest/ma_pagecache_consist.c:
Fix build errors on Windows
remove loop from get_len()
storage/maria/unittest/ma_pagecache_single.c:
Fix build errors on Windows
storage/maria/unittest/ma_test_loghandler-t.c:
Windows fixes
-Avoid division by 0 in expressions like
x/(RAND_MAX/y), where y is larger than RAND_MAX(==0x7fff on Windows)
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Windows fixes
-Avoid division by 0 in expressions like
x/(RAND_MAX/y), where y is larger than RAND_MAX(==0x7fff on Windows)
-remove loop in get_len()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Windows fixes
-Avoid division by 0 in expressions like
x/(RAND_MAX/y), where y is larger than RAND_MAX(==0x7fff on Windows)
-remove loop in get_len()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Fix build errors on Windows
storage/maria/unittest/test_file.c:
Correct the code to get file size on Windows.
stat() information can be outdated and thus cannot be trusted.
On Vista,stat() returns file size=0 until the file is closed at the
first time.
storage/myisam/CMakeLists.txt:
Fix compiler errors on Windows
Build test executables
storage/myisam/mi_test2.c:
Rename variable to avoid naming conflict with Microsoft C runtime
function
storage/myisam/mi_test3.c:
Fix build errors on Windows
strings/CMakeLists.txt:
Add missing file
unittest/unit.pl:
Windows:
downport unittest changes from 5.2 bk tree
unittest/mysys/Makefile.am:
Windows:
downport unittest changes from 5.2 bk tree
unittest/mysys/my_atomic-t.c:
Windows:
downport unittest changes from 5.2 bk tree
unittest/mytap/Makefile.am:
Windows:
downport unittest changes from 5.2 bk tree
unittest/mytap/tap.c:
Windows:
downport unittest changes from 5.2 bk tree
win/configure.js:
Add WITH_MARIA_STORAGE_ENGINE configure option
unittest/mytap/CMakeLists.txt:
Add missing file
unittest/mysys/CMakeLists.txt:
Add missing file
storage/maria/unittest/CMakeLists.txt:
Add missing file
BitKeeper/etc/ignore:
Added comments maria-win.patch to the ignore list
include/atomic/generic-msvc.h:
Implement atomic operations with MSVC intrinsics
storage/maria/ma_test1.c:
After merge fixes. Applied changes made in myisam manually to maria.
Fixed memory leak.
storage/maria/ma_test2.c:
After merge fixes. Applied changes made in myisam manually to maria.
Fixed memory leak.
added checksum of the file. Now we have size + magic string + checksum
to detect that all is ok.
Plus misc fixes for "make dist" to work and the resulting tarball to build
include/Makefile.am:
adding pagecache.h to help the tarball build.
The model of pagecache.h, keycache.h, is in pkginclude_HEADERS,
wonder why. Adding pagecache.h to noinst_HEADERS for now.
storage/maria/Makefile.am:
adding ma_control_file.h to help the tarball build
storage/maria/ma_control_file.c:
adding a simple checksum to the control file.
We protect against corruption of this file like this:
- test size
- test magic string at start
- test checksum
I also add some simple my_message() errors (to be changed to a better
reporting later).
storage/maria/ma_control_file.h:
comments
storage/maria/ma_control_file_test.c:
test of wrong checksum in control file
storage/maria/CMakeLists.txt:
just to make "make dist" happy for now.
