- Updated prototype for is_binary_frm_header().
- Added extra argument to ma_control_file_open().
- Added ma_control_file_open_or_create() for usage by tests.
(to make test a bit simpler).
I spent 4 hours on work and 12 hours of testing to try to find
the reason for aria crashing in recovery when starting a new test,
in which case the 'data directory' should be a copy of "install.db",
but aria_log.00000001 content was not correct.
The following changes are mostly done to make it a bit easier to find out
more in case of future similar crashes:
- Mark last_checkpoint_lsn volatile (safety).
- Write checkpoint message to aria_recovery.trace
- When compling with DBUG and with HAVE_DBUG_TRANSLOG_SRC,
use checksum's for Aria log pages. We cannot have it on by default
for DBUG servers yet as there is bugs when changing CRC between
restarts.
- Added a message to mtr --verbose when copying the data directory.
- Removed extra linefeed in Aria recovery message (cleanup)
... '/var/lib/mysql/aria_log_control' for exclusive use, error: 11
As such don't lock under aria_readonly (which is set by opt_help in the handler
initialization.
Fixes: 8eba777c2b
This fixed the MySQL bug# 20338 about misuse of double underscore
prefix __WIN__, which was old MySQL's idea of identifying Windows
Replace it by _WIN32 standard symbol for targeting Windows OS
(both 32 and 64 bit)
Not that connect storage engine is not fixed in this patch (must be
fixed in "upstream" branch)
Changes:
- Initalize Aria early to allow it to load mysql.plugin table with --help
- Don't print 'aborting' when doing --help
- Don't write 'loose' error messages on log_warning < 2 (2 is default)
- Don't write warnings about disabled plugings when doing --help
- Don't write aria_log_control or aria log files when doing --help
- When using --help, open all Aria tables in read only mode (safety)
- If aria_init() fails, do a cleanup(). (Frees used memory)
- If aria_log_control is locked with --help, then don't wait 30 seconds
but instead return at once without initialzing Aria plugin.
This commit is based on the work of Michal Schorm, rebased on the
earliest MariaDB version.
Th command line used to generate this diff was:
find ./ -type f \
-exec sed -i -e 's/Foundation, Inc., 59 Temple Place, Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/Foundation, Inc. 59 Temple Place.* Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/MA.*.....-1307.*USA/MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/Foundation, Inc., 59 Temple/Foundation, Inc., 51 Franklin/g' {} \; \
-exec sed -i -e 's/Place, Suite 330, Boston, MA.*02111-1307.*USA/Street, Fifth Floor, Boston, MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/MA.*.....-1307/MA 02110-1335/g' {} \;
- Made output to be aligned in aria_chk -d
- Aria engine error texts are now written instead of "Undefined error"
- When running with --check --force, tables with wrong TRN's but otherwise
correct are now zerofilled
- Fixed several bugs in check and recovery related to fulltext
- When doing recovery, store highest found TRID in aria_control_file
Before this, the
- Fix win64 pointer truncation warnings
(usually coming from misusing 0x%lx and long cast in DBUG)
- Also fix printf-format warnings
Make the above mentioned warnings fatal.
- fix pthread_join on Windows to set return value.
maria_chk & maria_read_log now reads block size from control file.
mysql-test/suite/maria/r/maria.result:
Updated results after trivial change of maria_chk's output
storage/maria/ma_bitmap.c:
More DBUG_PRINT
storage/maria/ma_blockrec.c:
Fixed bug that we didn't mark page full in bitmap if directory is full
storage/maria/ma_check.c:
Write out if directory is full for errors in bitmap
storage/maria/ma_control_file.c:
Don't give error for wrong block size if block size is 0
storage/maria/maria_chk.c:
Read block size from control file
In case of -dvv, write also out bitmap information (good for debugging)
storage/maria/maria_read_log.c:
Read block size from control file
Fixed that maria_read_log works with different page size than TRANSLOG_PAGE_SIZE
The old way to store the length prefix was (256 - length); This is now changed to (length -249)
Fixed also that some defines to have a MARIA_ prefix
storage/maria/ma_control_file.c:
Added comment
storage/maria/ma_key.c:
Added MARIA_ prefix to some defines
Changed how packed transid length was stored
storage/maria/ma_open.c:
Added MARIA_ prefix to some defines
storage/maria/maria_def.h:
Added MARIA_ prefix to some defines
Changed how packed transid length was stored
http://forge.mysql.com/worklog/task.php?id=4374
new option --maria-force-start-after-recovery-failures=N; number of consecutive recovery failures (failures
of log reading or recovery processing, anything in [translog_init(),maria_recovery_from_log()])
is stored in the control file; if at a Maria start they are more than N, logs are removed. This is for automated
systems which have to run whatever happens. As tables risk staying corrupted, --maria-recover should also
be used on them: this revision makes maria-recover work (it was disabled).
Fixed bug in translog_is_log_files(). translog_init() now prints message to error log if failed.
Removed \0 in the output of SHOW ENGINE MARIA LOGS; removed hard-coded engine name there.
KNOWN_BUGS.txt:
As option --maria-force-start-after-recovery-failures is added, it corresponds to the wish "we should fix that if this happens etc".
LOAD INDEX is not ignored since a few weeks. Listed concurrency bugs have been fixed some time ago.
Recovery of fulltext and GIS indexes works since a few weeks.
mysql-test/include/maria_make_snapshot.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_make_snapshot_for_comparison.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_make_snapshot_for_feeding_recovery.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/include/maria_verify_recovery.inc:
configurable prefix in table's name (so far 't' or 't_corrupted')
mysql-test/lib/mtr_report.pl:
new test maria-recover.test generates expected corruption warnings in the error log. maria-recovery.test's corrupted table is renamed to t_corrupted1 instead of t1.
mysql-test/r/maria-preload.result:
result update. maria_pagecache_read* values are similar to the previous version of this file, though a bit bigger
because using the information_schema and the join leads to some internal maria temp table being used, and thus some
blocks of it being read.
mysql-test/r/maria-purge.result:
engine's name in SHOW ENGINE MARIA LOGS changed.
mysql-test/r/maria-recover.result:
result for new test. We see corruption messages at first SELECT and then none at second SELECT, expected.
mysql-test/r/maria-recovery.result:
result update
mysql-test/r/maria.result:
new variables show up
mysql-test/t/disabled.def:
BUG#34911 is not fixed but the test had been made independent of the bug (workaround). A new bug (crash) has popped recently, so it has to stay
disabled (BUG#35107).
mysql-test/t/maria-preload.test:
Work around BUG#34911 "FLUSH STATUS doesn't flush what it should":
compute differences in status variables before and after relevant queries
mysql-test/t/maria-recover-master.opt:
test --maria-recover
mysql-test/t/maria-recover.test:
Test of the --maria-recover option (build a corrupted table and see if it is auto-repaired)
mysql-test/t/maria-recovery-big.test:
update for new API of include/maria*.inc
mysql-test/t/maria-recovery-bitmap.test:
update for new API of include/maria*.inc
mysql-test/t/maria-recovery.test:
update for new API of include/maria*.inc. Corrupted table t1 renamed to t_corrupted1, so that mtr_report.pl
does not blindly remove all corruption messages for t1 which is
a common name.
storage/maria/ha_maria.cc:
Enabling maria-recover.
Adding option and global variable --maria_force_start_after_recovery_failures: ha_maria_init()
calls mark_recovery_start() and mark_recovery_success() to keep track of failed consecutive recoveries
and remove logs if needed.
Removed \0 in the output of SHOW ENGINE MARIA LOGS; removed hard-coded engine name there.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Storing in one byte in the control file, the number of consecutive recovery failures.
storage/maria/ma_control_file.h:
new prototype
storage/maria/ma_init.c:
new prototype
storage/maria/ma_locking.c:
Need to update open_count on disk at first write and close for transactional tables, like we already did for
non-transactional tables, otherwise we cannot notice that the table is dubious.
storage/maria/ma_loghandler.c:
translog_is_log_files() is made more generic to serve either to search or to delete logs (the latter is
for --maria-force-start-after-recovery-failures). It also had a bug (always returned FALSE).
storage/maria/ma_loghandler.h:
export function because ha_maria::mark_recovery_start() needs it
storage/maria/ma_recovery.c:
changing name of maria_recover() to distinguish from the maria-recover option.
storage/maria/ma_recovery.h:
changing name of maria_recover() to distinguish from the maria-recover option.
storage/maria/ma_test_force_start.pl:
Test of --maria-force-start-after-recovery-failures (and also, to be realistic, of --maria-recover).
This is standalone because mysql-test-run does not support testing that multiple mysqld restarts expectedly failed.
I'll have to run it on my machine and also on a Windows machine.
storage/maria/unittest/ma_control_file-t.c:
adding recovery_failures to the test
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
fix for compiler warning (unused variable in non-debug build)
into mysql.com:/home/my/mysql-maria
mysql-test/r/maria.result:
Auto merged
mysql-test/suite/ndb/r/ndb_auto_increment.result:
Auto merged
mysql-test/t/maria.test:
Auto merged
mysys/hash.c:
Auto merged
mysys/thr_lock.c:
Auto merged
sql/field.cc:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/ha_ndbcluster.h:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/handler.cc:
Auto merged
sql/handler.h:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/protocol.cc:
Auto merged
sql/sql_load.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/csv/ha_tina.cc:
Auto merged
storage/federated/ha_federated.cc:
Auto merged
storage/maria/Makefile.am:
Auto merged
storage/maria/ma_check.c:
Auto merged
storage/maria/ma_control_file.c:
Auto merged
storage/maria/ma_delete_all.c:
Auto merged
storage/maria/ma_dynrec.c:
Auto merged
storage/maria/ma_init.c:
Auto merged
storage/maria/ma_key_recover.c:
Auto merged
storage/maria/ma_open.c:
Auto merged
storage/maria/ma_page.c:
Auto merged
storage/maria/ma_range.c:
Auto merged
storage/maria/ma_recovery.c:
Auto merged
storage/maria/ma_test1.c:
Auto merged
storage/maria/maria_read_log.c:
Auto merged
storage/maria/unittest/ma_test_all-t:
Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Auto merged
storage/maria/unittest/ma_test_recovery.pl:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
storage/myisam/myisamdef.h:
Auto merged
include/my_base.h:
Manual merge where error code are kept same as in 5.1
mysys/my_handler.c:
No changes
sql/item.cc:
Manual merge
sql/sql_class.cc:
Manual merge
sql/sql_insert.cc:
Manual merge
storage/maria/ha_maria.cc:
Manual merge
storage/maria/ma_blockrec.c:
Manual merge
storage/maria/ma_delete.c:
Manual merge
storage/maria/ma_write.c:
Manual merge
Will in future changeset (soon) av versioning of status variables (number of rows) and index
Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings
Removed some not needed variables (as noticed by Guilhem)
include/maria.h:
Added prototypes for maria_chk_init_for_check(), maria_versioning() and maria_ignore_trids()
include/my_base.h:
Add new error HA_ERR_ROW_NOT_VISIBLE
include/myisamchk.h:
Added variables for checking visibility of rows during maria_chk
include/thr_lock.h:
Changed argument type from int to my_bool for get_status
Added variable allow_multiple_concurrent_insert, to signal if table supports multiple concurrent inserts
mysql-test/r/maria-page-checksum.result:
Added missing drop table
mysql-test/t/maria-page-checksum.test:
Added missing drop table
mysys/my_handler.c:
Added new error messages
mysys/thr_lock.c:
Added support for multiple concurrent inserts, if table handler supports it
sql/sql_yacc.yy:
Added LOCK TABLE table_name WRITE CONCURRENT
This was added (temporarly?) to be able to check versioning with Maria
storage/csv/ha_tina.cc:
Updated parameter for get_status
storage/maria/ha_maria.cc:
Added calls to maria_chk_init_status()
Fixed call to ma_control_file_open()
storage/maria/ma_blockrec.c:
Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings
Changed back some 'header' parameters to const char*
Removed some casts
Added support for versioning:
- If info->row_flag & ROW_FLAG_TRANSID is set, store transaction id together with the row
- When reading rows, check if rows are visible. Give error if not
- When scanning table, ignore not visible rows
- Added function parameters to some functions, to be able to call _ma_compact_block_page() with different parameters depending of if the page is a HEAD or TAIL page
- _ma_compact_block_page() deletes transaction id's that are visible by all running transactions
- Added functions for thr_lock() to enable multiple concurrent inserts
- Added helper function 'mysql_versioning()' to enable/disable versioning
- Added helper function maria_ignore_trids(), used by maria_chk and maria_pack to see all rows.
storage/maria/ma_blockrec.h:
Updated parameters for some functions.
Added new functions to read/store state with thr_lock
storage/maria/ma_check.c:
Enable handling of transaction id's in rows
Give a readable error if a table contains a transation id that makes rows not visible
storage/maria/ma_control_file.c:
Added option to not give warning if control file doesn't exists.
storage/maria/ma_control_file.h:
Updated parameter lists for ma_control_file_open()
storage/maria/ma_delete.c:
Removed not used variable (suggestion by Guilhem)
storage/maria/ma_locking.c:
Changed type of argument from int -> my_bool
storage/maria/ma_open.c:
Removed not used variables 'key_write_undo_lsn' and 'key_delete_undo_lsn'
Added new thr_lock interface functions for BLOCK_RECORD to enable multiple concurrent insert
storage/maria/ma_test1.c:
Added option --versioning (-C) to check versioning
storage/maria/ma_test2.c:
Added option -C to check versioning
storage/maria/ma_test_recovery:
Forward argumetns to ma_test_recovery.pl
storage/maria/ma_write.c:
Removed not used variable key_write_undo_lsn
storage/maria/maria_chk.c:
Always read control file (if exist) at start
Initialize checking of tables by calling maria_chk_init_for_check()
In verbose mode and in case of error, print max found transaction id
storage/maria/maria_def.h:
Added Trid to MARIA_ROW to be able to check transaction id for found row
Moved 'base_length' from MARIA_ROW to MARIA_HA to be able to handle different base length (with and without TRANSID) without if's
Added default row_flag to MARIA_HA for the same reason
Changed LEX_STRING -> LEX_CUSTRING to avoid casts in ma_blockrec.c
Removed not needed variables key_write_undo_lsn and key_delete_undo_lsn
Added prototypes for new functions and fixed those that had changed
storage/maria/maria_pack.c:
Ensure we can read all rows from the file, independent of the used transaction id
storage/maria/maria_read_log.c:
Updated arguments to ma_control_file_open()
storage/maria/trnman.c:
If we have only one transaction, fixed that min_read_from contains current transaction
Fixed that trnman_can_read_from() returns that row is readable if it was written by current transaction
storage/maria/unittest/ma_control_file-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_all-t:
Added test of versioning
Removed printing of one extra space
storage/maria/unittest/ma_test_loghandler-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_recovery.expected:
Updated file with result from new tests
storage/maria/unittest/ma_test_recovery.pl:
Added options --abort-on-error and --verbose
In case of --verbose, print all excuted shell commands
Added test of versioning
storage/myisam/mi_locking.c:
Updated type of parameter
storage/myisam/myisamdef.h:
Updated type of parameter
mysql-test/r/maria-mvcc.result:
New BitKeeper file ``mysql-test/r/maria-mvcc.result''
mysql-test/t/maria-mvcc.test:
New BitKeeper file ``mysql-test/t/maria-mvcc.test''
When applying a REDO_INDEX_[FREE|NEW]_PAGE, update state members only
if record is newer than current state.
Print a warning when an old Maria is zero-ing out portions of the
control file which it does not know.
storage/maria/ma_control_file.c:
Print a warning when an old Maria is zero-ing out portions of the
control file which it does not know; it may help us when helping
troubleshoot customers' and users' problems.
storage/maria/ma_key_recover.c:
When applying a REDO_INDEX_[FREE|NEW]_PAGE, we should update the state
members (key_root, key_del) only if this REDO is newer than the current
state. We cannot just update it unconditionally, because that may
change it to an old state, which may not be later corrected if all later
REDOs are entirely skipped due to their page not being in the dirty pages
list of the checkpoint record.
it couldn't entirely write its new information (it could not write
more bytes than the old format wrote).
storage/maria/ma_control_file.c:
Let ma_control_file_write_and_force() return error if called when
file is not open.
When the file was of an old version (no room for our changeable part)
we upgrade this part. When the file was of a new version we re-use its
format but zero the parts which we can't maintain.
storage/maria/ma_init.c:
ma_control_file_write_and_force() now does not assert if called
when control file is not open.
storage/maria/ma_recovery.c:
missing ";" (thanks Peter Zaitsev for the bug report and Monty for the patch)
This can serve to maria_chk to check that trids found in rows and keys
are not too big. Also used by Recovery when logs are lost.
Options --require-control-file, --datadir, --log-dir (yes, the dashes are
inconsistent but I imitated mysqld --datadir and --maria-log-dir) for
maria_chk.
Lock control file _before_ reading its content.
storage/maria/ha_maria.cc:
new prototype
storage/maria/ma_check.c:
A function to find the max trid in the system (consults transaction
manager and control file), to check tables.
storage/maria/ma_checkpoint.c:
new prototype
storage/maria/ma_control_file.c:
Store max trid into control file, in a backward-compatible way
(can still read old control files).
Parameter to ma_control_file_open(), to not create the log if it's
missing (maria_chk needs that).
Lock control file _before_ reading its content.
Fix for a segfault when reading an old control file (bzero() with a
negative second argument)
storage/maria/ma_control_file.h:
changes to the control file module's API
storage/maria/ma_init.c:
When Maria shuts down cleanly, store max trid into control file.
storage/maria/ma_loghandler.c:
new prototype
storage/maria/ma_recovery.c:
During recovery, consult max trid stored in control file, in case it is
bigger than what we found in log (case of logs manually removed by user).
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/maria_chk.c:
New option --require-control-file (abort if control file not found),
--datadir (path for control file (and for logs if --log-dir not specified)),
--log-dir (path for logs).
Try to open control file when maria_chk starts.
storage/maria/maria_read_log.c:
new prototype
storage/maria/trnman.c:
A new function to know max trid in transaction manager
storage/maria/trnman_public.h:
New function
storage/maria/unittest/ma_control_file-t.c:
new prototypes. Testing storing and retrieving the max trid to/from
control file
storage/maria/unittest/ma_test_loghandler-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype
storage/maria/unittest/ma_test_loghandler_purge-t.c:
new prototype
ma_test_loghandler_pagecache-t on Windows.
storage/maria/ma_control_file.c:
stat() is unreliable on Windows (does not reflect process' own writes)
storage/maria/ma_loghandler.c:
translog_set_lsn_for_files() didn't close its file descriptor; it was
a real problem as non-closed files could not be purged.
Same for translog_truncate_log() in case of error.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
stat() is unreliable on Windows (does not reflect process' own writes)
into oleksandr-byelkins-powerbook-g4-15.local:/Users/bell/mysql/bk/work-maria-wrn
storage/maria/ma_blockrec.c:
Auto merged
storage/maria/ma_check.c:
Auto merged
storage/maria/ma_control_file.c:
Auto merged
storage/maria/ma_key_recover.c:
Auto merged
storage/maria/ma_page.c:
Auto merged
storage/maria/ma_pagecache.c:
Auto merged
storage/maria/ma_recovery.c:
Auto merged
include/maria.h:
mac compiler warnings fixed.
storage/maria/ma_bitmap.c:
mac compiler warnings fixed.
storage/maria/ma_blockrec.c:
mac compiler warnings fixed.
storage/maria/ma_check.c:
mac compiler warnings fixed.
storage/maria/ma_control_file.c:
mac compiler warnings fixed.
storage/maria/ma_create.c:
mac compiler warnings fixed.
storage/maria/ma_delete.c:
mac compiler warnings fixed.
storage/maria/ma_ft_boolean_search.c:
mac compiler warnings fixed.
storage/maria/ma_page.c:
mac compiler warnings fixed.
storage/maria/ma_pagecache.c:
mac compiler warnings fixed.
storage/maria/ma_recovery.c:
mac compiler warning fixed.
storage/maria/ma_rt_test.c:
mac compiler warnings fixed.
storage/maria/ma_search.c:
mac compiler warning fixed.
storage/maria/ma_write.c:
mac compiler warnings fixed.
storage/maria/unittest/ma_control_file-t.c:
mac compiler warnings fixed.
storage/maria/unittest/ma_pagecache_consist.c:
mac compiler warnings fixed.
storage/maria/unittest/ma_pagecache_single.c:
mac compiler warnings fixed.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
mac compiler warning fixed.
storage/maria/unittest/test_file.c:
mac compiler warning fixed.
Windows.
include/my_dbug.h:
a DBUG expression to force a flush of the trace file then an abort of the process
mysql-test/include/wait_until_connected_again.inc:
mysqladmin waits for pid file to be gone only under Unix; so
maria_empty_logs.inc cannot wait for mysqld to be gone, so
wait_until_connected_again.inc may send its "show status" to a
not-yet-dead server hence the 1053 error ("server shutdown in progress")
mysys/my_thr_init.c:
overload abort() under Windows, to not have an annoying CRT popup
("ignore/abort/retry" buttons) each time a test intentionally
crashes mysqld
sql/handler.cc:
use new expression
sql/log.cc:
use new expression
sql/mysql_priv.h:
use new expression
storage/maria/ha_maria.cc:
use new expression
storage/maria/ma_blockrec.c:
use new expression
storage/maria/ma_check.c:
use new expression
storage/maria/ma_checkpoint.c:
use new expression
storage/maria/ma_control_file.c:
Can't yet lock control file under Windows (test suite problems,
plus concerns about stray lock preventing a fast restart after crash).
storage/maria/ma_loghandler.c:
A file which should be closed, otherwise translog_purge() (the caller)
cannot delete logs.
storage/maria/ma_control_file.c:
Even if the file is just created we have to lock it, otherwise some
other Maria instance may write to it.
storage/maria/ma_pagecache.c:
catch too big page numbers when they enter the pagecache; this helps
debugging BUG#34250.
Debug asserts/prints from Monty.
No check of mutex operations (by Monty).
storage/maria/ma_control_file.c:
New external function to check mutex ownership.
storage/maria/ma_loghandler.h:
No check of mutex operations (by Monty).
New external function to check mutex ownership.
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
Fix of unit tests.
2 new tests for new features.
storage/maria/ma_control_file.c:
Getting the definition from the file for unit-test.
storage/maria/ma_loghandler.c:
Fix bug with skipped syncing.
storage/maria/unittest/ma_control_file-t.c:
Fix of unit test of control file.
2 new tests for new features of control file.
New extendable format for maria_log_control file
Fixed some compiler warnings
include/maria.h:
Added maria_disable_logging() and maria_enable_logging()
mysql-test/include/maria_verify_recovery.inc:
Updated tests now when key redo/undo works
mysql-test/r/maria-recovery.result:
Updated tests now when key redo/undo works
storage/maria/ma_blockrec.c:
Use unified CLR code
Added rec_lsn for full pages
Moved clr write hook to ma_key_recover.c
Changed REDO code to keep pages pinned until undo
Mark page_link's as changed
storage/maria/ma_blockrec.h:
Moved write_hook_for_clr_end() to ma_key_recover.c
storage/maria/ma_check.c:
Changed key check code to use PAGECACHE_READ_UNKNOWN_PAGE
Fixed wrong warning when checking files after maria_pack
When unpacking files, we have to use new keypos_to_recpos method
When doing repair, we can disregard index key file pages in page cache
storage/maria/ma_commit.c:
Added simple enable/disable logging functions
(Needed for recovery)
storage/maria/ma_control_file.c:
Make maria control file extendable without having to make it incompatible for older versions
storage/maria/ma_control_file.h:
New error messages
Added CONTROL_FILE_VERSION
storage/maria/ma_delete.c:
Added redo/undo for key pages
change_length -> changed_length to make things similar
More comments & more DBUG
storage/maria/ma_key_recover.c:
Unified CLR method
Moved here write_hook_for_clr_end() and common keypage log functions
Changed REDO to keep pages pinned until undo
Changed UNDO code to change key_root under log mutex
storage/maria/ma_key_recover.h:
New structures and functions
storage/maria/ma_loghandler.c:
Include needed files
storage/maria/ma_open.c:
Change maria_open() to use pread() instead of read()
storage/maria/ma_page.c:
Fixed bug in key_del handling
Clear pages if IDENTICAL_PAGES_AFTER_RECOVERY is defined
storage/maria/ma_pagecache.c:
Indentation and spelling fixes
More DBUG
Added helper function: pagecache_block_link_to_buffer()
storage/maria/ma_pagecache.h:
Added pagecache_block_link_to_buffer()
storage/maria/ma_recovery.c:
Fixed state.changed
Fixed that REDO keeps pages pinned until UNDO
Some bug fixes from previous commit
Fixes for UNDO/REDO of key pages
storage/maria/ma_search.c:
Fixed packing and storing of keys to provide more information to caller so
that we can do efficent REDO logging of the changes.
storage/maria/ma_test1.c:
Fixed bug with not initialized variable
storage/maria/ma_test2.c:
Removed not used code
storage/maria/ma_test_all.res:
Updated results
storage/maria/ma_test_all.sh:
Changed one test to test more
Removed timing tests as not relevant here
storage/maria/ma_test_recovery.expected:
Updated test result after redo/undo if key pages works
storage/maria/ma_test_recovery:
Updated test after redo/undo if key pages works
storage/maria/ma_write.c:
Moved some general log functions to ma_key_recover.c
Fixed some bugs in undo
Moved ma_log_split() to _ma_split_page()
Small changes in some function arguments to be able to do redo logging
storage/maria/maria_chk.c:
disable logging while doing repair table
storage/maria/maria_def.h:
New function prototypes
Move some structs and functions to ma_key_recover.c
storage/maria/unittest/ma_control_file-t.c:
Updated with patch from Sanja
NOTE: This is not complete and need to be updated to new control file format
storage/maria/unittest/ma_test_loghandler-t.c:
Fixed compiler warning
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
