mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-17 12:32:27 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
43365 commits 2784 branches 343 tags 1.6 GiB
Commit graph

13 commits

Author SHA1 Message Date
unknown
6bbca54d7d WL#3072 - Maria recovery 
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.


sql/handler.cc:
  typo
storage/maria/Makefile.am:
  Adding ma_test_recovery (which ma_test_all invokes, and which can
  also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
  comments
storage/maria/ma_bitmap.c:
  fixing comments. 2 -> sizeof(maria_bitmap_marker).
  Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
  Now putting the "bm" signature when creating the first bitmap page
  (it used to happen only at next open, but that
  caused an annoying difference when testing Recovery if the original
  run didn't open the table, and it looks more
  logical like this: it goes to disk only with its signature correct);
  see the "QQ" comment towards the _ma_initialize_data_file() call
  in ma_create.c for more).
  When reading a bitmap page, verify its signature (happens when normally
  using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
  * no need to sync the data file if table is not transactional
  * Comments, code cleanup (log-related data moved to log-related code
  block, int5store->page_store).
  * Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
  do for other records (though this record will soon be replaced
  with a CLR).
  * If "page" is 1 it means the page which extends from byte
  page*block_size+1 to (page+1)*block_size (byte number 1 being
  the first byte of the file). The last byte of the file is
  data_file_length (same convention).
  A new page needs to be created if the last byte of the page is
  beyond the last byte of the file, i.e.
   (page+1)*block_size+1 > data_file_length, so we correct the test
  (bug found when testing log applying for ma_test1 -M -T --skip-update).
  * update the page's LSN when removing a row from it during
  execution of a REDO_PURGE_ROW record (bug found when testing log
  applying for ma_test1 -M -T --skip-update).
  * applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
  new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
  we can always flush the table's state when closing the last instance
  of the table. And it is needed for maria_read_log (as it does
  not use maria_lock_database()).
storage/maria/ma_control_file.c:
  when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
  double-inclusion safe
storage/maria/ma_create.c:
  during recovery, don't log records. Comments.
  Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
  during recovery, don't log records. Log the end-zero of the dropped
  table's name, so that recovery can use the string in place without
  extending it to fit an end zero.
storage/maria/ma_loghandler.c:
  * inwrite_rec_hook also needs access to the MARIA_SHARE, like
  prewrite_rec_hook. This will be needed to update
  share->records_diff (in the upcoming patch "recovery of the state").
  * LOG_DESC::record_ends_group changed to an enum.
  * LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
  corrected
  * Sanja please see the @todo LOG BUG
  * avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
  - log write hooks called while the log's lock is held (inwrite_rec_hook)
  now need the MARIA_SHARE, like prewrite_rec_hook already had
  - instead of a bool saying if this record's type ends groups or not,
  we refine: it may not end a group, it may end a group, or it may
  be a group in itself. Imagine that we had a physical write failure
  to a table before we log the UNDO, we still end up in
  external_lock(F_UNLCK) and then we log a COMMIT: we don't want
  to consider this COMMIT as ending the group of REDOs (don't want
  to execute those REDOs during Recovery), that's why we say "COMMIT
  is a group in itself, it aborts any previous group". This also
  gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
  New Recovery code, replacing the old pseudocode.
  Most of maria_read_log moved here.
  Call-able from ha_maria, but not enabled yet.
  Compared to the previous version of maria_read_log, some bugs have
  been fixed, debugging output can go to stdout or a disk file (for now
  it's useful for me, later it can be changed), execution of
  REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
  has been factored into functions. We abort an unfinished group
  of records if we see a record which is a group in itself (like COMMIT).
  No need for maria_panic() after a bug (which caused tables to not
  be closed) was fixed; if there is yet another bug I prefer to see it.
  When opening a table for Recovery, set data_file_length
  and key_file_length to their real physical value (these are the
  easiest state members to restore :). Warn us if the last page
  was truncated (but Recovery handles it).
  MARIA_SHARE::state::state::records is now partly recovered (not
  idempotent, but works if recreating tables from scracth).
  When applying a REDO to a page, stamp it with the UNDO's LSN
  (current_group_end_lsn), not with the REDO's LSN; it makes
  the table more identical to the original table (easier to compare
  the two tables in the end).
  Big thing missing: some types of REDOs are not handled,
  and the UNDO phase does not exist (missing functions to execute UNDOs
  to actually rollback). So for now tests are only inserting/deleting
  a few 100 rows, closing the table and seeing if the log is applied ok;
  it works. UPDATE not handled.
storage/maria/ma_recovery.h:
  new functions: ma_recover() for recovery from inside ha_maria;
  _ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
  Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
  don't write log records during recovery
storage/maria/ma_test2.c:
  - fail if maria_info() or other subtests find some wrong information
  - new option -g to skip updates.
  - init the translog before creating the table, so that log applying
  can work.
  - in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
  cleanup files. Test log applying.
storage/maria/maria_read_log.c:
  most of the logic moves to ma_recovery.c to be shared between
  maria_read_log and recovery-from-inside-mysqld.
  See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
  unit test for Recovery. Tests insert and delete,
  REDO_UPDATE not yet coded.
  Script is called from ma_test_all. Can run standalone.
 2007-07-26 11:56:21 +02:00
unknown
adac9798bf WL#3072 Maria Recovery 
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes


storage/maria/Makefile.am:
  new program maria_read_log
storage/maria/ha_maria.cc:
  create control file if missing
storage/maria/ma_blockrec.c:
  0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
  preparations for Checkpoint module
storage/maria/ma_close.c:
  comment
storage/maria/ma_control_file.c:
  renaming constants.
  Possibility to say "open control file but don't create it if it's
  missing" (used by maria_read_log which does not want to create
  anything)
storage/maria/ma_control_file.h:
  renaming constants
storage/maria/ma_create.c:
  I had duplicated "linkname" and "linkname_ptr", now I see it's not
  needed, reverting. Indeed those variables don't contain interesting
  information; fixing log record accordingly (the links are in
  ci->data/index_file_name). Storing keystart in log record is needed,
  to know at which size we must extend the file if we replay
  LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
  some structures need to be known to maria_read_log.c, taking
  them to ma_loghandler.h
storage/maria/ma_loghandler.h:
  we have page_store, adding page_korr.
  translog_lock() made public, because Checkpoint will need it (to
  write to control file).
  Some structures moved from ma_loghandler.c because maria_read_log.c
  needs them (needs to know the execute-in-REDO-phase hooks of each
  record).
storage/maria/ma_loghandler_lsn.h:
  constants defined in ma_control_file.h serve everywhere,
  and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
  Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
  renaming constants
storage/maria/ma_recovery.h:
  copyright
storage/maria/ma_test1.c:
  new prototype
storage/maria/ma_test2.c:
  new prototype
storage/maria/trnman_public.h:
  double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  constants renamed, new prototype
storage/myisam/mi_close.c:
  comment
storage/maria/maria_read_log.c:
  program to read and print log records from a Maria transaction log,
  and optionally apply them to tables. Very basic, early version.
  Should serve as a base for Recovery's code. Designed to be idempotent.
  Create a log by running maria.test, then cd to var/master-data
  and run "maria_read_log --only-display" to see info about records;
  run "maria_read_log --display-and-apply" to also apply the records
  to tables (it's more interesting if you first wipe out the
  tables in var/master-data/test, to see how they get re-created).
  Only a few records are handled by now: LONG_TRANSACTION_ID,
  COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
  REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
  "Monty" in the file). Note: changes to the index pages, index's header
  and bitmap pages are not properly logged yet, so don't expect
  the program to work with that.
 2007-06-26 16:49:23 +02:00
unknown
1a96259191 - WL#3239 "log CREATE TABLE in Maria" 
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.


storage/maria/Makefile.am:
  more files to build
storage/maria/ha_maria.cc:
  - logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
  - ha_maria::data_file_type does not have to be set in every info()
  call, just do it once in open().
  - if caller said that transactionality can be disabled (like if
  caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
  temporarily disable transactionality of the table in external_lock();
  that will ensure that no REDOs/UNDOs are logged for this possibly
  massive write operation (they are not needed, as if any write fails,
  the table will be dropped). We re-enable in external_lock(F_UNLCK),
  which in ALTER TABLE happens before the tmp table replaces the original
  one (which is good, as thus the final table will have a REDO RENAME
  and a correct create_rename_lsn).
  - when we commit we also have to write a log record, so
  trnman_commit_trn() calls become ma_commit() calls
  - at end of engine's initialization, we are potentially entering a
  multi-threaded dangerous world (clients are going to be accepted)
  and so some assertions of mutex-owning become enforceable, for that
  we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
  new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
  - fixing comments according to discussion with Monty
  - if a table is transactional but temporarily non-transactional
  (like in ALTER TABLE), we need to give a sensible LSN to the pages
  (and, if we give 0, pagecache asserts).
  - translog_write_record() now takes care of storing the share's
  2-byte-id in the log record
storage/maria/ma_blockrec.h:
  fixing comment according to discussion with Monty
storage/maria/ma_check.c:
  When REPAIR/OPTIMIZE modify the data/index file, if this is a
  transactional table, they must sync it; if they remove files or rename
  files, they must sync the directory, so that everything is durable.
  This is just applying to REPAIR/OPTIMIZE the logic already implemented
  in CREATE/DROP/RENAME a few months ago.
  Adding a function to write a LOGREC_REPAIR_TABLE at end of
  REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
  to update the table's create_rename_lsn.
storage/maria/ma_close.c:
  fix for a future bug
storage/maria/ma_control_file.c:
  ensuring that if Maria is running in multi-threaded mode, anybody
  wanting to write to the control file and update
  last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
  see ma_control_file.c
storage/maria/ma_create.c:
  when creating a table:
  - sync it and its directory only if this is a transactional table
  and there is a log (no point in syncing in maria_chk)
  - decouple the two uses of linkname/linkname_ptr (for index file and
  for data file) into more variables, as we need to know all links
  until the moment we write the LOGREC_CREATE_TABLE.
  - set share.data_file_type early so that _ma_initialize_data_file()
  knows it (Monty's bugfix so that a table always has at least a bitmap
  page when it is created; so data-file is not 0 bytes anymore).
  - log a LOGREC_CREATE_TABLE; it contains the bytes which we have
  just written to the index file's header. Update table's
  create_rename_lsn.
  - syncing of kfile had been bugified in a previous merge, correcting
  - syncing of dfile is now needed as it's not empty anymore
  - in _ma_initialize_data_file(), use share's block_size and not the
  global one. This is a gratuitous change, both variables are equal,
  just that I find it more future-proof to use share-bound variable
  rather than global one.
storage/maria/ma_delete_all.c:
  log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
  update create_rename_lsn then.
storage/maria/ma_delete_table.c:
  - logging LOGREC_DROP_TABLE; knowing if this is needed, requires
  knowing if the table is transactional, which requires opening the
  table.
  - we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
  questions
storage/maria/ma_init.c:
  when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
  - translog_inited has to be visible to ma_create() (see how it is used
  in ma_create())
  - checkpoint record will be a single record, not three
  - no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
  log a REDO_CREATE)
  - adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
  truncating the files), REPAIR.
  - MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
  - in translog_write_record(), if MARIA_SHARE does not yet have a
  2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
  store this short id into log records.
  - in translog_write_record(), if transaction has not logged its
  long trid, log LOGREC_LONG_TRANSACTION_ID.
  - For Checkpoint, we need to know the current end-of-log: adding
  translog_get_horizon().
  - For Control File, adding an assertion that the thread owns the
  log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
  Changes in log records (see ma_loghandler.c).
  new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
  adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
  where the most significant byte is used for flags.
storage/maria/ma_open.c:
  storing the create_rename_lsn in the index file's header (in the
  state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
  - my set_if_bigger was wrong, correcting it
  - if the first_in_switch list is not empty, it means that
  changed_blocks misses some dirty pages, so Checkpoint cannot run and
  needs to wait. A variable missing_blocks_in_changed_list is added to
  tell that (should it be named missing_blocks_in_changed_blocks?)
  - pagecache_collect_changed_blocks_with_lsn() now also tells the
  minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
  see ma_pagecache.c
storage/maria/ma_panic.c:
  comment
storage/maria/ma_range.c:
  comment
storage/maria/ma_rename.c:
  - logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
  knowing if the table is transactional, which requires opening the
  table.
  - update create_rename_lsn
  - we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
  comment
storage/maria/ma_test_all.sh:
  - tip for Valgrind-ing ma_test_all
  - do "export maria_path=somepath" before calling ma_test_all,
  if you want to run ma_test_all out of storage/maria (useful
  to have parallel runs, like one normal and one Valgrind, they
  must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
  - state now contains, in memory and on disk, the create_rename_lsn
  - share now contains a 2-byte-id
storage/maria/trnman.c:
  preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
  minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
  using most significant byte of first_undo_lsn to hold miscellaneous
  flags, for now TRANSACTION_LOGGED_LONG_ID.
  dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
  dummy_transaction_object was declared in all files including
  trnman_public.h, while in fact it's a single object.
  new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
  update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  update for new prototype
storage/maria/ma_commit.c:
  function which wraps:
  - writing a LOGREC_COMMIT record (==commit on disk)
  - calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
  new header file
.tree-is-private:
  this file is now needed to keep our tree private (don't push it
  to public trees). When 5.1 is merged into mysql-maria, we can abandon
  our maria-specific post-commit trigger; .tree_is_private will take
  care of keeping commit mails private. Don't push this file to public
  trees.
 2007-06-22 14:49:37 +02:00
unknown
1a77ebd14c Maria: a stronger checksum in the control file, to test integrity. 
4 bytes, using my_checksum() (the old checksum was one byte and just
a sum of the bytes - that was before I saw we have my_checksum :)


storage/maria/ma_control_file.c:
  stronger checksum (4 bytes instead of 1, and using CRC instead of
  simple byte sum).
storage/maria/unittest/ma_control_file-t.c:
  Checksum is now 4 bytes (total length of control file is now 23),
  so LSN and LAST_LOGNO move.
 2007-03-02 12:15:19 +01:00
unknown
46922b5125 GPL license update (same change as was done for all files in 5.1). 
storage/maria/Makefile.am:
  GPL license update
storage/maria/ft_maria.c:
  GPL license update
storage/maria/ha_maria.cc:
  GPL license update
storage/maria/ha_maria.h:
  GPL license update
storage/maria/lockman.c:
  GPL license update
storage/maria/lockman.h:
  GPL license update
storage/maria/ma_bitmap.c:
  GPL license update
storage/maria/ma_blockrec.c:
  GPL license update
storage/maria/ma_blockrec.h:
  GPL license update
storage/maria/ma_cache.c:
  GPL license update
storage/maria/ma_changed.c:
  GPL license update
storage/maria/ma_check.c:
  GPL license update
storage/maria/ma_checkpoint.c:
  GPL license update
storage/maria/ma_checkpoint.h:
  GPL license update
storage/maria/ma_checksum.c:
  GPL license update
storage/maria/ma_close.c:
  GPL license update
storage/maria/ma_control_file.c:
  GPL license update
storage/maria/ma_control_file.h:
  GPL license update
storage/maria/ma_create.c:
  GPL license update
storage/maria/ma_dbug.c:
  GPL license update
storage/maria/ma_delete.c:
  GPL license update
storage/maria/ma_delete_all.c:
  GPL license update
storage/maria/ma_delete_table.c:
  GPL license update
storage/maria/ma_dynrec.c:
  GPL license update
storage/maria/ma_extra.c:
  GPL license update
storage/maria/ma_ft_boolean_search.c:
  GPL license update
storage/maria/ma_ft_eval.c:
  GPL license update
storage/maria/ma_ft_eval.h:
  GPL license update
storage/maria/ma_ft_nlq_search.c:
  GPL license update
storage/maria/ma_ft_parser.c:
  GPL license update
storage/maria/ma_ft_stem.c:
  GPL license update
storage/maria/ma_ft_test1.c:
  GPL license update
storage/maria/ma_ft_test1.h:
  GPL license update
storage/maria/ma_ft_update.c:
  GPL license update
storage/maria/ma_ftdefs.h:
  GPL license update
storage/maria/ma_fulltext.h:
  GPL license update
storage/maria/ma_info.c:
  GPL license update
storage/maria/ma_init.c:
  GPL license update
storage/maria/ma_key.c:
  GPL license update
storage/maria/ma_keycache.c:
  GPL license update
storage/maria/ma_least_recently_dirtied.c:
  GPL license update
storage/maria/ma_least_recently_dirtied.h:
  GPL license update
storage/maria/ma_locking.c:
  GPL license update
storage/maria/ma_open.c:
  GPL license update
storage/maria/ma_packrec.c:
  GPL license update
storage/maria/ma_page.c:
  GPL license update
storage/maria/ma_panic.c:
  GPL license update
storage/maria/ma_preload.c:
  GPL license update
storage/maria/ma_range.c:
  GPL license update
storage/maria/ma_recovery.c:
  GPL license update
storage/maria/ma_recovery.h:
  GPL license update
storage/maria/ma_rename.c:
  GPL license update
storage/maria/ma_rfirst.c:
  GPL license update
storage/maria/ma_rkey.c:
  GPL license update
storage/maria/ma_rlast.c:
  GPL license update
storage/maria/ma_rnext.c:
  GPL license update
storage/maria/ma_rnext_same.c:
  GPL license update
storage/maria/ma_rprev.c:
  GPL license update
storage/maria/ma_rrnd.c:
  GPL license update
storage/maria/ma_rsame.c:
  GPL license update
storage/maria/ma_rsamepos.c:
  GPL license update
storage/maria/ma_rt_index.c:
  GPL license update
storage/maria/ma_rt_index.h:
  GPL license update
storage/maria/ma_rt_key.c:
  GPL license update
storage/maria/ma_rt_key.h:
  GPL license update
storage/maria/ma_rt_mbr.c:
  GPL license update
storage/maria/ma_rt_mbr.h:
  GPL license update
storage/maria/ma_rt_split.c:
  GPL license update
storage/maria/ma_rt_test.c:
  GPL license update
storage/maria/ma_scan.c:
  GPL license update
storage/maria/ma_search.c:
  GPL license update
storage/maria/ma_sort.c:
  GPL license update
storage/maria/ma_sp_defs.h:
  GPL license update
storage/maria/ma_sp_key.c:
  GPL license update
storage/maria/ma_sp_test.c:
  GPL license update
storage/maria/ma_static.c:
  GPL license update
storage/maria/ma_statrec.c:
  GPL license update
storage/maria/ma_test1.c:
  GPL license update
storage/maria/ma_test2.c:
  GPL license update
storage/maria/ma_test3.c:
  GPL license update
storage/maria/ma_unique.c:
  GPL license update
storage/maria/ma_update.c:
  GPL license update
storage/maria/ma_write.c:
  GPL license update
storage/maria/maria_chk.c:
  GPL license update
storage/maria/maria_def.h:
  GPL license update
storage/maria/maria_ftdump.c:
  GPL license update
storage/maria/maria_pack.c:
  GPL license update
storage/maria/tablockman.c:
  GPL license update
storage/maria/tablockman.h:
  GPL license update
storage/maria/trnman.c:
  GPL license update
storage/maria/trnman.h:
  GPL license update
 2007-03-02 11:20:23 +01:00
unknown
3bc8f629dd Postreview changes. 
Fixed befaviour when loghandler flags changed from one
run to another one.
Description of maria transaction log and control file
added to the file commandÍs magic number file.


mysys/mf_pagecache.c:
  postreview changes
storage/maria/ma_control_file.c:
  Postreview changes.
storage/maria/ma_control_file.h:
  Postreview changes.
storage/maria/ma_loghandler.c:
  Postreview changes.
  Fixed befaviour when loghandler flags changed from
  one run to another one.
storage/maria/ma_loghandler.h:
  Postreview changes.
  Functions comment left only near the function body.
storage/maria/ma_loghandler_lsn.h:
  Postreview changes.
storage/maria/unittest/ma_test_loghandler-t.c:
  Postreview changes.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Postreview changes.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Postreview changes.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Postreview changes.
support-files/magic:
  Description of maria transaction log and control file
  added to the file commandÕs magic number file.
 2007-02-19 23:01:27 +02:00
unknown
91a8199773 Postmerge fix (including changing type of LSN) 
Some debug info and comments added


include/pagecache.h:
  postmerge fix
mysys/mf_pagecache.c:
  Postmerge fix (including changing type of LSN)
  Additional DBUG_ASSERTs added
  Comment about pinning mechanism added
storage/maria/ma_control_file.c:
  Used the same LSN storing procedure everywhere
  Postmerge fix (including changing type of LSN)
storage/maria/ma_control_file.h:
  Postmerge fix (including changing type of LSN)
storage/maria/ma_loghandler.c:
  Postmerge fix (including changing type of LSN)
storage/maria/ma_loghandler.h:
  Postmerge fix (including changing type of LSN)
storage/maria/ma_loghandler_lsn.h:
  Postmerge fix (including changing type of LSN)
storage/maria/unittest/Makefile.am:
  Postmerge fix
storage/maria/unittest/ma_control_file-t.c:
  Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler-t.c:
  Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Postmerge fix (including changing type of LSN)
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Postmerge fix (including changing type of LSN)
storage/maria/unittest/mf_pagecache_consist.c:
  Postmerge fix (including changing type of LSN)
storage/maria/unittest/mf_pagecache_single.c:
  Postmerge fix (including changing type of LSN)
 2007-02-12 14:23:43 +02:00
unknown
a41ac15b96 Maria - various fixes around durability of files: 
1) on Mac OS X >=10.3, fcntl() is recommended over fsync (from the
man page: "[With fsync()] the disk drive may also re-order the data
so that later writes may be present while earlier writes are not.
Applications such as databases that require a strict ordering of writes
should use F_FULLFSYNC to ensure their data is written in the order
they expect"). I have seen two other pieces of software changing from
fsync to F_FULLFSYNC on Mac OS X.
2) to make a file creation/deletion/renaming durable on Linux (at least
ext2 as I have tested) (see "man fsync"), a fsync() on the directory
is needed: new functions to do that, and a flag MY_SYNC_DIR to do
it in my_create/my_delete/my_rename.
3) now using this directory syncing when creating he frm if
opt_sync_frm, and for Maria's control file when it is created.


include/my_sys.h:
  new flag to my_create/my_delete/my_rename, which asks to sync the
  directory after the operation is done (currently does nothing except
  on Linux)
libmysql/CMakeLists.txt:
  my_create() now depends on my_sync() so my_sync is needed for libmysql
libmysql/Makefile.shared:
  my_create() now depends on my_sync() so my_sync is needed for libmysql
mysys/my_create.c:
  my_create() can now sync the directory if asked for
mysys/my_delete.c:
  my_delete() can now sync the directory if asked for
mysys/my_open.c:
  it was a bug that my_close() is done on fd but a positive fd would
  still be returned, by my_register_filename().
mysys/my_rename.c:
  my_rename() can now sync the two directories (the one of "from" and
  the one of "to") if asked for.
mysys/my_sync.c:
  On recent Mac OS X, fcntl(F_FULLFSYNC) is recommended over fsync()
  (see "man fsync" on Mac OS X 10.3).
  my_sync_dir(): to sync a directory after a file creation/deletion/
  renaming; can be called directly or via MY_SYNC_DIR in my_create/
  my_delete/my_rename(). No-op except on Linux (see "man fsync" on Linux).
  my_sync_dir_from_file(): same as above, just more practical when the
  caller has a file name but no directory name ready.
  Should the #warning even be a #error? I mean do we want to release
  binaries which don't guarantee any durability?
sql/log.cc:
  a TODO for the future.
sql/unireg.cc:
  If we sync the frm it makes sense to also sync its creation in the
  directory.
storage/maria/ma_control_file.c:
  control file is vital, try to make it to disk
 2006-11-21 22:22:59 +01:00
unknown
09a7f30973 WL#3234 Maria Control file manager 
last round of fixes to the storage engines' and plugins' unit tests
structure. Will extract a total patch and push it in 5.1 as has been
approved.


Makefile.am:
  unittest must be before storage and plugin, because engine and plugin
  may have unit tests which link with libtap which is found in
  unitttest.
config/ac-macros/plugins.m4:
  When enabling an engine/plugin, add its directory to the list
  of directories where unit tests should be searched. That is,
  its directory will be recursively searched by our unit test framework
  which will execute any executable *-t file.
storage/maria/ma_control_file.c:
  those my_message pollute the output of unit tests.
storage/maria/plug.in:
  When Maria is enabled, add its unittest Makefile.
unittest/Makefile.am:
  plugins too
 2006-09-15 11:05:35 +02:00
unknown
cdf831cf94 WL#3071 Maria checkpoint: 
changing pseudocode to use the structures of the Maria pagecache
    ("pagecache->changed_blocks" etc) and other Maria structures
    inherited from MyISAM (THR_LOCK_maria etc).


mysys/mf_pagecache.c:
  comment
storage/maria/ma_checkpoint.c:
  changing pseudocode to use the structures of the Maria pagecache
  ("pagecache->changed_blocks" etc) and other Maria structures
  inherited from MyISAM (THR_LOCK_maria etc).
storage/maria/ma_checkpoint.h:
  copyright
storage/maria/ma_control_file.c:
  copyright
storage/maria/ma_control_file.h:
  copyright
storage/maria/ma_least_recently_dirtied.c:
  copyright
storage/maria/ma_least_recently_dirtied.h:
  copyright
storage/maria/ma_recovery.c:
  copyright
storage/maria/ma_recovery.h:
  copyright
storage/maria/unittest/Makefile.am:
  copyright
 2006-09-14 19:06:51 +02:00
unknown
26fb36067a WL#3234 Maria control file manager. 
Fitting ma_control_file_test into the mytap unittest framework:
new directories:
- unittest/storage/ for unit tests of any storage engine
- unittest/storage/maria for ... Maria, containing ma_control_file-t.
Later, older tests like ma_test*, ma_test_all (but which is Unix
dependent in its current form) could move here too.
The plugins macro enable building of unittest/storage/X for any
enabled engine X which has such a directory.
If Falcon wants to have unit tests there too, I may have to merge
this patch into 5.x one day.


config/ac-macros/plugins.m4:
  If a storage engine has a directory in unittest/storage, build this
  directory.
configure.in:
  build storage engines' unit tests.
storage/maria/Makefile.am:
  ma_control_file_test moves to unittest/storage/maria
storage/maria/ma_control_file.c:
  more error codes when opening the control file fails.
  ma_control_file_end() may now return an error if my_close() failed.
storage/maria/ma_control_file.h:
  more error codes when opening the control file fails.
unittest/Makefile.am:
  adding unit tests for storage engines.
  Note that unit.pl simply recurses into "storage", so if a unit test for
  storage engine X has been built previously, and now you re-configure
  (without making clean) to disable this engine, then the unit test of
  X will not be rebuilt but will still be present in storage/X, so will
  be run.
unittest/storage/maria/ma_control_file-t.c:
  Making the test fit the mytap framework (return all the way up
  the stack instead of assert(); use the mytap functions plan(), ok() etc).
  Adding test of file too short/long.
unittest/storage/maria/Makefile.am:
  a_control_file-t is added to the Maria unit tests.
  Later, older tests (ma_test1 etc) could also move here.
unittest/storage/Makefile.am:
  New BitKeeper file ``unittest/storage/Makefile.am''
 2006-09-11 16:12:31 +02:00
unknown
fa793bca31 WL#3234 "Maria - control file manager": 
added checksum of the file. Now we have size + magic string + checksum
to detect that all is ok.
Plus misc fixes for "make dist" to work and the resulting tarball to build


include/Makefile.am:
  adding pagecache.h to help the tarball build.
  The model of pagecache.h, keycache.h, is in pkginclude_HEADERS,
  wonder why. Adding pagecache.h to noinst_HEADERS for now.
storage/maria/Makefile.am:
  adding ma_control_file.h to help the tarball build
storage/maria/ma_control_file.c:
  adding a simple checksum to the control file.
  We protect against corruption of this file like this:
  - test size
  - test magic string at start
  - test checksum
  I also add some simple my_message() errors (to be changed to a better
  reporting later).
storage/maria/ma_control_file.h:
  comments
storage/maria/ma_control_file_test.c:
  test of wrong checksum in control file
storage/maria/CMakeLists.txt:
  just to make "make dist" happy for now.
 2006-09-04 16:53:09 +02:00
unknown
a1f25544d5 WL#3234 "Maria - control file manager" 
- fixes to the control file module
- unit test for it
- renames of all Maria files I created to start with ma_


storage/maria/ma_checkpoint.c:
  Rename: storage/maria/checkpoint.c -> storage/maria/ma_checkpoint.c
storage/maria/ma_checkpoint.h:
  Rename: storage/maria/checkpoint.h -> storage/maria/ma_checkpoint.h
storage/maria/ma_least_recently_dirtied.c:
  Rename: storage/maria/least_recently_dirtied.c -> storage/maria/ma_least_recently_dirtied.c
storage/maria/ma_least_recently_dirtied.h:
  Rename: storage/maria/least_recently_dirtied.h -> storage/maria/ma_least_recently_dirtied.h
storage/maria/ma_recovery.c:
  Rename: storage/maria/recovery.c -> storage/maria/ma_recovery.c
storage/maria/ma_recovery.h:
  Rename: storage/maria/recovery.h -> storage/maria/ma_recovery.h
storage/maria/Makefile.am:
  control file module and its unit test program
storage/maria/ma_control_file.c:
  DBUG_ tags. Fix for gcc warnings.
  log_no -> logno (I felt "_no" sounded like a standalone "No" word).
  ma_ prefix for some functions.
  last_checkpoint_lsn_at_startup -> last_checkpoint_lsn (no need
  to make special vars for the values at startup). Same for last_logno.
  ma_control_file_write_and_force() now updates last_checkpoint_lsn
  and last_logno, the idea being that they belong to the module,
  others should not update them.
  And thus when the module shuts down, it zeroes those vars.
storage/maria/ma_control_file.h:
  importing structs from Sanja to get the control file module to compile;
  we'll remove that when Sanja pushes the log handler.
  CONTROL_FILE_IMPOSSIBLE_LOGNO is 0, not FFFFFFFF.
storage/maria/ma_control_file_test.c:
  Unit test program for the Maria control file module.
  Modelled after other ma_test* files in this directory (so, does
  not follow the unit test framework recently introduced with libtap;
  TODO as a task on all ma_test* programs).
  We test that writing to the control file works, and re-reading from it
  too, we check (by reading the file by ourselves) that its content
  on disk is correct, and check that a corrupted control file is detected.
 2006-09-01 17:53:10 +02:00
Renamed from storage/maria/control_file.c (Browse further)