Commit graph

150 commits

Author SHA1 Message Date
unknown
422375fc1b Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_bitmap.c:
  Auto merged
storage/maria/ma_checkpoint.c:
  Auto merged
storage/maria/ma_close.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_loghandler.h:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_pagecache.h:
  Auto merged
storage/maria/ma_write.c:
  Auto merged
storage/maria/maria_def.h:
  Auto merged
storage/maria/unittest/ma_pagecache_single.c:
  Auto merged
storage/maria/ma_blockrec.c:
  Manual merge
storage/maria/ma_page.c:
  Manual merge
storage/maria/ma_pagecache.c:
  Manual merge
storage/maria/ma_preload.c:
  Manual merge
storage/maria/ma_recovery.c:
  Manual merge
  Add _ma_unpin_all_pages() to all new UNDO redo_exec_hook's
2007-11-14 19:57:58 +02:00
unknown
21fd2a5a36 First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable


include/maria.h:
  Made min page cache larger (needed for pinning key page)
  Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
  Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
  Added new error message to be used when handler initialization failed
include/my_global.h:
  Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
  Added const to some parameters
mysys/array.c:
  More DBUG
mysys/my_error.c:
  Fixed indentation
mysys/my_handler.c:
  Added const to some parameters
  Added missing error messages
sql/field.h:
  Renamed variables to avoid variable shadowing
sql/handler.h:
  Renamed parameter to avoid variable name conflict
sql/item.h:
  Renamed variables to avoid variable shadowing
sql/log_event_old.h:
  Renamed variables to avoid variable shadowing
sql/set_var.h:
  Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
  Removed maria hack for temporary tables
  Fixed indentation
sql/sql_table.cc:
  Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
  This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
  Copy page_checksum from share
  Removed Maria hack
storage/maria/Makefile.am:
  Added new files
storage/maria/ha_maria.cc:
  Renamed records -> record_count and info -> create_info to avoid variable name conflicts
  Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
  Moved _ma_unpin_all_pages() to ma_key_recover.c
  Moved init of info->pinned_pages to ma_open.c
  Moved _ma_finalize_row() to maria_key_recover.h
  Renamed some variables to avoid variable name conflicts
  Mark page_link.changed for blocks we change directly
  Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
  Removed extra empty line
storage/maria/ma_checkpoint.c:
  Remove not needed trnman.h
storage/maria/ma_close.c:
  Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
  Give error message if we fail to open control file
storage/maria/ma_delete.c:
  Changes for redo logging (first part, logging of underflow not yet done)
  - Log undo-key-delete
  - Log delete of key
  - Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
  - Added new arguments to some functions to be able to write redo information
  - Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
  
  Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
  Changed 2 bmove_upp() to bmove() as this made code easer to understand
  More function comments
  Indentation fixes
storage/maria/ma_ft_update.c:
  New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
  Fixed some DBUG_PRINT messages
  Simplify code
  Added new log entrys for key page redo
  Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
  Moved some defines here
  Added define for storing key number on key pages
  Added new translog record types
  Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
  Always allocate info.pinned_pages (we need now also for normal key page usage)
  Update keyinfo->key_nr
  Added virtual functions to convert record position o number to be stored on key pages
  Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
  Added redo for key pages
  - Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
  - _ma_fetch_keypage() now pin's pages if needed
  - Extended _ma_write_keypage() with type of locks to be used
  - ma_dispose() now locks info->s->state.key_del from other threads
  - ma_dispose() writes redo log record
  - ma_new() locks info->s->state.key_del from other threads if it was used
  - ma_new() now pins read page
  
  Other things:
  - Removed some not needed arguments from _ma_new() and _ma_dispose)
  - Added some new variables to simplify code
  - If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
  Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
  Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
  Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
  - Added hooks for new translog types:
    REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
    UNDO_KEY_DELETE_WITH_ROOT.
  - Moved variable declarations to start of function (portability fixes)
  - Removed some not needed initializations
  - Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
  Removed end space
storage/maria/ma_check.c:
  Removed end space
storage/maria/ma_create.c:
  Removed end space
storage/maria/ma_locking.c:
  Removed end space
storage/maria/ma_packrec.c:
  Removed end space
storage/maria/ma_pagecache.c:
  Removed end space
storage/maria/ma_panic.c:
  Removed end space
storage/maria/ma_rt_index.c:
  Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
  Fixed indentation
storage/maria/ma_rt_key.c:
  Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
  Added new arguments for call to _ma_new()
  Use new keypage header
  Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
  Updated comments & indentation
  Added new arguments for call to _ma_fetch_keypage()
  Made some variables and arguments const
  Added virtual functions for converting row position to number to be stored in key
  use MARIA_RECORD_POS of record position instead of my_off_t
  Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
  Removed end space
storage/maria/ma_statrec.c:
  Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
  Fixed too small buffer to init_pagecache()
  Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
  Use more resonable pagecache size
  Remove not used code
  Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
  Fixed wrong test
storage/maria/ma_write.c:
  Lots of new code to handle REDO of key pages
  No logic changes because of REDO code, mostly adding new arguments and adding new code for logging 
  
  Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
  Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
  Zerofill new used pages for:
  - To remove possible sensitive data left in buffer
  - To get idenitical data on pages after running redo
  - Better compression of pages if archived
storage/maria/maria_chk.c:
  Added information if table is crash safe
storage/maria/maria_def.h:
  New virtual function to convert between record position on key and normal record position
  Aded mutex and extra variables to handle locking of share->state.key_del
  Moved some structure variables to get things more aligned
  Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
  Added argument to MARIA_PINNED_PAGE to indicate if page was changed
  Updated prototypes for functions
  Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
  Updated arguments for changed function calls
storage/myisam/mi_check.c:
  Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
  Update checksums to ignore null columns
storage/myisam/mi_create.c:
  Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
  Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
  Fixed bug
storage/myisam/myisamdef.h:
  Added virtual function for calculating checksum during check table
  Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
  New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
  New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
  New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 19:08:06 +02:00
unknown
5fbd5dafe7 * WL#4137 Maria- Framework for testing recovery in mysql-test-run
See test maria-recovery.test for a model; all include scripts have
an "API" section at start if they do take parameters from outside.
* Fixing bug reported by Jani and Monty (when two REDOs about the same
page in one group, see ma_blockrec.c). 
* Fixing small bugs in recovery


mysql-test/include/wait_until_connected_again.inc:
  be sure to enter the loop (the previous query by the caller may not have
  failed: it could be
  query;
  mysqladmin shutdown;
  call this script).
mysql-test/lib/mtr_process.pl:
  * Through the "expect" file a test can tell mtr that a server crash
  is expected. What the file contains is irrelevant. Now if its last
  line starts with "wait", mtr will wait before restarting (it will
  wait for the last line to not start with "wait"). This is for
  tests which need to mangle files under the feet of a dead mysqld.
  * Remove "expect" file before restarting; otherwise there could be a
  race condition: tests sees server restarted, does something, writes
  an "expect" file, and then mtr removes that file, then
  test kills mysqld, and then mtr will never restart it.
storage/maria/ma_blockrec.c:
  - when applying a REDO in recovery, we don't anymore put UNDO's LSN on the page
  at once; indeed if in this REDO's group there comes another REDO
  for the same page it would be wrongly skipped. Instead, we keep
  pages pinned, don't change their LSN. When done with all REDOs
  of the group we unpin them and stamp them with UNDO's LSN.
  - fixing bug in applying of REDO_PURGE_BLOCKS in recovery: page_range
  sometimes has TAIL_BIT set, need to turn it down to know the real page
  range.
  - Both bugs are covered in maria-recovery.test
storage/maria/ma_checkpoint.c:
  Capability to, in debug builds only, do some special operations
  (flush all bitmap and data pages, flush state, flush log)
  and crash mysqld, to later test recovery.
   Driven by some --debug=d, symbols.
storage/maria/ma_open.c:
  debugging info
storage/maria/ma_pagecache.c:
  Now that we can _ma_unpin_all_pages() during the REDO phase
  to set page's LSN, the assertion needs to be relaxed.
storage/maria/ma_recovery.c:
  - open trace file in append mode (useful when a test triggers several
  recoveries, we see them all).
  - fixing wrong error detection, it's possible that during recovery
  we want to open an already open table.
  - when applying a REDO in recovery, we don't anymore put UNDO's LSN on the page
  at once; indeed if in this REDO's group there comes another REDO
  for the same page it would be wrongly skipped. Instead, we keep
  pages pinned, don't change their LSN. When done with all REDOs
  of the group we unpin them and stamp them with UNDO's LSN.
  - we verify that all log records of a group are about the same table,
  for debugging.
mysql-test/r/maria-recovery.result:
  result
mysql-test/t/maria-recovery-master.opt:
  crash is expected, core file would take room, stack trace would
  wake pushbuild up.
mysql-test/t/maria-recovery.test:
  Test of recovery from mysql-test (it is already tested as unit tests
  in ma_test_recovery) (WL#4137)
  - test that, if recovery is made to start on an empty table it can
  replay the effects of committed and uncommitted statements (having only
  the committed ones in the end result). This should be the first test
  for someone writing code of new REDOs.
  - test that, if mysqld is crashed and recovery runs we have only
  committed statements in the end result. Crashes are done in different
  ways: flush nothing (so, uncommitted statement is often missing
  from the log => no rollback to do); flush pagecache (implicitely flushes
  log (WAL)) and flush log, both causes rollbacks; flush log can also
  flush state (state.records etc) to test recovery of the state
  (not tested well now as we repair the index anyway).
  - test of bug found by Jani and Monty in recovery (two REDO about
  the same page in one group).
mysql-test/include/maria_empty_logs.inc:
  removes logs, to have a clean sheet for testing recovery.
mysql-test/include/maria_make_snapshot.inc:
  copies a table to another directory, or back, or compares both
  (comparison is not implemented as physical comparison is impossible
  if an UNDO phase happened).
mysql-test/include/maria_make_snapshot_for_comparison.inc:
  copies tables to another directory so that they can later
  serve as a comparison reference (they are the good tables,
  recovery should produce similar ones).
mysql-test/include/maria_make_snapshot_for_feeding_recovery.inc:
  When we want to force recovery to start on old tables, we prepare
  old tables with this script: we put them in a spare directory.
  They are later copied back over mysqltest tables while mysqld is dead.
  We also need to copy back the control file, otherwise mysqld,
  in recovery, would start from the latest checkpoint: latest
  checkpoint plus old tables is not a recovery-possible scenario of course.
mysql-test/include/maria_verify_recovery.inc:
  causes mysqld to crash, restores old tables if requested,
  lets recovery run, compares resulting tables with reference tables
  by using CHECKSUM TABLE.
  We don't do any sanity checks on page's LSN in resulting tables, yet.
2007-11-13 17:12:29 +01:00
unknown
5e2aaf69d8 comments 2007-11-11 19:01:53 +01:00
unknown
f3e957b7c1 Support of rec_lsn added to pagecache_write call.
storage/maria/ma_pagecache.c:
  Support of rec_lsn added to pagecache_write call.
  A function for setting rec_lsn made.
2007-11-05 15:07:50 +02:00
unknown
301ee8d9a3 Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


include/my_sys.h:
  Auto merged
mysql-test/r/maria.result:
  Auto merged
mysql-test/t/maria.test:
  Auto merged
sql/handler.h:
  Auto merged
sql/mysqld.cc:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_bitmap.c:
  Auto merged
storage/maria/ma_blockrec.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/ma_test_recovery.expected:
  Auto merged
storage/maria/ma_test_recovery:
  Auto merged
sql/mysql_priv.h:
  manual merge
storage/maria/ma_recovery.c:
  manual merge
storage/maria/ma_test2.c:
  manual merge
2007-10-20 01:33:12 +03:00
unknown
13d53bf657 Merge some changes from sql directory in 5.1 tree
Changed format for REDO_INSERT_ROWS_BLOBS
Fixed several bugs in handling of big blobs
Added redo_free_head_or_tail() & redo_insert_row_blobs()
Added uuid to control file
maria_checks now verifies that not used part of bitmap is 0
REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
Added REDO_FREE_HEAD_OR_TAIL
Fixes problem when trying to read block outside of file during REDO


include/my_global.h:
  STACK_DIRECTION is already set by configure
mysql-test/r/maria.result:
  Updated results
mysql-test/t/maria.test:
  Test shrinking of VARCHAR
mysys/my_realloc.c:
  Fixed indentation
mysys/safemalloc.c:
  Fixed indentation
sql/filesort.cc:
  Removed some casts
sql/mysqld.cc:
  Added missing setting of myisam_stats_method_str
sql/uniques.cc:
  Removed some casts
storage/maria/ma_bitmap.c:
  Added printing of bitmap (for debugging)
  Renamed _ma_print_bitmap() -> _ma_print_bitmap_changes()
  Added _ma_set_full_page_bits()
  Fixed bug in ma_bitmap_find_new_place() (affecting updates) when using big files
storage/maria/ma_blockrec.c:
  Changed format for REDO_INSERT_ROWS_BLOBS
  Fixed several bugs in handling of big blobs
  Added code to fix some cases where redo when using blobs didn't produce idenital .MAD files as normal usage
  REDO_FREE_ROW_BLOCKS doesn't anymore change pages; We only mark things free in bitmap
  Remove TAIL and filler extents from REDO_FREE_BLOCKS log entry. (Fixed some asserts)
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Delete tails in update. (Fixed bug when doing update that shrinks blob/varchar length)
  Fixed bug when doing insert in block outside of file size.
  Added redo_free_head_or_tail() & redo_insert_row_blobs()
  Added pagecache_unlock_by_link() when read fails.
  Much more comments, DBUG and ASSERT entries
storage/maria/ma_blockrec.h:
  Prototypes of new functions
  Define of SUB_RANGE_SIZE & BLOCK_FILLER_SIZE
storage/maria/ma_check.c:
  Verify that not used part of bitmap is 0
storage/maria/ma_control_file.c:
  Added uuid to control file
storage/maria/ma_loghandler.c:
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_loghandler.h:
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_pagecache.c:
  If we write full block, remove error flag for block.
  (Fixes problem when trying to read block outside of file)
storage/maria/ma_recovery.c:
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_test1.c:
  Allow option after 'b' to be compatible with ma_test2
  (This is just to simplify test scripts like ma_test_recovery)
storage/maria/ma_test2.c:
  Default size of blob is now 1000 instead of 1
storage/maria/ma_test_all.sh:
  Added test for bigger blobs
storage/maria/ma_test_recovery.expected:
  Updated results
storage/maria/ma_test_recovery:
  Added test for bigger blobs
2007-10-20 00:24:22 +03:00
unknown
77017191de WL#3071 - Maria checkpoint
- serializing calls to flush_pagecache_blocks_int() on the same file
to avoid known concurrency bugs
- having that, we can now enable the background thread, as the
flushes it does are now supposedly safe in concurrent situations.
- new type of flush FLUSH_KEEP_LAZY: when the background checkpoint
thread is flushing a packet of dirty pages between two checkpoints,
it uses this flush type, indeed if a file is already being flushed
by another thread it's smarter to move on to the next file than wait.
- maria_checkpoint_frequency renamed to maria_checkpoint_interval.


include/my_sys.h:
  new type of flushing for the page cache: FLUSH_KEEP_LAZY
mysql-test/r/maria.result:
  result update
mysys/mf_keycache.c:
  indentation. No FLUSH_KEEP_LAZY support in key cache.
storage/maria/ha_maria.cc:
  maria_checkpoint_frequency was somehow a hidden part of the
  Checkpoint API and that was not good. Now we have checkpoint_interval,
  local to ha_maria.cc, which serves as container for the user-visible
  maria_checkpoint_interval global variable; setting it calls
  update_checkpoint_interval which passes the new value to
  ma_checkpoint_init(). There is no hiding anymore.
  By default, enable background thread which does checkpoints
  every 30 seconds, and dirty page flush in between. That thread takes
  a checkpoint when it ends, so no need for maria_hton_panic to take one.
  The | is | and not ||, because maria_panic() must always be called.
  frequency->interval.
storage/maria/ma_checkpoint.c:
  Use FLUSH_KEEP_LAZY for background thread when it flushes packets of
  dirty pages between two checkpoints: it is smarter to move on to
  the next file than wait for it to have been completely flushed, which
  may take long.
  Comments about flush concurrency bugs moved from ma_pagecache.c.
  Removing out-of-date comment.
  frequency->interval.
  create_background_thread -> (interval>0).
  In ma_checkpoint_background(), some variables need to be preserved
  between iterations.
storage/maria/ma_checkpoint.h:
  new prototype
storage/maria/ma_pagecache.c:
  - concurrent calls of flush_pagecache_blocks_int() on the same file
  cause bugs (see @note in that function); we fix them by serializing
  in this situation. For that we use a global hash of (file, wqueue).
  When flush_pagecache_blocks_int() starts it looks into the hash,
  using the file as key. If not found, it inserts (file,wqueue) into the
  hash, flushes the file, and finally removes itself from the hash and
  wakes up any waiter in the queue. If found, it adds itself to the
  wqueue and waits.
  - As a by-product, we can remove changed_blocks_is_incomplete
  and replace it by scanning the hash, replace the sleep() by a queue wait.
  - new type of flush FLUSH_KEEP_LAZY: when flushing a file, if it's
  already being flushed by another thread (even partially), return
  immediately.
storage/maria/ma_pagecache.h:
  In pagecache, a hash of files currently being flushed (i.e. there
  is a call to flush_pagecache_blocks_int() for them).
storage/maria/ma_recovery.c:
  new prototype
storage/maria/ma_test1.c:
  new prototype
storage/maria/ma_test2.c:
  new prototype
2007-10-19 14:15:13 +02:00
unknown
0f1feefa03 WL#3071 Maria checkpoint
Ability for flush_pagecache_blocks() to flush only certain pages of
a file, as instructed by an option "filter" pointer-to-function argument;
Checkpoint and background dirty page flushing use that to flush only
pages which have been dirty for long enough and bitmap pages.
Fix for a bug in flush_cached_blocks() (no idea if it could produce
a bug in real life, but theoretically it is).
Testing checkpoint in ma_test_recovery via ma_test1 and ma_test2.
Background checkpoint & dirty pages flush thread is still disabled
by default in ha_maria.


mysql-test/r/maria.result:
  result update
storage/maria/ha_maria.cc:
  blank after function comment
storage/maria/ma_checkpoint.c:
  Using an enum instead of 0/1/2 (applying Sanja's review comments).
  The comment about "this is an horizon" can be removed as Sanja
  created translog_next_LSN() which parse_checkpoint_record() uses.
  Variables in ma_checkpoint_background() cannot be declared in the
  for() as their value must not be reset at each iteration!
storage/maria/ma_pagecache.c:
  adding to flush_pagecache_blocks() optional arguments 'filter'
  (pointer to function) and 'filter_arg'; if filter!=NULL this function
  will be called for each block of the file and will reply if this
  block and following ones should be flushed or not (3 possible
  replies).
  Fixing a bug when flush_cached_blocks() skips a pinned page: it has
  to unset PCBLOCK_IN_FLUSH set by flush_pagecache_blocks_int().
storage/maria/ma_pagecache.h:
  flush_pagecache_blocks() is changed to take "filter" and "filter_arg"
  arguments. "filter", if it is not NULL, may return one value
  among enum pagecache_flush_filter_result.
storage/maria/ma_recovery.c:
  open_count=0 when closing tables at the end of recovery.
storage/maria/ma_test1.c:
  Optional checkpoints (-H#) at various stages (stages similar
  to --testflag), for testing of checkpoints.
storage/maria/ma_test2.c:
  Optional checkpoints (-H#) at various stages (stages similar
  to -t), for testing of checkpoints.
storage/maria/ma_test_recovery.expected:
  Result update: the results of the additional test run with -H#
  (checkpoints) are added here. They are exactly identical to without
  checkpoints except that the index's Root (printed by maria_chk)
  is more correct when using checkpoints. This is because checkpoint
  flushed the state, so it happens to be correct, while no-checkpoint
  does not flush the state, and recovery does not recover indexes
  so Root is never fixed. When we recover indices, this will go away.
storage/maria/ma_test_recovery:
  We duplicate the loop of tests to add an additional run with
  checkpoints at various stages, to see if maria_read_log
  uses them fine.
2007-10-17 16:55:26 +02:00
unknown
08b7152893 WL#3072 Maria recovery
- fixes (in recovery, checkpoint, log handler) of bugs found
during testing.
- new option --check for maria_read_log: with --only-display (which only
reads the header) it reads the full record, for debugging.


storage/maria/ma_loghandler.c:
  importing patch from Sanja for bug of translog_next_LSN() found
  during recovery
storage/maria/ma_loghandler_lsn.h:
  better types (0L is 4 bytes on some platforms, it causes problems
  when used into lsn_store(): right shift >= width of type.
storage/maria/ma_pagecache.c:
  work around infamous "PAGECACHE_PLAIN_PAGE used for transactional
  tables in specialm case"; REDO phase disables logging and this causes
  pages to be PAGECACHE_PLAIN_PAGE, thus ignored wrongly by the
  checkpoint taken at the end of the REDO phase.
storage/maria/ma_recovery.c:
  - a #ifdef which broke maria_read_log in non-debug builds (no output!)
  - support for maria_read_log --check
  - detect record corruption before opening the table
  - updating is_of_horizon requires writing the state
  - fix for wrong parsing of checkpoint record by recovery
storage/maria/ma_recovery.h:
  support for maria_read_log --check
storage/maria/maria_read_log.c:
  Option --check: --only-display only looks at the header;
  adding --check tries a translog_read_record() to see if record can
  be fully read (this is to find bugs).
2007-10-16 14:56:38 +02:00
unknown
212313de55 Optimisation todo for loghandler added.
Debug output information fixed.
Fixed direct page referencing in write mode.


storage/maria/ma_loghandler.c:
  TODO added.
storage/maria/ma_pagecache.c:
  Debug output information fixed.
  Fixed direct page referencing in write mode.
2007-10-07 23:27:03 +03:00
unknown
e814261700 Fix for pushbuild failure (noticable only on 64-bit)
storage/maria/ma_pagecache.c:
  pagecache->blocks is now long, takes 8 bytes on some platforms.
  The cast to ulonglong in int8store is for those platforms where
  ulong is 32-bit and int8store uses some << shifts, if there are
  (x<<40 is undefined if x is 32-bit).
storage/maria/ma_recovery.c:
  this change corresponds to the one done in ma_pagecache.c: number
  of dirty pages is stored in 8 bytes.
2007-10-02 19:16:20 +02:00
unknown
d0b9387b88 WL#3072 - Maria recovery.
* Recovery of the table's live checksum (CREATE TABLE ... CHECKSUM=1)
is achieved in this patch. The table's live checksum
(info->s->state.state.checksum) is updated in inwrite_rec_hook's
under the log mutex when writing UNDO_ROW_INSERT|UPDATE|DELETE
and REDO_DELETE_ALL. The checksum variation caused by the operation
is stored in these UNDOs, so that the REDO phase, when it sees such
UNDOs, can update the live checksum if it is older (state.is_of_lsn is
lower) than the record. It is also used, as a nice add-on with no
cost, to do less row checksum computation during the UNDO phase
(as we have it in the record already).
Doing this work, it became pressing to move in-write hooks
(write_hook_for_redo() et al) to ma_blockrec.c.
The 'parts' argument of inwrite_rec_hook is unpredictable (it comes
mangled at this stage, for example by LSN compression) so it is
replaced by a 'void* hook_arg', which is used to pass down information,
currently only to write_hook_for_clr_end() (previous undo_lsn and
type of undone record).
* If from ha_maria, we print to stderr how many seconds (with one
fractional digit) the REDO phase took, same for UNDO phase and for
final table close. Just to give an indication for debugging and maybe
also for Support.


storage/maria/ha_maria.cc:
  question for Monty
storage/maria/ma_blockrec.c:
  * log in-write hooks (write_hook_for_redo() etc) move from
  ma_loghandler.c to here; this is natural: the hooks are coupled
  to their callers (functions in ma_blockrec.c).
  * translog_write_record() now has a new argument "hook_arg";
  using it to pass down to write_hook_for_clr_end() the transaction's
  previous_undo_lsn and the type of the being undone record, and also
  to pass down to all UNDOs the live checksum variation caused by the
  operation.
  * If table has live checksum, store in UNDO_ROW_INSERT|UPDATE|DELETE
  and in CLR_END the checksum variation ("delta") caused by the
  operation. For example if a DELETE caused the table's live checksum
  to change from 123 to 456, we store in the UNDO_ROW_DELETE, in 4 bytes,
  the value 333 (456-123).
  * Instead of hard-coded "1" as length of the place where we store
  the undone record's type in CLR_END, use a symbol CLR_TYPE_STORE_SIZE;
  use macros clr_type_store and clr_type_korr.
  * write_block_record() has a new parameter 'old_record_checksum'
  which is the pre-computed checksum of old_record; that value is used
  to update the table's live checksum when writing UNDO_ROW_UPDATE|CLR_END.
  * In allocate_write_block_record(), if we are executing UNDO_ROW_DELETE
  the row's checksum is already computed.
  * _ma_update_block_record2() now expect the new row's checksum into
  cur_row.checksum (was already true) and the old row's checksum into
  new_row.checksum (that's new). Its two callers, maria_update() and
  _ma_apply_undo_row_update(), honour this.
  * When executing an UNDO_ROW_INSERT|UPDATE|DELETE in UNDO phase, pick
  up the checksum delta from the log record. It is then used to update
  the table's live checksum when writing CLR_END, and saves us a
  computation of record.
storage/maria/ma_blockrec.h:
  in-write hooks move from ma_loghandler.c
storage/maria/ma_check.c:
  more straightforward size of buffer
storage/maria/ma_checkpoint.c:
  <= is enough
storage/maria/ma_commit.c:
  new prototype of translog_write_record()
storage/maria/ma_create.c:
  new prototype of translog_write_record()
storage/maria/ma_delete.c:
  The row's checksum must be computed before calling(*delete_record)(),
  not after, because it must be known inside _ma_delete_block_record()
  (to update the table's live checksum when writing UNDO_ROW_DELETE).
  If deleting from a transactional table, live checksum was already updated
  when writing UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
  @todo is now done (in ma_loghandler.c)
storage/maria/ma_delete_table.c:
  new prototype of translog_write_record()
storage/maria/ma_loghandler.c:
  * in-write hooks move to ma_blockrec.c.
  * translog_write_record() gets a new argument 'hook_arg' which is
  passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
  for those hooks, because when those hooks are called, 'parts' has
  possibly been mangled (like with LSN compression) and is so
  unpredictable.
  * fix for compiler warning (unused buffer_start when compiling without
  debug support)
  * Because checksum delta is stored into UNDO_ROW_INSERT|UPDATE|DELETE
  and CLR_END, but only if the table has live checksum, these records
  are not PSEUDOFIXEDLENGTH anymore, they are now VARIABLE_LENGTH (their
  length is X if no live checksum and X+4 otherwise).
  * add an inwrite_rec_hook for UNDO_ROW_UPDATE, which updates the
  table's live checksum. Update it also in hooks of UNDO_ROW_INSERT|
  DELETE and REDO_DELETE_ALL and CLR_END.
  * Bugfix: when reading a record in translog_read_record(), it happened
  that "length" became negative, because the function assumed that
  the record extended beyond the page's end, whereas it may be shorter.
storage/maria/ma_loghandler.h:
  * Instead of hard-coded "1" and "4", use symbols and macros
  to store/retrieve the type of record which the CLR_END corresponds
  to, and the checksum variation caused by the operation which logs the
  record
  * translog_write_record() gets a new argument 'hook_arg' which is
  passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
  for those hooks, because when those hooks are called, 'parts' has
  possibly been mangled (like with LSN compression) and is so
  unpredictable.
storage/maria/ma_open.c:
  fix for "empty body in if() statement" (when compiling without safemutex)
storage/maria/ma_pagecache.c:
  <= is enough
storage/maria/ma_recovery.c:
  * print the time that each recovery phase (REDO/UNDO/flush) took;
  this is enabled only when recovering from ha_maria. Is it printed
  n seconds with a fractional part of one digit (like 123.4 seconds).
  * In the REDO phase, update the table's live checksum by using
  the checksum delta stored in UNDO_ROW_INSERT|DELETE|UPDATE and CLR_END.
  Update it too when seeing REDO_DELETE_ALL.
  * In the UNDO phase, when executing UNDO_ROW_INSERT, if the table does
  not have live checksum then reading the record's header (as done by
  the master loop of run_undo_phase()) is enough; otherwise we
  do a translog_read_record() to have the checksum delta ready
  for _ma_apply_undo_row_insert().
  * When at the end of the REDO phase we notice that there is an unfinished
  group of REDOs, don't assert in debug binaries, as I verified that it
  can happen in real life (with kill -9)
  * removing ' in #error as it confuses gcc3
storage/maria/ma_rename.c:
  new prototype of translog_write_record()
storage/maria/ma_test_recovery.expected:
  Change in output of ma_test_recovery: now all live checksums of
  original tables equal those of tables recreated by the REDO phase
  and those of tables fixed by the UNDO phase. I.e. recovery of
  the live checksum looks like working (which was after all the only
  goal of this changeset).
  I checked by hand that it's not just all live checksums which are
  now 0 and that's why they match. They are the old values like
  3757530372. maria.test has hard-coded checksum values in its result
  file so checks this too.
storage/maria/ma_update.c:
  * It's useless to put up HA_STATE_CHANGED in 'key_changed',
  as we put up HA_STATE_CHANGED in info->update anyway.
  * We need to compute the old and new rows' checksum before calling
  (*update_record)(), as checksum delta must be known when logging
  UNDO_ROW_UPDATE which is done by _ma_update_block_record(). Note that
  some functions change the 'newrec' record (at least _ma_check_unique()
  does) so we cannot move the checksum computation too early in the
  function.
storage/maria/ma_write.c:
  If inserting into a transactional table, live's checksum was
  already updated when writing UNDO_ROW_INSERT. The multiplication
  is a trick to save an if().
storage/maria/unittest/ma_test_loghandler-t.c:
  new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  new prototype of translog_write_record()
storage/myisam/sort.c:
  fix for compiler warnings in pushbuild (write_merge_key* functions
  didn't have their declaration match MARIA_HA::write_key).
2007-10-02 18:02:09 +02:00
unknown
0de4c55623 Faster translog_filename_by_fileno
Fixed compiler warning
PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*


storage/maria/ma_loghandler.h:
  PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
storage/maria/ma_pagecache.c:
  PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
storage/maria/ma_pagecache.h:
  PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
storage/maria/maria_def.h:
  PAGECACHE_PAGE_LINK -> PAGECACHE_BLOCK_LINK*
2007-09-27 17:41:21 +03:00
unknown
c9c58163a9 Remove SAFE_MODE for opt_range as it disables UPDATE to use keys
REDO optimization (Bascily avoid moving blocks from/to pagecache)
More command line arguments to maria_read_log
Fixed recovery bug when recreating table


sql/opt_range.cc:
  Remove SAFE_MODE for opt_range as it disables UPDATE to use keys
storage/maria/ma_blockrec.c:
  REDO optimization
  Use new interface for pagecache_reads to avoid copying page buffers
storage/maria/ma_loghandler.c:
  Patch from Sanja:
  - Added new parameter to translog_get_page to use direct links to pagecache
  - Changed scanner to be able to use direct links
  
  This avoids a lot of calls to bmove512() in page cache.
storage/maria/ma_loghandler.h:
  Added direct link to pagecache objects
storage/maria/ma_open.c:
  Added const to parameter
  Added missing braces
storage/maria/ma_pagecache.c:
  From Sanja:
  - Added direct links to pagecache (from pagecache_read())
    Dirrect link means that on pagecache_read we get back a pointer to the pagecache buffer
  
  
  From Monty:
  - Fixed arguments to init_page_cache to handle big page caches
  - Fixed compiler warnings
  - Replaced PAGECACHE_PAGE_LINK with PAGECACHE_BLOCK_LINK * to catch errors
storage/maria/ma_pagecache.h:
  Changed block numbers from int to long to be able to handle big page caches
  Changed some PAGECACHE_PAGE_LINK to PAGECACHE_BLOCK_LINK
storage/maria/ma_recovery.c:
  Fixed recovery bug when recreating table (table was kept open)
  Moved some variables to function start (portability)
  Added space to some print messages
storage/maria/maria_chk.c:
  key_buffer_size -> page_buffer_size
storage/maria/maria_def.h:
  Changed default page_buffer_size to 10M
storage/maria/maria_read_log.c:
  Added more startup options:
  --version
  --undo (apply undo)
  --page_cache_size (to run with big cache sizes)
  --silent (to not get any output from --apply)
storage/maria/unittest/ma_control_file-t.c:
  Fixed compiler warning
storage/maria/unittest/ma_test_loghandler-t.c:
  Added new argument to translog_init_scanner()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Added new argument to translog_init_scanner()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Added new argument to translog_init_scanner()
2007-09-27 14:18:28 +03:00
unknown
a303f5b2c8 Fixes of the empty log problem.
storage/maria/ma_checkpoint.c:
  The new macro for easier printing LSN added.
storage/maria/ma_loghandler.c:
  The assertion returned.
  The new macro for easier printing LSN added.
storage/maria/ma_loghandler_lsn.h:
  The new macro for easier printing LSN added.
storage/maria/ma_pagecache.c:
  The new macro for easier printing LSN added.
storage/maria/ma_recovery.c:
  Recovery checks empty log state.
  RECHEADER_READ_ERROR means some real error.
storage/maria/maria_read_log.c:
  Read log starts from real beggining of the log and precess
  error and empty log states.
  The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler-t.c:
  The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  The new macro for easier printing LSN added.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  The new macro for easier printing LSN added.
2007-09-13 10:37:51 +03:00
unknown
cec8ac3e07 WL#3071 Maria checkpoint
Finally this is the real checkpoint code.
It however exhibits unstabilities when a checkpoint runs concurrently
with data-modifying clients (table corruption, transaction log's
assertions) so for now a checkpoint is taken only at startup after
recovery and at shutdown, i.e. not in concurrent situations. Later
we will let it run periodically, as well as flush dirty pages
periodically (almost all needed code is there already, only pagecache
code is written but not committed).
WL#3072 Maria recovery
* replacing UNDO_ROW_PURGE with CLR_END; testing of those CLR_END via
ma_test2 which has INSERTs failing with duplicate keys.
* replaying of REDO_RENAME_TABLE
Now, off to test Recovery in ha_maria :)


BitKeeper/deleted/.del-ma_least_recently_dirtied.c:
  Delete: storage/maria/ma_least_recently_dirtied.c
BitKeeper/deleted/.del-ma_least_recently_dirtied.h:
  Delete: storage/maria/ma_least_recently_dirtied.h
storage/maria/Makefile.am:
  compile Checkpoint module
storage/maria/ha_maria.cc:
  When ha_maria starts, do a recovery from last checkpoint.
  Take a checkpoint when that recovery has ended and when ha_maria
  shuts down cleanly.
storage/maria/ma_blockrec.c:
  * even if my_sync() fails we have to my_close() (otherwise we leak
  a descriptor)
  * UNDO_ROW_PURGE is replaced by a simple CLR_END for UNDO_ROW_INSERT,
  as promised in the old comment; it gives us skipping during the
  UNDO phase.
storage/maria/ma_check.c:
  All REDOs before create_rename_lsn are ignored by Recovery. So
  create_rename_lsn must be set only after all data/index has been
  flushed and forced to disk. We thus move write_log_record_for_repair()
  to after _ma_flush_tables_files_after_repair().
storage/maria/ma_checkpoint.c:
  Checkpoint module.
storage/maria/ma_checkpoint.h:
  optional argument if caller wants a thread to periodically take
  checkpoints and flush dirty pages.
storage/maria/ma_create.c:
  * no need to init some vars as the initial bzero(share) takes care of this.
  * update to new function's name
  * even if we fail in my_sync() we have to my_close()
storage/maria/ma_extra.c:
  Checkpoint reads share->last_version under intern_lock, so we make
  maria_extra() update it under intern_lock. THR_LOCK_maria still needed
  because of _ma_test_if_reopen().
storage/maria/ma_init.c:
  destroy checkpoint module when Maria shuts down.
storage/maria/ma_loghandler.c:
  * UNDO_ROW_PURGE gone (see ma_blockrec.c)
  * we need to remember the LSN of the LOGREC_FILE_ID for a share,
  because this LSN is needed into the checkpoint record (Recovery wants
  to know the validity domain of an id->name mapping)
  * translog_get_horizon_no_lock() needed for Checkpoint
  * comment about failing assertion (Sanja knows)
  * translog_init_reader_data() thought that translog_read_record_header_scan()
  returns 0 in case of error, but 0 just means "0-length header".
  * translog_assign_id_to_share() now needs the MARIA_HA because
  LOGREC_FILE_ID uses a log-write hook.
  * Verify that (de)assignment of share->id happens only under intern_lock,
  as Checkpoint reads this id with intern_lock.
  * translog_purge() can accept TRANSLOG_ADDRESS, not necessarily
  a real LSN.
storage/maria/ma_loghandler.h:
  prototype updates
storage/maria/ma_open.c:
  no need to initialize "res"
storage/maria/ma_pagecache.c:
  When taking a checkpoint, we don't need to know the maximum rec_lsn
  of dirty pages; this LSN was intended to be used in the two-checkpoint
  rule, but last_checkpoint_lsn is as good.
  4 bytes for stored_list_size is enough as PAGECACHE::blocks (number
  of blocks which the pagecache can contain) is int.
storage/maria/ma_pagecache.h:
  new prototype
storage/maria/ma_recovery.c:
  * added replaying of REDO_RENAME_TABLE
  * UNDO_ROW_PURGE gone (see ma_blockrec.c), replaced by CLR_END
  * Recovery from the last checkpoint record now possible
  * In new_table() we skip the table if the id->name mapping is older than
  create_rename_lsn (mapping dates from lsn_of_file_id).
  * in get_MARIA_HA_from_REDO_record() we skip the record
  if the id->name mapping is newer than the record (can happen if processing
  a record which is before the checkpoint record).
  * parse_checkpoint_record() has to return a LSN, that's what caller expects
storage/maria/ma_rename.c:
  new function's name; log end zeroes of tables' names (ease recovery)
storage/maria/ma_test2.c:
  * equivalent of ma_test1's --test-undo added (named -u here).
  * -t=1 now stops right after creating the table, so that
  we can test undoing of INSERTs with duplicate keys (which tests the
  CLR_END logged by _ma_write_abort_block_record()).
storage/maria/ma_test_recovery.expected:
  Result of testing undoing of INSERTs with duplicate keys; there are
  some differences in maria_chk -dvv but they are normal (removing
  records does not shrink data/index file, does not put back the
  "analyzed, optimized keys"(etc) index state.
storage/maria/ma_test_recovery:
  Test undoing of INSERTs with duplicate keys, using ma_test2;
  when such INSERT happens, it logs REDO_INSERT, UNDO_INSERT, REDO_DELETE,
  CLR_END; we abort after that, and test that CLR_END causes recovery
  to jump over UNDO_INSERT.
storage/maria/ma_write.c:
  comment
storage/maria/maria_chk.c:
  comment
storage/maria/maria_def.h:
  * a new bit in MARIA_SHARE::in_checkpoint, used to build a list
  of unique shares during Checkpoint.
  * MARIA_SHARE::lsn_of_file_id added: the LSN of the last LOGREC_FILE_ID
  for this share; needed to know to which LSN domain the mappings
  found in the Checkpoint record apply (new mappings should not apply
  to old REDOs).
storage/maria/trnman.c:
  * small changes to how trnman_collect_transactions() fills its buffer;
  it also uses a non-dummy lsn_read_non_atomic() found in ma_checkpoint.h
2007-09-12 11:27:34 +02:00
unknown
044c4103ba Added undo of deleted row
Added part of undo of update row
Extended ma_test1 for recovery testing
Some bug fixes


storage/maria/ha_maria.cc:
  Ignore 'state.split' in case of block records
storage/maria/ma_bitmap.c:
  Added return value for _ma_bitmap_find_place() for how much data we should put on head page
storage/maria/ma_blockrec.c:
  Added undo of deleted row.
  - Added logging of CLR_END records in write_block_record()
  - Split ma_write_init_block_record() to two functions to get better code reuse
  - Added _ma_apply_undo_row_delete()
  - Added ma_get_length()
  
  Added 'empty' prototype for undo_row_update()
  
  Fixed bug when moving data withing a head/tail page.
  Fixed bug when reading a page with bigger LSN but of different type than was expected.
  Store undo_lsn first in CLR_END record
  
  Simplified some code by adding local variables.
  Changed log format for UNDO_ROW_DELETE to store total length of used blobs
storage/maria/ma_blockrec.h:
  Added prototypes for undo code.
storage/maria/ma_pagecache.c:
  Allow plain page to change to LSN page (needed in recovery to apply UNDO)
storage/maria/ma_recovery.c:
  Added undo handling of UNDO_ROW_DELETE and UNDO_ROW_UPDATE
storage/maria/ma_test1.c:
  Extended --test-undo option to allow us to die after insert or after delete.
  Fixed bug in printing key values when using -v
storage/maria/maria_def.h:
  Moved some variables around to be getter alignment
  Added length_buff buffer to be used during undo handling
2007-09-05 02:57:53 +03:00
unknown
b4b0bf47ef Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


storage/maria/ma_pagecache.c:
  Auto merged
2007-09-04 10:54:24 +03:00
unknown
03437ea043 Added UNDO handling of insert during recovery
storage/maria/ma_blockrec.c:
  Added UNDO handling of insert during recovery
  To do this, I also had to add write locking of tail pages during undo phase (As we need to access the same page twice if extents are split over two pages)
  Another way to handle the undo of insert would be to store the extent information as part of the UNDO_INSERT block.
storage/maria/ma_blockrec.h:
  Added new prototype
storage/maria/ma_loghandler.c:
  Changed type of CLR_END (to avoid crash in log handler)
  Removed not used variable
storage/maria/ma_loghandler.h:
  Added TRN argument to record_execute_in_undo_phase()
storage/maria/ma_pagecache.c:
  Hack for undo phase of recovery.  During REDO we work with PLAIN pages, but UNDO works with LSN pages, which caused an abort when trying to access a cached page.
storage/maria/ma_recovery.c:
  Added execution of UNDO_ROW_INSERT
storage/maria/ma_test1.c:
  Added option --test-undo for testing recovery with undo
storage/maria/maria_read_log.c:
  Added processing of undos
2007-09-04 10:53:52 +03:00
unknown
fe8920365b Spelling of comments fixed. 2007-09-04 08:38:52 +03:00
unknown
459943d797 fixed possible problem with multiply write locking (write counter added). 2007-09-03 16:55:46 +03:00
unknown
5183a4b00b Fixed several bugs found by running *.test with maria engine
Renamed HA_EXTRA_PREAPRE_FOR_DELETE to HA_EXTRA_PERPARE_FOR_DROP
Added HA_EXTRA_PREPARE_FOR_RENAME (as we in the code before used HA_EXTRA_PREPARE_FOR_DELETE also for renames which confused things)
Allow multiple write locks for same page by same file handle
Don't write table state if table is not changed


include/my_base.h:
  Renamed HA_EXTRA_PREAPRE_FOR_DELETE to HA_EXTRA_PERPARE_FOR_DROP
  Added HA_EXTRA_PREPARE_FOR_RENAME (as we in the code before used HA_EXTRA_PREPARE_FOR_DELETE also for renames which confused things)
mysql-test/r/maria.result:
  More tests of things that failed in other tests
mysql-test/t/maria.test:
  More tests of things that failed in other tests
sql/ha_partition.cc:
  HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
  Use HA_EXTRA_PREPARE_FOR_RENAME for renames
sql/ha_partition.h:
  HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
  Use HA_EXTRA_PREPARE_FOR_RENAME for renames
sql/lock.cc:
  Fixed comment
sql/sql_table.cc:
  Fixed wrong usage of HA_EXTRA_PREAPRE_FOR_DELETE
storage/maria/ha_maria.cc:
  Added missing _ma_renable_logging_for_table()  (When using with ALTER TABLE + repair index)
  Enabled fast generation of index
storage/maria/ma_bitmap.c:
  Fixed bug when resetting full pages when page was a tail page
storage/maria/ma_blockrec.c:
  Fixed several bugs found by running *.test with maria engine:
  During update we keep old changed pages locked with a write lock to be able to reuse them.
  - Fixed bug with allocated but not used tail part
  - Fixed bug with blob that only had tail part
  - Fixed bug when update reused a page (needed multiple write locks for same page)
  - Fixed bug when first extent was a tail block
storage/maria/ma_check.c:
  Better error message when bitmap is destroyed
storage/maria/ma_close.c:
  Only write status if file was changed.
  Fixed bug when maria_chk -e file_name changed the file.
storage/maria/ma_dynrec.c:
  Removed not used argument to _ma_state_info_read_dsk
storage/maria/ma_extra.c:
  HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
  Use HA_EXTRA_PREPARE_FOR_RENAME for renames
  Only ignore flushing of pages for DROP (not rename)
storage/maria/ma_locking.c:
  Removed not used argument to _ma_state_info_read_dsk
storage/maria/ma_open.c:
  Removed not used argument to _ma_state_info_read_dsk
storage/maria/ma_pagecache.c:
  Allow multiple write locks for same page by same file handle
  (Not yet complete, Sanja will fix)
storage/maria/ma_recovery.c:
  HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
storage/maria/maria_def.h:
  Removed not used argument to _ma_state_info_read_dsk
storage/myisam/mi_extra.c:
  HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
  Use HA_EXTRA_PREPARE_FOR_RENAME for renames
  Only ignore flushing of pages for DROP (not rename)
storage/myisammrg/ha_myisammrg.cc:
  HA_EXTRA_PREPARE_FOR_DELETE -> HA_EXTRA_PREPARE_FOR_DROP
  Use HA_EXTRA_PREPARE_FOR_RENAME for renames
2007-09-03 12:05:17 +03:00
unknown
fddf505081 Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-noflush
into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria


storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler-t.c:
  Auto merged
storage/maria/ma_loghandler.c:
  merge
storage/maria/ma_pagecache.h:
  merge
2007-08-13 15:19:31 +03:00
unknown
7c32eac40a Ability to read unflushed data added (only problem with CRC left and have to be fixed).
Already written pages injection to the cache fixed.


storage/maria/ma_loghandler.c:
  Ability to read unflushed data added.
storage/maria/ma_page.c:
  Parameters added
storage/maria/ma_pagecache.c:
  Already written pages injection to the cache fixed.
  Validator for case of page content injection added.
storage/maria/ma_pagecache.h:
  Validator for case of page content injection added.
storage/maria/unittest/Makefile.am:
  Test of reading unflushed data
storage/maria/unittest/ma_test_loghandler-t.c:
  Define fixed.
  Restart of the log removed.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  New BitKeeper file ``storage/maria/unittest/ma_test_loghandler_noflush-t.c''
2007-08-13 15:17:49 +03:00
unknown
91f03a4d31 WL#3072 Maria checkpoint
Preparation: for the two-checkpoint rule we will flush all pages
which have block->rec_lsn <= last_checkpoint_lsn. Pages with
rec_lsn not yet set (i.e. ==0) were dirtied very recently and so should
not be flushed (otherwise it's not efficient: we should try to let
a page be modified several times in memory before we flush it to disk).
To make this easy, "block->rec_lsn not yet set" is now expressed
with block->rec_lsn==LSN_MAX, not block->rec_lsn==0 anymore.
It is easier this way because LSN_MAX>last_checkpoint_lsn whereas
0<=last_checkpoint_lsn.


storage/maria/ma_blockrec.c:
  typo
storage/maria/ma_loghandler.c:
  typo
storage/maria/ma_loghandler_lsn.h:
  LSN_MAX
storage/maria/ma_pagecache.c:
  "block->rec_lsn not yet set" is now expressed by block->rec_lsn==LSN_MAX,
  not block_rec_lsn==0.
2007-08-02 12:42:49 +02:00
unknown
9554b40df2 Maria: it is allowed to change unknown type to any legal type of page
in the pagecache: fixing wrong assertion and a test case.
maria.test and ps_maria.test still fail.


mysql-test/r/maria.result:
  result update
mysql-test/t/maria.test:
  added test for incorrect assert on page's type
storage/maria/ma_pagecache.c:
  It is allowed to change unknown type to any legal type of page in the
  pagecache.
2007-07-30 12:01:52 +02:00
unknown
662002fc8f post-merge fixes, and fixes for some of the 16 compiler warnings
found in pushbuild on sapsrv1. Some not fixed as not repeatable
on my machine (32/64 bit issue?).
Fixes for some test failures:
- "maria-connect" now passes;
- "maria": after fixing the obvious reasons for failures, the test
went further and hit a more complex issues: difference in the output
of EXPLAIN output; not fixed;
- "ps_maria" still crashes in assertion
mysqld: ha_maria.cc:1627: virtual int ha_maria::index_read(uchar*, const uchar*, uint, ha_rkey_function): Ass
ertion `inited == INDEX' failed, as already observable in pushbuild.
All this might just be due to an incomplete merge of MyISAM changes
into Maria when 5.1 was last merged to mysql-maria.


include/my_global.h:
  temporary fix until next merge of 5.1; without this it does not build
mysql-test/r/maria-connect.result:
  position changed
mysql-test/t/maria-connect.test:
  If one wants to use the binlog it has to ask for it.
  1582 is not used for dup entry error anymore (it was in older 5.1).
  Size of first event in binlog was increased by 4 (when the new type
  of event "gap" was added).
mysql-test/t/maria.test:
  1582 not used anymore in this case
storage/maria/ha_maria.cc:
  engine now has to say what binlogging it supports
storage/maria/ma_blockrec.c:
  fix for compiler warnings ("comparison is always true" or "always
  false")
storage/maria/ma_loghandler.c:
  fix for compiler warnings (comparing char* to uchar*)
storage/maria/ma_packrec.c:
  fix for compiler warning (fix simply merged from MyISAM)
storage/maria/ma_pagecache.c:
  info_check_pin() was not used so gave a compiler warning.
storage/maria/ma_pagecache.h:
  fixing typo from the last 5.1->maria merge.
storage/maria/ma_recovery.c:
  my_free() has a void* argument, so why cast. byte->uchar.
storage/maria/ma_search.c:
  fix for compiler warning (fix simply merged from MyISAM)
storage/maria/maria_read_log.c:
  gptr->uchar*
storage/maria/trnman.c:
  probable fix for warning found in pushbuild (but not on my machine):
  storage/maria/trnman.c: 142
   passing argument 6 of \u2018lf_hash_init\u2019 from incompatible pointer type
  on sapsrv1.
2007-07-26 17:51:49 +02:00
unknown
8ba21c616e Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-maria
into  a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-maria.prod


BitKeeper/etc/ignore:
  auto-union
include/maria.h:
  Auto merged
include/my_base.h:
  Auto merged
mysql-test/include/ps_conv.inc:
  Auto merged
mysql-test/r/ps_2myisam.result:
  Auto merged
mysql-test/r/ps_3innodb.result:
  Auto merged
mysql-test/r/ps_4heap.result:
  Auto merged
mysql-test/r/ps_5merge.result:
  Auto merged
mysql-test/r/ps_7ndb.result:
  Auto merged
mysys/Makefile.am:
  Auto merged
mysys/mf_iocache.c:
  Auto merged
mysys/mf_tempfile.c:
  Auto merged
mysys/my_error.c:
  Auto merged
mysys/my_init.c:
  Auto merged
mysys/my_open.c:
  Auto merged
sql/handler.cc:
  Auto merged
sql/handler.h:
  Auto merged
sql/lex.h:
  Auto merged
sql/sql_show.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
sql/table.cc:
  Auto merged
sql/table.h:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ha_maria.h:
  Auto merged
storage/maria/ma_bitmap.c:
  Auto merged
storage/maria/ma_blockrec.h:
  Auto merged
storage/maria/ma_create.c:
  Auto merged
storage/maria/ma_dynrec.c:
  Auto merged
storage/maria/ma_loghandler.h:
  Auto merged
storage/maria/ma_loghandler_lsn.h:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/ma_pagecache.h:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/ma_test2.c:
  Auto merged
storage/maria/ma_update.c:
  Auto merged
storage/maria/maria_chk.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Auto merged
storage/myisam/ft_stopwords.c:
  Auto merged
storage/myisam/mi_close.c:
  Auto merged
support-files/compiler_warnings.supp:
  Auto merged
mysys/mf_keycache.c:
  Pulled from mysql-maria tree, manual merge.
mysys/my_handler.c:
  Pulled from mysql-maria tree, manual merge.
mysys/my_symlink2.c:
  Pulled from mysql-maria tree, manual merge.
sql/mysqld.cc:
  Pulled from mysql-maria tree, manual merge.
sql/sql_table.cc:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_blockrec.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_check.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_close.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_loghandler.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/maria_def.h:
  Pulled from mysql-maria tree, manual merge.
2007-07-04 12:20:53 +03:00
unknown
ad3e38f88e Marking the block dirty requires linking it into the changed_blocks[]
list (for flush_pagecache*() functions and Checkpoint to see it)
2007-07-03 16:00:05 +02:00
unknown
979b32746d Merge gbichot@bk-internal.mysql.com:/home/bk/mysql-maria
into  gbichot3.local:/home/mysql_src/mysql-maria-clean


storage/maria/ma_pagecache.c:
  merge
2007-07-03 15:24:18 +02:00
unknown
388122558c Maria:
* Don't modify share->base.born_transactional; now it is a value carved
in stone at creation time. share->now_transactional is what can be
modified: it starts at born_transactional, can become false during
ALTER TABLE (when we want no logging), and restored later.
* Not resetting create_rename_lsn to 0 during delete_all or repair.
* when we temporarily disable transactionality, we also change
the page type to PAGECACHE_PLAIN_PAGE: it bypasses some work in the
page cache (optimization), and avoids assertions related to LSNs.
* Disable INSERT DELAYED for transactional tables, because
durability could not be guaranteed (insertion may even not happen)


mysys/mf_keycache.c:
  comment
storage/maria/ha_maria.cc:
  * a transactional table cannot do INSERT DELAYED
  * ha_maria::save_transactional not needed anymore, as now instead
  we don't modify MARIA_SHARE::MARIA_BASE_INFO::born_transactional
  (born_transactional plays the role of save_transactional), and modify
  MARIA_SHARE::now_transactional.
  * REPAIR_TABLE log record is now logged by maria_repair()
  * comment why we rely on born_transactional to know if we should
  skipping a transaction.
  * putting together two if()s which test for F_UNLCK
storage/maria/ha_maria.h:
  ha_maria::save_transactional not needed anymore (moved to the C layer)
storage/maria/ma_blockrec.c:
  * For the block record's code (writing/updating/deleting records),
  all that counts is now_transactional, not born_transactional.
  * As we now set the page type to PAGECACHE_PLAIN_PAGE for tables
  which have now_transactional==FALSE, pagecache will not expect
  a meaningful LSN for them in pagecache_unlock_by_link(), so
  we can pass it LSN_IMPOSSIBLE.
storage/maria/ma_check.c:
  * writing LOGREC_REPAIR_TABLE moves from ha_maria::repair()
  to maria_repair(), sounds cleaner (less functions to export).
  * when opening a table during REPAIR, don't use the realpath-ed name,
  as this may fail if the table has symlinked files (maria_open()
  would try to find the data and index file in the directory
  of unique_file_name, it would fail if data and index files are in
  different dirs); use the unresolved name, open_file_name, which is
  the argument which was passed to the maria_open() which created 'info'.
storage/maria/ma_close.c:
  assert that when a statement is done with a table, it cleans up
storage/maria/ma_create.c:
  new name
storage/maria/ma_delete_all.c:
  * using now_transactional
  * no reason to reset create_rename_lsn during delete_all (a bug);
  also no reason to do it during repair: it was put there because
  a positive create_rename_lsn caused a call to check_and_set_lsn()
  which asserted in DBUG_ASSERT(block->type == PAGECACHE_LSN_PAGE);
  first solution was to use LSN_IMPOSSIBLE in _ma_unpin_all_pages() if
  not transactional; but then in the case of ALTER TABLE, with
  transactionality temporarily disabled, it asserted in
  DBUG_ASSERT(LSN_VALID(lsn)) in pagecache_fwrite() (PAGECACHE_LSN_PAGE
  page with zero LSN - bad). The additional solution is to use
  PAGECACHE_PLAIN_PAGE when we disable transactionality temporarily: this
  avoids checks on the LSN, and also bypasses (optimization) the "flush
  log up to LSN" call when the pagecache flushes our page (in other
  words, no WAL needed).
storage/maria/ma_delete_table.c:
  use now_transactional
storage/maria/ma_locking.c:
  assert that when a statement is done with a table, it cleans up.
storage/maria/ma_loghandler.c:
  * now_transactional should be used to test if we want a log record.
  * Assertions to make sure dummy_transaction_object is not spoilt
  by its many users.
storage/maria/ma_open.c:
  base.transactional -> base.born_transactional
storage/maria/ma_pagecache.c:
  missing name for page's type. Comment for future.
storage/maria/ma_rename.c:
  use now_transactional
storage/maria/maria_chk.c:
  use born_transactional
storage/maria/maria_def.h:
  MARIA_BASE_INFO::transactional renamed to born_transactional.
  MARIA_SHARE::now_transactional introduced.
  _ma_repair_write_log_record() is made local to ma_check.c.
  Macros to temporarily disable, and re-enable, transactionality for a
  table.
storage/maria/maria_read_log.c:
  assertions and using the new macros. Adding a forgotten resetting
  when we finally close all tables.
2007-07-03 15:20:41 +02:00
unknown
bca70f9f6e Mark the page dirty if we store LSN on it.
New type of page in the page cache fixes.


storage/maria/ma_pagecache.c:
  Mark the page dirty if we store LSN on it.
  Symbolic representation of new page type added (for debugging output).
  Asserts added (unknown type can't be used if we write the page).
2007-07-03 01:19:26 +03:00
unknown
631ecaabea Merged with mysql-5.1 main tree.
BUILD/compile-pentium-debug-max:
  Added definition after macro was removed from main tree. This will
  be fixed back in main tree later.
2007-07-02 20:45:15 +03:00
unknown
e2904e77f1 Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


include/maria.h:
  Auto merged
sql/handler.h:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_close.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/ma_pagecache.h:
  Auto merged
storage/maria/maria_chk.c:
  Auto merged
storage/maria/ma_blockrec.c:
  Manual merge (No changes)
storage/maria/ma_check.c:
  Manual merge
storage/maria/ma_create.c:
  Manual merge
storage/maria/ma_delete_all.c:
  Manual merge
storage/maria/ma_init.c:
  Manual merge (no changes)
storage/maria/ma_test_all.sh:
  Manual merge
storage/maria/maria_def.h:
  Manual merge
2007-07-01 16:48:57 +03:00
unknown
d6f2fda680 Fixed REPAIR/CHECK/ANALYZE TABLE for tables with new BLOCK-ROW format.
Fixed maria_chk to repair BLOCK-ROW tables.
Added CREATE options ROW_FORMAT=PAGE & TRANSACTIONAL= 0|1
More DBUG information in a lot of functions
Some minor code cleanups
Enable handler errors earlier for better clear text error messages at handler startup / standalone usage.
Don't print NULL strings in my_create_with_symlink();  Fixes core dump when used with --debug


include/maria.h:
  Added extra variables needed for REPAIR with BLOCK records
include/my_base.h:
  Added argument for opening copy of maria table without a shared object
include/my_handler.h:
  Prototypes for my_handler_error_register() & my_handler_error_unregister()
include/pagecache.h:
  Added PAGECACHE_READ_UNKNOWN_PAGE
mysql-test/include/ps_conv.inc:
  Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/maria.result:
  Moved some things to maria-connect.test
  Updared results as REPAIR now works
  Added tests for creation option TRANSACTIONAL
mysql-test/r/ps_2myisam.result:
  Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_3innodb.result:
  Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_4heap.result:
  Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_5merge.result:
  Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_7ndb.result:
  Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/r/ps_maria.result:
  Enforce creation of table as MyISAM (to allow one to use --default-storage-engine)
mysql-test/t/maria.test:
  Moved some things to maria-connect.test
  Updared results as REPAIR now works
  Added tests for creation option TRANSACTIONAL
mysys/mf_iocache.c:
  More debugging
mysys/mf_tempfile.c:
  Added missing close()
mysys/my_error.c:
  init_glob_errs() is now done in my_init()
mysys/my_handler.c:
  Added functions to initialize handler error messages
mysys/my_init.c:
  Moevd init_glob_errs() here.
mysys/my_open.c:
  More comments
  More debugging
  Code cleanup (join multiple code paths) and indentation fixes. No change in logic.
mysys/my_symlink2.c:
  Don't print NULL strings
sql/handler.cc:
  Added printing of PAGE row type
  Moved out initializing of handler errors to allow handler to give better error messages at startup
sql/handler.h:
  ROW_TYPE_PAGES -> ROW_TYPE_PAGE
sql/lex.h:
  Added 'PAGE' and 'TRANSACTIONAL'
sql/mysqld.cc:
  Initialize handler error messages early to get better error messages from handler startup
sql/sql_show.cc:
  ROW_TYPE_PAGES -> ROW_TYPE_PAGE
sql/sql_table.cc:
  Removed not needed initializer
sql/sql_yacc.yy:
  Added CREATE options ROW_FORMAT=PAGE and TRANSACTIONAL=[0|1]
sql/table.cc:
  Store transactional flag in .frm
  More comments
sql-bench/example:
  Better example
sql/table.h:
  Added transactional table option
storage/maria/ha_maria.cc:
  More debug information
  Enable REPAIR
  Detect usage of TRANSACTIONAL table option
storage/maria/ma_bitmap.c:
  More comments (from Guilhem)
storage/maria/ma_blockrec.c:
  SANITY_CHECK -> SANITY_CHECKS (fixed typo)
  Write out pages on delete even if there is no rows. (Fixed problem with REPAIR)
  Removed some ASSERTS to runtime checks (for better REPAIR)
  Fixed bug when scanning rows
  More DBUG information
storage/maria/ma_check.c:
  Partial rewrite to allow REPAIR of BLOCK/PAGE format.
  Repair of BLOCK format rows is for now only done with 'maria_repair()' (= repair through key cache)
  The new logic to repair rows with BLOCK format is:
  
  - Create new, unrelated MARIA_HA of the table
  - Create new datafile and associate it with new handler
  - Reset all statistic information in new handler
  - Copy all data to new handler with normal write operations
  - Move state of new handler to old handler
  - Close new handler
  - Close data file in old handler
  - Rename old data file to new data file.
  - Reopen data file in old handler
storage/maria/ma_close.c:
  REmoved not needed block
storage/maria/ma_create.c:
  Swap arguments to _ma_initialize_data_file()
storage/maria/ma_delete_all.c:
  Split maria_delete_all_rows() to two functions to allow REPAIR to easily reset all status information.
storage/maria/ma_dynrec.c:
  Added checksum argument to _ma_rec_check (multi-thread fix)
storage/maria/ma_info.c:
  Indentation fix
storage/maria/ma_init.c:
  Register error message to get better error message on init and when using as standalone module.
storage/maria/ma_loghandler.c:
  Fixed typo that disabled some error detection by valgrind
storage/maria/ma_open.c:
  Added 'calc_check_checksum()'
  Don't log things during repair
  Added option HA_OPEN_COPY to allow one to open a Maria table with an independent share (required by REPAIR)
storage/maria/ma_pagecache.c:
  Fixed some compiler warnings
  Added support for PAGECACHE_READ_UNKNOWN_PAGE (used for scanning file without knowing page types)
storage/maria/ma_test_all.sh:
  More test of REPAIR
storage/maria/ma_update.c:
  Optimized checksum code
storage/maria/maria_chk.c:
  Use DBUG_SET_INITIAL() to get DBUG to work with --parallel-repair
  Ensure we always use maria_repair() for BLOCK format (for now)
  More DBUG information
storage/maria/maria_def.h:
  For now, always run with more checkings (SANITY_CHECKS)
  Added share->calc_check_checksum to be used with REPAIR / CHECK table.
  Swaped arguments to _ma_initialize_data_file()
storage/myisam/ft_stopwords.c:
  Added DBUG information
mysql-test/r/maria-connect.result:
  New BitKeeper file ``mysql-test/r/maria-connect.result''
mysql-test/t/maria-connect.test:
  New BitKeeper file ``mysql-test/t/maria-connect.test''
2007-07-01 16:20:57 +03:00
unknown
adac9798bf WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes


storage/maria/Makefile.am:
  new program maria_read_log
storage/maria/ha_maria.cc:
  create control file if missing
storage/maria/ma_blockrec.c:
  0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
  preparations for Checkpoint module
storage/maria/ma_close.c:
  comment
storage/maria/ma_control_file.c:
  renaming constants.
  Possibility to say "open control file but don't create it if it's
  missing" (used by maria_read_log which does not want to create
  anything)
storage/maria/ma_control_file.h:
  renaming constants
storage/maria/ma_create.c:
  I had duplicated "linkname" and "linkname_ptr", now I see it's not
  needed, reverting. Indeed those variables don't contain interesting
  information; fixing log record accordingly (the links are in
  ci->data/index_file_name). Storing keystart in log record is needed,
  to know at which size we must extend the file if we replay
  LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
  some structures need to be known to maria_read_log.c, taking
  them to ma_loghandler.h
storage/maria/ma_loghandler.h:
  we have page_store, adding page_korr.
  translog_lock() made public, because Checkpoint will need it (to
  write to control file).
  Some structures moved from ma_loghandler.c because maria_read_log.c
  needs them (needs to know the execute-in-REDO-phase hooks of each
  record).
storage/maria/ma_loghandler_lsn.h:
  constants defined in ma_control_file.h serve everywhere,
  and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
  Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
  renaming constants
storage/maria/ma_recovery.h:
  copyright
storage/maria/ma_test1.c:
  new prototype
storage/maria/ma_test2.c:
  new prototype
storage/maria/trnman_public.h:
  double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  constants renamed, new prototype
storage/myisam/mi_close.c:
  comment
storage/maria/maria_read_log.c:
  program to read and print log records from a Maria transaction log,
  and optionally apply them to tables. Very basic, early version.
  Should serve as a base for Recovery's code. Designed to be idempotent.
  Create a log by running maria.test, then cd to var/master-data
  and run "maria_read_log --only-display" to see info about records;
  run "maria_read_log --display-and-apply" to also apply the records
  to tables (it's more interesting if you first wipe out the
  tables in var/master-data/test, to see how they get re-created).
  Only a few records are handled by now: LONG_TRANSACTION_ID,
  COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
  REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
  "Monty" in the file). Note: changes to the index pages, index's header
  and bitmap pages are not properly logged yet, so don't expect
  the program to work with that.
2007-06-26 16:49:23 +02:00
unknown
1a96259191 - WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.


storage/maria/Makefile.am:
  more files to build
storage/maria/ha_maria.cc:
  - logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
  - ha_maria::data_file_type does not have to be set in every info()
  call, just do it once in open().
  - if caller said that transactionality can be disabled (like if
  caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
  temporarily disable transactionality of the table in external_lock();
  that will ensure that no REDOs/UNDOs are logged for this possibly
  massive write operation (they are not needed, as if any write fails,
  the table will be dropped). We re-enable in external_lock(F_UNLCK),
  which in ALTER TABLE happens before the tmp table replaces the original
  one (which is good, as thus the final table will have a REDO RENAME
  and a correct create_rename_lsn).
  - when we commit we also have to write a log record, so
  trnman_commit_trn() calls become ma_commit() calls
  - at end of engine's initialization, we are potentially entering a
  multi-threaded dangerous world (clients are going to be accepted)
  and so some assertions of mutex-owning become enforceable, for that
  we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
  new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
  - fixing comments according to discussion with Monty
  - if a table is transactional but temporarily non-transactional
  (like in ALTER TABLE), we need to give a sensible LSN to the pages
  (and, if we give 0, pagecache asserts).
  - translog_write_record() now takes care of storing the share's
  2-byte-id in the log record
storage/maria/ma_blockrec.h:
  fixing comment according to discussion with Monty
storage/maria/ma_check.c:
  When REPAIR/OPTIMIZE modify the data/index file, if this is a
  transactional table, they must sync it; if they remove files or rename
  files, they must sync the directory, so that everything is durable.
  This is just applying to REPAIR/OPTIMIZE the logic already implemented
  in CREATE/DROP/RENAME a few months ago.
  Adding a function to write a LOGREC_REPAIR_TABLE at end of
  REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
  to update the table's create_rename_lsn.
storage/maria/ma_close.c:
  fix for a future bug
storage/maria/ma_control_file.c:
  ensuring that if Maria is running in multi-threaded mode, anybody
  wanting to write to the control file and update
  last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
  see ma_control_file.c
storage/maria/ma_create.c:
  when creating a table:
  - sync it and its directory only if this is a transactional table
  and there is a log (no point in syncing in maria_chk)
  - decouple the two uses of linkname/linkname_ptr (for index file and
  for data file) into more variables, as we need to know all links
  until the moment we write the LOGREC_CREATE_TABLE.
  - set share.data_file_type early so that _ma_initialize_data_file()
  knows it (Monty's bugfix so that a table always has at least a bitmap
  page when it is created; so data-file is not 0 bytes anymore).
  - log a LOGREC_CREATE_TABLE; it contains the bytes which we have
  just written to the index file's header. Update table's
  create_rename_lsn.
  - syncing of kfile had been bugified in a previous merge, correcting
  - syncing of dfile is now needed as it's not empty anymore
  - in _ma_initialize_data_file(), use share's block_size and not the
  global one. This is a gratuitous change, both variables are equal,
  just that I find it more future-proof to use share-bound variable
  rather than global one.
storage/maria/ma_delete_all.c:
  log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
  update create_rename_lsn then.
storage/maria/ma_delete_table.c:
  - logging LOGREC_DROP_TABLE; knowing if this is needed, requires
  knowing if the table is transactional, which requires opening the
  table.
  - we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
  questions
storage/maria/ma_init.c:
  when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
  - translog_inited has to be visible to ma_create() (see how it is used
  in ma_create())
  - checkpoint record will be a single record, not three
  - no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
  log a REDO_CREATE)
  - adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
  truncating the files), REPAIR.
  - MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
  - in translog_write_record(), if MARIA_SHARE does not yet have a
  2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
  store this short id into log records.
  - in translog_write_record(), if transaction has not logged its
  long trid, log LOGREC_LONG_TRANSACTION_ID.
  - For Checkpoint, we need to know the current end-of-log: adding
  translog_get_horizon().
  - For Control File, adding an assertion that the thread owns the
  log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
  Changes in log records (see ma_loghandler.c).
  new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
  adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
  where the most significant byte is used for flags.
storage/maria/ma_open.c:
  storing the create_rename_lsn in the index file's header (in the
  state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
  - my set_if_bigger was wrong, correcting it
  - if the first_in_switch list is not empty, it means that
  changed_blocks misses some dirty pages, so Checkpoint cannot run and
  needs to wait. A variable missing_blocks_in_changed_list is added to
  tell that (should it be named missing_blocks_in_changed_blocks?)
  - pagecache_collect_changed_blocks_with_lsn() now also tells the
  minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
  see ma_pagecache.c
storage/maria/ma_panic.c:
  comment
storage/maria/ma_range.c:
  comment
storage/maria/ma_rename.c:
  - logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
  knowing if the table is transactional, which requires opening the
  table.
  - update create_rename_lsn
  - we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
  comment
storage/maria/ma_test_all.sh:
  - tip for Valgrind-ing ma_test_all
  - do "export maria_path=somepath" before calling ma_test_all,
  if you want to run ma_test_all out of storage/maria (useful
  to have parallel runs, like one normal and one Valgrind, they
  must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
  - state now contains, in memory and on disk, the create_rename_lsn
  - share now contains a 2-byte-id
storage/maria/trnman.c:
  preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
  minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
  using most significant byte of first_undo_lsn to hold miscellaneous
  flags, for now TRANSACTION_LOGGED_LONG_ID.
  dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
  dummy_transaction_object was declared in all files including
  trnman_public.h, while in fact it's a single object.
  new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
  update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  update for new prototype
storage/maria/ma_commit.c:
  function which wraps:
  - writing a LOGREC_COMMIT record (==commit on disk)
  - calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
  new header file
.tree-is-private:
  this file is now needed to keep our tree private (don't push it
  to public trees). When 5.1 is merged into mysql-maria, we can abandon
  our maria-specific post-commit trigger; .tree_is_private will take
  care of keeping commit mails private. Don't push this file to public
  trees.
2007-06-22 14:49:37 +02:00
unknown
704b39a158 - moving pagecache.h from include/ to storage/maria as it is Maria-
specific
- adding TRN::first_undo_lsn, needed to know when a log can be deleted;
this variable must be set under log's mutex and that leads to setting
TRN::rec_lsn, TRN::undo_lsn and TRN::first_undo_lsn in a
inwrite_rec_hook; adding implementation of one hook for REDOs and one
for UNDOs. Thus translog_write_record() always uses TRN and so does
not need a short_id argument, can find it from TRN.
- Monty's patch for the last Valgrind error in the tree.
- Log handler's unit tests fail but Sanja says it's known


include/Makefile.am:
  pagecache.h moved and renamed
include/maria.h:
  pagecache.h moved and renamed
sql/handler.h:
  pagecache.h moved and renamed
storage/maria/Makefile.am:
  pagecache.h moved and renamed
storage/maria/ha_maria.cc:
  adding an assertion which sounds logical
storage/maria/ma_blockrec.c:
  trn->rec_lsn and trn->undo_lsn are now set via hooks inside the log
  record's writing; this allows to also set trn->first_undo_lsn
  needed to compute the log's low-water mark.
  The PAGERANGE_STORE_SIZE -> PAGE_STORE_SIZE is Monty's fix to a
  Valgrind error.
storage/maria/ma_loghandler.c:
  "tcb" renamed to "trn". Log handler now knows what is a transaction,
  and finds short_id from trn. trn's rec_lsn, undo_lsn, first_undo_lsn
  are now set by some inwrite_rec_hookS (one for REDOs, one for UNDOs).
  The HAVE_purify blocks are Monty's fix to a Valgrind error.
storage/maria/ma_loghandler.h:
  Log handler functions use TRN, that needs a forward declaration
storage/maria/ma_pagecache.c:
  pagecache.h was moved and renamed
storage/maria/ma_pagecache.h:
  pagecache.h was moved and renamed
storage/maria/ma_pagecaches.c:
  pagecache.h was moved and renamed
storage/maria/trnman.c:
  initializing some members of TRN.
storage/maria/trnman.h:
  TRN::first_undo_lsn needed for log's low-water mark calculation
  (which will serve to know which logs can be deleted)
storage/maria/unittest/ma_test_loghandler-t.c:
  translog_write_record() now needs a valid TRN
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  translog_write_record() now needs a valid TRN
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  translog_write_record() now needs a valid TRN
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  translog_write_record() now needs a valid TRN
storage/maria/unittest/test_file.h:
  pagecache.h was moved and renamed
2007-06-11 16:29:53 +02:00
unknown
ecb816683a Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
2007-06-07 01:02:56 +03:00
unknown
42cde8a7b2 rec_lsn (first REDO LSN( is now given to the page cache on unpinning
Added maria_clone(), needed by future REPAIR code


storage/maria/unittest/ma_pagecache_consist.c:
  Change mode to -rw-rw-r--
storage/maria/unittest/lockman-t.c:
  Change mode to -rw-rw-r--
storage/maria/unittest/lockman1-t.c:
  Change mode to -rw-rw-r--
storage/maria/unittest/lockman2-t.c:
  Change mode to -rw-rw-r--
storage/maria/unittest/trnman-t.c:
  Change mode to -rw-rw-r--
include/maria.h:
  Added prototype for maria_clone (for future)
storage/maria/ha_maria.cc:
  Move filename to share structure
storage/maria/ma_blockrec.c:
  rec_lsn (first REDO LSN( is now given to the page cache on unpinning
  Removed impossible lock handling in get_head_or_tail_page()
  Changed calls ot translog_write_record() to remember rec_lsn
  Removed some logging in csse of not transactions
storage/maria/ma_delete.c:
  info->filename -> info->s->open_file_name
storage/maria/ma_loghandler.c:
  Indentation fixes
storage/maria/ma_open.c:
  Added maria_clone(), needed by future REPAIR code
storage/maria/ma_packrec.c:
  Fixed typo in comment
storage/maria/ma_pagecache.c:
  Added comment.
  Allow setting REC_LSN in case of read lock
storage/maria/ma_update.c:
  info->filename -> info->s->open_file_name
storage/maria/ma_write.c:
  info->filename -> info->s->open_file_name
storage/maria/maria_def.h:
  info->filename -> info->s->open_file_name
  Added have_rtree to simplify test in ma_clone()
storage/maria/maria_ftdump.c:
  info->filename -> info->s->open_file_name
storage/maria/maria_pack.c:
  info->filename -> info->s->open_file_name
storage/maria/trnman.h:
  Added rec_lsn
2007-06-07 01:01:43 +03:00
unknown
1a7ee974d4 Fixed pagecache unittest
storage/maria/ma_pagecache.c:
  - remove_reader() call is removed from unlock/unpin operations
    which uses direct link (because find_block() was not called)
  - patch which broke pagecache unittest is reverted
2007-06-06 09:51:30 +03:00
unknown
ef00dbaa7e postmerge changes 2007-06-05 12:14:05 +03:00
unknown
1d2b4e48f3 Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria
into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-pagecache


storage/maria/unittest/ma_pagecache_consist.c:
  Auto merged
include/pagecache.h:
  Merge
storage/maria/ma_pagecache.c:
  Merge
storage/maria/unittest/ma_pagecache_single.c:
  Merge
2007-06-05 10:57:04 +03:00
unknown
ffe437be41 Fixed that ma_test_all works with -T (simple transaction/logging support)
Some fixes from Sanja


BitKeeper/etc/ignore:
  added storage/maria/maria_log.*
include/pagecache.h:
  Always have enum PAGECACHE_EMPTY_PAGE available
  (Simpler code)
storage/maria/ma_bitmap.c:
  Reset 'debugging' bitmap when creating new one (fixes valgrind warning)
storage/maria/ma_blockrec.c:
  Removed duplicate (wrong) initialization
  Reset not initialized variable
storage/maria/ma_check.c:
  Use right page type
  (Patch from Sanja)
storage/maria/ma_init.c:
  Reset logging in maria_end()
  (Fixes memory leak)
storage/maria/ma_loghandler.c:
  Add missing copyright header
  Added checking of duplicate calls or calls without init to translog_destroy()
  Don't lock mutex before destroying them (not needed as you can't use a destroyed mutex anyway)
storage/maria/ma_pagecache.c:
  Added extra page type text
  Trivial indentation fixes
storage/maria/ma_test1.c:
  Added transaction setup
  (Patch from Sanja)
storage/maria/ma_test2.c:
  Added transaction setup
  (Patch from Sanja)
2007-06-04 14:07:18 +03:00
unknown
8f39541e7d This patch is a collection of patches from from Sanja, Sergei and Monty.
Added logging and pinning of pages to block format.
Integration of transaction manager, log handler.
Better page cache intergration
Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs.
Renaming of structures, more comments, more debugging etc.
Fixed problem with small head block + long varchar.
Added extra argument to delete_record() and update_record() (needed for UNDO logging)
Small changes to interface of pagecache and log handler.
Change initialization of log_record_type_descriptors to not be depending on enum order.
Use array of LEX_STRING's to send data to log handler
Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists.


include/lf.h:
  Interface fixes
  Rename of structures
  (Patch from Sergei via Sanja)
include/my_atomic.h:
  More comments
include/my_global.h:
  Added MY_ERRPTR
include/pagecache.h:
  Added undo LSN when unlocking pages
mysql-test/r/maria.result:
  Updated results
mysql-test/t/maria.test:
  Added autocommit around lock tables
  (Patch from Sanja)
mysys/lf_alloc-pin.c:
  Post-review fixes, simple optimizations
  More comments
  Struct slot renames
  Check amount of memory on stack
  (Patch from Sergei)
mysys/lf_dynarray.c:
  More comments
mysys/lf_hash.c:
  More comments
  After review fixes
  (Patch from Sergei)
storage/maria/ha_maria.cc:
  Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program.
  (Temporary fix to avoid bug in gcc)
  Move out all deferencing of the transaction structure.
  Transaction manager integrated (Patch from Sergei)
storage/maria/ha_maria.h:
  Added prototype for start_stmt()
storage/maria/lockman.c:
  Function call rename
storage/maria/ma_bitmap.c:
  Mark deleted pages free from page cache
storage/maria/ma_blockrec.c:
  Offset -> rownr
  More debugging
  Fixed problem with small head block + long varchar
  Added logging of changed pages
  Added logging of undo (Including only loggging of changed fields in case of update)
  Added pinning/unpinning of all changed pages
  More comments
  Added free_full_pages() as the same code was used in several places.
  fill_rows_parts() renamed as fill_insert_undo_parts()
  offset -> rownr
  Added some optimization of not transactional tables
  _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update
storage/maria/ma_blockrec.h:
  Added ROW_EXTENTS_ON_STACK
  Changed prototype for update and delete of row
storage/maria/ma_check.c:
  Added original row to delete_record() call
storage/maria/ma_control_file.h:
  Added ifdefs for C++
storage/maria/ma_delete.c:
  Added original row to delete_record() call
  (Needed for efficent undo logging)
storage/maria/ma_dynrec.c:
  Added extra argument to delete_record() and update_record()
  Removed not used variable
storage/maria/ma_init.c:
  Initialize log handler
storage/maria/ma_loghandler.c:
  Removed not used variable
  Change initialization of log_record_type_descriptors to not be depending on enum order
  Use array of LEX_STRING's to send data to log handler
storage/maria/ma_loghandler.h:
  New defines
  Use array of LEX_STRING's to send data to log handler
storage/maria/ma_open.c:
  Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists.
  Store in MARIA_SHARE->page_type if pages will have up to date LSN's
storage/maria/ma_pagecache.c:
  Don't decrease number of readers when using pagecache_write()/pagecache_read()
  In pagecache_write() decrement request count if page was left pinned
  Added pagecache_delete_pages()
  Removed some casts
  Make trace output consistent with rest of code
  Simplify calling of DBUG_ASSERT(0)
  Only update LSN if the LSN is bigger than what's already on the page
  Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock()
  (Part of patch from Sanja)
storage/maria/ma_static.c:
  Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists.
  Added default page cache
storage/maria/ma_statrec.c:
  Added extra argument to delete_record() and update_record()
storage/maria/ma_test1.c:
  Added option -T for transactions
storage/maria/ma_test2.c:
  Added option -T for transactions
storage/maria/ma_test_all.sh:
  Test with transactions
storage/maria/ma_update.c:
  Changed prototype for update of row
storage/maria/maria_def.h:
  Changed prototype for update & delete of row as block records need to access the old row
  Store in MARIA_SHARE->page_type if pages will have up to date LSN's
  Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need.
  Removed not used 'empty_bits_buffer'
  Added pointer to transaction object
  Added array for pinned pages
  Added log_row_parts array for logging of field data.
  Added MARIA_PINNED_PAGE to store pinned pages
storage/maria/trnman.c:
  Added accessor functions to transaction object
  Added missing DBUG_RETURN()
  More debugging
  More comments
  Changed // comment of code to #ifdef NOT_USED
  Transaction manager integrated.
  Post review fixes
  Part of patch originally from Sergei
storage/maria/trnman.h:
  Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program.
  (Temporary fix to avoid bug in gcc)
storage/maria/unittest/ma_pagecache_single.c:
  Added missing argument
  Added SKIP_BIG_TESTS
  (Patch from Sanja)
storage/maria/unittest/ma_test_loghandler-t.c:
  Test logging with new LEX_STRING parameter
  (Patch from Sanja)
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Test logging with new LEX_STRING parameter
  (Patch from Sanja)
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Test logging with new LEX_STRING parameter
  (Patch from Sanja)
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Test logging with new LEX_STRING parameter
  (Patch from Sanja)
storage/maria/unittest/trnman-t.c:
  Stack overflow detection
  (Patch from Sergei)
unittest/unit.pl:
  Command-line options --big and --verbose
  (Patch from Sergei)
unittest/mytap/tap.c:
  Detect --big
  (Patch from Sergei)
unittest/mytap/tap.h:
  Skip_big_tests and SKIP_BIG_TESTS
  (Patch from Sergei)
storage/maria/trnman_public.h:
  New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 20:13:56 +03:00
unknown
6686a3ee53 After merge fixes
Read blocks through page cache in check_block_record()
Don't read first bitmap on ma_open()
Don't require that a files block_size is equal to maria_block_size, if page cache is not setup yet.
Changed ma_test1, ma_test2, maria_chk and maria_pack to always create a page cache.
The above fixes so that ma_test_all now works again



BitKeeper/etc/ignore:
  added storage/maria/unittest/ma_pagecache_consist_1k-t-big storage/maria/unittest/ma_pagecache_consist_1kHC-t-big storage/maria/unittest/ma_pagecache_consist_1kRD-t-big storage/maria/unittest/ma_pagecache_consist_1kWR-t-big storage/maria/unittest/ma_pagecache_consist_64k-t-big storage/maria/unittest/ma_pagecache_consist_64kHC-t-big storage/maria/unittest/ma_pagecache_consist_64kRD-t-big storage/maria/unittest/ma_pagecache_consist_64kWR-t-big storage/maria/unittest/ma_pagecache_single_64k-t-big
include/maria.h:
  Added MARIA_MIN_PAGE_CACHE_SIZE
include/pagecache.h:
  Filedescriptors should be of type File
storage/maria/ma_bitmap.c:
  After merge fixes
  Create dummy bitmap on startup (can't read first bitmap becasue page cache may not be set up yet)
storage/maria/ma_blockrec.c:
  After merge fixes
storage/maria/ma_check.c:
  Use page cache to read rows-in-block rows.
  Don't initialize page cache; It's now done in maria_chk
storage/maria/ma_dynrec.c:
  Trivial code reorganization
storage/maria/ma_open.c:
  Don't give error for conflicting block size if page cache is not initalized.
  (Needed for maria_chk to be able to work on tables with different page sizes)
  After merge fixes
storage/maria/ma_page.c:
  Fix compiler warning
  Remove net needed asserts (Guranteed by ma_create())
storage/maria/ma_pagecache.c:
  Allow one to create a page cache with just one block
  (For trivail scan of table)
  Trivial code simplication
storage/maria/ma_test1.c:
  Always create a page cache (Maria now requires a page cache to work)
storage/maria/ma_test2.c:
  Always create a page cache (Maria now requires a page cache to work)
storage/maria/maria_chk.c:
  Remove command line options --maria_block_size and --pagecache_block_size.
  Set the global maria_block_size from the data file. This allows maria_chk to work with tables of different block sizes.
  Simply DESCRIPT handling; Allows us to remove one indentation level in maria_chk().
  Always initialize page cache if we are doing check/repair.
  (Most of the patch is reindentation of the code)
storage/maria/maria_def.h:
  After merge fix
storage/maria/maria_pack.c:
  Set maria_block_size based on the files block_size.
  Initalize page cache (needed for getting rows-in-blocks to works)
2007-04-19 18:48:36 +03:00
unknown
92e99ce424 Postmerge fixes.
added forgoten file.

The patch broke maria.test (will be fixed later)


sql/handler.cc:
  Pagecache block should be equal maria block.
sql/mysqld.cc:
  parameters Fixed.
storage/maria/ma_bitmap.c:
  fixed typo.
storage/maria/ma_blockrec.c:
  fixed typo.
storage/maria/ma_delete_all.c:
  fixed typo.
storage/maria/ma_page.c:
  fixed typo.
storage/maria/ma_pagecache.c:
  pin/lock debugging protection activated by default.
storage/maria/ma_pagecaches.c:
  parameters Fixed.
storage/maria/ma_preload.c:
  fixed typo.
mysys/my_safehash.c:
  New BitKeeper file ``mysys/my_safehash.c''
2007-04-18 12:55:09 +03:00
unknown
e10fe77b0a Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria.bak
into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-test


include/maria.h:
  Auto merged
mysys/Makefile.am:
  Auto merged
mysys/mf_keycaches.c:
  Auto merged
sql/handler.h:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/set_var.h:
  Auto merged
storage/maria/Makefile.am:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_bitmap.c:
  Auto merged
storage/maria/ma_check.c:
  Auto merged
storage/maria/ma_close.c:
  Auto merged
storage/maria/ma_dynrec.c:
  Auto merged
storage/maria/ma_extra.c:
  Auto merged
storage/maria/ma_info.c:
  Auto merged
storage/maria/ma_keycache.c:
  Auto merged
storage/maria/ma_locking.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_packrec.c:
  Auto merged
storage/maria/ma_page.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/ma_panic.c:
  Auto merged
storage/maria/ma_preload.c:
  Auto merged
storage/maria/ma_static.c:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/ma_test2.c:
  Auto merged
storage/maria/ma_test3.c:
  Auto merged
storage/maria/ma_write.c:
  Auto merged
storage/maria/maria_chk.c:
  Auto merged
storage/maria/maria_def.h:
  Auto merged
storage/maria/maria_ftdump.c:
  Auto merged
storage/maria/maria_pack.c:
  Auto merged
sql/handler.cc:
  merge
storage/maria/ma_blockrec.c:
  merge
storage/maria/ma_delete_all.c:
  merge
2007-04-16 13:19:43 +03:00
unknown
39d64a1d25 Pagecache integration for review.
storage/maria/unittest/ma_pagecache_single.c:
  Rename: storage/maria/unittest/mf_pagecache_single.c -> storage/maria/unittest/ma_pagecache_single.c
include/maria.h:
  Pagecache integration.
include/myisamchk.h:
  Pagecache integration.
include/pagecache.h:
  removed WRITE_NOW mode
  Pagecache parameters management.
mysys/Makefile.am:
  Safe hash procedures moved to the separate file.
  Pagecache moved to maria engine directory.
mysys/mf_keycaches.c:
  Safe hash procedures moved to the separate file.
sql/handler.cc:
  Pageccahe integration.
sql/handler.h:
  Pagecache integration.
sql/mysql_priv.h:
  pagecache integration
sql/mysqld.cc:
  pagecache integration
sql/set_var.cc:
  Pagecache integration.
sql/set_var.h:
  Pagecache integration.
storage/maria/Makefile.am:
  Pagecache integration and moving to maria engine directory.
storage/maria/ha_maria.cc:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_bitmap.c:
  Pagecache integration.
storage/maria/ma_blockrec.c:
  Pagecache integration.
storage/maria/ma_check.c:
  File changed on PAGECCAHE_FILE.
  Pagecache integration.
storage/maria/ma_close.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_delete_all.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_dynrec.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_extra.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_info.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_keycache.c:
  Pagecache integration.
storage/maria/ma_locking.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_loghandler.c:
  Assert added.
storage/maria/ma_loghandler.h:
  extern specifier added.
storage/maria/ma_open.c:
  Pagecache integration.
  File changed on PAGECCAHE_FILE.
storage/maria/ma_packrec.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_page.c:
  Pagecache integration.
storage/maria/ma_pagecache.c:
  Pagecache renamed and moved to the maria directory.
  BLOCK_* defines renamed to avoid conflict with BLOCK_ERROR defined in maria_def.h
storage/maria/ma_panic.c:
  File changed on PAGECCAHE_FILE.
storage/maria/ma_preload.c:
  Pagecache integration.
  File changed on PAGECCAHE_FILE.
storage/maria/ma_static.c:
  Pagecache integration.
storage/maria/ma_test1.c:
  Pagecache integration.
storage/maria/ma_test2.c:
  Pagecache integration.
storage/maria/ma_test3.c:
  Pagecache integration.
storage/maria/ma_write.c:
  File changed on PAGECCAHE_FILE.
storage/maria/maria_chk.c:
  Pagecache integration.
  File changed on PAGECCAHE_FILE.
storage/maria/maria_def.h:
  Pagecache integration.
  File changed on PAGECCAHE_FILE.
storage/maria/maria_ftdump.c:
  Pagecache integration.
storage/maria/maria_pack.c:
  File changed on PAGECCAHE_FILE.
storage/maria/unittest/Makefile.am:
  Pagecache moved to the maria directory.
storage/maria/unittest/ma_pagecache_consist.c:
  fixed using uninitialized variable
storage/maria/ma_pagecaches.c:
  New BitKeeper file ``storage/maria/ma_pagecaches.c''
mysys/my_safehash.h:
  New BitKeeper file ``mysys/my_safehash.h''
2007-04-04 23:37:09 +03:00
Renamed from mysys/mf_pagecache.c (Browse further)