Commit graph

63 commits

Author SHA1 Message Date
iangilfillan
f0ec34002a Correct FSF address 2017-03-10 18:21:29 +01:00
Monty
05ed9fec03 Use MEM_CHECK_DEFINED to check if blocks contain uninitialized data
Fixed failure in tests when running optimized code
- Some assert() was using code that had to be executed
Fixed copying of some uninitialized data (fixed valgrind warning)
2015-11-10 13:46:57 +02:00
Michael Widenius
f0f2072d1f Fixed problem with very slow shutdown when using 100,000 MyISAM tables with delay_key_write
Reason for the problem was that the hash of changed files in the key cache was too small (was 128). Fixed by making the hash size larger and changeable.

- Introduced key-cache-file-hash-size (default 512) for MyISAM and aria_pagecache_file_hash_size (default 512) for Aria.
- Added new status variable "Feature_delay_key_write" which counts number of tables opened that are using delay_key_write


mysql-test/r/features.result:
  Added test of Feature_delay_key_write
mysql-test/r/key_cache.result:
  Updated tests as the number of blocks has changed
mysql-test/r/mysqld--help.result:
  Updated result
mysql-test/suite/maria/maria3.result:
  Updated result
mysql-test/suite/sys_vars/r/key_cache_file_hash_size_basic.result:
  Test new variable
mysql-test/suite/sys_vars/t/aria_pagecache_file_hash_size_basic.test:
  Test new variable
mysql-test/suite/sys_vars/t/key_cache_file_hash_size_basic.test:
  Test new variable
mysql-test/t/features.test:
  Added test of Feature_delay_key_write
mysql-test/t/key_cache.test:
  Updated tests as the number of blocks has changed
mysys/mf_keycache.c:
  Made CHANGED_BLOCKS_HASH dynamic
sql/handler.cc:
  Updated call to init_key_cache()
sql/mysqld.cc:
  Added "Feature_delay_key_write"
  Added support for key-cache-file-hash-size
sql/mysqld.h:
  Added support for key-cache-file-hash-size
sql/sql_class.h:
  Added feature_files_opened_with_delayed_keys
sql/sys_vars.cc:
  Added key_cache_file_hash_size
storage/maria/ha_maria.cc:
  Added pagecache_file_hash_size
  Added counting of files with delay_key_write
storage/maria/ma_checkpoint.c:
  Fixed compiler warning
storage/maria/ma_pagecache.c:
  Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_pagecache.h:
  Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_rt_test.c:
  Updated parameters for init_pagecache()
storage/maria/ma_test1.c:
  Updated parameters for init_pagecache()
storage/maria/ma_test2.c:
  Updated parameters for init_pagecache()
storage/maria/ma_test3.c:
  Updated parameters for init_pagecache()
storage/maria/maria_chk.c:
  Updated parameters for init_pagecache()
storage/maria/maria_ftdump.c:
  Updated parameters for init_pagecache()
storage/maria/maria_pack.c:
  Updated parameters for init_pagecache()
storage/maria/maria_read_log.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_consist.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist2.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_single.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Updated parameters for init_pagecache()
storage/myisam/ha_myisam.cc:
  Added counting of files with delay_key_write
storage/myisam/mi_check.c:
  Updated call to init_key_cache()
storage/myisam/mi_test1.c:
  Updated call to init_key_cache()
storage/myisam/mi_test2.c:
  Updated call to init_key_cache()
storage/myisam/mi_test3.c:
  Updated call to init_key_cache()
storage/myisam/mi_test_all.sh:
  Fixed broken test
storage/myisam/myisam_ftdump.c:
  Updated call to init_key_cache()
storage/myisam/myisamchk.c:
  Updated call to init_key_cache()
storage/myisam/myisamlog.c:
  Updated call to init_key_cache()
2014-07-19 17:46:08 +03:00
Sergey Vojtovich
d12c7adf71 MDEV-5314 - Compiling fails on OSX using clang
This is port of fix for MySQL BUG#17647863.

revno: 5572
revision-id: jon.hauglid@oracle.com-20131030232243-b0pw98oy72uka2sj
committer: Jon Olav Hauglid <jon.hauglid@oracle.com>
timestamp: Thu 2013-10-31 00:22:43 +0100
message:
  Bug#17647863: MYSQL DOES NOT COMPILE ON OSX 10.9 GM

  Rename test() macro to MY_TEST() to avoid conflict with libc++.
2014-02-19 14:05:15 +04:00
Sergei Golubchik
b7b5f6f1ab 10.0-monty merge
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
2013-07-21 16:39:19 +02:00
Michael Widenius
668306640e Fixed MDEV-4291: Assertion `trid >= info->s->state.create_trid' failure or data corruption (key points to record
outside datafile) on INSERT into an Aria table.

The isssue was that the check if a table was moved between systems didn't take into account that create_trid could be bigger than the current max trid on the new system.
This could only happen if one tried to move a table that one had just done a 'REPAIR TABLE' on.
Tables that one had run 'aria_chk --zerofill' on worked.

Fixed this by assuming that if create_trid is too big then the table has been moved from one system to another and we have to do an automatic zerofill.

Other fixed:

- Added a check to detect a wrong create_trid in 'check table'.
- aria_chk -dvv will now write out also the create_trid (to make future error finding easier)
- aria_chk --zerofill doesn't anymore require a aria_control_file
- Removed some warnings from safemalloc when using aria_chk, ma_test1 and ma_test2.


include/myisamchk.h:
  Removed wrong 'QQ' flags (the flags are used by myisamchk and aria_chk)
storage/maria/ha_maria.cc:
  maria_chk_status() can now also return an error.
storage/maria/ma_check.c:
  In maria_chk_status() check if create_trid value is too big.
storage/maria/ma_open.c:
  Changed check if table is moved so that we can detect wrong create_trid values.
  Don't set STATE_NOT_MOVABLE flag if we are doing repair/check. This was done so that aria_chk can print out the movable flag.
storage/maria/ma_test1.c:
  Added code to suppress memory leaks from safemalloc
storage/maria/ma_test2.c:
  Added code to suppress memory leaks from safemalloc
storage/maria/maria_chk.c:
  Added code to suppress memory leaks from safemalloc.
  Make help text a bit better for --HELP and --zerofill.
  Incresed version number.
  Don't require a control file if we are only doing --zerofill
  Print out 'create_trid' when doing --describe --verbose
storage/maria/unittest/ma_test_recovery.expected:
  Updated result file
2013-05-13 00:43:46 +03:00
Michael Widenius
068c61978e Temporary commit of 10.0-merge 2013-03-26 00:03:13 +02:00
Sergei Golubchik
76f0b94bb0 merge with 5.3
sql/sql_insert.cc:
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
  ******
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
  small cleanup
  ******
  small cleanup
2011-10-19 21:45:18 +02:00
Sergei Golubchik
b4a0b2c2f8 post-merge fixes.
most tests pass.
5.3 merge is next
2011-07-02 22:12:12 +02:00
Sergei Golubchik
0accbd0364 lots of post-merge changes 2011-04-25 17:22:25 +02:00
Michael Widenius
139a2b64bf Merge with 5.2 2011-03-09 15:47:59 +02:00
Michael Widenius
07b5d554ee - maria/ma_test_all.sh can now be run with --tmpdir=/dev/shm for faster testing
- Fixed mysql-test-run failures on window
- Fixed compiler warnings from my last push (sorry about that)
- Fixed that maria_chk --repair --extended works again
- Fixed compiler warnings about using not unitialized data

mysql-test/mysql-test-run.pl:
  Better output
mysql-test/suite/parts/inc/partition_check_drop.inc:
  Use remove_files_wildcard instead of rm
mysys/safemalloc.c:
  Fixed argument to printf
storage/maria/ma_cache.c:
  Don't give errors when running maria_chk
storage/maria/ma_dynrec.c:
  Don't give errors when running maria_chk
storage/maria/ma_rt_test.c:
  Added option --datadir for where to put logs and test data
storage/maria/ma_test1.c:
  Added option --datadir for where to put logs and test data
storage/maria/ma_test2.c:
  Added option --datadir for where to put logs and test data
storage/maria/maria_chk.c:
  If --datadir is used but --logdir is not, set --logdir from --datadir
  (this reflects how --help said how things should work)
storage/maria/maria_read_log.c:
  Changed short option for 'maria-log-dir-path' from -l to -h to be same as mysqld, maria_chk, ma_test1 etc..
storage/maria/unittest/ma_test_all-t:
  Allow one to specify --tmpdir for where to store logs and data
storage/xtradb/buf/buf0buf.c:
  Fixed compiler warnings about using not unitialized data
storage/xtradb/row/row0upd.c:
  Fixed compiler warnings about using not unitialized data
storage/xtradb/srv/srv0srv.c:
  Fixed compiler warnings about using not unitialized data
2011-02-25 14:55:40 +02:00
Sergei Golubchik
f5ea301acc Added support for NO_RECORD record format (don't store any row data) for Aria.
This makes the keys smaller (no row pointer) and gives us proper errors if we
use the table wrongly.

sql/sql_select.cc:
  Use NO_RECORD for tables that doesn't need row data.
storage/maria/Makefile.am:
  Added ma_norec.c
storage/maria/ma_check.c:
  Added support for NO_RECORD record format (don't store any row data)
storage/maria/ma_norec.c:
  Added support for NO_RECORD record format
storage/maria/ma_open.c:
  Added support for NO_RECORD record format
storage/maria/ma_search.c:
  Added support for 0 size row pointers (used with NO_RECORD)
storage/maria/ma_test1.c:
  Added testing of NO_RECORD record format.
storage/maria/maria_chk.c:
  Added support for NO_RECORD
storage/maria/maria_def.h:
  Added support for NO_RECORD
storage/maria/unittest/ma_test_all-t:
  Added testing of NO_RECORD record format
2011-01-14 11:54:39 +01:00
Michael Widenius
32f81bab7d WL#3262 add mutex lock order checking to safemutex (also called safe_mutex_deadlock_detector)
This writes a warning on stderr if one uses mutex in different order,
like if one in one case would lock mutex in the order A,B and in another case
would lock mutex in the order B,A
This is inspired by and loosely based on the LOCKDEP patch by Jonas
Wrong mutex order is either fixed or mutex are marked with MYF_NO_DEADLOCK_DETECTION
if used inconsistently (need to be fixed by server team)

KNOWN_BUGS.txt:
  Added information that one need to dump and restore Maria tables
include/hash.h:
  Added prototype function for walking over all elements in a hash
include/my_pthread.h:
  Added my_pthread_mutex_init() and my_pthread_mutex_lock(); These should be used if one wants to disable mutex order checking.
  Changed names of the nonposix mutex_init functions to not conflict with my_phread_mutex_init()
  Added and extended structures for mutex deadlock detection.
  New arguments to sage_mutex_init() and safe_mutex_lock() to allow one to disable mutex order checking.
  Added variable 'safe_mutex_deadlock_detector' to enable/disable deadlock detection for all pthread_mutex_init()
mysys/Makefile.am:
  Added cleaning of test files
  Added test_thr_mutex
mysys/hash.c:
  Added hash_iterate() to iterate over all elements in a hash
  More comments
mysys/my_init.c:
  Added calls to destory all mutex uses by mysys()
  Added waiting for threads to end before calling TERMINATE() to list not freed memory
mysys/my_pthread.c:
  Changed names to free my_pthread_mutex_init() for mutex-lock-order-checking
mysys/my_sleep.c:
  Fixed too long wait if using 1000000L as argument
mysys/my_thr_init.c:
  Mark THR_LOCK_threads and THR_LOCK_malloc to not have mutex deadlock detection.
  (We can't have it enabled for this as these are internal mutex used by the detector  
  Call my_thread_init() early as we need thread specific variables enabled for the following pthread_mutex_init()
  Move code to wait for threads to end to my_wait_for_other_threads_to_die()
  Don't destroy mutex and conditions unless all threads have died
  Added my_thread_destroy_mutex() to destroy all mutex used by the mysys thread system
  Name the thread specific mutex as "mysys_var->mutex"
  Added my_thread_var_mutex_in_use() to return pointer to mutex in use or 0 if thread variables are not initialized
mysys/mysys_priv.h:
  Added prototypes for functions used internally with mutex-wrong-usage detection
mysys/thr_mutex.c:
  Added runtime detection of mutex used in conflicting order
  See WL#3262 or test_thr_mutex.c for examples
  The base idea is for each mutex have two hashes:
  - mutex->locked_mutex points to all mutex used after this one
  - mutex->used_mutex points to all mutex which has this mutex in it's mutex->locked_mutex
  There is a wrong mutex order if any mutex currently locked before this mutex is in the mutex->locked_mutex hash
sql/event_queue.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/event_scheduler.cc:
  Declare the right order to take the mutex
sql/events.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/ha_ndbcluster_binlog.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/log.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/mysqld.cc:
  Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
  This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
  Added call to free sp cache.
  Add destruction of LOCK_server_started and COND_server_started.
  Added register_mutex_order() function to register in which order mutex should be taken
  (to initiailize mutex_deadlock_detector).
  Added option to turn off safe_mutex_deadlock_detector
sql/protocol.cc:
  Fixed wrong argument to DBUG_PRINT (found by valgrind)
sql/rpl_mi.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/set_var.cc:
  Remove wrong locking of LOCK_global_system_variables when reading and setting log variables
  (would cause inconsistent mutex order).
  Update global variables outside of logger.unlock() as LOCK_global_system_variables has to be taken before logger locks
  Reviewed by gluh
sql/sp_cache.cc:
  Added function to destroy mutex used by sp cache
sql/sp_cache.h:
  Added function to destroy mutex used by sp cache
sql/sql_class.cc:
  Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
  This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
  Register order in which LOCK_delete and mysys_var->mutex is taken
sql/sql_insert.cc:
  Give a name for Delayed_insert::mutex
  Mark mutex used inconsistently (need to be fixed by server team)
  Move closing of tables outside of di->mutex (to avoid wrong mutex order)
sql/sql_show.cc:
  Don't keep LOCK_global_system_variables locked over value->show_type() as this leads to wrong mutex order
storage/innobase/handler/ha_innodb.cc:
  Disable safe_muted_deadlock_detector for innobase intern mutex (to speed up page cache initialization)
storage/maria/ha_maria.cc:
  Added flag to ha_maria::info() to signal if we need to lock table share or not.
  This is needed to avoid locking mutex in wrong order
storage/maria/ha_maria.h:
  Added flag to ha_maria::info() to signal if we need to lock table share or not.
storage/maria/ma_close.c:
  Destroy key_del_lock
  Simplify freeing ftparser_param
storage/maria/ma_key.c:
  Better comment
storage/maria/ma_loghandler.c:
  Mark mutex used inconsistently (need to be fixed by sanja)
storage/maria/ma_state.c:
  More comments
storage/maria/ma_test1.c:
  Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
storage/maria/ma_test2.c:
  Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
2008-12-03 00:02:52 +02:00
Guilhem Bichot
0432bc1e95 Fix for unit test failures.
storage/maria/ma_test1.c:
  rec_length was too short by one byte: at line 299 memcpy(record,read_record,rec_length) thus didn't copy the
  last byte, so 'record' was not ok for use by update_record(), contained an incomplete blob address, sometimes
  crashed on Itanium.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Tests failed once, never before for hundreds of times, we don't know if test is correct, disabling.
2008-06-30 11:13:08 +02:00
Michael Widenius
52cb0c24a6 Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid

Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it

.bzrignore:
  Added missing temporary directory
extra/Makefile.am:
  comp_err is now deleted on make distclean
include/maria.h:
  Added structure MARIA_KEY, which is used for intern key objects in Maria.
  Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
  Changed some functions that always return true or false to my_bool.
  Added virtual function make_key() to avoid if in _ma_make_key()
  Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
  Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
  Added SEARCH_INSERT to be used when inserting new keys
  Added SEARCH_PART_KEY for partial searches
  Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
  Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
  Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
  Read and restore current database; Don't assume we are using mysqltest.
  Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
  Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
  New tests
mysql-test/r/maria-purge.result:
  New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
  New tests
mysql-test/r/variables-big.result:
  Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
  Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
  Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
  Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
  Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
  New tests
mysql-test/t/maria-purge.test:
  Remove logs to make test results predictable
mysql-test/t/maria.test:
  New tests for some possible problems
mysql-test/t/variables-big.test:
  Don't log id as it's not predictable
mysys/my_handler.c:
  Updated function comment to reflect old code
  Changed nextflag to be uint32 to ensure we can have flags > 16 bit
  Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
  Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
  Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
  Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
  Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
  Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
  Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
  Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
  Use 'file->start_state' as default state for transactional tables without versioning
  At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
  In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
  Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
  Updated comment
storage/maria/ma_check.c:
  Compact key pages (removes transid) when doing --zerofill
  Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
  Removed not needed argument from get_record_for_key()
  Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
  Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
  Use info->lastkey_buff instead of info->lastkey
  _ma_dpos() -> _ma_row_pos_from_key()
  _ma_make_key() -> keyinfo->make_key()
  _ma_print_key() -> _ma_print_keydata()
  _ma_move_key() -> ma_copy_copy()
  Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
  Ensure that data on page doesn't overwrite page checksum position
  Use DBUG_DUMP_KEY instead of DBUG_DUMP
  Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
  Fixed check if rowid points outside of BLOCK_RECORD data file
  Use info->lastkey_buff instead of key on stack in some safe places
  Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
  Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
  Removed assert that is already checked in maria_init()
  Force transactinal tables to be of type BLOCK_RECORD
  Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
  Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
  Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
  Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  info->lastkey2-> info->lastkey_buff2
  Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
  Use new interface for get_key(), _ma_get_last_key() and others
  _ma_dpos() -> ma_row_pos_from_key()
  Simplify setting of prev_key in del()
  Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
  Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
  Reset history state if maria_delete_all_rows() are called
  Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
  Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
  Check if blocksize is legal
  (Moved test here from ma_open())
storage/maria/ma_key.c:
  Added functions for storing/reading of transid 
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
  Add transid to keys if table is versioned
  Added _ma_copy_key()
storage/maria/ma_key_recover.c:
  Add logging of page_flag (holds information if there are keys with transid on page)
  Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
  Added lots of DBUG_PRINT()
  Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
  Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
  Fixed all logging of LSN to look similar in DBUG log
  Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
  Added new index page operations
storage/maria/ma_open.c:
  Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
  This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
  Changed info->lastkey to MARIA_KEY type
  Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
  Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
  Check on open that state.create_trid is correct
  Extend share->base.max_key_length in case of transactional table so that it can hold transid
  Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
  Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
  Initialize all new virtual function pointers
  Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
  Added comment to describe key page structure
  Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
  Added handling of pages with transid
storage/maria/ma_recovery.c:
  Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
  Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
  Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Don't use USE_WHOLE_KEY, use real length of key
  Use share->row_is_visible() to test if row is visible
  Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
  Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
  Updated comment
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use better variable names
  Removed not needed casts
  _ma_dpos() -> _ma_row_pos_from_key()
  Use info->last_rtree_keypos to save position to key instead of info->int_keypos
  Simplify err: condition
  Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
  Changed type of 'nextflag' to uint32
  Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
  Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
  Fixed wrong comment
  Reset recinfo to avoid valgrind varnings
  Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Added support of keys with optional trid
  Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
  _ma_dpos() -> _ma_row_pos_from_key()
  If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
  Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
  Combine similar code at end of _ma_get_binary_pack_key()
  Removed not used function _ma_move_key()
  In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
  Update info->cur_row.trid with trid for found key
  
  
  
  Removed some not needed casts
  Added _ma_trid_from_key()
  Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
  _ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
  _ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
  Don't test sizeof(double), test against 8 as we are using float8store()
  Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
  maria_versioning() now only calls _ma_block_get_status() if table supports versioning
  Added _ma_row_visible_xxx() functions for different occasions
  When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
  Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
  Indentation changes
storage/maria/ma_statrec.c:
  Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
  Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
  Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
  Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  In _ma_enlarge_root(), mark in page_flag if new key has transid
  _ma_dpos() -> _ma_row_pos_from_key()
  Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
  Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
  Added error if trying to sort with HA_BINARY_PACK_KEY
  Use new interface to get_key() and _ma_dpointer()
  _ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
  Added create_trid to MARIA_SHARE
  Removed not used state variables 'process' and 'unique'
  Added defines for handling TRID's in index pages
  Changed to use MARIA_SHARE instead of MARIA_HA for some functions
  Added 'have_versioning' flag if table supports versioning
  Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
  Changed last_key to be of type MARIA_KEY. Removed lastkey_length
  lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
  Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
  Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
  Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
  lastkey -> lastkey_buff
storage/maria/trnman.c:
  Fixed wrong initialization of min_read_from and max_commit_trid
  Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
  Added --start-from
storage/myisam/mi_check.c:
  Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
  Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
  Updated comment
storage/myisam/mi_write.c:
  Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
  Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 08:18:28 +03:00
unknown
f83bd712ae Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


mysql-test/r/maria.result:
  Auto merged
mysql-test/suite/ndb/r/ndb_auto_increment.result:
  Auto merged
mysql-test/t/maria.test:
  Auto merged
mysys/hash.c:
  Auto merged
mysys/thr_lock.c:
  Auto merged
sql/field.cc:
  Auto merged
sql/ha_ndbcluster.cc:
  Auto merged
sql/ha_ndbcluster.h:
  Auto merged
sql/ha_partition.cc:
  Auto merged
sql/ha_partition.h:
  Auto merged
sql/handler.cc:
  Auto merged
sql/handler.h:
  Auto merged
sql/log_event.cc:
  Auto merged
sql/log_event_old.cc:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/protocol.cc:
  Auto merged
sql/sql_load.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
storage/csv/ha_tina.cc:
  Auto merged
storage/federated/ha_federated.cc:
  Auto merged
storage/maria/Makefile.am:
  Auto merged
storage/maria/ma_check.c:
  Auto merged
storage/maria/ma_control_file.c:
  Auto merged
storage/maria/ma_delete_all.c:
  Auto merged
storage/maria/ma_dynrec.c:
  Auto merged
storage/maria/ma_init.c:
  Auto merged
storage/maria/ma_key_recover.c:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_page.c:
  Auto merged
storage/maria/ma_range.c:
  Auto merged
storage/maria/ma_recovery.c:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/maria_read_log.c:
  Auto merged
storage/maria/unittest/ma_test_all-t:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Auto merged
storage/maria/unittest/ma_test_recovery.pl:
  Auto merged
storage/myisam/ha_myisam.cc:
  Auto merged
storage/myisam/myisamdef.h:
  Auto merged
include/my_base.h:
  Manual merge where error code are kept same as in 5.1
mysys/my_handler.c:
  No changes
sql/item.cc:
  Manual merge
sql/sql_class.cc:
  Manual merge
sql/sql_insert.cc:
  Manual merge
storage/maria/ha_maria.cc:
  Manual merge
storage/maria/ma_blockrec.c:
  Manual merge
storage/maria/ma_delete.c:
  Manual merge
storage/maria/ma_write.c:
  Manual merge
2008-05-29 21:39:25 +03:00
unknown
c9a825810d WL#3072 - Maria Recovery
Recovery of R-tree and fulltext indices.
Fix for BUG#35551 "Maria: crash in REPAIR TABLE/ENABLE KEYS if using
repair-with-keycache method".
Fix for bug (see ma_rt_index.c) where we could have a wrong
page_link pointer causing wrong memory access during some R-tree
index insert/delete.
Making ma_rt_test work again (it had been neglected over time) and
adding options (record type etc) to prepare it for integration into
ma_test_all-t (but there is BUG#36321 about "ma_rt_test -M" crash)


mysql-test/r/maria.result:
  correct result
mysql-test/t/maria.test:
  now we get no error
storage/maria/ma_blockrec.c:
  delete_dir_entry() and delete_head_or_tail() don't use info->keyread_buff.
  ma_get_length() does not change **packet, marking it with 'const' to
  remove some casts in callers of this function. The
  (const uchar**)&header casts will be removed when Monty changes 'header'
  to const uchar*.
  _ma_apply_redo_purge_row_head_or_tail() sets 'buff' from pagecache_read()
  so its initialization was superfluous.
storage/maria/ma_check.c:
  Fix for BUG#35551 "Maria: crash in REPAIR TABLE/ENABLE KEYS if using repair-with-keycache method"
  (see comment in code)
storage/maria/ma_create.c:
  FULLTEXT and SPATIAL indices have logging now, they are recoverable.
storage/maria/ma_delete.c:
  Logging done by _ma_ck_delete() is moved to a function
  (_ma_write_undo_key_delete()), for reusal by R-tree logging.
  _ma_log_delete() is made non-static for same
  reason, and some of its parameters are made pointers to const.
  Removed wrong comment ("Note that for delete key" etc, contradicted by
  code and comment "Log also position to row" a few lines above)
storage/maria/ma_ft_update.c:
  unneeded cast, comment for future
storage/maria/ma_key_recover.c:
  Comment about possible deadlock.
  Write bad page to DBUG trace if KEY_OP_CHECK founds bad CRC.
  Support operation KEY_OP_MULTI_COPY.
  When we execute, in UNDO phase, UNDO_KEY_DELETE|INSERT, we must call
  the proper key insertion|deletion depending on if this is R-tree
  or B-tree.
  Explanation of of _ma_[un]lock_key_del() work, maybe useful for
  mortals like me.
storage/maria/ma_key_recover.h:
  change of prototypes
storage/maria/ma_loghandler.h:
  New operation which can be stored in REDO_INDEX log records: KEY_OP_MULTI_COPY
storage/maria/ma_page.c:
  Comments
storage/maria/ma_pagecache.c:
  typo
storage/maria/ma_rt_index.c:
  Fix for bug: the page_link pointer in maria_rtree_insert_req()
  could be wrong when we set its 'changed' member; for the solution
  see ma_key_recover.h. It is needed only in cases when we manipulate
  several pages.
  Logging of changes done to pages by key insert/delete.
  maria_rtree_delete()'s main work is moved to a new function
  maria_rtree_real_delete(), which is used by maria_rtree_delete()
  and by applying of UNDO_KEY_INSERT.
storage/maria/ma_rt_index.h:
  new prototypes and macros for ma_rt_index.c
storage/maria/ma_rt_key.c:
  Logging of maria_rtree_add_key() and maria_rtree_delete_key().
  When inserting, split is necessary if there is not enough room for key:
  take checksum's occupied space in this calculation.
storage/maria/ma_rt_key.h:
  new prototypes (those functions need to know the page's id
  because they do logging)
storage/maria/ma_rt_mbr.c:
  Comments about what the functions change.
storage/maria/ma_rt_split.c:
  maria_rtree_split_page() needs to know the page's id, because
  it does logging.
  Logging of what a split operation does to the split page (see
  comment of _ma_log_rt_split(): moves of keys inside the page,
  sometimes insertion of the new key, and shrinking of the page)
  and to the new page (receives some keys from split page, and
  sometimes the new key).
storage/maria/ma_rt_test.c:
  ma_rt_test had been forgotten when maria_rkey() was changed some months ago
  (0->HA_WHOLE_KEY change), and when calls to maria_rnd(,,HA_OFFSET_ERROR)
  were rewritten to maria_scan() calls (which implies maria_scan_init()).
  The 'max_i' change is to adapt to the fact that maria_scan() does
  not return deleted records for BLOCK_RECORD but does so for other formats;
  the initial code assumed a certain number of deleted records would be
  returned, we change it to rather count only non-deleted ones.
  We also add more features to this test, like ma_test1 (the plan
  is to run ma_rt_test in ma_test_all-t):
  options to choose records' format, table checksum, transactions,
  checkpoints, end at specific stages, abort without committing,
  and debug trace.
storage/maria/ma_test1.c:
  MY_INIT() does my_init().
storage/maria/ma_write.c:
  Logging done by _ma_ck_write_btree_with_log() is moved to a function
  (_ma_write_undo_key_insert()), for reusal by R-tree logging.
  _ma_log_new() and _ma_log_change() are made non-static for same
  reason. Some parameters of logging functions are made pointers to const.
  If EXTRA_DEBUG_KEY_CHANGES, we now log CRC in _ma_log_change() too
  (better checks, bigger record).
storage/maria/maria_read_log.c:
  Program takes no arguments, bail out if any, instead of silently discarding them
storage/myisam/rt_test.c:
  rt_test had been forgotten when mi_rkey() was changed some months ago
  (0->HA_WHOLE_KEY change).
  The 'max_i' change is to make it symmetric with ma_rt_test.c
mysql-test/r/maria-gis-rtree-dynamic.result:
  correct result
mysql-test/r/maria-gis-rtree-trans.result:
  correct result
mysql-test/r/maria-recovery-rtree-ft.result:
  almost correct result (hitting BUG# in the end)
mysql-test/t/maria-gis-rtree-dynamic.test:
  test R-tree & dynamic row format
mysql-test/t/maria-gis-rtree-trans.test:
  Test R-tree and page row format and transactional
mysql-test/t/maria-recovery-rtree-ft-master.opt:
  usual options for recovery testing
mysql-test/t/maria-recovery-rtree-ft.test:
  test of recovery of R-tree and fulltext indices.
2008-04-24 17:22:51 +02:00
unknown
126c1228f5 Added versioning of row data
Will in future changeset (soon) av versioning of status variables (number of rows) and index
Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings
Removed some not needed variables (as noticed by Guilhem)


include/maria.h:
  Added prototypes for maria_chk_init_for_check(), maria_versioning() and maria_ignore_trids()
include/my_base.h:
  Add new error HA_ERR_ROW_NOT_VISIBLE
include/myisamchk.h:
  Added variables for checking visibility of rows during maria_chk
include/thr_lock.h:
  Changed argument type from int to my_bool for get_status
  Added variable allow_multiple_concurrent_insert, to signal if table supports multiple concurrent inserts
mysql-test/r/maria-page-checksum.result:
  Added missing drop table
mysql-test/t/maria-page-checksum.test:
  Added missing drop table
mysys/my_handler.c:
  Added new error messages
mysys/thr_lock.c:
  Added support for multiple concurrent inserts, if table handler supports it
sql/sql_yacc.yy:
  Added LOCK TABLE table_name WRITE CONCURRENT
  This was added (temporarly?) to be able to check versioning with Maria
storage/csv/ha_tina.cc:
  Updated parameter for get_status
storage/maria/ha_maria.cc:
  Added calls to maria_chk_init_status()
  Fixed call to ma_control_file_open()
storage/maria/ma_blockrec.c:
  Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings
  Changed back some 'header' parameters to const char*
  Removed some casts
  
  Added support for versioning:
  - If info->row_flag & ROW_FLAG_TRANSID is set, store transaction id together with the row
  - When reading rows, check if rows are visible. Give error if not
  - When scanning table, ignore not visible rows
  - Added function parameters to some functions, to be able to call _ma_compact_block_page() with different parameters depending of if the page is a HEAD or TAIL page
  - _ma_compact_block_page() deletes transaction id's that are visible by all running transactions
  - Added functions for thr_lock() to enable multiple concurrent inserts
  - Added helper function 'mysql_versioning()' to enable/disable versioning
  - Added helper function maria_ignore_trids(), used by maria_chk and maria_pack to see all rows.
storage/maria/ma_blockrec.h:
  Updated parameters for some functions.
  Added new functions to read/store state with thr_lock
storage/maria/ma_check.c:
  Enable handling of transaction id's in rows
  Give a readable error if a table contains a transation id that makes rows not visible
storage/maria/ma_control_file.c:
  Added option to not give warning if control file doesn't exists.
storage/maria/ma_control_file.h:
  Updated parameter lists for ma_control_file_open()
storage/maria/ma_delete.c:
  Removed not used variable (suggestion by Guilhem)
storage/maria/ma_locking.c:
  Changed type of argument from int -> my_bool
storage/maria/ma_open.c:
  Removed not used variables 'key_write_undo_lsn' and 'key_delete_undo_lsn'
  Added new thr_lock interface functions for BLOCK_RECORD to enable multiple concurrent insert
storage/maria/ma_test1.c:
  Added option --versioning (-C) to check versioning
storage/maria/ma_test2.c:
  Added option -C to check versioning
storage/maria/ma_test_recovery:
  Forward argumetns to ma_test_recovery.pl
storage/maria/ma_write.c:
  Removed not used variable key_write_undo_lsn
storage/maria/maria_chk.c:
  Always read control file (if exist) at start
  Initialize checking of tables by calling maria_chk_init_for_check()
  In verbose mode and in case of error, print max found transaction id
storage/maria/maria_def.h:
  Added Trid to MARIA_ROW to be able to check transaction id for found row
  Moved 'base_length' from MARIA_ROW to MARIA_HA to be able to handle different base length (with and without TRANSID) without if's
  Added default row_flag to MARIA_HA for the same reason
  Changed LEX_STRING -> LEX_CUSTRING to avoid casts in ma_blockrec.c
  Removed not needed variables key_write_undo_lsn and key_delete_undo_lsn
  Added prototypes for new functions and fixed those that had changed
storage/maria/maria_pack.c:
  Ensure we can read all rows from the file, independent of the used transaction id
storage/maria/maria_read_log.c:
  Updated arguments to ma_control_file_open()
storage/maria/trnman.c:
  If we have only one transaction, fixed that min_read_from contains current transaction
  Fixed that trnman_can_read_from() returns that row is readable if it was written by current transaction
storage/maria/unittest/ma_control_file-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_all-t:
  Added test of versioning
  Removed printing of one extra space
storage/maria/unittest/ma_test_loghandler-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Updated arguments to ma_control_file_open()
storage/maria/unittest/ma_test_recovery.expected:
  Updated file with result from new tests
storage/maria/unittest/ma_test_recovery.pl:
  Added options --abort-on-error and --verbose
  In case of --verbose, print all excuted shell commands
  Added test of versioning
storage/myisam/mi_locking.c:
  Updated type of parameter
storage/myisam/myisamdef.h:
  Updated type of parameter
mysql-test/r/maria-mvcc.result:
  New BitKeeper file ``mysql-test/r/maria-mvcc.result''
mysql-test/t/maria-mvcc.test:
  New BitKeeper file ``mysql-test/t/maria-mvcc.test''
2008-04-10 05:26:36 +03:00
unknown
722a8ebe5b Store maximum transaction id into control file at clean shutdown.
This can serve to maria_chk to check that trids found in rows and keys
are not too big. Also used by Recovery when logs are lost.
Options --require-control-file, --datadir, --log-dir (yes, the dashes are
inconsistent but I imitated mysqld --datadir and --maria-log-dir) for
maria_chk.
Lock control file _before_ reading its content.


storage/maria/ha_maria.cc:
  new prototype
storage/maria/ma_check.c:
  A function to find the max trid in the system (consults transaction
  manager and control file), to check tables.
storage/maria/ma_checkpoint.c:
  new prototype
storage/maria/ma_control_file.c:
  Store max trid into control file, in a backward-compatible way
  (can still read old control files).
  Parameter to ma_control_file_open(), to not create the log if it's
  missing (maria_chk needs that).
  Lock control file _before_ reading its content.
  Fix for a segfault when reading an old control file (bzero() with a
  negative second argument)
storage/maria/ma_control_file.h:
  changes to the control file module's API
storage/maria/ma_init.c:
  When Maria shuts down cleanly, store max trid into control file.
storage/maria/ma_loghandler.c:
  new prototype
storage/maria/ma_recovery.c:
  During recovery, consult max trid stored in control file, in case it is
  bigger than what we found in log (case of logs manually removed by user).
storage/maria/ma_test1.c:
  new prototype
storage/maria/ma_test2.c:
  new prototype
storage/maria/maria_chk.c:
  New option --require-control-file (abort if control file not found),
  --datadir (path for control file (and for logs if --log-dir not specified)),
  --log-dir (path for logs).
  Try to open control file when maria_chk starts.
storage/maria/maria_read_log.c:
  new prototype
storage/maria/trnman.c:
  A new function to know max trid in transaction manager
storage/maria/trnman_public.h:
  New function
storage/maria/unittest/ma_control_file-t.c:
  new prototypes. Testing storing and retrieving the max trid to/from
  control file
storage/maria/unittest/ma_test_loghandler-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  new prototype
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  new prototype
2008-04-04 19:10:53 +02:00
unknown
dc36524546 Compiler warnings fixed.
storage/maria/ma_loghandler.c:
  Compiler warning fixed.
storage/maria/ma_test1.c:
  Compiler warning fixed.
storage/maria/ma_test2.c:
  Compiler warning fixed.
storage/maria/maria_chk.c:
  Compiler warning fixed.
storage/maria/unittest/ma_control_file-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Compiler warning fixed.
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Compiler warning fixed.
2008-01-30 08:01:45 +02:00
unknown
a1255fc299 Support of moving logs in other directory then data root.
Fixed typo in maria_read_log "help" output.
Compilation warning on Mac OS fixed.


mysql-test/include/maria_empty_logs.inc:
  Added support of moving maria log relatively
    to master data directory.
mysql-test/include/maria_make_snapshot_for_feeding_recovery.inc:
  Added support of moving maria log relatively
    to master data directory.
mysql-test/include/maria_verify_recovery.inc:
  Added support of moving maria log relatively
    to master data directory.
mysql-test/t/maria-recovery-master.opt:
  Check of recovery with logs in other place.
mysql-test/t/maria-recovery.test:
  Check of recovery with logs in other place.
storage/maria/ha_maria.cc:
  Support of moving logs in other directory then data root.
storage/maria/ma_loghandler.c:
  Compilation warning on Mac OS fixed.
storage/maria/ma_static.c:
  Support of moving logs in other directory then data root.
storage/maria/ma_test1.c:
  Support of moving logs in other directory then data root.
storage/maria/maria_def.h:
  Support of moving logs in other directory then data root.
storage/maria/maria_read_log.c:
  Support of moving logs in other directory then data root.
  Fixed typo in maria_read_log "help" output.
2008-01-26 23:03:23 +02:00
unknown
c719e1fd33 Added wrapper for chmod(): my_chmod()
Automaticly disable ma_test_recovery if not compiled with debugging. This fixes that make test works in Maria.
Fixed wrong merge of ma_init.c from 5.1
Portability fixes:
- Use my_chmod() instead of my_chmod()
- Use my_access() instead of my_stat() to test if file exists
- Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable
- No reason to test if file exists before we delete it



include/my_sys.h:
  Added my_chmod
include/mysys_err.h:
  Added error for my_chmod
mysys/Makefile.am:
  Added my_chmod
mysys/errors.c:
  Added error for my_chmod
mysys/my_init.c:
  Syncronize with 5.1 to fix setting of QueryPerformanceFrequency()
storage/maria/ma_test1.c:
  Changed short option of --skip-delete and --skip-update to be more logical
storage/maria/ma_test_recovery.expected:
  Updated results after adding more tests
storage/maria/ma_test_recovery:
  Abort test nicely if we are runnning without debugging
  Added more tests
  Changed temporary file names so that one can run maria_chk on them
  Removed some old comments
storage/maria/maria_read_log.c:
  Added note if maria_read_log will not be able to create byte-to-byte identical tables compared to normal execution
storage/maria/unittest/ma_pagecache_consist.c:
  Removed wrong setting of buff that caused memory overwrite
  Use my_chmod() instead of chmod()
  Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable
storage/maria/unittest/ma_pagecache_single.c:
  Use my_chmod() instead of chmod()
  Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  No reason to test if file exists before we delete it
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  No reason to test if file exists before we delete it
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
  Use my_access() instead of my_stat() to test if file exists
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  No reason to test if file exists before we delete it
  chmod -> my_chmod
mysys/my_chmod.c:
  Added wrapper for chmod()
2008-01-03 09:45:46 +02:00
unknown
30d3d8d3fc Fixed several bugs in page CRC handling
- Ignore CRC errors in REDO for potential new pages
- Ignore CRC errors when repairing tables
- Don't do readcheck callback on read error
- Set my_errno to HA_ERR_WRONG_CRC if we find page with wrong CRC
- Check index page for length before calculating CRC to catch bad pages
Fixed bugs where we used wrong file descriptor to read/write bitmaps
Fixed wrong hash key in 'files_in_flush'
Fixed wrong lock method when writing bitmap
Fixed some wrong printf statements in check/repair that caused core dumps
Fixed argument to translog_page_validator that cause reading of log files to fail
Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages.
Use fast 'dummy' pagecheck callbacks for temporary tables
Don't die silently if flush finds pinned pages
Give error (for now) if one tries to create a transactional table with fulltext or spatial keys
Removed some not needed calls to pagecache_file_init()
Added checking of pagecache checksums to ma_test1 and ma_test2
More DBUG
Fixed some DBUG_PRINT to be in line with rest of the code


include/my_base.h:
  Added HA_ERR_INTERNAL_ERROR (used for flush with pinned pages) and HA_ERR_WRONG_CRC
mysql-test/r/binlog_unsafe.result:
  Added missing DROP VIEW statement
mysql-test/r/maria.result:
  Added TRANSACTIONAL=0 when testing with fulltext keys
  Added test that verifies we can't yet create transactional test with fulltext or spatial keys
mysql-test/r/ps_maria.result:
  Added TRANSACTIONAL=0 when testing with fulltext keys
mysql-test/t/binlog_unsafe.test:
  Added missing DROP VIEW statement
mysql-test/t/maria.test:
  Added TRANSACTIONAL=0 when testing with fulltext keys
  Added test that verifies we can't yet create transactional test with fulltext or spatial keys
mysql-test/t/ps_maria.test:
  Added TRANSACTIONAL=0 when testing with fulltext keys
mysys/my_fopen.c:
  Fd: -> fd:
mysys/my_handler.c:
  Added new error messages
mysys/my_lock.c:
  Fd: -> fd:
mysys/my_pread.c:
  Fd: -> fd:
mysys/my_read.c:
  Fd: -> fd:
mysys/my_seek.c:
  Fd: -> fd:
mysys/my_sync.c:
  Fd: -> fd:
mysys/my_write.c:
  Fd: -> fd:
sql/mysqld.cc:
  Fixed wrong argument to my_uuid_init()
sql/sql_plugin.cc:
  Unified DBUG_PRINT (for convert-dbug-for-diff)
storage/maria/ma_bitmap.c:
  Fixed wrong lock method when writing bitmap
  Fixed valgrind error
  Use fast  'dummy' pagecheck callbacks for temporary tables
  Faster bitmap handling for non transational tables
storage/maria/ma_blockrec.c:
  Fixed that bitmap reading is done with the correct filehandle
  Handle reading of pages with wrong CRC when page contect doesn't matter
  Use the page buffer also when we get WRONG CRC or FILE_TOO_SHORT. (Faster and fixed a couple of bugs)
storage/maria/ma_check.c:
  Split long strings for readablity
  Fixed some wrong printf statements that caused core dumps
  Use bitmap.file for bitmaps
  Ignore pages with wrong CRC
storage/maria/ma_close.c:
  More DBUG_PRINT
storage/maria/ma_create.c:
  Give error (for now) if one tries to create a crash safe table with fulltext or spatial keys
storage/maria/ma_key_recover.c:
  Ignore HA_ERR_WRONG_CRC for new pages
  info->s  ->  share
  Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages.
storage/maria/ma_loghandler.c:
  Fixed argument to translog_page_validator()
storage/maria/ma_open.c:
  Removed old VMS specific code
  Added function to setup pagecache callbacks
  Moved code around to set 'share->temporary' early
  Removed some not needed calls to pagecache_file_init()
storage/maria/ma_page.c:
  Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages.
storage/maria/ma_pagecache.c:
  Don't do readcheck callback on read error
  Reset PCBLOCK_ERROR in pagecache_unlock_by_link() if we write page
  Set my_errno to HA_ER_INTERNAL_ERROR if flush() finds pinned pages
  Don't die silently if flush finds pinned pages.
  Use correct file descriptor when flushing pages
  Fixed wrong hash key in 'files_in_flush';  This must be the file descriptor, not the PAGECACHE_FILE as there may be several PAGECACHE_FILE for same file descriptor
  More DBUG_PRINT
storage/maria/ma_pagecrc.c:
  Removed inline from not tiny static function
  Set my_errno to HA_ERR_WRONG_CRC if we find page with wrong CRC
  (Otherwise my_errno may be 0, and a lot of other code will be confused)
  CRCerror -> error (to keep code uniform)
  Print crc with %lu, as in my_checksum()
  uchar* -> uchar *
  Check index page for length before calculating CRC to catch bad pages
  Added 'dummy' crc_check and filler functions that are used for temporary tables
storage/maria/ma_recovery.c:
  More DBUG
  More message to users to give information what phase failed
  Better error message if recovery failed
storage/maria/ma_test1.c:
  Added checking of page checksums (combined with 'c' to not have to add more test runs)
storage/maria/ma_test2.c:
  Added checking of page checksums (combined with 'c' to not have to add more test runs)
storage/maria/maria_chk.c:
  Fixed wrong argument to _ma_check_print_error()
storage/maria/maria_def.h:
  Added format information to _ma_check_print_xxxx functions
  uchar* -> uchar *
2007-12-18 03:21:32 +02:00
unknown
98a2c5e3cf Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria
into  desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-readonly


storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_create.c:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/ma_test2.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Auto merged
storage/maria/ma_check.c:
  Merge.
storage/maria/ma_loghandler.c:
  Merge.
storage/maria/ma_loghandler.h:
  Merge.
storage/maria/maria_read_log.c:
  Merge.
storage/maria/unittest/Makefile.am:
  Merge.
2007-12-15 16:07:16 +02:00
unknown
d225521c76 Transaction log behaviour in case of write
error fixed (switching to the read only mode).
Added read only mode of transactions log handler.


storage/maria/ha_maria.cc:
  Transaction log initialization parameters change.
storage/maria/ma_check.c:
  New status variable of transactional log.
storage/maria/ma_create.c:
  New status variable of transactional log.
storage/maria/ma_loghandler.c:
  - New status variable added.
  - Checking the status variable in the loghandler
  interface functions added
  - All fails of loghandler functions revised.
  - UNRECOVERABLE_ERROR() removed.
  - Switching to read only mode added where it nead.
  - Checking of log state added before writes log content
    and changing status variables like sent_to_file, n_buffers_only, flushed.
  - Readonly loghandler initialization added.
  - Fixed problem with example table transactional log initialization.
storage/maria/ma_loghandler.h:
  Readonly loghandler initialization added.
  Fixed problem with example table transactional log initialization.
  New status variable added.
storage/maria/ma_open.c:
  New status variable of transactional log.
storage/maria/ma_test1.c:
  Transaction log initialization parameters change.
storage/maria/ma_test2.c:
  Transaction log initialization parameters change.
storage/maria/maria_read_log.c:
  Transaction loghandler initialization in read only mode in
  case of only dysplay parameter.
storage/maria/unittest/Makefile.am:
  Test of readonly mode added.
storage/maria/unittest/ma_test_loghandler-t.c:
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
  Fixed incorrect fprintf call parameters.
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Useing this test also as read only loghandler test.
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Transaction log initialization parameters change.
  Fixed problem with example table transactional log initialization.
2007-12-15 15:44:29 +02:00
unknown
2f6f08ed88 Added MARIA_SHARE *share to a lot of places to make code simpler
Changed info->s -> share to get more efficent code
Updated arguments to page accessor functions to use MARIA_SHARE * instead of MARIA_HA *.
Tested running tests in quick mode (no balance page on insert and only when critical on delete)
Fixed bug in underflow handling in quick mode
Fixed bug in log handler where it accessed not initialized variable
Fixed bug in log handler where it didn't free mutex in unlikely error condition
Removed double write of page in case of of some underflow conditions
Added DBUG_PRINT in safemutex lock/unlock


dbug/dbug.c:
  Compile without SAFE_MUTEX (to be able to use DBUG_PRINT in safe_mutex code)
  Use calls to get/set my_thread_var->dbug. (Make dbug independent of compile time options for mysys)
include/my_pthread.h:
  Added prototypes for my_thread_var_get_dbug() & my_thread_var_set_dbug()
mysql-test/lib/mtr_report.pl:
  Don't check warnings in log files if we are using --extern
mysys/my_thr_init.c:
  Added my_thread_var_get_dbug() & my_thread_var_set_dbug()
mysys/thr_mutex.c:
  Added DBUG printing of addresses to mutex for lock/unlock
storage/maria/ma_blockrec.c:
  Fixed comment
storage/maria/ma_check.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_close.c:
  Indentation fixes
storage/maria/ma_create.c:
  Calculate min_key_length correctly
storage/maria/ma_dbug.c:
  Indentation fixes
storage/maria/ma_delete.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
  Removed some writing of key pages that underflow (will be written by caller)
  Fixed crashing bug in underflow handling when using quick mode
storage/maria/ma_delete_all.c:
  Indentation fixes
storage/maria/ma_dynrec.c:
  Indentation fixes
storage/maria/ma_extra.c:
  Fixed indentation
  Removed old useless code
  Reset share->changed if we have written state
storage/maria/ma_ft_update.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_info.c:
  Indentation fixes
storage/maria/ma_key_recover.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_locking.c:
  Indentation fixes
storage/maria/ma_loghandler.c:
  Removed wrapper functions translog_mutex_lock and translog_mutex_unlock (safemutex now does same kind of printing)
  Renamed LOGREC_REDO_INSERT_ROW_BLOB to LOGREC_REDO_INSERT_NOT_USED to mark it free
  Fixed some DBUG_PRINT to ensure that convert-dbug-for-diff works
  Fixed bug in translog_flush() that caused log to stop syncing to disk
  Added missing mutex_unlock in case of error
storage/maria/ma_loghandler.h:
  Renamed LOGREC_REDO_INSERT_ROW_BLOB to LOGREC_REDO_INSERT_NOT_USED to mark it free
storage/maria/ma_open.c:
  Indentation fixes
storage/maria/ma_packrec.c:
  Indentation fixes
storage/maria/ma_page.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
  Added check that we never write a key page without content (except in recovery where a key page may temporary be without content)
storage/maria/ma_preload.c:
  Updated arguments to page accessor functions
storage/maria/ma_range.c:
  Updated arguments to page accessor functions
storage/maria/ma_rkey.c:
  Indentation fixes
storage/maria/ma_rprev.c:
  Indentation fixes
storage/maria/ma_rt_index.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_rt_index.h:
  Updated arguments to page accessor functions
storage/maria/ma_rt_key.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_rt_mbr.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_rt_split.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_search.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/ma_sort.c:
  Indentation fixes
storage/maria/ma_statrec.c:
  Indentation fixes
storage/maria/ma_test1.c:
  Added extra undo test
  Flush also keys in -u1, to ensure that the full log is flushed
storage/maria/ma_test2.c:
  Added extra undo test
  Flush also keys in -u1, to ensure that the full log is flushed
storage/maria/ma_test_recovery.expected:
  Updated results
storage/maria/ma_test_recovery:
  Added extra undo test
storage/maria/ma_update.c:
  Indentation fixes
storage/maria/ma_write.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
  Prepare for quick mode for insert (don't balance page)
storage/maria/maria_chk.c:
  Added MARIA_SHARE *share to a lot of places to make code simpler
  info->s -> share
  Updated arguments to page accessor functions
storage/maria/maria_def.h:
  Updated arguments to page accessor functions
2007-12-10 02:32:00 +02:00
unknown
ebf7ab7bce Added error HA_ERR_FILE_TOO_SHORT to be used when files are shorter than expected (by my_read/my_pread)
Added debugger hook _my_dbug_put_break_here() that is called if we get a CRC that matches --debug-crc-break (my_crc_dbug_break)
Fixed REDO_REPAIR to use all repair modes (repair, repair_by_sort, repair_paralell
REDO_REPAIR now also logs used key map
Fixed some bugs in REDO logging of key pages
Better error messages from maria_read_log
Added my_readwrite_flags to init_pagecache() to be able to get better error messages and simplify code.
Don't allow pagecaches with less than 8 blocks (Causes strange crashes)
Added EXTRA_DEBUG_KEY_CHANGES. When this is defined some REDO_INDEX entries contains page checksums (these are calculated and checked in DBUG mode, ignored otherwise)
Fixed bug in ma_pagecache unit tests that caused program to sometimes fail
Added some missing calls to MY_INIT() that caused some unit tests to fail
Fixed that TRUNCATE works properly on temporary MyISAM files
Updates some result files to new table checksums results (checksum when NULL fields are ignored)
perl test-insert can be replayed with maria_read_log!


sql/share/Makefile.am:
  Change mode to -rw-rw-r--
BitKeeper/etc/ignore:
  added storage/maria/unittest/page_cache_test_file_1 storage/maria/unittest/pagecache_debug.log
include/maria.h:
  Added maria_tmpdir
include/my_base.h:
  Added error HA_ERR_FILE_TOO_SHORT
include/my_sys.h:
  Added variable my_crc_dbug_check
  Added function my_dbug_put_break_here()
include/myisamchk.h:
  Added org_key_map (Needed for writing REDO record for REPAIR)
mysql-test/r/innodb.result:
  Updated to new checksum algorithm (NULL ignored)
mysql-test/r/mix2_myisam.result:
  Updated to new checksum algorithm (NULL ignored)
mysql-test/r/myisam.result:
  Updated to new checksum algorithm (NULL ignored)
mysql-test/t/myisam.test:
  Added used table
mysys/checksum.c:
  Added DBUG for checksum results
  Added debugger hook so that _my_dbug_put_break_here() is called if we get matching CRC
mysys/lf_alloc-pin.c:
  Fixed compiler warning
mysys/my_handler.c:
  Added new error message
mysys/my_init.c:
  If my_progname is not given, use 'unknown' form my_progname_short
  Added debugger function my_debug_put_break_here()
mysys/my_pread.c:
  In case of too short file when MY_NABP or MY_FNABP is specified, give error HA_ERR_FILE_TO_SHORT
mysys/my_read.c:
  In case of too short file when MY_NABP or MY_FNABP is specified, give error HA_ERR_FILE_TO_SHORT
sql/mysqld.cc:
  Added debug option --debug-crc-break
sql/sql_parse.cc:
  Trivial optimization
storage/maria/ha_maria.cc:
  Renamed variable to be more logical
  Ensure that param.testflag is correct when calling repair
  Added extra argument to init_pagecache
  Set default value for maria_tempdir
storage/maria/ma_blockrec.c:
  Test for HA_ERR_FILE_TOO_SHORT instead for -1
storage/maria/ma_cache.c:
  Test for HA_ERR_FILE_TOO_SHORT instead for -1
storage/maria/ma_check.c:
  Set param->testflag to match how repair is run (needed for REDO logging)
  Simple optimization
  Moved flag if page is node from pagelength to keypage-flag byte
  Log used key map in REDO log.
storage/maria/ma_delete.c:
  Remember previous UNDO entry when writing undo (for future CLR records)
  Moved flag if page is node from pagelength to keypage-flag byte
  Fixed some bugs in redo logging
  Added CRC for some translog REDO_INDEX entries
storage/maria/ma_dynrec.c:
  Test for HA_ERR_FILE_TOO_SHORT instead for -1
storage/maria/ma_ft_update.c:
  Fixed call to _ma_store_page_used()
storage/maria/ma_key_recover.c:
  Added CRC for some translog REDO_INDEX entries
  Removed not needed pagecache_write() in _ma_apply_redo_index()
storage/maria/ma_locking.c:
  Test for HA_ERR_FILE_TOO_SHORT instead for -1
storage/maria/ma_loghandler.c:
  Added used key map to REDO_REPAIR_TABLE
storage/maria/ma_loghandler.h:
  Added operation for checksum of key pages
storage/maria/ma_open.c:
  Allocate storage for undo lsn pointers
storage/maria/ma_pagecache.c:
  Remove not needed include file
  Change logging to use fd: for file descritors as other code
  Added my_readwrite_flags to init_pagecache() to be able to get better error messages for maria_chk/maria_read_log
  Don't allow pagecaches with less than 8 blocks
  Remove wrong DBUG_ASSERT()
storage/maria/ma_pagecache.h:
  Added readwrite_flags
storage/maria/ma_recovery.c:
  Better error messages for maria_read_log:
  - Added eprint() for printing error messages
  - Print extra \n before error message if we are printing %0 %10 ...
  
  Added used key_map to REDO_REPAIR log entry
  More DBUG
  Call same repair method that was used by mysqld
storage/maria/ma_rt_index.c:
  Moved flag if page is node from pagelength to keypage-flag byte
storage/maria/ma_rt_key.c:
  Fixed call to _ma_store_page_used()
storage/maria/ma_rt_split.c:
  Moved flag if page is node from pagelength to keypage-flag byte
storage/maria/ma_static.c:
  Added maria_tmpdir
storage/maria/ma_test1.c:
  Updated call to init_pagecache()
storage/maria/ma_test2.c:
  Updated call to init_pagecache()
storage/maria/ma_test3.c:
  Updated call to init_pagecache()
storage/maria/ma_write.c:
  Removed #ifdef NOT_YET
  Moved flag if page is node from pagelength to keypage-flag byte
  Fixed bug in  _ma_log_del_prefix()
storage/maria/maria_chk.c:
  Fixed wrong min limit for page_buffer_size
  Updated call to init_pagecache()
storage/maria/maria_def.h:
  Added EXTRA_DEBUG_KEY_CHANGES. When this is defined some REDO_INDEX entries contains page checksums
  Moved flag if page is node from pagelength to keypage-flag byte
storage/maria/maria_ftdump.c:
  Updated call to init_pagecache()
storage/maria/maria_pack.c:
  Updated call to init_pagecache()
  Reset share->state.create_rename_lsn & share->state.is_of_horizon
storage/maria/maria_read_log.c:
  Better error messages
  Added --tmpdir option (needed to set temporary directory for REDO_REPAIR)
  Added --start-from-lsn
  Changed option for --display-only to 'd' (wanted to use -o for 'offset')
storage/maria/unittest/lockman2-t.c:
  Added missing call to MY_INIT()
storage/maria/unittest/ma_pagecache_consist.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_pagecache_single.c:
  Fixed bug that caused program to sometimes fail
  Added some DBUG_ASSERTS()
  Changed some calls to malloc()/free() to my_malloc()/my_free()
  Create extra file to expose original hard-to-find bug
storage/maria/unittest/ma_test_loghandler-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Updated call to init_pagecache()
storage/maria/unittest/test_file.c:
  Changed malloc()/free() to my_malloc()/my_free()
  Fixed memory leak
  Changd logic a bit while trying to find bug in reset_file()
storage/maria/unittest/trnman-t.c:
  Added missing call to MY_INIT()
storage/myisam/mi_cache.c:
  Test for HA_ERR_FILE_TOO_SHORT instead for -1
storage/myisam/mi_create.c:
  Removed O_EXCL to get TRUNCATE to work for temporary files
storage/myisam/mi_dynrec.c:
  Test for HA_ERR_FILE_TOO_SHORT instead for -1
storage/myisam/mi_locking.c:
  Test for HA_ERR_FILE_TOO_SHORT instead for -1
mysql-test/r/old-mode.result:
  New BitKeeper file ``mysql-test/r/old-mode.result''
mysql-test/t/old-mode-master.opt:
  New BitKeeper file ``mysql-test/t/old-mode-master.opt''
mysql-test/t/old-mode.test:
  New BitKeeper file ``mysql-test/t/old-mode.test''
2007-12-04 23:23:42 +02:00
unknown
6b3743f0aa Fixes for redo/undo logging of key pages
New extendable format for maria_log_control file
Fixed some compiler warnings


include/maria.h:
  Added maria_disable_logging() and maria_enable_logging()
mysql-test/include/maria_verify_recovery.inc:
  Updated tests now when key redo/undo works
mysql-test/r/maria-recovery.result:
  Updated tests now when key redo/undo works
storage/maria/ma_blockrec.c:
  Use unified CLR code
  Added rec_lsn for full pages
  Moved clr write hook to ma_key_recover.c
  Changed REDO code to keep pages pinned until undo
  Mark page_link's as changed
storage/maria/ma_blockrec.h:
  Moved write_hook_for_clr_end() to ma_key_recover.c
storage/maria/ma_check.c:
  Changed key check code to use PAGECACHE_READ_UNKNOWN_PAGE
  Fixed wrong warning when checking files after maria_pack
  When unpacking files, we have to use new keypos_to_recpos method
  When doing repair, we can disregard index key file pages in page cache
storage/maria/ma_commit.c:
  Added simple enable/disable logging functions
  (Needed for recovery)
storage/maria/ma_control_file.c:
  Make maria control file extendable without having to make it incompatible for older versions
storage/maria/ma_control_file.h:
  New error messages
  Added CONTROL_FILE_VERSION
storage/maria/ma_delete.c:
  Added redo/undo for key pages
  change_length -> changed_length to make things similar
  More comments & more DBUG
storage/maria/ma_key_recover.c:
  Unified CLR method
  Moved here write_hook_for_clr_end() and common keypage log functions
  Changed REDO to keep pages pinned until undo
  Changed UNDO code to change key_root under log mutex
storage/maria/ma_key_recover.h:
  New structures and functions
storage/maria/ma_loghandler.c:
  Include needed files
storage/maria/ma_open.c:
  Change maria_open() to use pread() instead of read()
storage/maria/ma_page.c:
  Fixed bug in key_del handling
  Clear pages if IDENTICAL_PAGES_AFTER_RECOVERY is defined
storage/maria/ma_pagecache.c:
  Indentation and spelling fixes
  More DBUG
  Added helper function: pagecache_block_link_to_buffer()
storage/maria/ma_pagecache.h:
  Added pagecache_block_link_to_buffer()
storage/maria/ma_recovery.c:
  Fixed state.changed
  Fixed that REDO keeps pages pinned until UNDO
  Some bug fixes from previous commit
  Fixes for UNDO/REDO of key pages
storage/maria/ma_search.c:
  Fixed packing and storing of keys to provide more information to caller so
  that we can do efficent REDO logging of the changes.
storage/maria/ma_test1.c:
  Fixed bug with not initialized variable
storage/maria/ma_test2.c:
  Removed not used code
storage/maria/ma_test_all.res:
  Updated results
storage/maria/ma_test_all.sh:
  Changed one test to test more
  Removed timing tests as not relevant here
storage/maria/ma_test_recovery.expected:
  Updated test result after redo/undo if key pages works
storage/maria/ma_test_recovery:
  Updated test after redo/undo if key pages works
storage/maria/ma_write.c:
  Moved some general log functions to ma_key_recover.c
  Fixed some bugs in undo
  Moved ma_log_split() to _ma_split_page()
  Small changes in some function arguments to be able to do redo logging
storage/maria/maria_chk.c:
  disable logging while doing repair table
storage/maria/maria_def.h:
  New function prototypes
  Move some structs and functions to ma_key_recover.c
storage/maria/unittest/ma_control_file-t.c:
  Updated with patch from Sanja
  NOTE: This is not complete and need to be updated to new control file format
storage/maria/unittest/ma_test_loghandler-t.c:
  Fixed compiler warning
2007-11-20 17:42:16 +02:00
unknown
21fd2a5a36 First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable


include/maria.h:
  Made min page cache larger (needed for pinning key page)
  Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
  Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
  Added new error message to be used when handler initialization failed
include/my_global.h:
  Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
  Added const to some parameters
mysys/array.c:
  More DBUG
mysys/my_error.c:
  Fixed indentation
mysys/my_handler.c:
  Added const to some parameters
  Added missing error messages
sql/field.h:
  Renamed variables to avoid variable shadowing
sql/handler.h:
  Renamed parameter to avoid variable name conflict
sql/item.h:
  Renamed variables to avoid variable shadowing
sql/log_event_old.h:
  Renamed variables to avoid variable shadowing
sql/set_var.h:
  Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
  Removed maria hack for temporary tables
  Fixed indentation
sql/sql_table.cc:
  Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
  This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
  Copy page_checksum from share
  Removed Maria hack
storage/maria/Makefile.am:
  Added new files
storage/maria/ha_maria.cc:
  Renamed records -> record_count and info -> create_info to avoid variable name conflicts
  Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
  Moved _ma_unpin_all_pages() to ma_key_recover.c
  Moved init of info->pinned_pages to ma_open.c
  Moved _ma_finalize_row() to maria_key_recover.h
  Renamed some variables to avoid variable name conflicts
  Mark page_link.changed for blocks we change directly
  Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
  Removed extra empty line
storage/maria/ma_checkpoint.c:
  Remove not needed trnman.h
storage/maria/ma_close.c:
  Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
  Give error message if we fail to open control file
storage/maria/ma_delete.c:
  Changes for redo logging (first part, logging of underflow not yet done)
  - Log undo-key-delete
  - Log delete of key
  - Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
  - Added new arguments to some functions to be able to write redo information
  - Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
  
  Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
  Changed 2 bmove_upp() to bmove() as this made code easer to understand
  More function comments
  Indentation fixes
storage/maria/ma_ft_update.c:
  New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
  Fixed some DBUG_PRINT messages
  Simplify code
  Added new log entrys for key page redo
  Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
  Moved some defines here
  Added define for storing key number on key pages
  Added new translog record types
  Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
  Always allocate info.pinned_pages (we need now also for normal key page usage)
  Update keyinfo->key_nr
  Added virtual functions to convert record position o number to be stored on key pages
  Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
  Added redo for key pages
  - Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
  - _ma_fetch_keypage() now pin's pages if needed
  - Extended _ma_write_keypage() with type of locks to be used
  - ma_dispose() now locks info->s->state.key_del from other threads
  - ma_dispose() writes redo log record
  - ma_new() locks info->s->state.key_del from other threads if it was used
  - ma_new() now pins read page
  
  Other things:
  - Removed some not needed arguments from _ma_new() and _ma_dispose)
  - Added some new variables to simplify code
  - If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
  Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
  Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
  Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
  - Added hooks for new translog types:
    REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
    UNDO_KEY_DELETE_WITH_ROOT.
  - Moved variable declarations to start of function (portability fixes)
  - Removed some not needed initializations
  - Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
  Removed end space
storage/maria/ma_check.c:
  Removed end space
storage/maria/ma_create.c:
  Removed end space
storage/maria/ma_locking.c:
  Removed end space
storage/maria/ma_packrec.c:
  Removed end space
storage/maria/ma_pagecache.c:
  Removed end space
storage/maria/ma_panic.c:
  Removed end space
storage/maria/ma_rt_index.c:
  Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
  Fixed indentation
storage/maria/ma_rt_key.c:
  Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
  Added new arguments for call to _ma_new()
  Use new keypage header
  Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
  Updated comments & indentation
  Added new arguments for call to _ma_fetch_keypage()
  Made some variables and arguments const
  Added virtual functions for converting row position to number to be stored in key
  use MARIA_RECORD_POS of record position instead of my_off_t
  Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
  Removed end space
storage/maria/ma_statrec.c:
  Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
  Fixed too small buffer to init_pagecache()
  Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
  Use more resonable pagecache size
  Remove not used code
  Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
  Fixed wrong test
storage/maria/ma_write.c:
  Lots of new code to handle REDO of key pages
  No logic changes because of REDO code, mostly adding new arguments and adding new code for logging 
  
  Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
  Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
  Zerofill new used pages for:
  - To remove possible sensitive data left in buffer
  - To get idenitical data on pages after running redo
  - Better compression of pages if archived
storage/maria/maria_chk.c:
  Added information if table is crash safe
storage/maria/maria_def.h:
  New virtual function to convert between record position on key and normal record position
  Aded mutex and extra variables to handle locking of share->state.key_del
  Moved some structure variables to get things more aligned
  Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
  Added argument to MARIA_PINNED_PAGE to indicate if page was changed
  Updated prototypes for functions
  Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
  Updated arguments for changed function calls
storage/myisam/mi_check.c:
  Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
  Update checksums to ignore null columns
storage/myisam/mi_create.c:
  Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
  Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
  Fixed bug
storage/myisam/myisamdef.h:
  Added virtual function for calculating checksum during check table
  Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
  New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
  New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
  New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 19:08:06 +02:00
unknown
301ee8d9a3 Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


include/my_sys.h:
  Auto merged
mysql-test/r/maria.result:
  Auto merged
mysql-test/t/maria.test:
  Auto merged
sql/handler.h:
  Auto merged
sql/mysqld.cc:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_bitmap.c:
  Auto merged
storage/maria/ma_blockrec.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/ma_test_recovery.expected:
  Auto merged
storage/maria/ma_test_recovery:
  Auto merged
sql/mysql_priv.h:
  manual merge
storage/maria/ma_recovery.c:
  manual merge
storage/maria/ma_test2.c:
  manual merge
2007-10-20 01:33:12 +03:00
unknown
13d53bf657 Merge some changes from sql directory in 5.1 tree
Changed format for REDO_INSERT_ROWS_BLOBS
Fixed several bugs in handling of big blobs
Added redo_free_head_or_tail() & redo_insert_row_blobs()
Added uuid to control file
maria_checks now verifies that not used part of bitmap is 0
REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
Added REDO_FREE_HEAD_OR_TAIL
Fixes problem when trying to read block outside of file during REDO


include/my_global.h:
  STACK_DIRECTION is already set by configure
mysql-test/r/maria.result:
  Updated results
mysql-test/t/maria.test:
  Test shrinking of VARCHAR
mysys/my_realloc.c:
  Fixed indentation
mysys/safemalloc.c:
  Fixed indentation
sql/filesort.cc:
  Removed some casts
sql/mysqld.cc:
  Added missing setting of myisam_stats_method_str
sql/uniques.cc:
  Removed some casts
storage/maria/ma_bitmap.c:
  Added printing of bitmap (for debugging)
  Renamed _ma_print_bitmap() -> _ma_print_bitmap_changes()
  Added _ma_set_full_page_bits()
  Fixed bug in ma_bitmap_find_new_place() (affecting updates) when using big files
storage/maria/ma_blockrec.c:
  Changed format for REDO_INSERT_ROWS_BLOBS
  Fixed several bugs in handling of big blobs
  Added code to fix some cases where redo when using blobs didn't produce idenital .MAD files as normal usage
  REDO_FREE_ROW_BLOCKS doesn't anymore change pages; We only mark things free in bitmap
  Remove TAIL and filler extents from REDO_FREE_BLOCKS log entry. (Fixed some asserts)
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Delete tails in update. (Fixed bug when doing update that shrinks blob/varchar length)
  Fixed bug when doing insert in block outside of file size.
  Added redo_free_head_or_tail() & redo_insert_row_blobs()
  Added pagecache_unlock_by_link() when read fails.
  Much more comments, DBUG and ASSERT entries
storage/maria/ma_blockrec.h:
  Prototypes of new functions
  Define of SUB_RANGE_SIZE & BLOCK_FILLER_SIZE
storage/maria/ma_check.c:
  Verify that not used part of bitmap is 0
storage/maria/ma_control_file.c:
  Added uuid to control file
storage/maria/ma_loghandler.c:
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_loghandler.h:
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_pagecache.c:
  If we write full block, remove error flag for block.
  (Fixes problem when trying to read block outside of file)
storage/maria/ma_recovery.c:
  REDO_PURGE_BLOCKS -> REDO_FREE_BLOCKS
  Added REDO_FREE_HEAD_OR_TAIL
storage/maria/ma_test1.c:
  Allow option after 'b' to be compatible with ma_test2
  (This is just to simplify test scripts like ma_test_recovery)
storage/maria/ma_test2.c:
  Default size of blob is now 1000 instead of 1
storage/maria/ma_test_all.sh:
  Added test for bigger blobs
storage/maria/ma_test_recovery.expected:
  Updated results
storage/maria/ma_test_recovery:
  Added test for bigger blobs
2007-10-20 00:24:22 +03:00
unknown
77017191de WL#3071 - Maria checkpoint
- serializing calls to flush_pagecache_blocks_int() on the same file
to avoid known concurrency bugs
- having that, we can now enable the background thread, as the
flushes it does are now supposedly safe in concurrent situations.
- new type of flush FLUSH_KEEP_LAZY: when the background checkpoint
thread is flushing a packet of dirty pages between two checkpoints,
it uses this flush type, indeed if a file is already being flushed
by another thread it's smarter to move on to the next file than wait.
- maria_checkpoint_frequency renamed to maria_checkpoint_interval.


include/my_sys.h:
  new type of flushing for the page cache: FLUSH_KEEP_LAZY
mysql-test/r/maria.result:
  result update
mysys/mf_keycache.c:
  indentation. No FLUSH_KEEP_LAZY support in key cache.
storage/maria/ha_maria.cc:
  maria_checkpoint_frequency was somehow a hidden part of the
  Checkpoint API and that was not good. Now we have checkpoint_interval,
  local to ha_maria.cc, which serves as container for the user-visible
  maria_checkpoint_interval global variable; setting it calls
  update_checkpoint_interval which passes the new value to
  ma_checkpoint_init(). There is no hiding anymore.
  By default, enable background thread which does checkpoints
  every 30 seconds, and dirty page flush in between. That thread takes
  a checkpoint when it ends, so no need for maria_hton_panic to take one.
  The | is | and not ||, because maria_panic() must always be called.
  frequency->interval.
storage/maria/ma_checkpoint.c:
  Use FLUSH_KEEP_LAZY for background thread when it flushes packets of
  dirty pages between two checkpoints: it is smarter to move on to
  the next file than wait for it to have been completely flushed, which
  may take long.
  Comments about flush concurrency bugs moved from ma_pagecache.c.
  Removing out-of-date comment.
  frequency->interval.
  create_background_thread -> (interval>0).
  In ma_checkpoint_background(), some variables need to be preserved
  between iterations.
storage/maria/ma_checkpoint.h:
  new prototype
storage/maria/ma_pagecache.c:
  - concurrent calls of flush_pagecache_blocks_int() on the same file
  cause bugs (see @note in that function); we fix them by serializing
  in this situation. For that we use a global hash of (file, wqueue).
  When flush_pagecache_blocks_int() starts it looks into the hash,
  using the file as key. If not found, it inserts (file,wqueue) into the
  hash, flushes the file, and finally removes itself from the hash and
  wakes up any waiter in the queue. If found, it adds itself to the
  wqueue and waits.
  - As a by-product, we can remove changed_blocks_is_incomplete
  and replace it by scanning the hash, replace the sleep() by a queue wait.
  - new type of flush FLUSH_KEEP_LAZY: when flushing a file, if it's
  already being flushed by another thread (even partially), return
  immediately.
storage/maria/ma_pagecache.h:
  In pagecache, a hash of files currently being flushed (i.e. there
  is a call to flush_pagecache_blocks_int() for them).
storage/maria/ma_recovery.c:
  new prototype
storage/maria/ma_test1.c:
  new prototype
storage/maria/ma_test2.c:
  new prototype
2007-10-19 14:15:13 +02:00
unknown
0f1feefa03 WL#3071 Maria checkpoint
Ability for flush_pagecache_blocks() to flush only certain pages of
a file, as instructed by an option "filter" pointer-to-function argument;
Checkpoint and background dirty page flushing use that to flush only
pages which have been dirty for long enough and bitmap pages.
Fix for a bug in flush_cached_blocks() (no idea if it could produce
a bug in real life, but theoretically it is).
Testing checkpoint in ma_test_recovery via ma_test1 and ma_test2.
Background checkpoint & dirty pages flush thread is still disabled
by default in ha_maria.


mysql-test/r/maria.result:
  result update
storage/maria/ha_maria.cc:
  blank after function comment
storage/maria/ma_checkpoint.c:
  Using an enum instead of 0/1/2 (applying Sanja's review comments).
  The comment about "this is an horizon" can be removed as Sanja
  created translog_next_LSN() which parse_checkpoint_record() uses.
  Variables in ma_checkpoint_background() cannot be declared in the
  for() as their value must not be reset at each iteration!
storage/maria/ma_pagecache.c:
  adding to flush_pagecache_blocks() optional arguments 'filter'
  (pointer to function) and 'filter_arg'; if filter!=NULL this function
  will be called for each block of the file and will reply if this
  block and following ones should be flushed or not (3 possible
  replies).
  Fixing a bug when flush_cached_blocks() skips a pinned page: it has
  to unset PCBLOCK_IN_FLUSH set by flush_pagecache_blocks_int().
storage/maria/ma_pagecache.h:
  flush_pagecache_blocks() is changed to take "filter" and "filter_arg"
  arguments. "filter", if it is not NULL, may return one value
  among enum pagecache_flush_filter_result.
storage/maria/ma_recovery.c:
  open_count=0 when closing tables at the end of recovery.
storage/maria/ma_test1.c:
  Optional checkpoints (-H#) at various stages (stages similar
  to --testflag), for testing of checkpoints.
storage/maria/ma_test2.c:
  Optional checkpoints (-H#) at various stages (stages similar
  to -t), for testing of checkpoints.
storage/maria/ma_test_recovery.expected:
  Result update: the results of the additional test run with -H#
  (checkpoints) are added here. They are exactly identical to without
  checkpoints except that the index's Root (printed by maria_chk)
  is more correct when using checkpoints. This is because checkpoint
  flushed the state, so it happens to be correct, while no-checkpoint
  does not flush the state, and recovery does not recover indexes
  so Root is never fixed. When we recover indices, this will go away.
storage/maria/ma_test_recovery:
  We duplicate the loop of tests to add an additional run with
  checkpoints at various stages, to see if maria_read_log
  uses them fine.
2007-10-17 16:55:26 +02:00
unknown
a1bb4afa7e after-merge fixes for 64bit
storage/maria/ma_test1.c:
  typo (?) fixed
2007-10-06 11:39:26 +02:00
unknown
4eb383e06c After merge fixes. Applied changes made in myisam manually to maria.
storage/maria/ma_test1.c:
  After merge fixes. Applied changes made in myisam manually to maria.
  
  Fixed memory leak.
storage/maria/ma_test2.c:
  After merge fixes. Applied changes made in myisam manually to maria.
  
  Fixed memory leak.
2007-10-04 20:33:42 +03:00
unknown
0b2ba820c3 WL#3072 Maria recovery
* testing of execution of UNDO_ROW_UPDATE
* when executing an UNDO_ROW_UPDATE, store "UNDO_ROW_UPDATE" as
"type of undone record" into the CLR_END record.


storage/maria/ma_blockrec.c:
  When logging a CLR_END in write_block_record(), it can be for
  a DELETE or for an UPDATE (now that Monty has coded execution of
  UNDO_UPDATE)
storage/maria/ma_loghandler.c:
  UNDO_ROW_UPDATE's execution coded, so no crash
storage/maria/ma_recovery.c:
  UNDO_ROW_UPDATE's execution now coded, so no crash
storage/maria/ma_test1.c:
  upper case letter
storage/maria/ma_test_recovery.expected:
  output of testing execution of UNDO_ROW_UPDATE. Table's checksum
  not recovered (known issue not specific to UPDATE).
storage/maria/ma_test_recovery:
  Test execution of UNDO_ROW_UPDATE: first we stop ma_test1 after
  deletes and commit, then we stop ma_test1 after updates and abort;
  we verify that updates are rolled back by comparing tables
2007-09-11 11:11:22 +02:00
unknown
6aef814d98 Fixed some bugs when using undo of VARCHAR fields
Fixed bug in undo_delete
Fixed wrong error output from maria_check


include/my_base.h:
  Added marker if we have null fields in table
mysql-test/r/maria.result:
  checksum in maria now ignore null fields that are null
sql/sql_table.cc:
  Ignore null fields that are now
  (Before enabling this, we have to change MyISAM to also skip null fields)
storage/maria/ma_blockrec.c:
  More logging
  After merge fixes
  Fixed some bugs when using undo of VARCHAR fields
  Fixed bug in undo_delete (We can't use info->rec_buff here as this is used in write_block_record())
storage/maria/ma_blockrec.h:
  ma_recordpos_to_dir_entry changed to return uint
storage/maria/ma_check.c:
  Fixed wrong output in case of errors
storage/maria/ma_create.c:
  Set share.base.pack_reclength more correct for block record
  Delete support for RAID
storage/maria/ma_open.c:
  Don't calculate checksum fields with value NULL
storage/maria/ma_test1.c:
  Fixed output from -v for VARCHAR keys
storage/maria/ma_test_recovery.expected:
  Update results after adding new printf
  New checksums (because we now ignore nulls)
  Some file lengths are different, but think they are ok (didn't have time to investigate)
storage/myisam/ha_myisam.cc:
  Fixed comment
storage/myisam/mi_test1.c:
  Fixed bug
2007-09-11 01:58:15 +03:00
unknown
cb2b22f0e0 Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  mysql.com:/home/my/mysql-maria


storage/maria/ma_check.c:
  Auto merged
storage/maria/ma_locking.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_recovery.c:
  Auto merged
storage/maria/maria_def.h:
  Auto merged
storage/maria/maria_read_log.c:
  Auto merged
storage/maria/ma_blockrec.c:
  Manual merge
storage/maria/ma_test1.c:
  Manual merge (using Guilhems code)
2007-09-09 19:22:20 +03:00
unknown
155193a6e2 Added applying of undo for updates
Fixed bug in duplicate key handling for block records during repair
All read-row methods now return error number in case of error
Don't calculate checksum for null fields
Fixed bug when running maria_read_log with -o


BUILD/SETUP.sh:
  Added STACK_DIRECTION
BUILD/compile-pentium-debug-max:
  Moved STACK_DIRECTION to SETUP
include/myisam.h:
  Added extra parameter to write_key
storage/maria/ma_blockrec.c:
  Added applying of undo for updates
  Fixed indentation
  Removed some not needed casts
  Fixed wrong logging of CLR record
  Split ma_update_block_record to two functions to be able to reuse it from undo-applying
  Simplify filling of packed fields
  ma_record_block_record) now returns error number on failure
  Sligtly changed log record information for undo-update
storage/maria/ma_check.c:
  Fixed bug in duplicate key handling for block records during repair
storage/maria/ma_checksum.c:
  Don't calculate checksum for null fields
storage/maria/ma_dynrec.c:
  _ma_read_dynamic_reocrd() now returns error number on error
  Rest of the changes are code simplification and indentation fixes
storage/maria/ma_locking.c:
  Added comment
storage/maria/ma_loghandler.c:
  More debugging
  Removed printing of total_record_length as this was always same as record_length
storage/maria/ma_open.c:
  Allocate bitmap for changed fields
storage/maria/ma_packrec.c:
  read_record now returns error number on error
storage/maria/ma_recovery.c:
  Fixed wrong arguments to undo_row_update
storage/maria/ma_statrec.c:
  read_record now returns error number on error (not 1)
  Code simplification
storage/maria/ma_test1.c:
  Added exit possibility after update phase (to test undo of updates)
storage/maria/maria_def.h:
  Include bitmap header file
storage/maria/maria_read_log.c:
  Fixed bug when running with -o
2007-09-09 19:15:10 +03:00
unknown
69d7db7758 WL#3072 - Maria Recovery
At the end of recovery, we initialize the transaction manager's
trid generator with the maximum trid seen during the REDO phase.
This ensures that trids always grow (needed for versioning),
even after a crash.
This patch is only preparation, as ma_recover() is not called
from ha_maria yet.


storage/maria/ha_maria.cc:
  trnman_init() needs argument now (soon trnman_init() will rather
  be done via ma_recover() and thus it will not be 0)
storage/maria/ma_recovery.c:
  During the REDO phase, remember the max long trid of transactions
  which we have seen (both in the checkpoint record and the
  LOGREC_LONG_TRANSACTION_ID records)
storage/maria/ma_test1.c:
  trnman_init() needs argument now
storage/maria/ma_test2.c:
  trnman_init() needs argument now
storage/maria/trnman.c:
  new argument to trnman_init() so that caller can decide which
  value the generator of trids starts from.
storage/maria/trnman_public.h:
  trnman_init() needs argument now
storage/maria/unittest/trnman-t.c:
  trnman_init() needs argument now
2007-09-07 15:52:25 +02:00
unknown
2291f932b2 - WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).


storage/maria/ha_maria.cc:
  protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
  * don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
  should wait until we have corrected the allocation in the bitmap
  (as the REDO can serve to correct the allocation during Recovery);
  introducing _ma_finalize_row() for that.
  * In a changeset yesterday I moved computation of the checksum
  into write_block_record(), to fix a bug in UPDATE. Now I notice
  that maria_update() already computes the checksum, it's just that
  it puts it into info->cur_row while _ma_update_block_record()
  uses info->new_row; so, removing the checksum computation from
  write_block_record(), putting it back into allocate_and_write_block_record()
  (which is called only by INSERT and UNDO_DELETE), and copying
  cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
  new prototypes, they will take intern_lock when writing the state;
  also take intern_lock when changing share->kfile. In both cases
  this is to protect against Checkpoint reading/writing the state or reading
  kfile at the same time.
  Not updating create_rename_lsn directly at end of write_log_record_for_repair()
  as it wouldn't have intern_lock.
storage/maria/ma_close.c:
  Checkpoint builds a list of shares (under THR_LOCK_maria), then it
  handles each such share (under intern_lock) (doing flushing etc);
  if maria_close() freed this share between the two, Checkpoint
  would see a bad pointer. To avoid this, when building the list Checkpoint
  marks each share, so that maria_close() knows it should not free it
  and Checkpoint will free it itself.
  Extending the zone covered by intern_lock to protect against
  Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
  When we update create_rename_lsn, we also update is_of_lsn to
  the same value: it is logical, and allows us to test in maria_open()
  that the former is not bigger than the latter (the contrary is a sign
  of index header corruption, or severe logging bug which hinders
  Recovery, table needs a repair).
  _ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
  it now operates under intern_lock (protect against Checkpoint),
  a shortcut function is available for cases where acquiring
  intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
  if table is transactional, "records" is already decremented
  when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
  comments
storage/maria/ma_extra.c:
  Protect modifications of the state, in memory and/or on disk,
  with intern_lock, against a concurrent Checkpoint.
  When state goes to disk, update it's is_of_lsn (by calling
  the new _ma_state_info_write()).
  In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
  a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
  no real code change here.
storage/maria/ma_loghandler.c:
  Log-write-hooks for updating "state.records" under log's mutex
  when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
  merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
  When opening a table verify that is_of_lsn >= create_rename_lsn; if
  false the header must be corrupted.
  _ma_state_info_write() is split in two: _ma_state_info_write_sub()
  which is the old _ma_state_info_write(), and _ma_state_info_write()
  which additionally takes intern_lock if requested (to protect
  against Checkpoint) and updates is_of_lsn.
  _ma_open_keyfile() should change kfile.file under intern_lock
  to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
  Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
  which has a LSN > state.is_of_lsn it increments state.records.
  Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
  When closing a table during Recovery, we know its state is at least
  as new as the current log record we are looking at, so increase
  is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
  update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
  update to new prototype
storage/maria/ma_test2.c:
  update to new prototype (actually prototype was changed days ago,
  but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
  new result file of ma_test_recovery. Improvements: record
  count read from index's header is now always correct.
storage/maria/ma_test_recovery:
  "rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
  if table is transactional, "records" is already incremented when
  logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
  update is_of_lsn too
storage/maria/maria_def.h:
  - MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
  into the index file's header.
  - Checkpoint can now mark a table as "don't free this", and maria_close()
  can reply "ok then you will free it".
  - new functions
storage/maria/maria_pack.c:
  update for new name
2007-09-07 15:02:30 +02:00
unknown
ac4ad9bdba WL#3072 Maria Recovery
misc fixes of execution of UNDOs in the UNDO phase:
- into the CLR_END, store the LSN of the _previous_ UNDO (we debated
what was best, so far we're going with "previous"; later we can change
to "current" if needed), and store the type of record which is being
undone (needed to know how to update state.records when we see the
CLR_END during the REDO phase).
- declaring all UNDOs and CLR_END as "compressed"
- when executing an UNDO in the UNDO phase, state.records is updated
as a hook when writing CLR_END (needed for "recovery of the state"),
and so is trn->undo_lsn (needed for when we have checkpoints).
- bugfix (execution of UNDO_ROW_DELETE didn't store the correct checksum
into the re-inserted row, maria_chk -r thus threw the row away).
- modifications of ma_test1: where to stop is now driven by --testflag;
--test-undo just tells how to stop (flush data, flush log, nothing).
- ma_test_recovery: testing of the UNDO phase, more testing of the
REDO phase, identification of a bug.


storage/maria/ma_blockrec.c:
  - bugfix: execution of UNDO_ROW_DELETE didn't store the correct
  checksum into the row (leading to "maria_chk -r" eliminating the
  re-inserted row, net effect was that rollback appeared to have
  rolled back no deletion). Reason was that write_block_record() used
  info->cur_row.checksum, while "row" can be != &info->cur_row
  (case of UNDO_ROW_DELETE). After fixing this, problems with
  _ma_update_block_record() appeared; indeed checksum was computed
  by  allocate_and_write_block_record() while _ma_update_block_record()
  directly calls write_block_record(). Solution is to compute checksum
  in write_block_record() instead.
  - when executing an UNDO, we now pass the LSN of the _previous_ UNDO
  to block_format functions. This LSN can be 0 (if the being-executed UNDO
  was the transaction's first UNDO), so "undo_lsn==0" cannot work
  anymore to indicate "this is not UNDO work". Using undo_lsn==LSN_ERROR
  instead (this is an impossible LSN).
  - store into CLR_END the type of log record which was undone
  (INSERT/UPDATE/DELETE); needed for Recovery to know if/how it has
  to update state.records if it sees this CLR_END in the REDO phase.
  - when writing the CLR_END in _ma_apply_undo_row_insert(),
  the place to store file's id is log_data+LSN_STORE_SIZE.
  - in _ma_apply_undo_row_insert(), the records-- is moved
  to a hook when writing the CLR_END (this way it is under log's mutex
  which is needed for "recovery of the state")
storage/maria/ma_loghandler.c:
  - all UNDOs, and CLR_END, start with the LSN of another UNDO; so
  we can declare them "compressed".
  - write_hook_for_clr_end() to set trn->undo_lsn (to the previous
  UNDO's LSN) under log's lock (like UNDOs set trn->undo_lsn under log's
  lock), and also update, if appropriate, state.records.
  - reset share->id to 0 when deassigning; not useful for now but
  sounds logical.
storage/maria/ma_recovery.c:
  - if no table is found for a REDO, it's not an error; for an UNDO, it is
  - in the REDO phase, when we see a CLR_END we must update trn->undo_lsn
  and sometimes state.records.
  - in the UNDO phase, when we execute an UNDO_ROW_INSERT:
    * update trn->undo_lsn only after executing the record
    * store the _previous_ undo_lsn into the CLR_END
  - at the end of the REDO phase, when we recreate TRN objects, they
  have already their long id in the log (either via a
  LOGREC_LONG_TRANSACTION_ID, or in a checkpoint record), don't write
  a new, useless LOGREC_LONG_TRANSACTION_ID for them.
storage/maria/ma_test1.c:
  * where to stop execution is now driven by --testflag and not --test-undo
  (ma_test2 already has --testflag for the same purpose). This allows
  us to do a clean stop (with commit) at any point.
  * --test-undo=# tells how to abort (flush all pages (which implies
  flushing log) or only log or nothing); all such "ways of crashing"
  are tested in ma_test_recovery
storage/maria/ma_test_recovery:
  * Testing execution of UNDOs, with and without BLOBs.
  * Testing idempotency of REDOs.
  * See @todo for a probable bug with BLOBs.
  * maria_chk -rq instead of -r, as with -q it nicely stops on any
  problem in the data file (like the checksum bug see comment of
  ma_blockrec.c).
  * Testing if log was written by UNDO phase (often expected),
  not written by REDO phase (always expected).
  * Less output on the screen, compares with expected output in the end.
  * some shell thingies like "set --" and $# are courtesy of
  Danny and Pekka.
storage/maria/maria_read_log.c:
  when only displaying the records, don't do an UNDO phase
storage/maria/ma_test_recovery.expected:
  This is the expected output of a great part of ma_test_recovery.
  ma_test_recovery compares its output to the expected output
  and tells if different.
  If we look at this file it mentions differences in checksum
  (normal, it's not recovered yet) and in records count
  (getting a correct records' count when recovery starts on an
  already existing table, like when testing rollback,
  is coded but not yet pushed).
2007-09-06 16:04:36 +02:00
unknown
044c4103ba Added undo of deleted row
Added part of undo of update row
Extended ma_test1 for recovery testing
Some bug fixes


storage/maria/ha_maria.cc:
  Ignore 'state.split' in case of block records
storage/maria/ma_bitmap.c:
  Added return value for _ma_bitmap_find_place() for how much data we should put on head page
storage/maria/ma_blockrec.c:
  Added undo of deleted row.
  - Added logging of CLR_END records in write_block_record()
  - Split ma_write_init_block_record() to two functions to get better code reuse
  - Added _ma_apply_undo_row_delete()
  - Added ma_get_length()
  
  Added 'empty' prototype for undo_row_update()
  
  Fixed bug when moving data withing a head/tail page.
  Fixed bug when reading a page with bigger LSN but of different type than was expected.
  Store undo_lsn first in CLR_END record
  
  Simplified some code by adding local variables.
  Changed log format for UNDO_ROW_DELETE to store total length of used blobs
storage/maria/ma_blockrec.h:
  Added prototypes for undo code.
storage/maria/ma_pagecache.c:
  Allow plain page to change to LSN page (needed in recovery to apply UNDO)
storage/maria/ma_recovery.c:
  Added undo handling of UNDO_ROW_DELETE and UNDO_ROW_UPDATE
storage/maria/ma_test1.c:
  Extended --test-undo option to allow us to die after insert or after delete.
  Fixed bug in printing key values when using -v
storage/maria/maria_def.h:
  Moved some variables around to be getter alignment
  Added length_buff buffer to be used during undo handling
2007-09-05 02:57:53 +03:00
unknown
03437ea043 Added UNDO handling of insert during recovery
storage/maria/ma_blockrec.c:
  Added UNDO handling of insert during recovery
  To do this, I also had to add write locking of tail pages during undo phase (As we need to access the same page twice if extents are split over two pages)
  Another way to handle the undo of insert would be to store the extent information as part of the UNDO_INSERT block.
storage/maria/ma_blockrec.h:
  Added new prototype
storage/maria/ma_loghandler.c:
  Changed type of CLR_END (to avoid crash in log handler)
  Removed not used variable
storage/maria/ma_loghandler.h:
  Added TRN argument to record_execute_in_undo_phase()
storage/maria/ma_pagecache.c:
  Hack for undo phase of recovery.  During REDO we work with PLAIN pages, but UNDO works with LSN pages, which caused an abort when trying to access a cached page.
storage/maria/ma_recovery.c:
  Added execution of UNDO_ROW_INSERT
storage/maria/ma_test1.c:
  Added option --test-undo for testing recovery with undo
storage/maria/maria_read_log.c:
  Added processing of undos
2007-09-04 10:53:52 +03:00
unknown
2cc2f3e757 Generalized the way update and redo extends the size of a directory record.
storage/maria/ma_blockrec.c:
  Generalized the way update and redo extends the size of a directory record.
  This will (for now) ensure that data files are idenitical after normal run and after a apply-log run.
storage/maria/ma_open.c:
  Disabled reservation of transid on rows (for now) as these are not yet used.
  (I had to disable this as otherwise update thougth rows had grown in size when they hadn't and we had thus different row sizes on update and redo, which caused different block information)
storage/maria/ma_test1.c:
  Added comment
storage/maria/ma_test2.c:
  Do commit on error/abort
storage/maria/ma_test_all.sh:
  Some more testing (to cover a bug that was not found in previous runs)
storage/maria/ma_test_recovery:
  More tests
2007-08-31 10:19:54 +03:00
unknown
f7b766c029 Added maria_commit() and maria_begin() to be used with external tests
Now ma_test1 -M -T and ma_test2 -M -T produces readable, applyable logs

Note: The .MAD file is not binary identical after applying redo compare to a an original file.
(This is becasue we don't have full information which function called PURGE_REDO_BLOCKS).
To verify if a file was correctly applied, we now instead compare row checksums


BitKeeper/etc/ignore:
  added storage/maria/tmp/*
include/maria.h:
  Added maria_commit() and maria_begin() to be used with external tests
storage/maria/ha_maria.cc:
  Ensure maria_def. is read in C mode
storage/maria/ma_blockrec.c:
  Fixed redo handling.
  _ma_apply_redo_purge_blocks() updated to handle any number of purged blocks
  Removed code to make data file idenitcal after redo (can't easily be done). See changeset comments
  
  Now ma_test1 -M -T and ma_test2 -M -T produces readable, applyable logs
storage/maria/ma_commit.c:
  More DBUG statements
  Moved variable declaration to start of function (portability fix)
  Added helper functions 'maria_commit()' and 'maria_begin()'
storage/maria/ma_loghandler.c:
  Fixed wrong REDO_PURGE_BLOCKS initialization
storage/maria/ma_recovery.c:
  Added UNDO_ROW_UPDATE
  Removed wrong setting of lsn (there was no lsn at the used position)
  Fixed REDO_PURGE_BLOCKS to handle any number of blocks
storage/maria/ma_test1.c:
  Added transaction support (via maria_begin() & maria_commit()) to get a log that can be applied with maria_read_log
storage/maria/ma_test2.c:
  Added transaction support (via maria_begin() & maria_commit()) to get a log that can be applied with maria_read_log
storage/maria/ma_test_recovery:
  Create temporary files in maria/tmp
  Verify files with checksums instead of byte comparisons
storage/maria/maria_chk.c:
  When using with -dss we only get  filename, records and checksum.
  This is useful to do a quick comparision if a files is identical to another one.
storage/maria/maria_def.h:
  Added ma_commit()
storage/maria/maria_read_log.c:
  Added --help
2007-08-29 09:03:10 +03:00
unknown
9b1e83dba7 porting Serg's fix for BUG#30094 to Maria. Now ma_test_all passes.
maria.test and ps_maria.test still fail.


mysys/mf_keycache.c:
  split string annoys some compilers
storage/maria/ha_maria.cc:
  fix for compiler warnings
storage/maria/ma_test1.c:
  porting Serg's fix for BUG#30094 to Maria
storage/maria/ma_test2.c:
  porting Serg's fix for BUG#30094 to Maria
storage/maria/ma_test3.c:
  porting Serg's fix for BUG#30094 to Maria
storage/maria/ma_test_recovery:
  don't print ma_test1's messages if no problem
2007-07-27 16:11:40 +02:00
unknown
8ba21c616e Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-maria
into  a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-maria.prod


BitKeeper/etc/ignore:
  auto-union
include/maria.h:
  Auto merged
include/my_base.h:
  Auto merged
mysql-test/include/ps_conv.inc:
  Auto merged
mysql-test/r/ps_2myisam.result:
  Auto merged
mysql-test/r/ps_3innodb.result:
  Auto merged
mysql-test/r/ps_4heap.result:
  Auto merged
mysql-test/r/ps_5merge.result:
  Auto merged
mysql-test/r/ps_7ndb.result:
  Auto merged
mysys/Makefile.am:
  Auto merged
mysys/mf_iocache.c:
  Auto merged
mysys/mf_tempfile.c:
  Auto merged
mysys/my_error.c:
  Auto merged
mysys/my_init.c:
  Auto merged
mysys/my_open.c:
  Auto merged
sql/handler.cc:
  Auto merged
sql/handler.h:
  Auto merged
sql/lex.h:
  Auto merged
sql/sql_show.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
sql/table.cc:
  Auto merged
sql/table.h:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ha_maria.h:
  Auto merged
storage/maria/ma_bitmap.c:
  Auto merged
storage/maria/ma_blockrec.h:
  Auto merged
storage/maria/ma_create.c:
  Auto merged
storage/maria/ma_dynrec.c:
  Auto merged
storage/maria/ma_loghandler.h:
  Auto merged
storage/maria/ma_loghandler_lsn.h:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/ma_pagecache.h:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/ma_test2.c:
  Auto merged
storage/maria/ma_update.c:
  Auto merged
storage/maria/maria_chk.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Auto merged
storage/myisam/ft_stopwords.c:
  Auto merged
storage/myisam/mi_close.c:
  Auto merged
support-files/compiler_warnings.supp:
  Auto merged
mysys/mf_keycache.c:
  Pulled from mysql-maria tree, manual merge.
mysys/my_handler.c:
  Pulled from mysql-maria tree, manual merge.
mysys/my_symlink2.c:
  Pulled from mysql-maria tree, manual merge.
sql/mysqld.cc:
  Pulled from mysql-maria tree, manual merge.
sql/sql_table.cc:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_blockrec.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_check.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_close.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_loghandler.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/maria_def.h:
  Pulled from mysql-maria tree, manual merge.
2007-07-04 12:20:53 +03:00
unknown
631ecaabea Merged with mysql-5.1 main tree.
BUILD/compile-pentium-debug-max:
  Added definition after macro was removed from main tree. This will
  be fixed back in main tree later.
2007-07-02 20:45:15 +03:00