mariadb/storage/maria/trnman_public.h

69 lines
2.6 KiB
C
Raw Normal View History

This patch is a collection of patches from from Sanja, Sergei and Monty. Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 19:13:56 +02:00
/* Copyright (C) 2006 MySQL AB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
/*
External definitions for trnman.h
We need to split this into two files as gcc 4.1.2 gives error if it tries
to include my_atomic.h in C++ code.
*/
WL#3072 Maria Recovery - new program maria_read_log to display and apply log records found in a Maria log (see file's revision comment) - minor, misc fixes storage/maria/Makefile.am: new program maria_read_log storage/maria/ha_maria.cc: create control file if missing storage/maria/ma_blockrec.c: 0 -> LSN_IMPOSSIBLE; comments storage/maria/ma_checkpoint.h: preparations for Checkpoint module storage/maria/ma_close.c: comment storage/maria/ma_control_file.c: renaming constants. Possibility to say "open control file but don't create it if it's missing" (used by maria_read_log which does not want to create anything) storage/maria/ma_control_file.h: renaming constants storage/maria/ma_create.c: I had duplicated "linkname" and "linkname_ptr", now I see it's not needed, reverting. Indeed those variables don't contain interesting information; fixing log record accordingly (the links are in ci->data/index_file_name). Storing keystart in log record is needed, to know at which size we must extend the file if we replay LOGREC_CREATE_TABLE. storage/maria/ma_loghandler.c: some structures need to be known to maria_read_log.c, taking them to ma_loghandler.h storage/maria/ma_loghandler.h: we have page_store, adding page_korr. translog_lock() made public, because Checkpoint will need it (to write to control file). Some structures moved from ma_loghandler.c because maria_read_log.c needs them (needs to know the execute-in-REDO-phase hooks of each record). storage/maria/ma_loghandler_lsn.h: constants defined in ma_control_file.h serve everywhere, and they relate to LSNs, so putting them in ma_loghandler_lsn.h. Stronger constraints in LSN_VALID(). storage/maria/ma_pagecache.c: renaming constants storage/maria/ma_recovery.h: copyright storage/maria/ma_test1.c: new prototype storage/maria/ma_test2.c: new prototype storage/maria/trnman_public.h: double-inclusion safe storage/maria/unittest/ma_control_file-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: constants renamed, new prototype storage/myisam/mi_close.c: comment storage/maria/maria_read_log.c: program to read and print log records from a Maria transaction log, and optionally apply them to tables. Very basic, early version. Should serve as a base for Recovery's code. Designed to be idempotent. Create a log by running maria.test, then cd to var/master-data and run "maria_read_log --only-display" to see info about records; run "maria_read_log --display-and-apply" to also apply the records to tables (it's more interesting if you first wipe out the tables in var/master-data/test, to see how they get re-created). Only a few records are handled by now: LONG_TRANSACTION_ID, COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for REDO_INSERT_ROW_HEAD where I could use Monty's help (search for "Monty" in the file). Note: changes to the index pages, index's header and bitmap pages are not properly logged yet, so don't expect the program to work with that.
2007-06-26 16:49:23 +02:00
#ifndef _trnman_public_h
#define _trnman_public_h
- WL#3239 "log CREATE TABLE in Maria" - WL#3240 "log DROP TABLE in Maria" - similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and DELETE no_WHERE_clause (== the DELETE which just truncates the files) - create_rename_lsn added to MARIA_SHARE's state - all these operations (except DROP TABLE) also update the table's create_rename_lsn, which is needed for the correctness of Recovery (see function comment of _ma_repair_write_log_record() in ma_check.c) - write a COMMIT record when transaction commits. - don't log REDOs/UNDOs if this is an internal temporary table like inside ALTER TABLE (I expect this to be a big win). There was already no logging for user-created "CREATE TEMPORARY" tables. - don't fsync files/directories if the table is not transactional - in translog_write_record(), autogenerate a 2-byte-id for the table and log the "id->name" pair (LOGREC_FILE_ID); log LOGREC_LONG_TRANSACTION_ID; automatically store the table's 2-byte-id in any log record. - preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint when some dirty pages are unknown; capturing trn->rec_lsn, trn->first_undo_lsn for Checkpoint and log's low-water-mark computing. - assertions, comments. storage/maria/Makefile.am: more files to build storage/maria/ha_maria.cc: - logging a REPAIR log record if REPAIR/OPTIMIZE was successful. - ha_maria::data_file_type does not have to be set in every info() call, just do it once in open(). - if caller said that transactionality can be disabled (like if caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we temporarily disable transactionality of the table in external_lock(); that will ensure that no REDOs/UNDOs are logged for this possibly massive write operation (they are not needed, as if any write fails, the table will be dropped). We re-enable in external_lock(F_UNLCK), which in ALTER TABLE happens before the tmp table replaces the original one (which is good, as thus the final table will have a REDO RENAME and a correct create_rename_lsn). - when we commit we also have to write a log record, so trnman_commit_trn() calls become ma_commit() calls - at end of engine's initialization, we are potentially entering a multi-threaded dangerous world (clients are going to be accepted) and so some assertions of mutex-owning become enforceable, for that we set maria_multi_threaded=TRUE (see ma_control_file.c) storage/maria/ha_maria.h: new member ha_maria::save_transactional (see also ha_maria.cc) storage/maria/ma_blockrec.c: - fixing comments according to discussion with Monty - if a table is transactional but temporarily non-transactional (like in ALTER TABLE), we need to give a sensible LSN to the pages (and, if we give 0, pagecache asserts). - translog_write_record() now takes care of storing the share's 2-byte-id in the log record storage/maria/ma_blockrec.h: fixing comment according to discussion with Monty storage/maria/ma_check.c: When REPAIR/OPTIMIZE modify the data/index file, if this is a transactional table, they must sync it; if they remove files or rename files, they must sync the directory, so that everything is durable. This is just applying to REPAIR/OPTIMIZE the logic already implemented in CREATE/DROP/RENAME a few months ago. Adding a function to write a LOGREC_REPAIR_TABLE at end of REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and to update the table's create_rename_lsn. storage/maria/ma_close.c: fix for a future bug storage/maria/ma_control_file.c: ensuring that if Maria is running in multi-threaded mode, anybody wanting to write to the control file and update last_checkpoint_lsn/last_logno owns the log's lock. storage/maria/ma_control_file.h: see ma_control_file.c storage/maria/ma_create.c: when creating a table: - sync it and its directory only if this is a transactional table and there is a log (no point in syncing in maria_chk) - decouple the two uses of linkname/linkname_ptr (for index file and for data file) into more variables, as we need to know all links until the moment we write the LOGREC_CREATE_TABLE. - set share.data_file_type early so that _ma_initialize_data_file() knows it (Monty's bugfix so that a table always has at least a bitmap page when it is created; so data-file is not 0 bytes anymore). - log a LOGREC_CREATE_TABLE; it contains the bytes which we have just written to the index file's header. Update table's create_rename_lsn. - syncing of kfile had been bugified in a previous merge, correcting - syncing of dfile is now needed as it's not empty anymore - in _ma_initialize_data_file(), use share's block_size and not the global one. This is a gratuitous change, both variables are equal, just that I find it more future-proof to use share-bound variable rather than global one. storage/maria/ma_delete_all.c: log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows(); update create_rename_lsn then. storage/maria/ma_delete_table.c: - logging LOGREC_DROP_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - we need to sync directories only if the table is transactional storage/maria/ma_extra.c: questions storage/maria/ma_init.c: when maria_end() is called, engine is not multithreaded storage/maria/ma_loghandler.c: - translog_inited has to be visible to ma_create() (see how it is used in ma_create()) - checkpoint record will be a single record, not three - no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will log a REDO_CREATE) - adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by truncating the files), REPAIR. - MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk - in translog_write_record(), if MARIA_SHARE does not yet have a 2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically store this short id into log records. - in translog_write_record(), if transaction has not logged its long trid, log LOGREC_LONG_TRANSACTION_ID. - For Checkpoint, we need to know the current end-of-log: adding translog_get_horizon(). - For Control File, adding an assertion that the thread owns the log's lock (control file is protected by this lock) storage/maria/ma_loghandler.h: Changes in log records (see ma_loghandler.c). new prototypes, new functions. storage/maria/ma_loghandler_lsn.h: adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn, where the most significant byte is used for flags. storage/maria/ma_open.c: storing the create_rename_lsn in the index file's header (in the state, precisely) and retrieving it from there. storage/maria/ma_pagecache.c: - my set_if_bigger was wrong, correcting it - if the first_in_switch list is not empty, it means that changed_blocks misses some dirty pages, so Checkpoint cannot run and needs to wait. A variable missing_blocks_in_changed_list is added to tell that (should it be named missing_blocks_in_changed_blocks?) - pagecache_collect_changed_blocks_with_lsn() now also tells the minimum rec_lsn (needed for low-water mark computation). storage/maria/ma_pagecache.h: see ma_pagecache.c storage/maria/ma_panic.c: comment storage/maria/ma_range.c: comment storage/maria/ma_rename.c: - logging LOGREC_RENAME_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - update create_rename_lsn - we need to sync directories only if the table is transactional storage/maria/ma_static.c: comment storage/maria/ma_test_all.sh: - tip for Valgrind-ing ma_test_all - do "export maria_path=somepath" before calling ma_test_all, if you want to run ma_test_all out of storage/maria (useful to have parallel runs, like one normal and one Valgrind, they must not use the same tables so need to run in different directories) storage/maria/maria_def.h: - state now contains, in memory and on disk, the create_rename_lsn - share now contains a 2-byte-id storage/maria/trnman.c: preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn; minimum first_undo_lsn needed to know log's low-water-mark storage/maria/trnman.h: using most significant byte of first_undo_lsn to hold miscellaneous flags, for now TRANSACTION_LOGGED_LONG_ID. dummy_transaction_object is already declared in ma_static.c. storage/maria/trnman_public.h: dummy_transaction_object was declared in all files including trnman_public.h, while in fact it's a single object. new prototype storage/maria/unittest/ma_test_loghandler-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: update for new prototype storage/maria/ma_commit.c: function which wraps: - writing a LOGREC_COMMIT record (==commit on disk) - calling trnman_commit_trn() (=commit in memory) storage/maria/ma_commit.h: new header file .tree-is-private: this file is now needed to keep our tree private (don't push it to public trees). When 5.1 is merged into mysql-maria, we can abandon our maria-specific post-commit trigger; .tree_is_private will take care of keeping commit mails private. Don't push this file to public trees.
2007-06-22 14:49:37 +02:00
#include "ma_loghandler_lsn.h"
This patch is a collection of patches from from Sanja, Sergei and Monty. Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 19:13:56 +02:00
C_MODE_START
typedef uint64 TrID; /* our TrID is 6 bytes */
typedef struct st_transaction TRN;
#define SHORT_TRID_MAX 65535
extern uint trnman_active_transactions, trnman_allocated_transactions;
- WL#3239 "log CREATE TABLE in Maria" - WL#3240 "log DROP TABLE in Maria" - similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and DELETE no_WHERE_clause (== the DELETE which just truncates the files) - create_rename_lsn added to MARIA_SHARE's state - all these operations (except DROP TABLE) also update the table's create_rename_lsn, which is needed for the correctness of Recovery (see function comment of _ma_repair_write_log_record() in ma_check.c) - write a COMMIT record when transaction commits. - don't log REDOs/UNDOs if this is an internal temporary table like inside ALTER TABLE (I expect this to be a big win). There was already no logging for user-created "CREATE TEMPORARY" tables. - don't fsync files/directories if the table is not transactional - in translog_write_record(), autogenerate a 2-byte-id for the table and log the "id->name" pair (LOGREC_FILE_ID); log LOGREC_LONG_TRANSACTION_ID; automatically store the table's 2-byte-id in any log record. - preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint when some dirty pages are unknown; capturing trn->rec_lsn, trn->first_undo_lsn for Checkpoint and log's low-water-mark computing. - assertions, comments. storage/maria/Makefile.am: more files to build storage/maria/ha_maria.cc: - logging a REPAIR log record if REPAIR/OPTIMIZE was successful. - ha_maria::data_file_type does not have to be set in every info() call, just do it once in open(). - if caller said that transactionality can be disabled (like if caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we temporarily disable transactionality of the table in external_lock(); that will ensure that no REDOs/UNDOs are logged for this possibly massive write operation (they are not needed, as if any write fails, the table will be dropped). We re-enable in external_lock(F_UNLCK), which in ALTER TABLE happens before the tmp table replaces the original one (which is good, as thus the final table will have a REDO RENAME and a correct create_rename_lsn). - when we commit we also have to write a log record, so trnman_commit_trn() calls become ma_commit() calls - at end of engine's initialization, we are potentially entering a multi-threaded dangerous world (clients are going to be accepted) and so some assertions of mutex-owning become enforceable, for that we set maria_multi_threaded=TRUE (see ma_control_file.c) storage/maria/ha_maria.h: new member ha_maria::save_transactional (see also ha_maria.cc) storage/maria/ma_blockrec.c: - fixing comments according to discussion with Monty - if a table is transactional but temporarily non-transactional (like in ALTER TABLE), we need to give a sensible LSN to the pages (and, if we give 0, pagecache asserts). - translog_write_record() now takes care of storing the share's 2-byte-id in the log record storage/maria/ma_blockrec.h: fixing comment according to discussion with Monty storage/maria/ma_check.c: When REPAIR/OPTIMIZE modify the data/index file, if this is a transactional table, they must sync it; if they remove files or rename files, they must sync the directory, so that everything is durable. This is just applying to REPAIR/OPTIMIZE the logic already implemented in CREATE/DROP/RENAME a few months ago. Adding a function to write a LOGREC_REPAIR_TABLE at end of REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and to update the table's create_rename_lsn. storage/maria/ma_close.c: fix for a future bug storage/maria/ma_control_file.c: ensuring that if Maria is running in multi-threaded mode, anybody wanting to write to the control file and update last_checkpoint_lsn/last_logno owns the log's lock. storage/maria/ma_control_file.h: see ma_control_file.c storage/maria/ma_create.c: when creating a table: - sync it and its directory only if this is a transactional table and there is a log (no point in syncing in maria_chk) - decouple the two uses of linkname/linkname_ptr (for index file and for data file) into more variables, as we need to know all links until the moment we write the LOGREC_CREATE_TABLE. - set share.data_file_type early so that _ma_initialize_data_file() knows it (Monty's bugfix so that a table always has at least a bitmap page when it is created; so data-file is not 0 bytes anymore). - log a LOGREC_CREATE_TABLE; it contains the bytes which we have just written to the index file's header. Update table's create_rename_lsn. - syncing of kfile had been bugified in a previous merge, correcting - syncing of dfile is now needed as it's not empty anymore - in _ma_initialize_data_file(), use share's block_size and not the global one. This is a gratuitous change, both variables are equal, just that I find it more future-proof to use share-bound variable rather than global one. storage/maria/ma_delete_all.c: log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows(); update create_rename_lsn then. storage/maria/ma_delete_table.c: - logging LOGREC_DROP_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - we need to sync directories only if the table is transactional storage/maria/ma_extra.c: questions storage/maria/ma_init.c: when maria_end() is called, engine is not multithreaded storage/maria/ma_loghandler.c: - translog_inited has to be visible to ma_create() (see how it is used in ma_create()) - checkpoint record will be a single record, not three - no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will log a REDO_CREATE) - adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by truncating the files), REPAIR. - MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk - in translog_write_record(), if MARIA_SHARE does not yet have a 2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically store this short id into log records. - in translog_write_record(), if transaction has not logged its long trid, log LOGREC_LONG_TRANSACTION_ID. - For Checkpoint, we need to know the current end-of-log: adding translog_get_horizon(). - For Control File, adding an assertion that the thread owns the log's lock (control file is protected by this lock) storage/maria/ma_loghandler.h: Changes in log records (see ma_loghandler.c). new prototypes, new functions. storage/maria/ma_loghandler_lsn.h: adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn, where the most significant byte is used for flags. storage/maria/ma_open.c: storing the create_rename_lsn in the index file's header (in the state, precisely) and retrieving it from there. storage/maria/ma_pagecache.c: - my set_if_bigger was wrong, correcting it - if the first_in_switch list is not empty, it means that changed_blocks misses some dirty pages, so Checkpoint cannot run and needs to wait. A variable missing_blocks_in_changed_list is added to tell that (should it be named missing_blocks_in_changed_blocks?) - pagecache_collect_changed_blocks_with_lsn() now also tells the minimum rec_lsn (needed for low-water mark computation). storage/maria/ma_pagecache.h: see ma_pagecache.c storage/maria/ma_panic.c: comment storage/maria/ma_range.c: comment storage/maria/ma_rename.c: - logging LOGREC_RENAME_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - update create_rename_lsn - we need to sync directories only if the table is transactional storage/maria/ma_static.c: comment storage/maria/ma_test_all.sh: - tip for Valgrind-ing ma_test_all - do "export maria_path=somepath" before calling ma_test_all, if you want to run ma_test_all out of storage/maria (useful to have parallel runs, like one normal and one Valgrind, they must not use the same tables so need to run in different directories) storage/maria/maria_def.h: - state now contains, in memory and on disk, the create_rename_lsn - share now contains a 2-byte-id storage/maria/trnman.c: preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn; minimum first_undo_lsn needed to know log's low-water-mark storage/maria/trnman.h: using most significant byte of first_undo_lsn to hold miscellaneous flags, for now TRANSACTION_LOGGED_LONG_ID. dummy_transaction_object is already declared in ma_static.c. storage/maria/trnman_public.h: dummy_transaction_object was declared in all files including trnman_public.h, while in fact it's a single object. new prototype storage/maria/unittest/ma_test_loghandler-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: update for new prototype storage/maria/ma_commit.c: function which wraps: - writing a LOGREC_COMMIT record (==commit on disk) - calling trnman_commit_trn() (=commit in memory) storage/maria/ma_commit.h: new header file .tree-is-private: this file is now needed to keep our tree private (don't push it to public trees). When 5.1 is merged into mysql-maria, we can abandon our maria-specific post-commit trigger; .tree_is_private will take care of keeping commit mails private. Don't push this file to public trees.
2007-06-22 14:49:37 +02:00
extern TRN dummy_transaction_object;
WL#3138: Maria - fast "SELECT COUNT(*) FROM t;" and "CHECKSUM TABLE t" Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation Fixed wrong call to strmake Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert Allow storing year 2155 in year field When running with purify/valgrind avoid copying structures over themself Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction Fixed that ndb doesn't crash on duplicate key error when start_bulk_insert/end_bulk_insert are not called include/maria.h: Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation include/my_tree.h: Added macro 'reset_free_element()' to be able to ignore calls to the external free function. Is used to optimize end-bulk-insert in case of failures, in which case we don't want write the remaining keys in the tree mysql-test/install_test_db.sh: Upgrade to new mysql_install_db options mysql-test/r/maria-mvcc.result: New tests mysql-test/r/maria.result: New tests mysql-test/suite/ndb/r/ndb_auto_increment.result: Fixed error message now when bulk insert is not always called mysql-test/suite/ndb/t/ndb_auto_increment.test: Fixed error message now when bulk insert is not always called mysql-test/t/maria-mvcc.test: Added testing of versioning of count(*) mysql-test/t/maria-page-checksum.test: Added comment mysql-test/t/maria.test: More tests mysys/hash.c: Code style change sql/field.cc: Allow storing year 2155 in year field sql/ha_ndbcluster.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_ndbcluster.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/handler.cc: Don't call get_dup_key() if there is no table object. This can happen if the handler generates a duplicate key error on commit sql/handler.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored (ie, the table will be deleted) sql/item.cc: Style fix Removed compiler warning sql/log_event.cc: Added new argument to ha_end_bulk_insert() sql/log_event_old.cc: Added new argument to ha_end_bulk_insert() sql/mysqld.cc: Removed compiler warning sql/protocol.cc: Added DBUG sql/sql_class.cc: Added DBUG Fixed wrong call to strmake sql/sql_insert.cc: Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert involves a lot of if's) Added new argument to ha_end_bulk_insert() sql/sql_load.cc: Added new argument to ha_end_bulk_insert() sql/sql_parse.cc: Style fixes Avoid goto in common senario sql/sql_select.cc: When running with purify/valgrind avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_select.h: Avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_table.cc: Call HA_EXTRA_PREPARE_FOR_DROP if table created by ALTER TABLE is going to be dropped Added new argument to ha_end_bulk_insert() storage/archive/ha_archive.cc: Added new argument to end_bulk_insert() storage/archive/ha_archive.h: Added new argument to end_bulk_insert() storage/federated/ha_federated.cc: Added new argument to end_bulk_insert() storage/federated/ha_federated.h: Added new argument to end_bulk_insert() storage/maria/Makefile.am: Added ma_state.c and ma_state.h storage/maria/ha_maria.cc: Versioning of count(*) and checksum - share->state.state is now assumed to be correct, not handler->state - Call _ma_setup_live_state() in external lock to get count(*)/checksum versioning. In case of not versioned and not concurrent insertable table, file->s->state.state contains the correct state information Other things: - file->s -> share - Added DBUG_ASSERT() for unlikely case - Optimized end_bulk_insert() to not write anything if table is going to be deleted (as in failed alter table) - Indentation changes in external_lock becasue of removed 'goto' caused a big conflict even if very little was changed storage/maria/ha_maria.h: New argument to end_bulk_insert() storage/maria/ma_blockrec.c: Update for versioning of count(*) and checksum Keep share->state.state.data_file_length up to date (not info->state->data_file_length) Moved _ma_block_xxxx_status() and maria_versioning() functions to ma_state.c storage/maria/ma_check.c: Update and use share->state.state instead of info->state info->s to share Update info->state at end of repair Call _ma_reset_state() to update share->state_history at end of repair storage/maria/ma_checkpoint.c: Call _ma_remove_not_visible_states() on checkpoint to clean up not visible state history from tables storage/maria/ma_close.c: Remember state history for running transaction even if table is closed storage/maria/ma_commit.c: Ensure we always call trnman_commit_trn() even if other calls fails. If we don't do that, the translog and state structures will not be freed storage/maria/ma_delete.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/ma_delete_all.c: Versioning of count(*) and checksum: - Ensure that share->state.state is updated, as here is where we store the primary information storage/maria/ma_dynrec.c: Use lock_key_trees instead of concurrent_insert to check if trees should be locked. This allows us to lock trees both for concurrent_insert and for index versioning. storage/maria/ma_extra.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - share->concurrent_insert -> share->non_transactional_concurrent_insert - Don't update share->state.state from info->state if transactional table Optimization: - Don't flush io_cache or bitmap if we are using FLUSH_IGNORE_CHANGED storage/maria/ma_info.c: Get most state information from current state storage/maria/ma_init.c: Add hash table and free function to store states for closed tables Install hook for transaction commit/rollback to update history state storage/maria/ma_key_recover.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state storage/maria/ma_locking.c: Versioning of count(*) and checksum: - Call virtual functions (if exists) to restore/update status - Move _ma_xxx_status() functions to ma_state.c info->s -> share storage/maria/ma_open.c: Versioning of count(*) and checksum: - For not transactional tables, set info->state to point to new allocated state structure. - Initialize new info->state_start variable that points to state at start of transaction - Copy old history states from hash table (maria_stored_states) first time the table is opened - Split flag share->concurrent_insert to non_transactional_concurrent_insert & lock_key_tree - For now, only enable versioning of tables without keys (to be fixed in soon!) - Added new virtual function to restore status in maria_lock_database) More DBUG storage/maria/ma_page.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Modify share->state.state.key_file_length under share->intern_lock storage/maria/ma_range.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees info->s -> share storage/maria/ma_recovery.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Update state information on close and when reenabling logging storage/maria/ma_rkey.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext_same.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees - Only skip rows based on file length if non_transactional_concurrent_insert is set storage/maria/ma_rprev.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rsame.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_sort.c: Use share->state.state instead of info->state Fixed indentation storage/maria/ma_static.c: Added maria_stored_state storage/maria/ma_update.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records - Remove optimization for index file update as it doesn't work for transactional tables storage/maria/ma_write.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/maria_def.h: Move MARIA_STATUS_INFO to ma_state.h Changes to MARIA_SHARE: - Added state_history to store count(*)/checksum states - Added in_trans as counter if table is used by running transactions - Split concurrent_insert into lock_key_trees and on_transactional_concurrent_insert. - Added virtual function lock_restore_status Changes to MARIA_HA: - save_state -> state_save - Added state_start to store state at start of transaction storage/maria/maria_pack.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state Indentation fixes storage/maria/trnman.c: Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction More DBUG Changed return type of trnman_end_trn() to my_bool Added trnman_get_min_trid() to get minimum trid in use. Added trnman_exists_active_transactions() to check if there exist a running transaction started between two commit id storage/maria/trnman.h: Added 'used_tables' Moved all pointers into same groups to get better memory alignment storage/maria/trnman_public.h: Added prototypes for new functions and variables Chagned return type of trnman_end_trn() to my_bool storage/myisam/ha_myisam.cc: Added argument to end_bulk_insert() if operation should be aborted storage/myisam/ha_myisam.h: Added argument to end_bulk_insert() if operation should be aborted storage/maria/ma_state.c: Functions to handle state of count(*) and checksum storage/maria/ma_state.h: Structures and declarations to handle state of count(*) and checksum
2008-05-29 17:33:33 +02:00
extern my_bool (*trnman_end_trans_hook)(TRN *trn, my_bool commit,
my_bool active_transactions);
This patch is a collection of patches from from Sanja, Sergei and Monty. Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 19:13:56 +02:00
int trnman_init(TrID);
This patch is a collection of patches from from Sanja, Sergei and Monty. Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 19:13:56 +02:00
void trnman_destroy(void);
TRN *trnman_new_trn(pthread_mutex_t *, pthread_cond_t *, void *);
WL#3138: Maria - fast "SELECT COUNT(*) FROM t;" and "CHECKSUM TABLE t" Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation Fixed wrong call to strmake Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert Allow storing year 2155 in year field When running with purify/valgrind avoid copying structures over themself Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction Fixed that ndb doesn't crash on duplicate key error when start_bulk_insert/end_bulk_insert are not called include/maria.h: Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation include/my_tree.h: Added macro 'reset_free_element()' to be able to ignore calls to the external free function. Is used to optimize end-bulk-insert in case of failures, in which case we don't want write the remaining keys in the tree mysql-test/install_test_db.sh: Upgrade to new mysql_install_db options mysql-test/r/maria-mvcc.result: New tests mysql-test/r/maria.result: New tests mysql-test/suite/ndb/r/ndb_auto_increment.result: Fixed error message now when bulk insert is not always called mysql-test/suite/ndb/t/ndb_auto_increment.test: Fixed error message now when bulk insert is not always called mysql-test/t/maria-mvcc.test: Added testing of versioning of count(*) mysql-test/t/maria-page-checksum.test: Added comment mysql-test/t/maria.test: More tests mysys/hash.c: Code style change sql/field.cc: Allow storing year 2155 in year field sql/ha_ndbcluster.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_ndbcluster.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/handler.cc: Don't call get_dup_key() if there is no table object. This can happen if the handler generates a duplicate key error on commit sql/handler.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored (ie, the table will be deleted) sql/item.cc: Style fix Removed compiler warning sql/log_event.cc: Added new argument to ha_end_bulk_insert() sql/log_event_old.cc: Added new argument to ha_end_bulk_insert() sql/mysqld.cc: Removed compiler warning sql/protocol.cc: Added DBUG sql/sql_class.cc: Added DBUG Fixed wrong call to strmake sql/sql_insert.cc: Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert involves a lot of if's) Added new argument to ha_end_bulk_insert() sql/sql_load.cc: Added new argument to ha_end_bulk_insert() sql/sql_parse.cc: Style fixes Avoid goto in common senario sql/sql_select.cc: When running with purify/valgrind avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_select.h: Avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_table.cc: Call HA_EXTRA_PREPARE_FOR_DROP if table created by ALTER TABLE is going to be dropped Added new argument to ha_end_bulk_insert() storage/archive/ha_archive.cc: Added new argument to end_bulk_insert() storage/archive/ha_archive.h: Added new argument to end_bulk_insert() storage/federated/ha_federated.cc: Added new argument to end_bulk_insert() storage/federated/ha_federated.h: Added new argument to end_bulk_insert() storage/maria/Makefile.am: Added ma_state.c and ma_state.h storage/maria/ha_maria.cc: Versioning of count(*) and checksum - share->state.state is now assumed to be correct, not handler->state - Call _ma_setup_live_state() in external lock to get count(*)/checksum versioning. In case of not versioned and not concurrent insertable table, file->s->state.state contains the correct state information Other things: - file->s -> share - Added DBUG_ASSERT() for unlikely case - Optimized end_bulk_insert() to not write anything if table is going to be deleted (as in failed alter table) - Indentation changes in external_lock becasue of removed 'goto' caused a big conflict even if very little was changed storage/maria/ha_maria.h: New argument to end_bulk_insert() storage/maria/ma_blockrec.c: Update for versioning of count(*) and checksum Keep share->state.state.data_file_length up to date (not info->state->data_file_length) Moved _ma_block_xxxx_status() and maria_versioning() functions to ma_state.c storage/maria/ma_check.c: Update and use share->state.state instead of info->state info->s to share Update info->state at end of repair Call _ma_reset_state() to update share->state_history at end of repair storage/maria/ma_checkpoint.c: Call _ma_remove_not_visible_states() on checkpoint to clean up not visible state history from tables storage/maria/ma_close.c: Remember state history for running transaction even if table is closed storage/maria/ma_commit.c: Ensure we always call trnman_commit_trn() even if other calls fails. If we don't do that, the translog and state structures will not be freed storage/maria/ma_delete.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/ma_delete_all.c: Versioning of count(*) and checksum: - Ensure that share->state.state is updated, as here is where we store the primary information storage/maria/ma_dynrec.c: Use lock_key_trees instead of concurrent_insert to check if trees should be locked. This allows us to lock trees both for concurrent_insert and for index versioning. storage/maria/ma_extra.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - share->concurrent_insert -> share->non_transactional_concurrent_insert - Don't update share->state.state from info->state if transactional table Optimization: - Don't flush io_cache or bitmap if we are using FLUSH_IGNORE_CHANGED storage/maria/ma_info.c: Get most state information from current state storage/maria/ma_init.c: Add hash table and free function to store states for closed tables Install hook for transaction commit/rollback to update history state storage/maria/ma_key_recover.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state storage/maria/ma_locking.c: Versioning of count(*) and checksum: - Call virtual functions (if exists) to restore/update status - Move _ma_xxx_status() functions to ma_state.c info->s -> share storage/maria/ma_open.c: Versioning of count(*) and checksum: - For not transactional tables, set info->state to point to new allocated state structure. - Initialize new info->state_start variable that points to state at start of transaction - Copy old history states from hash table (maria_stored_states) first time the table is opened - Split flag share->concurrent_insert to non_transactional_concurrent_insert & lock_key_tree - For now, only enable versioning of tables without keys (to be fixed in soon!) - Added new virtual function to restore status in maria_lock_database) More DBUG storage/maria/ma_page.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Modify share->state.state.key_file_length under share->intern_lock storage/maria/ma_range.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees info->s -> share storage/maria/ma_recovery.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Update state information on close and when reenabling logging storage/maria/ma_rkey.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext_same.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees - Only skip rows based on file length if non_transactional_concurrent_insert is set storage/maria/ma_rprev.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rsame.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_sort.c: Use share->state.state instead of info->state Fixed indentation storage/maria/ma_static.c: Added maria_stored_state storage/maria/ma_update.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records - Remove optimization for index file update as it doesn't work for transactional tables storage/maria/ma_write.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/maria_def.h: Move MARIA_STATUS_INFO to ma_state.h Changes to MARIA_SHARE: - Added state_history to store count(*)/checksum states - Added in_trans as counter if table is used by running transactions - Split concurrent_insert into lock_key_trees and on_transactional_concurrent_insert. - Added virtual function lock_restore_status Changes to MARIA_HA: - save_state -> state_save - Added state_start to store state at start of transaction storage/maria/maria_pack.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state Indentation fixes storage/maria/trnman.c: Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction More DBUG Changed return type of trnman_end_trn() to my_bool Added trnman_get_min_trid() to get minimum trid in use. Added trnman_exists_active_transactions() to check if there exist a running transaction started between two commit id storage/maria/trnman.h: Added 'used_tables' Moved all pointers into same groups to get better memory alignment storage/maria/trnman_public.h: Added prototypes for new functions and variables Chagned return type of trnman_end_trn() to my_bool storage/myisam/ha_myisam.cc: Added argument to end_bulk_insert() if operation should be aborted storage/myisam/ha_myisam.h: Added argument to end_bulk_insert() if operation should be aborted storage/maria/ma_state.c: Functions to handle state of count(*) and checksum storage/maria/ma_state.h: Structures and declarations to handle state of count(*) and checksum
2008-05-29 17:33:33 +02:00
my_bool trnman_end_trn(TRN *trn, my_bool commit);
This patch is a collection of patches from from Sanja, Sergei and Monty. Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 19:13:56 +02:00
#define trnman_commit_trn(T) trnman_end_trn(T, TRUE)
#define trnman_abort_trn(T) trnman_end_trn(T, FALSE)
#define trnman_rollback_trn(T) trnman_end_trn(T, FALSE)
void trnman_free_trn(TRN *trn);
int trnman_can_read_from(TRN *trn, TrID trid);
void trnman_new_statement(TRN *trn);
void trnman_rollback_statement(TRN *trn);
- WL#3239 "log CREATE TABLE in Maria" - WL#3240 "log DROP TABLE in Maria" - similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and DELETE no_WHERE_clause (== the DELETE which just truncates the files) - create_rename_lsn added to MARIA_SHARE's state - all these operations (except DROP TABLE) also update the table's create_rename_lsn, which is needed for the correctness of Recovery (see function comment of _ma_repair_write_log_record() in ma_check.c) - write a COMMIT record when transaction commits. - don't log REDOs/UNDOs if this is an internal temporary table like inside ALTER TABLE (I expect this to be a big win). There was already no logging for user-created "CREATE TEMPORARY" tables. - don't fsync files/directories if the table is not transactional - in translog_write_record(), autogenerate a 2-byte-id for the table and log the "id->name" pair (LOGREC_FILE_ID); log LOGREC_LONG_TRANSACTION_ID; automatically store the table's 2-byte-id in any log record. - preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint when some dirty pages are unknown; capturing trn->rec_lsn, trn->first_undo_lsn for Checkpoint and log's low-water-mark computing. - assertions, comments. storage/maria/Makefile.am: more files to build storage/maria/ha_maria.cc: - logging a REPAIR log record if REPAIR/OPTIMIZE was successful. - ha_maria::data_file_type does not have to be set in every info() call, just do it once in open(). - if caller said that transactionality can be disabled (like if caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we temporarily disable transactionality of the table in external_lock(); that will ensure that no REDOs/UNDOs are logged for this possibly massive write operation (they are not needed, as if any write fails, the table will be dropped). We re-enable in external_lock(F_UNLCK), which in ALTER TABLE happens before the tmp table replaces the original one (which is good, as thus the final table will have a REDO RENAME and a correct create_rename_lsn). - when we commit we also have to write a log record, so trnman_commit_trn() calls become ma_commit() calls - at end of engine's initialization, we are potentially entering a multi-threaded dangerous world (clients are going to be accepted) and so some assertions of mutex-owning become enforceable, for that we set maria_multi_threaded=TRUE (see ma_control_file.c) storage/maria/ha_maria.h: new member ha_maria::save_transactional (see also ha_maria.cc) storage/maria/ma_blockrec.c: - fixing comments according to discussion with Monty - if a table is transactional but temporarily non-transactional (like in ALTER TABLE), we need to give a sensible LSN to the pages (and, if we give 0, pagecache asserts). - translog_write_record() now takes care of storing the share's 2-byte-id in the log record storage/maria/ma_blockrec.h: fixing comment according to discussion with Monty storage/maria/ma_check.c: When REPAIR/OPTIMIZE modify the data/index file, if this is a transactional table, they must sync it; if they remove files or rename files, they must sync the directory, so that everything is durable. This is just applying to REPAIR/OPTIMIZE the logic already implemented in CREATE/DROP/RENAME a few months ago. Adding a function to write a LOGREC_REPAIR_TABLE at end of REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and to update the table's create_rename_lsn. storage/maria/ma_close.c: fix for a future bug storage/maria/ma_control_file.c: ensuring that if Maria is running in multi-threaded mode, anybody wanting to write to the control file and update last_checkpoint_lsn/last_logno owns the log's lock. storage/maria/ma_control_file.h: see ma_control_file.c storage/maria/ma_create.c: when creating a table: - sync it and its directory only if this is a transactional table and there is a log (no point in syncing in maria_chk) - decouple the two uses of linkname/linkname_ptr (for index file and for data file) into more variables, as we need to know all links until the moment we write the LOGREC_CREATE_TABLE. - set share.data_file_type early so that _ma_initialize_data_file() knows it (Monty's bugfix so that a table always has at least a bitmap page when it is created; so data-file is not 0 bytes anymore). - log a LOGREC_CREATE_TABLE; it contains the bytes which we have just written to the index file's header. Update table's create_rename_lsn. - syncing of kfile had been bugified in a previous merge, correcting - syncing of dfile is now needed as it's not empty anymore - in _ma_initialize_data_file(), use share's block_size and not the global one. This is a gratuitous change, both variables are equal, just that I find it more future-proof to use share-bound variable rather than global one. storage/maria/ma_delete_all.c: log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows(); update create_rename_lsn then. storage/maria/ma_delete_table.c: - logging LOGREC_DROP_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - we need to sync directories only if the table is transactional storage/maria/ma_extra.c: questions storage/maria/ma_init.c: when maria_end() is called, engine is not multithreaded storage/maria/ma_loghandler.c: - translog_inited has to be visible to ma_create() (see how it is used in ma_create()) - checkpoint record will be a single record, not three - no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will log a REDO_CREATE) - adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by truncating the files), REPAIR. - MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk - in translog_write_record(), if MARIA_SHARE does not yet have a 2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically store this short id into log records. - in translog_write_record(), if transaction has not logged its long trid, log LOGREC_LONG_TRANSACTION_ID. - For Checkpoint, we need to know the current end-of-log: adding translog_get_horizon(). - For Control File, adding an assertion that the thread owns the log's lock (control file is protected by this lock) storage/maria/ma_loghandler.h: Changes in log records (see ma_loghandler.c). new prototypes, new functions. storage/maria/ma_loghandler_lsn.h: adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn, where the most significant byte is used for flags. storage/maria/ma_open.c: storing the create_rename_lsn in the index file's header (in the state, precisely) and retrieving it from there. storage/maria/ma_pagecache.c: - my set_if_bigger was wrong, correcting it - if the first_in_switch list is not empty, it means that changed_blocks misses some dirty pages, so Checkpoint cannot run and needs to wait. A variable missing_blocks_in_changed_list is added to tell that (should it be named missing_blocks_in_changed_blocks?) - pagecache_collect_changed_blocks_with_lsn() now also tells the minimum rec_lsn (needed for low-water mark computation). storage/maria/ma_pagecache.h: see ma_pagecache.c storage/maria/ma_panic.c: comment storage/maria/ma_range.c: comment storage/maria/ma_rename.c: - logging LOGREC_RENAME_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - update create_rename_lsn - we need to sync directories only if the table is transactional storage/maria/ma_static.c: comment storage/maria/ma_test_all.sh: - tip for Valgrind-ing ma_test_all - do "export maria_path=somepath" before calling ma_test_all, if you want to run ma_test_all out of storage/maria (useful to have parallel runs, like one normal and one Valgrind, they must not use the same tables so need to run in different directories) storage/maria/maria_def.h: - state now contains, in memory and on disk, the create_rename_lsn - share now contains a 2-byte-id storage/maria/trnman.c: preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn; minimum first_undo_lsn needed to know log's low-water-mark storage/maria/trnman.h: using most significant byte of first_undo_lsn to hold miscellaneous flags, for now TRANSACTION_LOGGED_LONG_ID. dummy_transaction_object is already declared in ma_static.c. storage/maria/trnman_public.h: dummy_transaction_object was declared in all files including trnman_public.h, while in fact it's a single object. new prototype storage/maria/unittest/ma_test_loghandler-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: update for new prototype storage/maria/ma_commit.c: function which wraps: - writing a LOGREC_COMMIT record (==commit on disk) - calling trnman_commit_trn() (=commit in memory) storage/maria/ma_commit.h: new header file .tree-is-private: this file is now needed to keep our tree private (don't push it to public trees). When 5.1 is merged into mysql-maria, we can abandon our maria-specific post-commit trigger; .tree_is_private will take care of keeping commit mails private. Don't push this file to public trees.
2007-06-22 14:49:37 +02:00
my_bool trnman_collect_transactions(LEX_STRING *str_act, LEX_STRING *str_com,
LSN *min_rec_lsn,
LSN *min_first_undo_lsn);
This patch is a collection of patches from from Sanja, Sergei and Monty. Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 19:13:56 +02:00
uint trnman_increment_locked_tables(TRN *trn);
uint trnman_decrement_locked_tables(TRN *trn);
WL#3072 Maria Recovery All statements doing an implicit commit now also do one in Maria. This is useful because LOCK TABLES; REPAIR; crash; is not rollback-able, the implicit commit of REPAIR avoid that Recovery tries to rollback and fails. Fix for BUG#33827 "COMMIT AND CHAIN causes serious Valgrind error" (maybe not the definite one, depends on the assigned dev). mysql-test/t/maria-recovery.test: test of REPAIR's implicit commit. I cannot commit the result file because maria-recovery fails in vanilla tree (seen in pushbuild) but its new section looks like: repair table t1; Table Op Msg_type Msg_text mysqltest.t1 repair status OK insert into t1 values(2); select * from t1; a 1 2 3 SET SESSION debug="+d,maria_flush_whole_log,maria_flush_whole_page_cache,maria_crash"; * crashing mysqld intentionally set global maria_checkpoint_interval=1; ERROR HY000: Lost connection to MySQL server during query * recovery happens check table t1 extended; Table Op Msg_type Msg_text mysqltest.t1 check status OK * testing that checksum after recovery is as expected Checksum-check failure use mysqltest; select * from t1; a 1 3 Which is as it should be. sql/rpl_injector.cc: fix for BUG#33827 sql/sql_parse.cc: - All DDLs and mysql_admin_table() (REPAIR etc) use end_actrive_trans() to do an implicit commit so we add there an implicit commit of the Maria transaction. - Fix for BUG#33827 storage/maria/ha_maria.cc: - A method to do implicit commit in Maria - After an implicit commit, if it was under LOCK TABLES, the locked tables have a stale file->trn: update it. storage/maria/ha_maria.h: new static method storage/maria/ma_check.c: bugfix: this disabling of transactionality had the effect that if LOCK TABLES; REPAIR; INSERT then the INSERT ran non-transactional (so couldn't be undone in case of crash, if, by bad chance, its effect on pages went to disk). storage/maria/ma_checkpoint.c: indentation storage/maria/ma_recovery.c: dbug statements storage/maria/trnman.c: When doing an implicit commit we need to know the number of locked tables of the committed transaction and copy it to the new transaction storage/maria/trnman_public.h: prototype change
2008-01-11 22:48:54 +01:00
uint trnman_has_locked_tables(TRN *trn);
void trnman_reset_locked_tables(TRN *trn, uint locked_tables);
WL#3072 Maria recovery * create page cache before initializing engine and not after, because Maria's recovery needs a page cache * make the creation of a bitmap page more crash-resistent * bugfix (see ma_blockrec.c) * back to old way: create an 8k bitmap page when creating table * preparations for the UNDO phase: recreate TRNs * preparations for Checkpoint: list of dirty pages, testing of rec_lsn to know if page should be skipped during Recovery (unused in this patch as no Checkpoint module pushed yet) * maria_chk tags repaired table with a special LSN * reworking all around in ma_recovery.c (less duplication) mysys/my_realloc.c: noted an issue in my_realloc() sql/mysqld.cc: page cache needs to be created before engines are initialized, because Maria's initialization may do a recovery which needs the page cache. storage/maria/ha_maria.cc: update to new prototype storage/maria/ma_bitmap.c: when creating the first bitmap page we used chsize to 8192 bytes then pwrite (overwrite) the last 2 bytes (8191-8192). If crash between the two operations, this leaves a bitmap page full without its end marker. A later recovery may try to read this page and find it exists and misses a marker and conclude it's corrupted and fail. Changing the chsize to only 8190 bytes: recovery will then find the page is too short and recreate it entirely. storage/maria/ma_blockrec.c: Fix for a bug: when executing a REDO, if the data page is created, data_file_length was increased before _ma_bitmap_set(): _ma_bitmap_set() called _ma_read_bitmap_page() which, due to the increased data_file_length, expected to find a bitmap page on disk with a correct end marker; if the bitmap page didn't exist already in fact, this failed. Fixed by increasing data_file_length only after _ma_read_bitmap_page() has created the new bitmap page correctly. This bug could happen every time a REDO is about creating a new bitmap page. storage/maria/ma_check.c: empty data file has a bitmap page storage/maria/ma_control_file.c: useless parameter to ma_control_file_create_or_open(), just test if this is recovery. storage/maria/ma_control_file.h: new prototype storage/maria/ma_create.c: Back to how it was before: maria_create() creates an 8k bitmap page. Thus (bugfix) data_file_length needs to reflect this instead of being 0. storage/maria/ma_loghandler.c: as ma_test1 and ma_test2 now use real transactions and not dummy_transaction_object, REDO for INSERT/UPDATE/DELETE are always about real transactions, can assert this. A function for Recovery to assign a short id to a table. storage/maria/ma_loghandler.h: new function storage/maria/ma_loghandler_lsn.h: maria_chk tags repaired tables with this LSN storage/maria/ma_open.c: * enforce that DMLs on transactional tables use real transactions and not dummy_transaction_object. * test if table was repaired with maria_chk (which has to been seen as an import of an external table into the server), test validity of create_rename_lsn (header corruption detection) * comments. storage/maria/ma_recovery.c: * preparations for the UNDO phase: recreate TRNs * preparations for Checkpoint: list of dirty pages, testing of rec_lsn to know if page should be skipped during Recovery (unused in this patch as no Checkpoint module pushed yet) * reworking all around (less duplication) storage/maria/ma_recovery.h: a parameter to say if the UNDO phase should be skipped storage/maria/maria_chk.c: tag repaired tables with a special LSN storage/maria/maria_read_log.c: * update to new prototype * no UNDO phase in maria_read_log for now storage/maria/trnman.c: * a function for Recovery to create a transaction (TRN), needed in the UNDO phase * a function for Recovery to grab an existing transaction, needed in the UNDO phase (rollback all existing transactions) storage/maria/trnman_public.h: new functions
2007-08-29 16:43:01 +02:00
TRN *trnman_recreate_trn_from_recovery(uint16 shortid, TrID longid);
Store maximum transaction id into control file at clean shutdown. This can serve to maria_chk to check that trids found in rows and keys are not too big. Also used by Recovery when logs are lost. Options --require-control-file, --datadir, --log-dir (yes, the dashes are inconsistent but I imitated mysqld --datadir and --maria-log-dir) for maria_chk. Lock control file _before_ reading its content. storage/maria/ha_maria.cc: new prototype storage/maria/ma_check.c: A function to find the max trid in the system (consults transaction manager and control file), to check tables. storage/maria/ma_checkpoint.c: new prototype storage/maria/ma_control_file.c: Store max trid into control file, in a backward-compatible way (can still read old control files). Parameter to ma_control_file_open(), to not create the log if it's missing (maria_chk needs that). Lock control file _before_ reading its content. Fix for a segfault when reading an old control file (bzero() with a negative second argument) storage/maria/ma_control_file.h: changes to the control file module's API storage/maria/ma_init.c: When Maria shuts down cleanly, store max trid into control file. storage/maria/ma_loghandler.c: new prototype storage/maria/ma_recovery.c: During recovery, consult max trid stored in control file, in case it is bigger than what we found in log (case of logs manually removed by user). storage/maria/ma_test1.c: new prototype storage/maria/ma_test2.c: new prototype storage/maria/maria_chk.c: New option --require-control-file (abort if control file not found), --datadir (path for control file (and for logs if --log-dir not specified)), --log-dir (path for logs). Try to open control file when maria_chk starts. storage/maria/maria_read_log.c: new prototype storage/maria/trnman.c: A new function to know max trid in transaction manager storage/maria/trnman_public.h: New function storage/maria/unittest/ma_control_file-t.c: new prototypes. Testing storing and retrieving the max trid to/from control file storage/maria/unittest/ma_test_loghandler-t.c: new prototype storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: new prototype storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: new prototype storage/maria/unittest/ma_test_loghandler_noflush-t.c: new prototype storage/maria/unittest/ma_test_loghandler_nologs-t.c: new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: new prototype storage/maria/unittest/ma_test_loghandler_purge-t.c: new prototype
2008-04-04 19:10:53 +02:00
TRN *trnman_get_any_trn(void);
WL#3138: Maria - fast "SELECT COUNT(*) FROM t;" and "CHECKSUM TABLE t" Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation Fixed wrong call to strmake Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert Allow storing year 2155 in year field When running with purify/valgrind avoid copying structures over themself Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction Fixed that ndb doesn't crash on duplicate key error when start_bulk_insert/end_bulk_insert are not called include/maria.h: Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation include/my_tree.h: Added macro 'reset_free_element()' to be able to ignore calls to the external free function. Is used to optimize end-bulk-insert in case of failures, in which case we don't want write the remaining keys in the tree mysql-test/install_test_db.sh: Upgrade to new mysql_install_db options mysql-test/r/maria-mvcc.result: New tests mysql-test/r/maria.result: New tests mysql-test/suite/ndb/r/ndb_auto_increment.result: Fixed error message now when bulk insert is not always called mysql-test/suite/ndb/t/ndb_auto_increment.test: Fixed error message now when bulk insert is not always called mysql-test/t/maria-mvcc.test: Added testing of versioning of count(*) mysql-test/t/maria-page-checksum.test: Added comment mysql-test/t/maria.test: More tests mysys/hash.c: Code style change sql/field.cc: Allow storing year 2155 in year field sql/ha_ndbcluster.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_ndbcluster.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/handler.cc: Don't call get_dup_key() if there is no table object. This can happen if the handler generates a duplicate key error on commit sql/handler.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored (ie, the table will be deleted) sql/item.cc: Style fix Removed compiler warning sql/log_event.cc: Added new argument to ha_end_bulk_insert() sql/log_event_old.cc: Added new argument to ha_end_bulk_insert() sql/mysqld.cc: Removed compiler warning sql/protocol.cc: Added DBUG sql/sql_class.cc: Added DBUG Fixed wrong call to strmake sql/sql_insert.cc: Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert involves a lot of if's) Added new argument to ha_end_bulk_insert() sql/sql_load.cc: Added new argument to ha_end_bulk_insert() sql/sql_parse.cc: Style fixes Avoid goto in common senario sql/sql_select.cc: When running with purify/valgrind avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_select.h: Avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_table.cc: Call HA_EXTRA_PREPARE_FOR_DROP if table created by ALTER TABLE is going to be dropped Added new argument to ha_end_bulk_insert() storage/archive/ha_archive.cc: Added new argument to end_bulk_insert() storage/archive/ha_archive.h: Added new argument to end_bulk_insert() storage/federated/ha_federated.cc: Added new argument to end_bulk_insert() storage/federated/ha_federated.h: Added new argument to end_bulk_insert() storage/maria/Makefile.am: Added ma_state.c and ma_state.h storage/maria/ha_maria.cc: Versioning of count(*) and checksum - share->state.state is now assumed to be correct, not handler->state - Call _ma_setup_live_state() in external lock to get count(*)/checksum versioning. In case of not versioned and not concurrent insertable table, file->s->state.state contains the correct state information Other things: - file->s -> share - Added DBUG_ASSERT() for unlikely case - Optimized end_bulk_insert() to not write anything if table is going to be deleted (as in failed alter table) - Indentation changes in external_lock becasue of removed 'goto' caused a big conflict even if very little was changed storage/maria/ha_maria.h: New argument to end_bulk_insert() storage/maria/ma_blockrec.c: Update for versioning of count(*) and checksum Keep share->state.state.data_file_length up to date (not info->state->data_file_length) Moved _ma_block_xxxx_status() and maria_versioning() functions to ma_state.c storage/maria/ma_check.c: Update and use share->state.state instead of info->state info->s to share Update info->state at end of repair Call _ma_reset_state() to update share->state_history at end of repair storage/maria/ma_checkpoint.c: Call _ma_remove_not_visible_states() on checkpoint to clean up not visible state history from tables storage/maria/ma_close.c: Remember state history for running transaction even if table is closed storage/maria/ma_commit.c: Ensure we always call trnman_commit_trn() even if other calls fails. If we don't do that, the translog and state structures will not be freed storage/maria/ma_delete.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/ma_delete_all.c: Versioning of count(*) and checksum: - Ensure that share->state.state is updated, as here is where we store the primary information storage/maria/ma_dynrec.c: Use lock_key_trees instead of concurrent_insert to check if trees should be locked. This allows us to lock trees both for concurrent_insert and for index versioning. storage/maria/ma_extra.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - share->concurrent_insert -> share->non_transactional_concurrent_insert - Don't update share->state.state from info->state if transactional table Optimization: - Don't flush io_cache or bitmap if we are using FLUSH_IGNORE_CHANGED storage/maria/ma_info.c: Get most state information from current state storage/maria/ma_init.c: Add hash table and free function to store states for closed tables Install hook for transaction commit/rollback to update history state storage/maria/ma_key_recover.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state storage/maria/ma_locking.c: Versioning of count(*) and checksum: - Call virtual functions (if exists) to restore/update status - Move _ma_xxx_status() functions to ma_state.c info->s -> share storage/maria/ma_open.c: Versioning of count(*) and checksum: - For not transactional tables, set info->state to point to new allocated state structure. - Initialize new info->state_start variable that points to state at start of transaction - Copy old history states from hash table (maria_stored_states) first time the table is opened - Split flag share->concurrent_insert to non_transactional_concurrent_insert & lock_key_tree - For now, only enable versioning of tables without keys (to be fixed in soon!) - Added new virtual function to restore status in maria_lock_database) More DBUG storage/maria/ma_page.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Modify share->state.state.key_file_length under share->intern_lock storage/maria/ma_range.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees info->s -> share storage/maria/ma_recovery.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Update state information on close and when reenabling logging storage/maria/ma_rkey.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext_same.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees - Only skip rows based on file length if non_transactional_concurrent_insert is set storage/maria/ma_rprev.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rsame.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_sort.c: Use share->state.state instead of info->state Fixed indentation storage/maria/ma_static.c: Added maria_stored_state storage/maria/ma_update.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records - Remove optimization for index file update as it doesn't work for transactional tables storage/maria/ma_write.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/maria_def.h: Move MARIA_STATUS_INFO to ma_state.h Changes to MARIA_SHARE: - Added state_history to store count(*)/checksum states - Added in_trans as counter if table is used by running transactions - Split concurrent_insert into lock_key_trees and on_transactional_concurrent_insert. - Added virtual function lock_restore_status Changes to MARIA_HA: - save_state -> state_save - Added state_start to store state at start of transaction storage/maria/maria_pack.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state Indentation fixes storage/maria/trnman.c: Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction More DBUG Changed return type of trnman_end_trn() to my_bool Added trnman_get_min_trid() to get minimum trid in use. Added trnman_exists_active_transactions() to check if there exist a running transaction started between two commit id storage/maria/trnman.h: Added 'used_tables' Moved all pointers into same groups to get better memory alignment storage/maria/trnman_public.h: Added prototypes for new functions and variables Chagned return type of trnman_end_trn() to my_bool storage/myisam/ha_myisam.cc: Added argument to end_bulk_insert() if operation should be aborted storage/myisam/ha_myisam.h: Added argument to end_bulk_insert() if operation should be aborted storage/maria/ma_state.c: Functions to handle state of count(*) and checksum storage/maria/ma_state.h: Structures and declarations to handle state of count(*) and checksum
2008-05-29 17:33:33 +02:00
TrID trnman_get_min_trid(void);
Store maximum transaction id into control file at clean shutdown. This can serve to maria_chk to check that trids found in rows and keys are not too big. Also used by Recovery when logs are lost. Options --require-control-file, --datadir, --log-dir (yes, the dashes are inconsistent but I imitated mysqld --datadir and --maria-log-dir) for maria_chk. Lock control file _before_ reading its content. storage/maria/ha_maria.cc: new prototype storage/maria/ma_check.c: A function to find the max trid in the system (consults transaction manager and control file), to check tables. storage/maria/ma_checkpoint.c: new prototype storage/maria/ma_control_file.c: Store max trid into control file, in a backward-compatible way (can still read old control files). Parameter to ma_control_file_open(), to not create the log if it's missing (maria_chk needs that). Lock control file _before_ reading its content. Fix for a segfault when reading an old control file (bzero() with a negative second argument) storage/maria/ma_control_file.h: changes to the control file module's API storage/maria/ma_init.c: When Maria shuts down cleanly, store max trid into control file. storage/maria/ma_loghandler.c: new prototype storage/maria/ma_recovery.c: During recovery, consult max trid stored in control file, in case it is bigger than what we found in log (case of logs manually removed by user). storage/maria/ma_test1.c: new prototype storage/maria/ma_test2.c: new prototype storage/maria/maria_chk.c: New option --require-control-file (abort if control file not found), --datadir (path for control file (and for logs if --log-dir not specified)), --log-dir (path for logs). Try to open control file when maria_chk starts. storage/maria/maria_read_log.c: new prototype storage/maria/trnman.c: A new function to know max trid in transaction manager storage/maria/trnman_public.h: New function storage/maria/unittest/ma_control_file-t.c: new prototypes. Testing storing and retrieving the max trid to/from control file storage/maria/unittest/ma_test_loghandler-t.c: new prototype storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: new prototype storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: new prototype storage/maria/unittest/ma_test_loghandler_noflush-t.c: new prototype storage/maria/unittest/ma_test_loghandler_nologs-t.c: new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: new prototype storage/maria/unittest/ma_test_loghandler_purge-t.c: new prototype
2008-04-04 19:10:53 +02:00
TrID trnman_get_max_trid(void);
WL#3138: Maria - fast "SELECT COUNT(*) FROM t;" and "CHECKSUM TABLE t" Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation Fixed wrong call to strmake Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert Allow storing year 2155 in year field When running with purify/valgrind avoid copying structures over themself Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction Fixed that ndb doesn't crash on duplicate key error when start_bulk_insert/end_bulk_insert are not called include/maria.h: Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation include/my_tree.h: Added macro 'reset_free_element()' to be able to ignore calls to the external free function. Is used to optimize end-bulk-insert in case of failures, in which case we don't want write the remaining keys in the tree mysql-test/install_test_db.sh: Upgrade to new mysql_install_db options mysql-test/r/maria-mvcc.result: New tests mysql-test/r/maria.result: New tests mysql-test/suite/ndb/r/ndb_auto_increment.result: Fixed error message now when bulk insert is not always called mysql-test/suite/ndb/t/ndb_auto_increment.test: Fixed error message now when bulk insert is not always called mysql-test/t/maria-mvcc.test: Added testing of versioning of count(*) mysql-test/t/maria-page-checksum.test: Added comment mysql-test/t/maria.test: More tests mysys/hash.c: Code style change sql/field.cc: Allow storing year 2155 in year field sql/ha_ndbcluster.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_ndbcluster.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/handler.cc: Don't call get_dup_key() if there is no table object. This can happen if the handler generates a duplicate key error on commit sql/handler.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored (ie, the table will be deleted) sql/item.cc: Style fix Removed compiler warning sql/log_event.cc: Added new argument to ha_end_bulk_insert() sql/log_event_old.cc: Added new argument to ha_end_bulk_insert() sql/mysqld.cc: Removed compiler warning sql/protocol.cc: Added DBUG sql/sql_class.cc: Added DBUG Fixed wrong call to strmake sql/sql_insert.cc: Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert involves a lot of if's) Added new argument to ha_end_bulk_insert() sql/sql_load.cc: Added new argument to ha_end_bulk_insert() sql/sql_parse.cc: Style fixes Avoid goto in common senario sql/sql_select.cc: When running with purify/valgrind avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_select.h: Avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_table.cc: Call HA_EXTRA_PREPARE_FOR_DROP if table created by ALTER TABLE is going to be dropped Added new argument to ha_end_bulk_insert() storage/archive/ha_archive.cc: Added new argument to end_bulk_insert() storage/archive/ha_archive.h: Added new argument to end_bulk_insert() storage/federated/ha_federated.cc: Added new argument to end_bulk_insert() storage/federated/ha_federated.h: Added new argument to end_bulk_insert() storage/maria/Makefile.am: Added ma_state.c and ma_state.h storage/maria/ha_maria.cc: Versioning of count(*) and checksum - share->state.state is now assumed to be correct, not handler->state - Call _ma_setup_live_state() in external lock to get count(*)/checksum versioning. In case of not versioned and not concurrent insertable table, file->s->state.state contains the correct state information Other things: - file->s -> share - Added DBUG_ASSERT() for unlikely case - Optimized end_bulk_insert() to not write anything if table is going to be deleted (as in failed alter table) - Indentation changes in external_lock becasue of removed 'goto' caused a big conflict even if very little was changed storage/maria/ha_maria.h: New argument to end_bulk_insert() storage/maria/ma_blockrec.c: Update for versioning of count(*) and checksum Keep share->state.state.data_file_length up to date (not info->state->data_file_length) Moved _ma_block_xxxx_status() and maria_versioning() functions to ma_state.c storage/maria/ma_check.c: Update and use share->state.state instead of info->state info->s to share Update info->state at end of repair Call _ma_reset_state() to update share->state_history at end of repair storage/maria/ma_checkpoint.c: Call _ma_remove_not_visible_states() on checkpoint to clean up not visible state history from tables storage/maria/ma_close.c: Remember state history for running transaction even if table is closed storage/maria/ma_commit.c: Ensure we always call trnman_commit_trn() even if other calls fails. If we don't do that, the translog and state structures will not be freed storage/maria/ma_delete.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/ma_delete_all.c: Versioning of count(*) and checksum: - Ensure that share->state.state is updated, as here is where we store the primary information storage/maria/ma_dynrec.c: Use lock_key_trees instead of concurrent_insert to check if trees should be locked. This allows us to lock trees both for concurrent_insert and for index versioning. storage/maria/ma_extra.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - share->concurrent_insert -> share->non_transactional_concurrent_insert - Don't update share->state.state from info->state if transactional table Optimization: - Don't flush io_cache or bitmap if we are using FLUSH_IGNORE_CHANGED storage/maria/ma_info.c: Get most state information from current state storage/maria/ma_init.c: Add hash table and free function to store states for closed tables Install hook for transaction commit/rollback to update history state storage/maria/ma_key_recover.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state storage/maria/ma_locking.c: Versioning of count(*) and checksum: - Call virtual functions (if exists) to restore/update status - Move _ma_xxx_status() functions to ma_state.c info->s -> share storage/maria/ma_open.c: Versioning of count(*) and checksum: - For not transactional tables, set info->state to point to new allocated state structure. - Initialize new info->state_start variable that points to state at start of transaction - Copy old history states from hash table (maria_stored_states) first time the table is opened - Split flag share->concurrent_insert to non_transactional_concurrent_insert & lock_key_tree - For now, only enable versioning of tables without keys (to be fixed in soon!) - Added new virtual function to restore status in maria_lock_database) More DBUG storage/maria/ma_page.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Modify share->state.state.key_file_length under share->intern_lock storage/maria/ma_range.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees info->s -> share storage/maria/ma_recovery.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Update state information on close and when reenabling logging storage/maria/ma_rkey.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext_same.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees - Only skip rows based on file length if non_transactional_concurrent_insert is set storage/maria/ma_rprev.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rsame.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_sort.c: Use share->state.state instead of info->state Fixed indentation storage/maria/ma_static.c: Added maria_stored_state storage/maria/ma_update.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records - Remove optimization for index file update as it doesn't work for transactional tables storage/maria/ma_write.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/maria_def.h: Move MARIA_STATUS_INFO to ma_state.h Changes to MARIA_SHARE: - Added state_history to store count(*)/checksum states - Added in_trans as counter if table is used by running transactions - Split concurrent_insert into lock_key_trees and on_transactional_concurrent_insert. - Added virtual function lock_restore_status Changes to MARIA_HA: - save_state -> state_save - Added state_start to store state at start of transaction storage/maria/maria_pack.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state Indentation fixes storage/maria/trnman.c: Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction More DBUG Changed return type of trnman_end_trn() to my_bool Added trnman_get_min_trid() to get minimum trid in use. Added trnman_exists_active_transactions() to check if there exist a running transaction started between two commit id storage/maria/trnman.h: Added 'used_tables' Moved all pointers into same groups to get better memory alignment storage/maria/trnman_public.h: Added prototypes for new functions and variables Chagned return type of trnman_end_trn() to my_bool storage/myisam/ha_myisam.cc: Added argument to end_bulk_insert() if operation should be aborted storage/myisam/ha_myisam.h: Added argument to end_bulk_insert() if operation should be aborted storage/maria/ma_state.c: Functions to handle state of count(*) and checksum storage/maria/ma_state.h: Structures and declarations to handle state of count(*) and checksum
2008-05-29 17:33:33 +02:00
my_bool trnman_exists_active_transactions(TrID min_id, TrID max_id,
my_bool trnman_is_locked);
WL#3072 - Maria recovery. * fix for bitmap vs checkpoint bug which could lead to corrupted tables in case of crashes at certain moments: a bitmap could be flushed to disk even though it was inconsistent with the log (it could be flushed before REDO-UNDO are written to the log). One bug remains, need code from others. Tests added. Fix is to pin unflushable bitmap pages, and let checkpoint wait for them to be flushable. * fix for long_trid!=0 assertion failure at Recovery. * less useless wakeups in the background flush|checkpoint thread. * store global_trid_generator in checkpoint record. mysql-test/r/maria-recovery.result: result update mysql-test/t/maria-recovery.test: make it easier to locate subtests storage/maria/ma_bitmap.c: When we send a bitmap to the pagecache, if this bitmap is not in a flushable state we keep it pinned and add it to a list, it will be unpinned when the bitmap is flushable again. A new function _ma_bitmap_flush_all() used by checkpoint. A new function _ma_bitmap_flushable() used by block format to signal when it starts modifying a bitmap and when it is done with it. storage/maria/ma_blockrec.c: When starting a row operation (insert/update/delete), mark that the bitmap is not flushable (because for example INSERT is going to over-allocate in the bitmap to prevent other threads from using our data pages). If a checkpoint comes at this moment it will wait for the bitmap to be flushable before flushing it. When the operation ends, bitmap becomes flushable again; that transition is done under the bitmap's mutex (needed for correct synchro with a concurrent checkpoint); but for INSERT/UPDATE this happens inside _ma_bitmap_release_unused() at a place where it already has the mutex, so the only penalty (mutex adding) is in DELETE and UNDO of INSERT. In case of errors after setting the bitmap unflushable, we must always set it back to flushable or checkpoint would block. Debug possibilities to force a sleep while the bitmap is over-allocated. In case of error in get_head_or_tail() in allocate_and_write_block_record(), we still need to unpin all pages. Bugfix: _ma_apply_redo_insert_row_blobs() produced wrong data_file_length. storage/maria/ma_blockrec.h: new bitmap calls. storage/maria/ma_checkpoint.c: filter_flush_indirect not needed anymore (flushing bitmap pages happens in _ma_bitmap_flush_all() now). So st_filter_param::is_data_file|pages_covered_by_bitmap not needed. Other filter_flush* don't need to flush bitmap anymore. Add debug possibility to flush all bitmap pages outside of a checkpoint, to simulate pagecache LRU eviction. When the background flush/checkpoint thread notices it has nothing to flush, it now sleeps directly until the next potential checkpoint moment instead of waking up every second. When in checkpoint we decide to not store a table in the checkpoint record (because it has logged no writes for example), we can also skip flushing this table. storage/maria/ma_commit.c: comment is out-of-date storage/maria/ma_key_recover.c: comment fix storage/maria/ma_loghandler.c: comment is out-of-date storage/maria/ma_open.c: comment is out-of-date storage/maria/ma_pagecache.c: comment for bug to fix. And we don't take checkpoints at end of REDO phase yet so can trust block->type. storage/maria/ma_recovery.c: Comments. Now-unneeded code for incomplete REDO-UNDO groups removed. When we forget about an old transaction we must really forget about it with bzero() (fixes the "long_trid!=0 assertion" recovery bug). When we delete a row with maria_delete() we turn on STATE_NOT_OPTIMIZED_ROWS so we do the same when we see a CLR_END for an UNDO_ROW_INSERT or when we execute an UNDO_ROW_INSERT (in both cases a row was deleted). Pick up max_long_trid from the checkpoint record. storage/maria/maria_chk.c: comment storage/maria/maria_def.h: MARIA_FILE_BITMAP gets new members: 'flushable', 'bitmap_cond' and 'pinned_pages'. storage/maria/trnman.c: I used to think that recovery only needs to know the maximum TrID of the lists of active and committed transactions. But no, sometimes both lists can even be empty and their TrID should not be reused. So Checkpoint now saves global_trid_generator in the checkpoint record. storage/maria/trnman_public.h: macros to read/store a TrID mysql-test/r/maria-recovery-bitmap.result: result is ok. Without the code fix, we would get a corruption message about the bitmap page in CHECK TABLE EXTENDED. mysql-test/t/maria-recovery-bitmap-master.opt: usual when we crash mysqld in tests mysql-test/t/maria-recovery-bitmap.test: test of recovery problems specific of the bitmap pages.
2007-12-14 16:14:12 +01:00
#define TRANSID_SIZE 6
#define transid_store(dst, id) int6store(dst,id)
#define transid_korr(P) uint6korr(P)
This patch is a collection of patches from from Sanja, Sergei and Monty. Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h''
2007-05-29 19:13:56 +02:00
C_MODE_END
WL#3072 Maria Recovery - new program maria_read_log to display and apply log records found in a Maria log (see file's revision comment) - minor, misc fixes storage/maria/Makefile.am: new program maria_read_log storage/maria/ha_maria.cc: create control file if missing storage/maria/ma_blockrec.c: 0 -> LSN_IMPOSSIBLE; comments storage/maria/ma_checkpoint.h: preparations for Checkpoint module storage/maria/ma_close.c: comment storage/maria/ma_control_file.c: renaming constants. Possibility to say "open control file but don't create it if it's missing" (used by maria_read_log which does not want to create anything) storage/maria/ma_control_file.h: renaming constants storage/maria/ma_create.c: I had duplicated "linkname" and "linkname_ptr", now I see it's not needed, reverting. Indeed those variables don't contain interesting information; fixing log record accordingly (the links are in ci->data/index_file_name). Storing keystart in log record is needed, to know at which size we must extend the file if we replay LOGREC_CREATE_TABLE. storage/maria/ma_loghandler.c: some structures need to be known to maria_read_log.c, taking them to ma_loghandler.h storage/maria/ma_loghandler.h: we have page_store, adding page_korr. translog_lock() made public, because Checkpoint will need it (to write to control file). Some structures moved from ma_loghandler.c because maria_read_log.c needs them (needs to know the execute-in-REDO-phase hooks of each record). storage/maria/ma_loghandler_lsn.h: constants defined in ma_control_file.h serve everywhere, and they relate to LSNs, so putting them in ma_loghandler_lsn.h. Stronger constraints in LSN_VALID(). storage/maria/ma_pagecache.c: renaming constants storage/maria/ma_recovery.h: copyright storage/maria/ma_test1.c: new prototype storage/maria/ma_test2.c: new prototype storage/maria/trnman_public.h: double-inclusion safe storage/maria/unittest/ma_control_file-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: constants renamed, new prototype storage/myisam/mi_close.c: comment storage/maria/maria_read_log.c: program to read and print log records from a Maria transaction log, and optionally apply them to tables. Very basic, early version. Should serve as a base for Recovery's code. Designed to be idempotent. Create a log by running maria.test, then cd to var/master-data and run "maria_read_log --only-display" to see info about records; run "maria_read_log --display-and-apply" to also apply the records to tables (it's more interesting if you first wipe out the tables in var/master-data/test, to see how they get re-created). Only a few records are handled by now: LONG_TRANSACTION_ID, COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for REDO_INSERT_ROW_HEAD where I could use Monty's help (search for "Monty" in the file). Note: changes to the index pages, index's header and bitmap pages are not properly logged yet, so don't expect the program to work with that.
2007-06-26 16:49:23 +02:00
#endif