2006-04-11 15:45:10 +02:00
|
|
|
/* Copyright (C) 2006 MySQL AB & MySQL Finland AB & TCX DataKonsult AB
|
|
|
|
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
|
|
it under the terms of the GNU General Public License as published by
|
2007-03-02 11:20:23 +01:00
|
|
|
the Free Software Foundation; version 2 of the License.
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
GNU General Public License for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
|
|
along with this program; if not, write to the Free Software
|
2017-02-10 12:26:55 +01:00
|
|
|
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02111-1301 USA */
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
/* Create a MARIA table */
|
|
|
|
|
|
|
|
#include "ma_ftdefs.h"
|
|
|
|
#include "ma_sp_defs.h"
|
2006-08-11 01:16:51 +02:00
|
|
|
#include <my_bit.h>
|
2007-01-18 20:38:14 +01:00
|
|
|
#include "ma_blockrec.h"
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
#include "trnman_public.h"
|
2014-12-22 15:53:17 +01:00
|
|
|
#include "ma_crypt.h"
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
#if defined(MSDOS) || defined(__WIN__)
|
|
|
|
#ifdef __WIN__
|
|
|
|
#include <fcntl.h>
|
|
|
|
#else
|
|
|
|
#include <process.h> /* Prototype for getpid */
|
|
|
|
#endif
|
|
|
|
#endif
|
|
|
|
#include <m_ctype.h>
|
|
|
|
|
2007-01-18 20:38:14 +01:00
|
|
|
static int compare_columns(MARIA_COLUMNDEF **a, MARIA_COLUMNDEF **b);
|
|
|
|
|
2017-09-26 18:36:19 +02:00
|
|
|
|
|
|
|
static ulonglong update_tot_length(ulonglong tot_length, ulonglong max_rows, uint length)
|
|
|
|
{
|
|
|
|
ulonglong tot_length_part;
|
|
|
|
|
|
|
|
if (tot_length == ULONGLONG_MAX)
|
|
|
|
return ULONGLONG_MAX;
|
|
|
|
|
|
|
|
tot_length_part= (max_rows/(ulong) ((maria_block_size -
|
|
|
|
MAX_KEYPAGE_HEADER_SIZE - KEYPAGE_CHECKSUM_SIZE)/
|
|
|
|
(length*2)));
|
|
|
|
if (tot_length_part >= ULONGLONG_MAX / maria_block_size)
|
|
|
|
return ULONGLONG_MAX;
|
|
|
|
|
|
|
|
if (tot_length > ULONGLONG_MAX - tot_length_part * maria_block_size)
|
|
|
|
return ULONGLONG_MAX;
|
|
|
|
|
|
|
|
return tot_length + tot_length_part * maria_block_size;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
/*
|
|
|
|
Old options is used when recreating database, from maria_chk
|
|
|
|
*/
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2007-04-19 12:18:56 +02:00
|
|
|
int maria_create(const char *name, enum data_file_type datafile_type,
|
2007-01-18 20:38:14 +01:00
|
|
|
uint keys,MARIA_KEYDEF *keydefs,
|
2007-04-19 12:18:56 +02:00
|
|
|
uint columns, MARIA_COLUMNDEF *columndef,
|
2007-01-18 20:38:14 +01:00
|
|
|
uint uniques, MARIA_UNIQUEDEF *uniquedefs,
|
|
|
|
MARIA_CREATE_INFO *ci,uint flags)
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
|
|
|
register uint i,j;
|
2011-10-29 20:40:03 +02:00
|
|
|
File UNINIT_VAR(dfile), UNINIT_VAR(file);
|
2007-04-05 13:38:05 +02:00
|
|
|
int errpos,save_errno, create_mode= O_RDWR | O_TRUNC, res;
|
2006-04-11 15:45:10 +02:00
|
|
|
myf create_flag;
|
2007-01-18 20:38:14 +01:00
|
|
|
uint length,max_key_length,packed,pack_bytes,pointer,real_length_diff,
|
2007-12-10 01:32:00 +01:00
|
|
|
key_length,info_length,key_segs,options,min_key_length,
|
2018-01-18 00:41:52 +01:00
|
|
|
base_pos,long_varchar_count,
|
2007-06-05 13:35:34 +02:00
|
|
|
unique_key_parts,fulltext_keys,offset, not_block_record_extra_length;
|
2007-10-09 20:09:50 +02:00
|
|
|
uint max_field_lengths, extra_header_size, column_nr;
|
2013-03-25 23:03:13 +01:00
|
|
|
uint internal_table= flags & HA_CREATE_INTERNAL_TABLE;
|
2006-04-11 15:45:10 +02:00
|
|
|
ulong reclength, real_reclength,min_pack_length;
|
2017-02-20 22:40:47 +01:00
|
|
|
char kfilename[FN_REFLEN], klinkname[FN_REFLEN], *klinkname_ptr;
|
2017-05-07 17:26:10 +02:00
|
|
|
char dfilename[FN_REFLEN], dlinkname[FN_REFLEN], *dlinkname_ptr= 0;
|
2006-04-11 15:45:10 +02:00
|
|
|
ulong pack_reclength;
|
|
|
|
ulonglong tot_length,max_rows, tmp;
|
|
|
|
enum en_fieldtype type;
|
2007-04-19 12:18:56 +02:00
|
|
|
enum data_file_type org_datafile_type= datafile_type;
|
2006-04-11 15:45:10 +02:00
|
|
|
MARIA_SHARE share;
|
|
|
|
MARIA_KEYDEF *keydef,tmp_keydef;
|
|
|
|
MARIA_UNIQUEDEF *uniquedef;
|
|
|
|
HA_KEYSEG *keyseg,tmp_keyseg;
|
2007-04-19 12:18:56 +02:00
|
|
|
MARIA_COLUMNDEF *column, *end_column;
|
2007-10-09 20:09:50 +02:00
|
|
|
double *rec_per_key_part;
|
|
|
|
ulong *nulls_per_key_part;
|
|
|
|
uint16 *column_array;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
my_off_t key_root[HA_MAX_POSSIBLE_KEY], kfile_size_before_extension;
|
2006-04-11 15:45:10 +02:00
|
|
|
MARIA_CREATE_INFO tmp_create_info;
|
WL#3072 Maria Recovery. Making DDLs durable in Maria:
Sync table files after CREATE (of non-temp table), DROP, RENAME,
TRUNCATE, sync directories and symlinks (for the 3 first commands).
Comments for future log records.
In ma_rename(), if rename of index works and then rename of data fails,
try to undo the rename of the index to leave a consistent state.
mysys/my_symlink.c:
sync directory after creation of a symbolic link in it, if asked
mysys/my_sync.c:
comment. Fix for when the file's name has no directory in it.
storage/maria/ma_create.c:
sync files and links and dirs when creating a non-temporary table.
Optimizations of the above to reduce syncs in the common cases:
* if index file and data file have the exact same paths (regular
and link), sync the directories (of regular and link) only once
after creating the last file (the data file).
* don't sync the data file if we didn't write to it (always true
in our builds).
storage/maria/ma_delete_all.c:
sync files after truncating a table
storage/maria/ma_delete_table.c:
sync files and symbolic links and dirs after dropping a table
storage/maria/ma_extra.c:
a function which wraps the sync of the index file and the sync of the
data file.
storage/maria/ma_locking.c:
using a wrapper function
storage/maria/ma_rename.c:
sync files and symbolic links and dirs after renaming a table.
If rename of index works and then rename of data fails, try to undo
the rename of the index to leave a consistent state. That is just a
try, it may fail...
storage/maria/ma_test3.c:
warning to not pay attention to this test.
storage/maria/maria_def.h:
declaration for the function added to ma_extra.c
2006-11-27 22:01:29 +01:00
|
|
|
my_bool tmp_table= FALSE; /* cache for presence of HA_OPTION_TMP_TABLE */
|
2007-06-05 13:35:34 +02:00
|
|
|
my_bool forced_packed;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
myf sync_dir= 0;
|
|
|
|
uchar *log_data= NULL;
|
2015-03-26 17:00:49 +01:00
|
|
|
my_bool encrypted= maria_encrypt_tables && datafile_type == BLOCK_RECORD;
|
2015-03-26 17:16:37 +01:00
|
|
|
my_bool insert_order= MY_TEST(flags & HA_PRESERVE_INSERT_ORDER);
|
2014-12-22 15:53:17 +01:00
|
|
|
uint crypt_page_header_space= 0;
|
2006-04-11 15:45:10 +02:00
|
|
|
DBUG_ENTER("maria_create");
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
DBUG_PRINT("enter", ("keys: %u columns: %u uniques: %u flags: %u",
|
|
|
|
keys, columns, uniques, flags));
|
2006-04-11 15:45:10 +02:00
|
|
|
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
DBUG_ASSERT(maria_inited);
|
2007-01-18 20:38:14 +01:00
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
if (!ci)
|
|
|
|
{
|
|
|
|
bzero((char*) &tmp_create_info,sizeof(tmp_create_info));
|
|
|
|
ci=&tmp_create_info;
|
|
|
|
}
|
|
|
|
|
2008-01-21 21:03:24 +01:00
|
|
|
if (keys + uniques > MARIA_MAX_KEY)
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
|
|
|
DBUG_RETURN(my_errno=HA_WRONG_CREATE_OPTION);
|
|
|
|
}
|
|
|
|
errpos=0;
|
|
|
|
options=0;
|
2007-07-02 19:45:15 +02:00
|
|
|
bzero((uchar*) &share,sizeof(share));
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
if (flags & HA_DONT_TOUCH_DATA)
|
|
|
|
{
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
/* We come here from recreate table */
|
2007-04-19 12:18:56 +02:00
|
|
|
org_datafile_type= ci->org_data_file_type;
|
2006-04-11 15:45:10 +02:00
|
|
|
if (!(ci->old_options & HA_OPTION_TEMP_COMPRESS_RECORD))
|
2008-01-31 02:06:04 +01:00
|
|
|
options= (ci->old_options &
|
|
|
|
(HA_OPTION_COMPRESS_RECORD | HA_OPTION_PACK_RECORD |
|
|
|
|
HA_OPTION_READ_ONLY_DATA | HA_OPTION_CHECKSUM |
|
|
|
|
HA_OPTION_TMP_TABLE | HA_OPTION_DELAY_KEY_WRITE |
|
|
|
|
HA_OPTION_LONG_BLOB_PTR | HA_OPTION_PAGE_CHECKSUM));
|
2006-04-11 15:45:10 +02:00
|
|
|
else
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
|
|
|
/* Uncompressing rows */
|
2008-01-31 02:06:04 +01:00
|
|
|
options= (ci->old_options &
|
|
|
|
(HA_OPTION_CHECKSUM | HA_OPTION_TMP_TABLE |
|
|
|
|
HA_OPTION_DELAY_KEY_WRITE | HA_OPTION_LONG_BLOB_PTR |
|
|
|
|
HA_OPTION_PAGE_CHECKSUM));
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
else
|
|
|
|
{
|
|
|
|
/* Transactional tables must be of type BLOCK_RECORD */
|
|
|
|
if (ci->transactional)
|
|
|
|
datafile_type= BLOCK_RECORD;
|
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
if (!(rec_per_key_part=
|
2007-10-09 20:09:50 +02:00
|
|
|
(double*) my_malloc((keys + uniques)*HA_MAX_KEY_SEG*sizeof(double) +
|
|
|
|
(keys + uniques)*HA_MAX_KEY_SEG*sizeof(ulong) +
|
|
|
|
sizeof(uint16) * columns,
|
|
|
|
MYF(MY_WME | MY_ZEROFILL))))
|
2006-04-11 15:45:10 +02:00
|
|
|
DBUG_RETURN(my_errno);
|
2007-10-09 20:09:50 +02:00
|
|
|
nulls_per_key_part= (ulong*) (rec_per_key_part +
|
|
|
|
(keys + uniques) * HA_MAX_KEY_SEG);
|
|
|
|
column_array= (uint16*) (nulls_per_key_part +
|
|
|
|
(keys + uniques) * HA_MAX_KEY_SEG);
|
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2007-12-18 02:21:32 +01:00
|
|
|
/* Start by checking fields and field-types used */
|
2018-01-18 00:41:52 +01:00
|
|
|
long_varchar_count=packed= not_block_record_extra_length=
|
2007-01-18 20:38:14 +01:00
|
|
|
pack_reclength= max_field_lengths= 0;
|
|
|
|
reclength= min_pack_length= ci->null_bytes;
|
2007-06-05 13:35:34 +02:00
|
|
|
forced_packed= 0;
|
2007-10-09 20:09:50 +02:00
|
|
|
column_nr= 0;
|
2007-01-18 20:38:14 +01:00
|
|
|
|
2014-12-22 15:53:17 +01:00
|
|
|
if (encrypted)
|
|
|
|
{
|
|
|
|
DBUG_ASSERT(datafile_type == BLOCK_RECORD);
|
|
|
|
crypt_page_header_space= ma_crypt_get_data_page_header_space();
|
|
|
|
}
|
|
|
|
|
2007-04-19 12:18:56 +02:00
|
|
|
for (column= columndef, end_column= column + columns ;
|
|
|
|
column != end_column ;
|
|
|
|
column++)
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
2007-01-18 20:38:14 +01:00
|
|
|
/* Fill in not used struct parts */
|
2007-10-09 20:09:50 +02:00
|
|
|
column->column_nr= column_nr++;
|
2007-04-19 12:18:56 +02:00
|
|
|
column->offset= reclength;
|
|
|
|
column->empty_pos= 0;
|
|
|
|
column->empty_bit= 0;
|
|
|
|
column->fill_length= column->length;
|
2007-09-11 00:58:15 +02:00
|
|
|
if (column->null_bit)
|
|
|
|
options|= HA_OPTION_NULL_FIELDS;
|
2007-04-19 12:18:56 +02:00
|
|
|
|
|
|
|
reclength+= column->length;
|
|
|
|
type= column->type;
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
if (datafile_type == BLOCK_RECORD)
|
|
|
|
{
|
|
|
|
if (type == FIELD_SKIP_PRESPACE)
|
2008-01-07 17:54:41 +01:00
|
|
|
type= column->type= FIELD_NORMAL; /* SKIP_PRESPACE not supported */
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
if (type == FIELD_NORMAL &&
|
2014-12-22 15:53:17 +01:00
|
|
|
column->length > FULL_PAGE_SIZE2(maria_block_size,
|
|
|
|
crypt_page_header_space))
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
{
|
|
|
|
/* FIELD_NORMAL can't be split over many blocks, convert to a CHAR */
|
|
|
|
type= column->type= FIELD_SKIP_ENDSPACE;
|
|
|
|
}
|
|
|
|
}
|
2007-01-18 20:38:14 +01:00
|
|
|
|
|
|
|
if (type != FIELD_NORMAL && type != FIELD_CHECK)
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
2007-04-19 12:18:56 +02:00
|
|
|
column->empty_pos= packed/8;
|
|
|
|
column->empty_bit= (1 << (packed & 7));
|
2006-04-11 15:45:10 +02:00
|
|
|
if (type == FIELD_BLOB)
|
|
|
|
{
|
2007-06-05 13:35:34 +02:00
|
|
|
forced_packed= 1;
|
2007-01-18 20:38:14 +01:00
|
|
|
packed++;
|
2006-04-11 15:45:10 +02:00
|
|
|
share.base.blobs++;
|
|
|
|
if (pack_reclength != INT_MAX32)
|
|
|
|
{
|
2007-04-19 12:18:56 +02:00
|
|
|
if (column->length == 4+portable_sizeof_char_ptr)
|
2006-04-11 15:45:10 +02:00
|
|
|
pack_reclength= INT_MAX32;
|
|
|
|
else
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
|
|
|
/* Add max possible blob length */
|
2007-04-19 12:18:56 +02:00
|
|
|
pack_reclength+= (1 << ((column->length-
|
|
|
|
portable_sizeof_char_ptr)*8));
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
2007-04-19 12:18:56 +02:00
|
|
|
max_field_lengths+= (column->length - portable_sizeof_char_ptr);
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
else if (type == FIELD_SKIP_PRESPACE ||
|
|
|
|
type == FIELD_SKIP_ENDSPACE)
|
|
|
|
{
|
2007-06-05 13:35:34 +02:00
|
|
|
forced_packed= 1;
|
2007-04-19 12:18:56 +02:00
|
|
|
max_field_lengths+= column->length > 255 ? 2 : 1;
|
2007-06-05 13:35:34 +02:00
|
|
|
not_block_record_extra_length++;
|
2007-01-18 20:38:14 +01:00
|
|
|
packed++;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
else if (type == FIELD_VARCHAR)
|
|
|
|
{
|
|
|
|
pack_reclength++;
|
2007-06-05 13:35:34 +02:00
|
|
|
not_block_record_extra_length++;
|
2007-01-18 20:38:14 +01:00
|
|
|
max_field_lengths++;
|
2011-06-10 01:02:58 +02:00
|
|
|
if (datafile_type != DYNAMIC_RECORD)
|
|
|
|
packed++;
|
2007-04-19 12:18:56 +02:00
|
|
|
column->fill_length= 1;
|
First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 18:08:06 +01:00
|
|
|
options|= HA_OPTION_NULL_FIELDS; /* Use ma_checksum() */
|
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
/* We must test for 257 as length includes pack-length */
|
2014-02-19 11:05:15 +01:00
|
|
|
if (MY_TEST(column->length >= 257))
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
|
|
|
long_varchar_count++;
|
2007-01-18 20:38:14 +01:00
|
|
|
max_field_lengths++;
|
2007-04-19 12:18:56 +02:00
|
|
|
column->fill_length= 2;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
}
|
2007-04-05 13:38:05 +02:00
|
|
|
else if (type == FIELD_SKIP_ZERO)
|
|
|
|
packed++;
|
|
|
|
else
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
2007-06-05 13:35:34 +02:00
|
|
|
if (!column->null_bit)
|
2007-04-19 12:18:56 +02:00
|
|
|
min_pack_length+= column->length;
|
2007-06-05 13:35:34 +02:00
|
|
|
else
|
2007-12-12 22:57:28 +01:00
|
|
|
{
|
|
|
|
/* Only BLOCK_RECORD skips NULL fields for all field values */
|
2007-06-05 13:35:34 +02:00
|
|
|
not_block_record_extra_length+= column->length;
|
2007-12-12 22:57:28 +01:00
|
|
|
}
|
2007-04-19 12:18:56 +02:00
|
|
|
column->empty_pos= 0;
|
|
|
|
column->empty_bit= 0;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
else /* FIELD_NORMAL */
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
2007-04-19 12:18:56 +02:00
|
|
|
if (!column->null_bit)
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
2007-06-05 13:35:34 +02:00
|
|
|
min_pack_length+= column->length;
|
2007-01-18 20:38:14 +01:00
|
|
|
share.base.fixed_not_null_fields++;
|
2007-04-19 12:18:56 +02:00
|
|
|
share.base.fixed_not_null_fields_length+= column->length;
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
2007-06-05 13:35:34 +02:00
|
|
|
else
|
|
|
|
not_block_record_extra_length+= column->length;
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
2007-06-05 13:35:34 +02:00
|
|
|
|
|
|
|
if (datafile_type == STATIC_RECORD && forced_packed)
|
|
|
|
{
|
|
|
|
/* Can't use fixed length records, revert to block records */
|
|
|
|
datafile_type= BLOCK_RECORD;
|
|
|
|
}
|
|
|
|
|
2011-01-14 11:54:39 +01:00
|
|
|
if (datafile_type == NO_RECORD && uniques)
|
|
|
|
{
|
|
|
|
/* Can't do unique without data, revert to block records */
|
|
|
|
datafile_type= BLOCK_RECORD;
|
|
|
|
}
|
|
|
|
|
2014-12-22 15:53:17 +01:00
|
|
|
if (encrypted)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
datafile_type is set (finally?)
|
|
|
|
update encryption that is only supported for BLOCK_RECORD
|
|
|
|
*/
|
|
|
|
if (datafile_type != BLOCK_RECORD)
|
|
|
|
{
|
|
|
|
encrypted= FALSE;
|
|
|
|
crypt_page_header_space= 0;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2007-06-05 13:35:34 +02:00
|
|
|
if (datafile_type == DYNAMIC_RECORD)
|
|
|
|
options|= HA_OPTION_PACK_RECORD; /* Must use packed records */
|
|
|
|
|
2011-01-14 11:54:39 +01:00
|
|
|
if (datafile_type == STATIC_RECORD || datafile_type == NO_RECORD)
|
2007-06-05 13:35:34 +02:00
|
|
|
{
|
|
|
|
/* We can't use checksum with static length rows */
|
|
|
|
flags&= ~HA_CREATE_CHECKSUM;
|
|
|
|
options&= ~HA_OPTION_CHECKSUM;
|
2007-12-12 22:57:28 +01:00
|
|
|
min_pack_length= reclength;
|
2007-06-05 13:35:34 +02:00
|
|
|
packed= 0;
|
|
|
|
}
|
2007-12-12 22:57:28 +01:00
|
|
|
else if (datafile_type != BLOCK_RECORD)
|
2007-06-05 13:35:34 +02:00
|
|
|
min_pack_length+= not_block_record_extra_length;
|
2007-12-12 22:57:28 +01:00
|
|
|
else
|
|
|
|
min_pack_length+= 5; /* Min row overhead */
|
2007-06-05 13:35:34 +02:00
|
|
|
|
2007-06-08 16:03:39 +02:00
|
|
|
if (flags & HA_CREATE_TMP_TABLE)
|
|
|
|
{
|
|
|
|
options|= HA_OPTION_TMP_TABLE;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
tmp_table= TRUE;
|
First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 18:08:06 +01:00
|
|
|
create_mode|= O_NOFOLLOW;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
/* "CREATE TEMPORARY" tables are not crash-safe (dropped at restart) */
|
2007-06-08 16:03:39 +02:00
|
|
|
ci->transactional= FALSE;
|
First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 18:08:06 +01:00
|
|
|
flags&= ~HA_CREATE_PAGE_CHECKSUM;
|
2007-06-08 16:03:39 +02:00
|
|
|
}
|
2007-01-18 20:38:14 +01:00
|
|
|
share.base.null_bytes= ci->null_bytes;
|
|
|
|
share.base.original_null_bytes= ci->null_bytes;
|
Maria:
* Don't modify share->base.born_transactional; now it is a value carved
in stone at creation time. share->now_transactional is what can be
modified: it starts at born_transactional, can become false during
ALTER TABLE (when we want no logging), and restored later.
* Not resetting create_rename_lsn to 0 during delete_all or repair.
* when we temporarily disable transactionality, we also change
the page type to PAGECACHE_PLAIN_PAGE: it bypasses some work in the
page cache (optimization), and avoids assertions related to LSNs.
* Disable INSERT DELAYED for transactional tables, because
durability could not be guaranteed (insertion may even not happen)
mysys/mf_keycache.c:
comment
storage/maria/ha_maria.cc:
* a transactional table cannot do INSERT DELAYED
* ha_maria::save_transactional not needed anymore, as now instead
we don't modify MARIA_SHARE::MARIA_BASE_INFO::born_transactional
(born_transactional plays the role of save_transactional), and modify
MARIA_SHARE::now_transactional.
* REPAIR_TABLE log record is now logged by maria_repair()
* comment why we rely on born_transactional to know if we should
skipping a transaction.
* putting together two if()s which test for F_UNLCK
storage/maria/ha_maria.h:
ha_maria::save_transactional not needed anymore (moved to the C layer)
storage/maria/ma_blockrec.c:
* For the block record's code (writing/updating/deleting records),
all that counts is now_transactional, not born_transactional.
* As we now set the page type to PAGECACHE_PLAIN_PAGE for tables
which have now_transactional==FALSE, pagecache will not expect
a meaningful LSN for them in pagecache_unlock_by_link(), so
we can pass it LSN_IMPOSSIBLE.
storage/maria/ma_check.c:
* writing LOGREC_REPAIR_TABLE moves from ha_maria::repair()
to maria_repair(), sounds cleaner (less functions to export).
* when opening a table during REPAIR, don't use the realpath-ed name,
as this may fail if the table has symlinked files (maria_open()
would try to find the data and index file in the directory
of unique_file_name, it would fail if data and index files are in
different dirs); use the unresolved name, open_file_name, which is
the argument which was passed to the maria_open() which created 'info'.
storage/maria/ma_close.c:
assert that when a statement is done with a table, it cleans up
storage/maria/ma_create.c:
new name
storage/maria/ma_delete_all.c:
* using now_transactional
* no reason to reset create_rename_lsn during delete_all (a bug);
also no reason to do it during repair: it was put there because
a positive create_rename_lsn caused a call to check_and_set_lsn()
which asserted in DBUG_ASSERT(block->type == PAGECACHE_LSN_PAGE);
first solution was to use LSN_IMPOSSIBLE in _ma_unpin_all_pages() if
not transactional; but then in the case of ALTER TABLE, with
transactionality temporarily disabled, it asserted in
DBUG_ASSERT(LSN_VALID(lsn)) in pagecache_fwrite() (PAGECACHE_LSN_PAGE
page with zero LSN - bad). The additional solution is to use
PAGECACHE_PLAIN_PAGE when we disable transactionality temporarily: this
avoids checks on the LSN, and also bypasses (optimization) the "flush
log up to LSN" call when the pagecache flushes our page (in other
words, no WAL needed).
storage/maria/ma_delete_table.c:
use now_transactional
storage/maria/ma_locking.c:
assert that when a statement is done with a table, it cleans up.
storage/maria/ma_loghandler.c:
* now_transactional should be used to test if we want a log record.
* Assertions to make sure dummy_transaction_object is not spoilt
by its many users.
storage/maria/ma_open.c:
base.transactional -> base.born_transactional
storage/maria/ma_pagecache.c:
missing name for page's type. Comment for future.
storage/maria/ma_rename.c:
use now_transactional
storage/maria/maria_chk.c:
use born_transactional
storage/maria/maria_def.h:
MARIA_BASE_INFO::transactional renamed to born_transactional.
MARIA_SHARE::now_transactional introduced.
_ma_repair_write_log_record() is made local to ma_check.c.
Macros to temporarily disable, and re-enable, transactionality for a
table.
storage/maria/maria_read_log.c:
assertions and using the new macros. Adding a forgotten resetting
when we finally close all tables.
2007-07-03 15:20:41 +02:00
|
|
|
share.base.born_transactional= ci->transactional;
|
2007-01-18 20:38:14 +01:00
|
|
|
share.base.max_field_lengths= max_field_lengths;
|
|
|
|
share.base.field_offsets= 0; /* for future */
|
First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 18:08:06 +01:00
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
if (flags & HA_CREATE_CHECKSUM || (options & HA_OPTION_CHECKSUM))
|
|
|
|
{
|
|
|
|
options|= HA_OPTION_CHECKSUM;
|
|
|
|
min_pack_length++;
|
2007-01-18 20:38:14 +01:00
|
|
|
pack_reclength++;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
2008-01-07 17:54:41 +01:00
|
|
|
if (pack_reclength < INT_MAX32)
|
|
|
|
pack_reclength+= max_field_lengths + long_varchar_count;
|
|
|
|
else
|
|
|
|
pack_reclength= INT_MAX32;
|
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
if (flags & HA_CREATE_DELAY_KEY_WRITE)
|
|
|
|
options|= HA_OPTION_DELAY_KEY_WRITE;
|
|
|
|
if (flags & HA_CREATE_RELIES_ON_SQL_LAYER)
|
|
|
|
options|= HA_OPTION_RELIES_ON_SQL_LAYER;
|
2007-10-09 20:09:50 +02:00
|
|
|
if (flags & HA_CREATE_PAGE_CHECKSUM)
|
|
|
|
options|= HA_OPTION_PAGE_CHECKSUM;
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2007-01-18 20:38:14 +01:00
|
|
|
pack_bytes= (packed + 7) / 8;
|
2006-04-11 15:45:10 +02:00
|
|
|
if (pack_reclength != INT_MAX32)
|
2007-01-18 20:38:14 +01:00
|
|
|
pack_reclength+= reclength+pack_bytes +
|
2014-02-19 11:05:15 +01:00
|
|
|
MY_TEST(test_all_bits(options, HA_OPTION_CHECKSUM |
|
|
|
|
HA_OPTION_PACK_RECORD));
|
2007-01-18 20:38:14 +01:00
|
|
|
min_pack_length+= pack_bytes;
|
|
|
|
/* Calculate min possible row length for rows-in-block */
|
|
|
|
extra_header_size= MAX_FIXED_HEADER_SIZE;
|
|
|
|
if (ci->transactional)
|
2007-08-09 10:51:15 +02:00
|
|
|
{
|
2007-01-18 20:38:14 +01:00
|
|
|
extra_header_size= TRANS_MAX_FIXED_HEADER_SIZE;
|
2007-08-09 10:51:15 +02:00
|
|
|
DBUG_PRINT("info",("creating a transactional table"));
|
|
|
|
}
|
2007-12-12 22:57:28 +01:00
|
|
|
share.base.min_block_length= (extra_header_size + share.base.null_bytes +
|
|
|
|
pack_bytes);
|
2006-04-11 15:45:10 +02:00
|
|
|
if (!ci->data_file_length && ci->max_rows)
|
|
|
|
{
|
2018-01-18 00:41:52 +01:00
|
|
|
set_if_bigger(ci->max_rows, ci->reloc_rows);
|
2006-04-11 15:45:10 +02:00
|
|
|
if (pack_reclength == INT_MAX32 ||
|
|
|
|
(~(ulonglong) 0)/ci->max_rows < (ulonglong) pack_reclength)
|
|
|
|
ci->data_file_length= ~(ulonglong) 0;
|
|
|
|
else
|
2011-07-24 10:25:28 +02:00
|
|
|
{
|
|
|
|
ci->data_file_length= _ma_safe_mul(ci->max_rows, pack_reclength);
|
|
|
|
if (datafile_type == BLOCK_RECORD)
|
|
|
|
{
|
|
|
|
/* Assume that blocks are only half full (very pessimistic!) */
|
|
|
|
ci->data_file_length= _ma_safe_mul(ci->data_file_length, 2);
|
|
|
|
set_if_bigger(ci->data_file_length, maria_block_size*2);
|
|
|
|
}
|
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
else if (!ci->max_rows)
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
2007-04-19 12:18:56 +02:00
|
|
|
if (datafile_type == BLOCK_RECORD)
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
2014-12-22 15:53:17 +01:00
|
|
|
uint rows_per_page=
|
|
|
|
((maria_block_size - PAGE_OVERHEAD_SIZE_RAW - crypt_page_header_space)
|
|
|
|
/ (min_pack_length + extra_header_size + DIR_ENTRY_SIZE));
|
2007-01-18 20:38:14 +01:00
|
|
|
ulonglong data_file_length= ci->data_file_length;
|
2007-04-19 12:18:56 +02:00
|
|
|
if (!data_file_length)
|
2007-01-18 20:38:14 +01:00
|
|
|
data_file_length= ((((ulonglong) 1 << ((BLOCK_RECORD_POINTER_SIZE-1) *
|
2011-07-24 10:25:28 +02:00
|
|
|
8))/2 -1) * maria_block_size);
|
2007-01-18 20:38:14 +01:00
|
|
|
if (rows_per_page > 0)
|
|
|
|
{
|
|
|
|
set_if_smaller(rows_per_page, MAX_ROWS_PER_PAGE);
|
2017-06-30 16:56:58 +02:00
|
|
|
ci->max_rows= (data_file_length / maria_block_size+1) * rows_per_page;
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
|
|
|
else
|
2007-04-19 12:18:56 +02:00
|
|
|
ci->max_rows= data_file_length / (min_pack_length +
|
|
|
|
extra_header_size +
|
2018-01-18 00:41:52 +01:00
|
|
|
DIR_ENTRY_SIZE);
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
ci->max_rows=(ha_rows) (ci->data_file_length/(min_pack_length +
|
|
|
|
((options &
|
|
|
|
HA_OPTION_PACK_RECORD) ?
|
2018-01-18 00:41:52 +01:00
|
|
|
3 : 0)));
|
|
|
|
set_if_smaller(ci->reloc_rows, ci->max_rows);
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
|
|
|
max_rows= (ulonglong) ci->max_rows;
|
2007-04-19 12:18:56 +02:00
|
|
|
if (datafile_type == BLOCK_RECORD)
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
2007-10-09 20:09:50 +02:00
|
|
|
/*
|
|
|
|
The + 1 is for record position withing page
|
2011-07-24 10:25:28 +02:00
|
|
|
The * 2 is because we need one bit for knowing if there is transid's
|
2007-10-09 20:09:50 +02:00
|
|
|
after the row pointer
|
|
|
|
*/
|
2007-01-18 20:38:14 +01:00
|
|
|
pointer= maria_get_pointer_length((ci->data_file_length /
|
2014-02-11 18:40:33 +01:00
|
|
|
maria_block_size) * 2, 4) + 1;
|
2007-01-18 20:38:14 +01:00
|
|
|
set_if_smaller(pointer, BLOCK_RECORD_POINTER_SIZE);
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2007-01-18 20:38:14 +01:00
|
|
|
if (!max_rows)
|
|
|
|
max_rows= (((((ulonglong) 1 << ((pointer-1)*8)) -1) * maria_block_size) /
|
2007-10-09 20:09:50 +02:00
|
|
|
min_pack_length / 2);
|
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
else
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
2011-01-14 11:54:39 +01:00
|
|
|
if (datafile_type == NO_RECORD)
|
|
|
|
pointer= 0;
|
|
|
|
else if (datafile_type != STATIC_RECORD)
|
2007-01-18 20:38:14 +01:00
|
|
|
pointer= maria_get_pointer_length(ci->data_file_length,
|
|
|
|
maria_data_pointer_size);
|
|
|
|
else
|
|
|
|
pointer= maria_get_pointer_length(ci->max_rows, maria_data_pointer_size);
|
|
|
|
if (!max_rows)
|
|
|
|
max_rows= ((((ulonglong) 1 << (pointer*8)) -1) / min_pack_length);
|
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
real_reclength=reclength;
|
2007-04-19 12:18:56 +02:00
|
|
|
if (datafile_type == STATIC_RECORD)
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
|
|
|
if (reclength <= pointer)
|
|
|
|
reclength=pointer+1; /* reserve place for delete link */
|
|
|
|
}
|
|
|
|
else
|
|
|
|
reclength+= long_varchar_count; /* We need space for varchar! */
|
|
|
|
|
|
|
|
max_key_length=0; tot_length=0 ; key_segs=0;
|
|
|
|
fulltext_keys=0;
|
2007-10-09 20:09:50 +02:00
|
|
|
share.state.rec_per_key_part= rec_per_key_part;
|
|
|
|
share.state.nulls_per_key_part= nulls_per_key_part;
|
2006-04-11 15:45:10 +02:00
|
|
|
share.state.key_root=key_root;
|
2007-01-18 20:38:14 +01:00
|
|
|
share.state.key_del= HA_OFFSET_ERROR;
|
2006-04-11 15:45:10 +02:00
|
|
|
if (uniques)
|
2007-01-18 20:38:14 +01:00
|
|
|
max_key_length= MARIA_UNIQUE_HASH_LENGTH + pointer;
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
for (i=0, keydef=keydefs ; i < keys ; i++ , keydef++)
|
|
|
|
{
|
|
|
|
share.state.key_root[i]= HA_OFFSET_ERROR;
|
2007-12-10 01:32:00 +01:00
|
|
|
length= real_length_diff= 0;
|
|
|
|
min_key_length= key_length= pointer;
|
2007-12-18 02:21:32 +01:00
|
|
|
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
if (keydef->key_alg == HA_KEY_ALG_RTREE)
|
|
|
|
keydef->flag|= HA_RTREE_INDEX; /* For easier tests */
|
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
if (keydef->flag & HA_SPATIAL)
|
|
|
|
{
|
|
|
|
#ifdef HAVE_SPATIAL
|
|
|
|
/* BAR TODO to support 3D and more dimensions in the future */
|
|
|
|
uint sp_segs=SPDIMS*2;
|
|
|
|
keydef->flag=HA_SPATIAL;
|
|
|
|
|
|
|
|
if (flags & HA_DONT_TOUCH_DATA)
|
|
|
|
{
|
|
|
|
/*
|
2007-01-18 20:38:14 +01:00
|
|
|
Called by maria_chk - i.e. table structure was taken from
|
|
|
|
MYI file and SPATIAL key *does have* additional sp_segs keysegs.
|
|
|
|
keydef->seg here points right at the GEOMETRY segment,
|
|
|
|
so we only need to decrease keydef->keysegs.
|
|
|
|
(see maria_recreate_table() in _ma_check.c)
|
2006-04-11 15:45:10 +02:00
|
|
|
*/
|
|
|
|
keydef->keysegs-=sp_segs-1;
|
|
|
|
}
|
|
|
|
|
|
|
|
for (j=0, keyseg=keydef->seg ; (int) j < keydef->keysegs ;
|
|
|
|
j++, keyseg++)
|
|
|
|
{
|
|
|
|
if (keyseg->type != HA_KEYTYPE_BINARY &&
|
|
|
|
keyseg->type != HA_KEYTYPE_VARBINARY1 &&
|
|
|
|
keyseg->type != HA_KEYTYPE_VARBINARY2)
|
|
|
|
{
|
|
|
|
my_errno=HA_WRONG_CREATE_OPTION;
|
2007-04-05 13:38:05 +02:00
|
|
|
goto err_no_lock;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
keydef->keysegs+=sp_segs;
|
|
|
|
key_length+=SPLEN*sp_segs;
|
2007-07-02 19:45:15 +02:00
|
|
|
length++; /* At least one length uchar */
|
2007-12-10 01:32:00 +01:00
|
|
|
min_key_length++;
|
2006-04-11 15:45:10 +02:00
|
|
|
#else
|
|
|
|
my_errno= HA_ERR_UNSUPPORTED;
|
2007-04-05 13:38:05 +02:00
|
|
|
goto err_no_lock;
|
2006-04-11 15:45:10 +02:00
|
|
|
#endif /*HAVE_SPATIAL*/
|
|
|
|
}
|
|
|
|
else if (keydef->flag & HA_FULLTEXT)
|
|
|
|
{
|
|
|
|
keydef->flag=HA_FULLTEXT | HA_PACK_KEY | HA_VAR_LENGTH_KEY;
|
|
|
|
options|=HA_OPTION_PACK_KEYS; /* Using packed keys */
|
|
|
|
|
|
|
|
for (j=0, keyseg=keydef->seg ; (int) j < keydef->keysegs ;
|
|
|
|
j++, keyseg++)
|
|
|
|
{
|
|
|
|
if (keyseg->type != HA_KEYTYPE_TEXT &&
|
|
|
|
keyseg->type != HA_KEYTYPE_VARTEXT1 &&
|
|
|
|
keyseg->type != HA_KEYTYPE_VARTEXT2)
|
|
|
|
{
|
|
|
|
my_errno=HA_WRONG_CREATE_OPTION;
|
2007-04-05 13:38:05 +02:00
|
|
|
goto err_no_lock;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
if (!(keyseg->flag & HA_BLOB_PART) &&
|
|
|
|
(keyseg->type == HA_KEYTYPE_VARTEXT1 ||
|
|
|
|
keyseg->type == HA_KEYTYPE_VARTEXT2))
|
|
|
|
{
|
|
|
|
/* Make a flag that this is a VARCHAR */
|
|
|
|
keyseg->flag|= HA_VAR_LENGTH_PART;
|
|
|
|
/* Store in bit_start number of bytes used to pack the length */
|
|
|
|
keyseg->bit_start= ((keyseg->type == HA_KEYTYPE_VARTEXT1)?
|
|
|
|
1 : 2);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
fulltext_keys++;
|
|
|
|
key_length+= HA_FT_MAXBYTELEN+HA_FT_WLEN;
|
2007-07-02 19:45:15 +02:00
|
|
|
length++; /* At least one length uchar */
|
2007-12-10 01:32:00 +01:00
|
|
|
min_key_length+= 1 + HA_FT_WLEN;
|
2006-04-11 15:45:10 +02:00
|
|
|
real_length_diff=HA_FT_MAXBYTELEN-FT_MAX_WORD_LEN_FOR_SORT;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* Test if prefix compression */
|
|
|
|
if (keydef->flag & HA_PACK_KEY)
|
|
|
|
{
|
|
|
|
/* Can't use space_compression on number keys */
|
|
|
|
if ((keydef->seg[0].flag & HA_SPACE_PACK) &&
|
|
|
|
keydef->seg[0].type == (int) HA_KEYTYPE_NUM)
|
|
|
|
keydef->seg[0].flag&= ~HA_SPACE_PACK;
|
|
|
|
|
|
|
|
/* Only use HA_PACK_KEY when first segment is a variable length key */
|
|
|
|
if (!(keydef->seg[0].flag & (HA_SPACE_PACK | HA_BLOB_PART |
|
|
|
|
HA_VAR_LENGTH_PART)))
|
|
|
|
{
|
|
|
|
/* pack relative to previous key */
|
|
|
|
keydef->flag&= ~HA_PACK_KEY;
|
|
|
|
keydef->flag|= HA_BINARY_PACK_KEY | HA_VAR_LENGTH_KEY;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
keydef->seg[0].flag|=HA_PACK_KEY; /* for easyer intern test */
|
|
|
|
keydef->flag|=HA_VAR_LENGTH_KEY;
|
|
|
|
options|=HA_OPTION_PACK_KEYS; /* Using packed keys */
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (keydef->flag & HA_BINARY_PACK_KEY)
|
|
|
|
options|=HA_OPTION_PACK_KEYS; /* Using packed keys */
|
|
|
|
|
|
|
|
if (keydef->flag & HA_AUTO_KEY && ci->with_auto_increment)
|
|
|
|
share.base.auto_key=i+1;
|
|
|
|
for (j=0, keyseg=keydef->seg ; j < keydef->keysegs ; j++, keyseg++)
|
|
|
|
{
|
|
|
|
/* numbers are stored with high by first to make compression easier */
|
|
|
|
switch (keyseg->type) {
|
|
|
|
case HA_KEYTYPE_SHORT_INT:
|
|
|
|
case HA_KEYTYPE_LONG_INT:
|
|
|
|
case HA_KEYTYPE_FLOAT:
|
|
|
|
case HA_KEYTYPE_DOUBLE:
|
|
|
|
case HA_KEYTYPE_USHORT_INT:
|
|
|
|
case HA_KEYTYPE_ULONG_INT:
|
|
|
|
case HA_KEYTYPE_LONGLONG:
|
|
|
|
case HA_KEYTYPE_ULONGLONG:
|
|
|
|
case HA_KEYTYPE_INT24:
|
|
|
|
case HA_KEYTYPE_UINT24:
|
|
|
|
case HA_KEYTYPE_INT8:
|
|
|
|
keyseg->flag|= HA_SWAP_KEY;
|
|
|
|
break;
|
|
|
|
case HA_KEYTYPE_VARTEXT1:
|
|
|
|
case HA_KEYTYPE_VARTEXT2:
|
|
|
|
case HA_KEYTYPE_VARBINARY1:
|
|
|
|
case HA_KEYTYPE_VARBINARY2:
|
|
|
|
if (!(keyseg->flag & HA_BLOB_PART))
|
|
|
|
{
|
|
|
|
/* Make a flag that this is a VARCHAR */
|
|
|
|
keyseg->flag|= HA_VAR_LENGTH_PART;
|
|
|
|
/* Store in bit_start number of bytes used to pack the length */
|
|
|
|
keyseg->bit_start= ((keyseg->type == HA_KEYTYPE_VARTEXT1 ||
|
|
|
|
keyseg->type == HA_KEYTYPE_VARBINARY1) ?
|
|
|
|
1 : 2);
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
if (keyseg->flag & HA_SPACE_PACK)
|
|
|
|
{
|
2007-12-10 01:32:00 +01:00
|
|
|
DBUG_ASSERT(!(keyseg->flag & (HA_VAR_LENGTH_PART | HA_BLOB_PART)));
|
2006-04-11 15:45:10 +02:00
|
|
|
keydef->flag |= HA_SPACE_PACK_USED | HA_VAR_LENGTH_KEY;
|
|
|
|
options|=HA_OPTION_PACK_KEYS; /* Using packed keys */
|
2007-07-02 19:45:15 +02:00
|
|
|
length++; /* At least one length uchar */
|
2008-01-22 18:49:05 +01:00
|
|
|
if (!keyseg->null_bit)
|
|
|
|
min_key_length++;
|
2007-12-10 01:32:00 +01:00
|
|
|
key_length+= keyseg->length;
|
2006-04-11 15:45:10 +02:00
|
|
|
if (keyseg->length >= 255)
|
2007-12-10 01:32:00 +01:00
|
|
|
{
|
|
|
|
/* prefix may be 3 bytes */
|
|
|
|
length+= 2;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
}
|
2007-12-10 01:32:00 +01:00
|
|
|
else if (keyseg->flag & (HA_VAR_LENGTH_PART | HA_BLOB_PART))
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
|
|
|
DBUG_ASSERT(!test_all_bits(keyseg->flag,
|
|
|
|
(HA_VAR_LENGTH_PART | HA_BLOB_PART)));
|
|
|
|
keydef->flag|=HA_VAR_LENGTH_KEY;
|
2007-07-02 19:45:15 +02:00
|
|
|
length++; /* At least one length uchar */
|
2008-01-22 18:49:05 +01:00
|
|
|
if (!keyseg->null_bit)
|
|
|
|
min_key_length++;
|
2006-04-11 15:45:10 +02:00
|
|
|
options|=HA_OPTION_PACK_KEYS; /* Using packed keys */
|
2007-12-10 01:32:00 +01:00
|
|
|
key_length+= keyseg->length;
|
2006-04-11 15:45:10 +02:00
|
|
|
if (keyseg->length >= 255)
|
2007-12-10 01:32:00 +01:00
|
|
|
{
|
|
|
|
/* prefix may be 3 bytes */
|
|
|
|
length+= 2;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
}
|
2007-12-10 01:32:00 +01:00
|
|
|
else
|
|
|
|
{
|
|
|
|
key_length+= keyseg->length;
|
|
|
|
if (!keyseg->null_bit)
|
|
|
|
min_key_length+= keyseg->length;
|
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
if (keyseg->null_bit)
|
|
|
|
{
|
|
|
|
key_length++;
|
2007-12-10 01:32:00 +01:00
|
|
|
/* min key part is 1 byte */
|
|
|
|
min_key_length++;
|
2006-04-11 15:45:10 +02:00
|
|
|
options|=HA_OPTION_PACK_KEYS;
|
|
|
|
keyseg->flag|=HA_NULL_PART;
|
|
|
|
keydef->flag|=HA_VAR_LENGTH_KEY | HA_NULL_PART_KEY;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
} /* if HA_FULLTEXT */
|
|
|
|
key_segs+=keydef->keysegs;
|
|
|
|
if (keydef->keysegs > HA_MAX_KEY_SEG)
|
|
|
|
{
|
|
|
|
my_errno=HA_WRONG_CREATE_OPTION;
|
2007-04-05 13:38:05 +02:00
|
|
|
goto err_no_lock;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
/*
|
|
|
|
key_segs may be 0 in the case when we only want to be able to
|
|
|
|
add on row into the table. This can happen with some DISTINCT queries
|
|
|
|
in MySQL
|
|
|
|
*/
|
|
|
|
if ((keydef->flag & (HA_NOSAME | HA_NULL_PART_KEY)) == HA_NOSAME &&
|
|
|
|
key_segs)
|
|
|
|
share.state.rec_per_key_part[key_segs-1]=1L;
|
|
|
|
length+=key_length;
|
2007-04-19 12:18:56 +02:00
|
|
|
/*
|
|
|
|
A key can't be longer than than half a index block (as we have
|
|
|
|
to be able to put at least 2 keys on an index block for the key
|
|
|
|
algorithms to work).
|
|
|
|
*/
|
2011-10-26 19:25:18 +02:00
|
|
|
if (length > _ma_max_key_length())
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
|
|
|
my_errno=HA_WRONG_CREATE_OPTION;
|
2007-04-05 13:38:05 +02:00
|
|
|
goto err_no_lock;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
2008-01-10 20:21:36 +01:00
|
|
|
keydef->block_length= (uint16) maria_block_size;
|
2006-04-11 15:45:10 +02:00
|
|
|
keydef->keylength= (uint16) key_length;
|
2007-12-10 01:32:00 +01:00
|
|
|
keydef->minlength= (uint16) min_key_length;
|
2006-04-11 15:45:10 +02:00
|
|
|
keydef->maxlength= (uint16) length;
|
|
|
|
|
|
|
|
if (length > max_key_length)
|
|
|
|
max_key_length= length;
|
2017-09-24 20:03:44 +02:00
|
|
|
|
2017-09-26 18:36:19 +02:00
|
|
|
tot_length= update_tot_length(tot_length, max_rows, length);
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
unique_key_parts=0;
|
|
|
|
for (i=0, uniquedef=uniquedefs ; i < uniques ; i++ , uniquedef++)
|
|
|
|
{
|
|
|
|
uniquedef->key=keys+i;
|
|
|
|
unique_key_parts+=uniquedef->keysegs;
|
|
|
|
share.state.key_root[keys+i]= HA_OFFSET_ERROR;
|
2017-09-24 20:03:44 +02:00
|
|
|
|
2017-09-26 18:36:19 +02:00
|
|
|
tot_length= update_tot_length(tot_length, max_rows, MARIA_UNIQUE_HASH_LENGTH + pointer);
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
keys+=uniques; /* Each unique has 1 key */
|
|
|
|
key_segs+=uniques; /* Each unique has 1 key seg */
|
|
|
|
|
|
|
|
base_pos=(MARIA_STATE_INFO_SIZE + keys * MARIA_STATE_KEY_SIZE +
|
2007-01-18 20:38:14 +01:00
|
|
|
key_segs * MARIA_STATE_KEYSEG_SIZE);
|
|
|
|
info_length= base_pos+(uint) (MARIA_BASE_INFO_SIZE+
|
|
|
|
keys * MARIA_KEYDEF_SIZE+
|
|
|
|
uniques * MARIA_UNIQUEDEF_SIZE +
|
|
|
|
(key_segs + unique_key_parts)*HA_KEYSEG_SIZE+
|
2007-10-09 20:09:50 +02:00
|
|
|
columns*(MARIA_COLUMNDEF_SIZE + 2));
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2014-12-22 15:53:17 +01:00
|
|
|
if (encrypted)
|
|
|
|
{
|
|
|
|
share.base.extra_options|= MA_EXTRA_OPTIONS_ENCRYPTED;
|
|
|
|
|
|
|
|
/* store crypt data in info */
|
|
|
|
info_length+= ma_crypt_get_file_length();
|
|
|
|
}
|
|
|
|
|
|
|
|
if (insert_order)
|
|
|
|
{
|
|
|
|
share.base.extra_options|= MA_EXTRA_OPTIONS_INSERT_ORDER;
|
|
|
|
}
|
|
|
|
|
|
|
|
DBUG_PRINT("info", ("info_length: %u", info_length));
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
/* There are only 16 bits for the total header length. */
|
|
|
|
if (info_length > 65535)
|
|
|
|
{
|
2009-01-15 23:25:53 +01:00
|
|
|
my_printf_error(HA_WRONG_CREATE_OPTION,
|
2010-09-12 18:40:01 +02:00
|
|
|
"Aria table '%s' has too many columns and/or "
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
"indexes and/or unique constraints.",
|
|
|
|
MYF(0), name + dirname_length(name));
|
|
|
|
my_errno= HA_WRONG_CREATE_OPTION;
|
2007-04-05 13:38:05 +02:00
|
|
|
goto err_no_lock;
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
}
|
|
|
|
|
2009-01-09 05:23:25 +01:00
|
|
|
bmove(share.state.header.file_version, maria_file_magic, 4);
|
2007-10-09 20:09:50 +02:00
|
|
|
ci->old_options=options | (ci->old_options & HA_OPTION_TEMP_COMPRESS_RECORD ?
|
|
|
|
HA_OPTION_COMPRESS_RECORD |
|
|
|
|
HA_OPTION_TEMP_COMPRESS_RECORD: 0);
|
2006-04-11 15:45:10 +02:00
|
|
|
mi_int2store(share.state.header.options,ci->old_options);
|
|
|
|
mi_int2store(share.state.header.header_length,info_length);
|
|
|
|
mi_int2store(share.state.header.state_info_length,MARIA_STATE_INFO_SIZE);
|
|
|
|
mi_int2store(share.state.header.base_info_length,MARIA_BASE_INFO_SIZE);
|
|
|
|
mi_int2store(share.state.header.base_pos,base_pos);
|
2007-06-28 14:01:57 +02:00
|
|
|
share.state.header.data_file_type= share.data_file_type= datafile_type;
|
2007-04-19 12:18:56 +02:00
|
|
|
share.state.header.org_data_file_type= org_datafile_type;
|
2016-12-02 13:02:30 +01:00
|
|
|
share.state.header.not_used= 0;
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
share.state.dellink = HA_OFFSET_ERROR;
|
2007-01-18 20:38:14 +01:00
|
|
|
share.state.first_bitmap_with_space= 0;
|
2011-07-04 03:32:53 +02:00
|
|
|
#ifdef MARIA_EXTERNAL_LOCKING
|
2006-04-11 15:45:10 +02:00
|
|
|
share.state.process= (ulong) getpid();
|
First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 18:08:06 +01:00
|
|
|
#endif
|
2006-04-11 15:45:10 +02:00
|
|
|
share.state.version= (ulong) time((time_t*) 0);
|
|
|
|
share.state.sortkey= (ushort) ~0;
|
|
|
|
share.state.auto_increment=ci->auto_increment;
|
|
|
|
share.options=options;
|
|
|
|
share.base.rec_reflength=pointer;
|
2007-01-18 20:38:14 +01:00
|
|
|
share.base.block_size= maria_block_size;
|
2016-12-02 13:02:30 +01:00
|
|
|
share.base.language= (ci->language ? ci->language :
|
|
|
|
default_charset_info->number);
|
2007-01-18 20:38:14 +01:00
|
|
|
|
2008-01-31 02:06:04 +01:00
|
|
|
/*
|
|
|
|
Get estimate for index file length (this may be wrong for FT keys)
|
|
|
|
This is used for pointers to other key pages.
|
|
|
|
*/
|
2017-09-24 20:03:44 +02:00
|
|
|
tmp= (tot_length / maria_block_size + keys * MARIA_INDEX_BLOCK_MARGIN);
|
2008-01-31 02:06:04 +01:00
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
/*
|
|
|
|
use maximum of key_file_length we calculated and key_file_length value we
|
2008-01-31 02:06:04 +01:00
|
|
|
got from MAI file header (see also mariapack.c:save_state)
|
2006-04-11 15:45:10 +02:00
|
|
|
*/
|
|
|
|
share.base.key_reflength=
|
2013-03-25 23:03:13 +01:00
|
|
|
maria_get_pointer_length(MY_MAX(ci->key_file_length,tmp),3);
|
2006-04-11 15:45:10 +02:00
|
|
|
share.base.keys= share.state.header.keys= keys;
|
|
|
|
share.state.header.uniques= uniques;
|
|
|
|
share.state.header.fulltext_keys= fulltext_keys;
|
|
|
|
mi_int2store(share.state.header.key_parts,key_segs);
|
|
|
|
mi_int2store(share.state.header.unique_key_parts,unique_key_parts);
|
|
|
|
|
|
|
|
maria_set_all_keys_active(share.state.key_map, keys);
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
share.base.keystart = share.state.state.key_file_length=
|
2007-01-18 20:38:14 +01:00
|
|
|
MY_ALIGN(info_length, maria_block_size);
|
|
|
|
share.base.max_key_block_length= maria_block_size;
|
2006-04-11 15:45:10 +02:00
|
|
|
share.base.max_key_length=ALIGN_SIZE(max_key_length+4);
|
|
|
|
share.base.records=ci->max_rows;
|
|
|
|
share.base.reloc= ci->reloc_rows;
|
|
|
|
share.base.reclength=real_reclength;
|
2014-02-19 11:05:15 +01:00
|
|
|
share.base.pack_reclength= reclength + MY_TEST(options & HA_OPTION_CHECKSUM);
|
2006-04-11 15:45:10 +02:00
|
|
|
share.base.max_pack_length=pack_reclength;
|
|
|
|
share.base.min_pack_length=min_pack_length;
|
2007-01-18 20:38:14 +01:00
|
|
|
share.base.pack_bytes= pack_bytes;
|
|
|
|
share.base.fields= columns;
|
|
|
|
share.base.pack_fields= packed;
|
2007-09-11 00:58:15 +02:00
|
|
|
|
|
|
|
if (share.data_file_type == BLOCK_RECORD)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
we are going to create a first bitmap page, set data_file_length
|
|
|
|
to reflect this, before the state goes to disk
|
|
|
|
*/
|
|
|
|
share.state.state.data_file_length= maria_block_size;
|
|
|
|
/* Add length of packed fields + length */
|
|
|
|
share.base.pack_reclength+= share.base.max_field_lengths+3;
|
2018-01-18 00:41:52 +01:00
|
|
|
share.base.max_pack_length= share.base.pack_reclength;
|
2007-09-11 00:58:15 +02:00
|
|
|
|
2008-01-07 17:54:41 +01:00
|
|
|
/* Adjust max_pack_length, to be used if we have short rows */
|
|
|
|
if (share.base.max_pack_length < maria_block_size)
|
|
|
|
{
|
|
|
|
share.base.max_pack_length+= FLAG_SIZE;
|
|
|
|
if (ci->transactional)
|
|
|
|
share.base.max_pack_length+= TRANSID_SIZE * 2;
|
|
|
|
}
|
2007-09-11 00:58:15 +02:00
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
/* max_data_file_length and max_key_file_length are recalculated on open */
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
if (tmp_table)
|
2007-01-18 20:38:14 +01:00
|
|
|
share.base.max_data_file_length= (my_off_t) ci->data_file_length;
|
2007-12-15 14:44:29 +01:00
|
|
|
else if (ci->transactional && translog_status == TRANSLOG_OK &&
|
|
|
|
!maria_in_recovery)
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
we have checked translog_inited above, because maria_chk may call us
|
|
|
|
(via maria_recreate_table()) and it does not have a log.
|
|
|
|
*/
|
|
|
|
sync_dir= MY_SYNC_DIR;
|
2009-01-16 22:00:39 +01:00
|
|
|
/*
|
|
|
|
If crash between _ma_state_info_write_sub() and
|
|
|
|
_ma_update_state__lsns_sub(), table should be ignored by Recovery (or
|
|
|
|
old REDOs would fail), so we cannot let LSNs be 0:
|
|
|
|
*/
|
|
|
|
share.state.skip_redo_lsn= share.state.is_of_horizon=
|
|
|
|
share.state.create_rename_lsn= LSN_MAX;
|
WL#3072 Maria Recovery. Making DDLs durable in Maria:
Sync table files after CREATE (of non-temp table), DROP, RENAME,
TRUNCATE, sync directories and symlinks (for the 3 first commands).
Comments for future log records.
In ma_rename(), if rename of index works and then rename of data fails,
try to undo the rename of the index to leave a consistent state.
mysys/my_symlink.c:
sync directory after creation of a symbolic link in it, if asked
mysys/my_sync.c:
comment. Fix for when the file's name has no directory in it.
storage/maria/ma_create.c:
sync files and links and dirs when creating a non-temporary table.
Optimizations of the above to reduce syncs in the common cases:
* if index file and data file have the exact same paths (regular
and link), sync the directories (of regular and link) only once
after creating the last file (the data file).
* don't sync the data file if we didn't write to it (always true
in our builds).
storage/maria/ma_delete_all.c:
sync files after truncating a table
storage/maria/ma_delete_table.c:
sync files and symbolic links and dirs after dropping a table
storage/maria/ma_extra.c:
a function which wraps the sync of the index file and the sync of the
data file.
storage/maria/ma_locking.c:
using a wrapper function
storage/maria/ma_rename.c:
sync files and symbolic links and dirs after renaming a table.
If rename of index works and then rename of data fails, try to undo
the rename of the index to leave a consistent state. That is just a
try, it may fail...
storage/maria/ma_test3.c:
warning to not pay attention to this test.
storage/maria/maria_def.h:
declaration for the function added to ma_extra.c
2006-11-27 22:01:29 +01:00
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2007-12-12 22:57:28 +01:00
|
|
|
if (datafile_type == DYNAMIC_RECORD)
|
2007-04-05 13:38:05 +02:00
|
|
|
{
|
|
|
|
share.base.min_block_length=
|
|
|
|
(share.base.pack_reclength+3 < MARIA_EXTEND_BLOCK_LENGTH &&
|
|
|
|
! share.base.blobs) ?
|
2013-03-25 23:03:13 +01:00
|
|
|
MY_MAX(share.base.pack_reclength,MARIA_MIN_BLOCK_LENGTH) :
|
2007-04-05 13:38:05 +02:00
|
|
|
MARIA_EXTEND_BLOCK_LENGTH;
|
|
|
|
}
|
2007-12-12 22:57:28 +01:00
|
|
|
else if (datafile_type == STATIC_RECORD)
|
|
|
|
share.base.min_block_length= share.base.pack_reclength;
|
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
if (! (flags & HA_DONT_TOUCH_DATA))
|
2010-01-14 17:51:00 +01:00
|
|
|
share.state.create_time= time((time_t*) 0);
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2013-03-25 23:03:13 +01:00
|
|
|
if (!internal_table)
|
|
|
|
mysql_mutex_lock(&THR_LOCK_maria);
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2007-07-27 12:06:39 +02:00
|
|
|
/*
|
|
|
|
NOTE: For test_if_reopen() we need a real path name. Hence we need
|
|
|
|
MY_RETURN_REAL_PATH for every fn_format(filename, ...).
|
|
|
|
*/
|
2006-04-11 15:45:10 +02:00
|
|
|
if (ci->index_file_name)
|
|
|
|
{
|
|
|
|
char *iext= strrchr(ci->index_file_name, '.');
|
|
|
|
int have_iext= iext && !strcmp(iext, MARIA_NAME_IEXT);
|
WL#3072 Maria Recovery. Making DDLs durable in Maria:
Sync table files after CREATE (of non-temp table), DROP, RENAME,
TRUNCATE, sync directories and symlinks (for the 3 first commands).
Comments for future log records.
In ma_rename(), if rename of index works and then rename of data fails,
try to undo the rename of the index to leave a consistent state.
mysys/my_symlink.c:
sync directory after creation of a symbolic link in it, if asked
mysys/my_sync.c:
comment. Fix for when the file's name has no directory in it.
storage/maria/ma_create.c:
sync files and links and dirs when creating a non-temporary table.
Optimizations of the above to reduce syncs in the common cases:
* if index file and data file have the exact same paths (regular
and link), sync the directories (of regular and link) only once
after creating the last file (the data file).
* don't sync the data file if we didn't write to it (always true
in our builds).
storage/maria/ma_delete_all.c:
sync files after truncating a table
storage/maria/ma_delete_table.c:
sync files and symbolic links and dirs after dropping a table
storage/maria/ma_extra.c:
a function which wraps the sync of the index file and the sync of the
data file.
storage/maria/ma_locking.c:
using a wrapper function
storage/maria/ma_rename.c:
sync files and symbolic links and dirs after renaming a table.
If rename of index works and then rename of data fails, try to undo
the rename of the index to leave a consistent state. That is just a
try, it may fail...
storage/maria/ma_test3.c:
warning to not pay attention to this test.
storage/maria/maria_def.h:
declaration for the function added to ma_extra.c
2006-11-27 22:01:29 +01:00
|
|
|
if (tmp_table)
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
{
|
|
|
|
char *path;
|
|
|
|
/* chop off the table name, tempory tables use generated name */
|
|
|
|
if ((path= strrchr(ci->index_file_name, FN_LIBCHAR)))
|
|
|
|
*path= '\0';
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(kfilename, name, ci->index_file_name, MARIA_NAME_IEXT,
|
2007-07-27 12:06:39 +02:00
|
|
|
MY_REPLACE_DIR | MY_UNPACK_FILENAME |
|
|
|
|
MY_RETURN_REAL_PATH | MY_APPEND_EXT);
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(kfilename, ci->index_file_name, "", MARIA_NAME_IEXT,
|
2007-07-27 12:06:39 +02:00
|
|
|
MY_UNPACK_FILENAME | MY_RETURN_REAL_PATH |
|
|
|
|
(have_iext ? MY_REPLACE_EXT : MY_APPEND_EXT));
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
}
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(klinkname, name, "", MARIA_NAME_IEXT,
|
2006-04-11 15:45:10 +02:00
|
|
|
MY_UNPACK_FILENAME|MY_APPEND_EXT);
|
2017-02-20 22:40:47 +01:00
|
|
|
klinkname_ptr= klinkname;
|
2006-04-11 15:45:10 +02:00
|
|
|
/*
|
|
|
|
Don't create the table if the link or file exists to ensure that one
|
|
|
|
doesn't accidently destroy another table.
|
WL#3072 Maria Recovery. Making DDLs durable in Maria:
Sync table files after CREATE (of non-temp table), DROP, RENAME,
TRUNCATE, sync directories and symlinks (for the 3 first commands).
Comments for future log records.
In ma_rename(), if rename of index works and then rename of data fails,
try to undo the rename of the index to leave a consistent state.
mysys/my_symlink.c:
sync directory after creation of a symbolic link in it, if asked
mysys/my_sync.c:
comment. Fix for when the file's name has no directory in it.
storage/maria/ma_create.c:
sync files and links and dirs when creating a non-temporary table.
Optimizations of the above to reduce syncs in the common cases:
* if index file and data file have the exact same paths (regular
and link), sync the directories (of regular and link) only once
after creating the last file (the data file).
* don't sync the data file if we didn't write to it (always true
in our builds).
storage/maria/ma_delete_all.c:
sync files after truncating a table
storage/maria/ma_delete_table.c:
sync files and symbolic links and dirs after dropping a table
storage/maria/ma_extra.c:
a function which wraps the sync of the index file and the sync of the
data file.
storage/maria/ma_locking.c:
using a wrapper function
storage/maria/ma_rename.c:
sync files and symbolic links and dirs after renaming a table.
If rename of index works and then rename of data fails, try to undo
the rename of the index to leave a consistent state. That is just a
try, it may fail...
storage/maria/ma_test3.c:
warning to not pay attention to this test.
storage/maria/maria_def.h:
declaration for the function added to ma_extra.c
2006-11-27 22:01:29 +01:00
|
|
|
Don't sync dir now if the data file has the same path.
|
2006-04-11 15:45:10 +02:00
|
|
|
*/
|
WL#3072 Maria Recovery. Making DDLs durable in Maria:
Sync table files after CREATE (of non-temp table), DROP, RENAME,
TRUNCATE, sync directories and symlinks (for the 3 first commands).
Comments for future log records.
In ma_rename(), if rename of index works and then rename of data fails,
try to undo the rename of the index to leave a consistent state.
mysys/my_symlink.c:
sync directory after creation of a symbolic link in it, if asked
mysys/my_sync.c:
comment. Fix for when the file's name has no directory in it.
storage/maria/ma_create.c:
sync files and links and dirs when creating a non-temporary table.
Optimizations of the above to reduce syncs in the common cases:
* if index file and data file have the exact same paths (regular
and link), sync the directories (of regular and link) only once
after creating the last file (the data file).
* don't sync the data file if we didn't write to it (always true
in our builds).
storage/maria/ma_delete_all.c:
sync files after truncating a table
storage/maria/ma_delete_table.c:
sync files and symbolic links and dirs after dropping a table
storage/maria/ma_extra.c:
a function which wraps the sync of the index file and the sync of the
data file.
storage/maria/ma_locking.c:
using a wrapper function
storage/maria/ma_rename.c:
sync files and symbolic links and dirs after renaming a table.
If rename of index works and then rename of data fails, try to undo
the rename of the index to leave a consistent state. That is just a
try, it may fail...
storage/maria/ma_test3.c:
warning to not pay attention to this test.
storage/maria/maria_def.h:
declaration for the function added to ma_extra.c
2006-11-27 22:01:29 +01:00
|
|
|
create_flag=
|
|
|
|
(ci->data_file_name &&
|
|
|
|
!strcmp(ci->index_file_name, ci->data_file_name)) ? 0 : sync_dir;
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2007-07-27 12:06:39 +02:00
|
|
|
char *iext= strrchr(name, '.');
|
|
|
|
int have_iext= iext && !strcmp(iext, MARIA_NAME_IEXT);
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(kfilename, name, "", MARIA_NAME_IEXT,
|
2007-07-27 12:06:39 +02:00
|
|
|
MY_UNPACK_FILENAME | MY_RETURN_REAL_PATH |
|
|
|
|
(have_iext ? MY_REPLACE_EXT : MY_APPEND_EXT));
|
2017-02-20 22:40:47 +01:00
|
|
|
klinkname_ptr= NullS;
|
WL#3072 Maria Recovery. Making DDLs durable in Maria:
Sync table files after CREATE (of non-temp table), DROP, RENAME,
TRUNCATE, sync directories and symlinks (for the 3 first commands).
Comments for future log records.
In ma_rename(), if rename of index works and then rename of data fails,
try to undo the rename of the index to leave a consistent state.
mysys/my_symlink.c:
sync directory after creation of a symbolic link in it, if asked
mysys/my_sync.c:
comment. Fix for when the file's name has no directory in it.
storage/maria/ma_create.c:
sync files and links and dirs when creating a non-temporary table.
Optimizations of the above to reduce syncs in the common cases:
* if index file and data file have the exact same paths (regular
and link), sync the directories (of regular and link) only once
after creating the last file (the data file).
* don't sync the data file if we didn't write to it (always true
in our builds).
storage/maria/ma_delete_all.c:
sync files after truncating a table
storage/maria/ma_delete_table.c:
sync files and symbolic links and dirs after dropping a table
storage/maria/ma_extra.c:
a function which wraps the sync of the index file and the sync of the
data file.
storage/maria/ma_locking.c:
using a wrapper function
storage/maria/ma_rename.c:
sync files and symbolic links and dirs after renaming a table.
If rename of index works and then rename of data fails, try to undo
the rename of the index to leave a consistent state. That is just a
try, it may fail...
storage/maria/ma_test3.c:
warning to not pay attention to this test.
storage/maria/maria_def.h:
declaration for the function added to ma_extra.c
2006-11-27 22:01:29 +01:00
|
|
|
/*
|
|
|
|
Replace the current file.
|
|
|
|
Don't sync dir now if the data file has the same path.
|
|
|
|
*/
|
2007-10-04 19:33:42 +02:00
|
|
|
create_flag= (flags & HA_CREATE_KEEP_FILES) ? 0 : MY_DELETE_OLD;
|
|
|
|
create_flag|= (!ci->data_file_name ? 0 : sync_dir);
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
If a MRG_MARIA table is in use, the mapped MARIA tables are open,
|
|
|
|
but no entry is made in the table cache for them.
|
|
|
|
A TRUNCATE command checks for the table in the cache only and could
|
|
|
|
be fooled to believe, the table is not open.
|
|
|
|
Pull the emergency brake in this situation. (Bug #8306)
|
2007-07-27 12:06:39 +02:00
|
|
|
|
|
|
|
|
|
|
|
NOTE: The filename is compared against unique_file_name of every
|
|
|
|
open table. Hence we need a real path here.
|
2006-04-11 15:45:10 +02:00
|
|
|
*/
|
2017-03-03 00:37:54 +01:00
|
|
|
if (!internal_table && _ma_test_if_reopen(kfilename))
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
2011-04-25 17:22:25 +02:00
|
|
|
my_printf_error(HA_ERR_TABLE_EXIST, "Aria table '%s' is in use "
|
2006-04-11 15:45:10 +02:00
|
|
|
"(most likely by a MERGE table). Try FLUSH TABLES.",
|
|
|
|
MYF(0), name + dirname_length(name));
|
2008-10-14 23:23:33 +02:00
|
|
|
my_errno= HA_ERR_TABLE_EXIST;
|
2006-04-11 15:45:10 +02:00
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
2017-02-20 22:40:47 +01:00
|
|
|
if ((file= mysql_file_create_with_symlink(key_file_kfile, klinkname_ptr,
|
|
|
|
kfilename, 0, create_mode,
|
2011-07-13 21:10:18 +02:00
|
|
|
MYF(MY_WME|create_flag))) < 0)
|
2006-04-11 15:45:10 +02:00
|
|
|
goto err;
|
|
|
|
errpos=1;
|
|
|
|
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
DBUG_PRINT("info", ("write state info and base info"));
|
2009-01-08 09:20:04 +01:00
|
|
|
if (_ma_state_info_write_sub(file, &share.state,
|
|
|
|
MA_STATE_INFO_WRITE_FULL_INFO) ||
|
2006-04-11 15:45:10 +02:00
|
|
|
_ma_base_info_write(file, &share.base))
|
|
|
|
goto err;
|
2007-01-18 20:38:14 +01:00
|
|
|
DBUG_PRINT("info", ("base_pos: %d base_info_size: %d",
|
|
|
|
base_pos, MARIA_BASE_INFO_SIZE));
|
2011-07-13 21:10:18 +02:00
|
|
|
DBUG_ASSERT(mysql_file_tell(file,MYF(0)) == base_pos+ MARIA_BASE_INFO_SIZE);
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
/* Write key and keyseg definitions */
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
DBUG_PRINT("info", ("write key and keyseg definitions"));
|
2006-04-11 15:45:10 +02:00
|
|
|
for (i=0 ; i < share.base.keys - uniques; i++)
|
|
|
|
{
|
|
|
|
uint sp_segs=(keydefs[i].flag & HA_SPATIAL) ? 2*SPDIMS : 0;
|
|
|
|
|
|
|
|
if (_ma_keydef_write(file, &keydefs[i]))
|
|
|
|
goto err;
|
|
|
|
for (j=0 ; j < keydefs[i].keysegs-sp_segs ; j++)
|
|
|
|
if (_ma_keyseg_write(file, &keydefs[i].seg[j]))
|
|
|
|
goto err;
|
|
|
|
#ifdef HAVE_SPATIAL
|
|
|
|
for (j=0 ; j < sp_segs ; j++)
|
|
|
|
{
|
|
|
|
HA_KEYSEG sseg;
|
|
|
|
sseg.type=SPTYPE;
|
|
|
|
sseg.language= 7; /* Binary */
|
|
|
|
sseg.null_bit=0;
|
|
|
|
sseg.bit_start=0;
|
|
|
|
sseg.bit_length= 0;
|
|
|
|
sseg.bit_pos= 0;
|
|
|
|
sseg.length=SPLEN;
|
|
|
|
sseg.null_pos=0;
|
|
|
|
sseg.start=j*SPLEN;
|
|
|
|
sseg.flag= HA_SWAP_KEY;
|
|
|
|
if (_ma_keyseg_write(file, &sseg))
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
/* Create extra keys for unique definitions */
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
offset= real_reclength - uniques*MARIA_UNIQUE_HASH_LENGTH;
|
2006-04-11 15:45:10 +02:00
|
|
|
bzero((char*) &tmp_keydef,sizeof(tmp_keydef));
|
|
|
|
bzero((char*) &tmp_keyseg,sizeof(tmp_keyseg));
|
|
|
|
for (i=0; i < uniques ; i++)
|
|
|
|
{
|
|
|
|
tmp_keydef.keysegs=1;
|
|
|
|
tmp_keydef.flag= HA_UNIQUE_CHECK;
|
2007-01-18 20:38:14 +01:00
|
|
|
tmp_keydef.block_length= (uint16) maria_block_size;
|
2006-04-11 15:45:10 +02:00
|
|
|
tmp_keydef.keylength= MARIA_UNIQUE_HASH_LENGTH + pointer;
|
|
|
|
tmp_keydef.minlength=tmp_keydef.maxlength=tmp_keydef.keylength;
|
|
|
|
tmp_keyseg.type= MARIA_UNIQUE_HASH_TYPE;
|
|
|
|
tmp_keyseg.length= MARIA_UNIQUE_HASH_LENGTH;
|
|
|
|
tmp_keyseg.start= offset;
|
|
|
|
offset+= MARIA_UNIQUE_HASH_LENGTH;
|
|
|
|
if (_ma_keydef_write(file,&tmp_keydef) ||
|
|
|
|
_ma_keyseg_write(file,(&tmp_keyseg)))
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Save unique definition */
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
DBUG_PRINT("info", ("write unique definitions"));
|
2006-04-11 15:45:10 +02:00
|
|
|
for (i=0 ; i < share.state.header.uniques ; i++)
|
|
|
|
{
|
|
|
|
HA_KEYSEG *keyseg_end;
|
|
|
|
keyseg= uniquedefs[i].seg;
|
|
|
|
if (_ma_uniquedef_write(file, &uniquedefs[i]))
|
|
|
|
goto err;
|
|
|
|
for (keyseg= uniquedefs[i].seg, keyseg_end= keyseg+ uniquedefs[i].keysegs;
|
|
|
|
keyseg < keyseg_end;
|
|
|
|
keyseg++)
|
|
|
|
{
|
|
|
|
switch (keyseg->type) {
|
|
|
|
case HA_KEYTYPE_VARTEXT1:
|
|
|
|
case HA_KEYTYPE_VARTEXT2:
|
|
|
|
case HA_KEYTYPE_VARBINARY1:
|
|
|
|
case HA_KEYTYPE_VARBINARY2:
|
|
|
|
if (!(keyseg->flag & HA_BLOB_PART))
|
|
|
|
{
|
|
|
|
keyseg->flag|= HA_VAR_LENGTH_PART;
|
|
|
|
keyseg->bit_start= ((keyseg->type == HA_KEYTYPE_VARTEXT1 ||
|
|
|
|
keyseg->type == HA_KEYTYPE_VARBINARY1) ?
|
|
|
|
1 : 2);
|
|
|
|
}
|
|
|
|
break;
|
|
|
|
default:
|
2007-01-18 20:38:14 +01:00
|
|
|
DBUG_ASSERT((keyseg->flag & HA_VAR_LENGTH_PART) == 0);
|
2006-04-11 15:45:10 +02:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
if (_ma_keyseg_write(file, keyseg))
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
}
|
Completion of merge of mysql-5.1 into mysql-maria.
Manually imported changes done to MyISAM (include/myisam.h,
storage/myisam/*, sql/ha_myisam.*, mysql-test/t/myisam.test,
mysql-test/t/ps_2myisam.test) the last
months into Maria (tedious, should do it more frequently in the
future), including those not done at the previous 5.1->Maria merge
(please in the future don't forget to apply MyISAM changes to Maria
when you merge 5.1 into Maria).
Note: I didn't try to import anything which could be MyISAM-related
in other tests of mysql-test (I didn't want to dig in all csets),
but as QA is working to make most tests re-usable for other engines
(Falcon), it is likely that we'll benefit from this and just have
to set engine=Maria somewhere to run those tests on Maria.
func_group and partition tests fail but they already do in main 5.1
on my machine. No Valgrind error in t/*maria*.test.
Monty: please see the commit comment of maria.result and check.
BitKeeper/deleted/.del-ha_maria.m4:
Delete: config/ac-macros/ha_maria.m4
configure.in:
fix for the new way of enabling engines
include/maria.h:
importing changes done to MyISAM the last months into Maria
include/my_handler.h:
importing changes done to MyISAM the last months into Maria
include/myisam.h:
importing changes done to MyISAM the last months into Maria
mysql-test/r/maria.result:
identical to myisam.result, except the engine name in some places
AND in the line testing key_block_size=1000000000000000000:
Maria gives a key block size of 8192 while MyISAM gives 4096;
is it explainable by the difference between MARIA_KEY_BLOCK_LENGTH
and the same constant in MyISAM? Monty?
mysql-test/r/ps_maria.result:
identical to ps_2myisam.result (except the engine name in some places)
mysql-test/t/maria.test:
instead of engine=maria everywhere, I use @@storage_engine (reduces
the diff with myisam.test).
importing changes done to MyISAM the last months into Maria
mysys/my_handler.c:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.cc:
importing changes done to MyISAM the last months into Maria
sql/ha_maria.h:
importing changes done to MyISAM the last months into Maria
sql/mysqld.cc:
unneeded
storage/maria/Makefile.am:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_check.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_create.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_delete_table.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_dynrec.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_extra.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_boolean_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_eval.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_nlq_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_parser.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ft_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_ftdefs.h:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_key.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_open.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_page.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rkey.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rsamepos.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_index.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_rt_mbr.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_search.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_sort.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test1.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test2.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_test3.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_update.c:
importing changes done to MyISAM the last months into Maria
storage/maria/ma_write.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_chk.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_def.h:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_ftdump.c:
importing changes done to MyISAM the last months into Maria
storage/maria/maria_pack.c:
importing changes done to MyISAM the last months into Maria
2006-08-10 16:36:54 +02:00
|
|
|
DBUG_PRINT("info", ("write field definitions"));
|
2007-04-19 12:18:56 +02:00
|
|
|
if (datafile_type == BLOCK_RECORD)
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
|
|
|
/* Store columns in a more efficent order */
|
2007-04-05 13:38:05 +02:00
|
|
|
MARIA_COLUMNDEF **col_order, **pos;
|
|
|
|
if (!(col_order= (MARIA_COLUMNDEF**) my_malloc(share.base.fields *
|
2007-04-19 12:18:56 +02:00
|
|
|
sizeof(MARIA_COLUMNDEF*),
|
|
|
|
MYF(MY_WME))))
|
2006-04-11 15:45:10 +02:00
|
|
|
goto err;
|
2007-04-19 12:18:56 +02:00
|
|
|
for (column= columndef, pos= col_order ;
|
|
|
|
column != end_column ;
|
|
|
|
column++, pos++)
|
|
|
|
*pos= column;
|
2007-04-05 13:38:05 +02:00
|
|
|
qsort(col_order, share.base.fields, sizeof(*col_order),
|
|
|
|
(qsort_cmp) compare_columns);
|
2007-01-18 20:38:14 +01:00
|
|
|
for (i=0 ; i < share.base.fields ; i++)
|
2007-04-05 13:38:05 +02:00
|
|
|
{
|
2007-10-09 20:09:50 +02:00
|
|
|
column_array[col_order[i]->column_nr]= i;
|
2007-04-19 12:18:56 +02:00
|
|
|
if (_ma_columndef_write(file, col_order[i]))
|
2007-04-05 13:38:05 +02:00
|
|
|
{
|
2011-04-25 17:22:25 +02:00
|
|
|
my_free(col_order);
|
2007-01-18 20:38:14 +01:00
|
|
|
goto err;
|
2007-04-05 13:38:05 +02:00
|
|
|
}
|
|
|
|
}
|
2011-04-25 17:22:25 +02:00
|
|
|
my_free(col_order);
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
for (i=0 ; i < share.base.fields ; i++)
|
2007-10-09 20:09:50 +02:00
|
|
|
{
|
|
|
|
column_array[i]= (uint16) i;
|
2007-04-19 12:18:56 +02:00
|
|
|
if (_ma_columndef_write(file, &columndef[i]))
|
2007-01-18 20:38:14 +01:00
|
|
|
goto err;
|
2007-10-09 20:09:50 +02:00
|
|
|
}
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
2007-10-09 20:09:50 +02:00
|
|
|
if (_ma_column_nr_write(file, column_array, columns))
|
|
|
|
goto err;
|
2006-04-11 15:45:10 +02:00
|
|
|
|
2014-12-22 15:53:17 +01:00
|
|
|
if (encrypted)
|
|
|
|
{
|
|
|
|
if (ma_crypt_create(&share) ||
|
|
|
|
ma_crypt_write(&share, file))
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
2011-07-13 21:10:18 +02:00
|
|
|
if ((kfile_size_before_extension= mysql_file_tell(file,MYF(0))) == MY_FILEPOS_ERROR)
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
goto err;
|
2006-04-11 15:45:10 +02:00
|
|
|
#ifndef DBUG_OFF
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
if (kfile_size_before_extension != info_length)
|
|
|
|
DBUG_PRINT("warning",("info_length: %u != used_length: %u",
|
|
|
|
info_length, (uint)kfile_size_before_extension));
|
|
|
|
#endif
|
|
|
|
|
|
|
|
if (sync_dir)
|
2006-04-11 15:45:10 +02:00
|
|
|
{
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
/*
|
|
|
|
we log the first bytes and then the size to which we extend; this is
|
|
|
|
not log 1 KB of mostly zeroes if this is a small table.
|
|
|
|
*/
|
|
|
|
char empty_string[]= "";
|
Injecting more "const" declarations into code which does not change
pointed data.
I ran gcc -Wcast-qual on storage/maria, this identified un-needed casts,
a couple of functions which said they had a const parameter though
they changed the pointed content! This is fixed here. Some suspicious
places receive a comment.
The original intention of running -Wcast-qual was to find what code
changes R-tree keys: I added const words, but hidden casts
like those of int2store (casts target to (uint16*)) removed const
checking; -Wcast-qual helped find those hidden casts.
Log handler does not change the content pointed by LEX_STRING::str it
receives, so we now use a struct which has a const inside, to emphasize
this and be able to pass "const uchar*" buffers to log handler
without fear of their content being changed by it.
One-line fix for a merge glitch (when merging from MyISAM).
include/m_string.h:
As Maria's log handler uses LEX_STRING but never changes the content
pointed by LEX_STRING::str, and assigns uchar* into this member most
of the time, we introduce a new struct LEX_CUSTRING
(C const U unsigned) for the log handler.
include/my_global.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
include/my_handler.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
ha_find_null() does not change *a.
include/my_sys.h:
insert_dynamic() does not change *element.
include/myisampack.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
mysys/array.c:
insert_dynamic() does not change *element
mysys/my_handler.c:
ha_find_null() does not change *a
storage/maria/ma_bitmap.c:
Log handler receives const strings now
storage/maria/ma_blockrec.c:
Log handler receives const strings now.
_ma_apply_undo_row_delete/update() do change *header.
storage/maria/ma_blockrec.h:
correct prototype
storage/maria/ma_check.c:
Log handler receives const strings now. Un-needed casts
storage/maria/ma_checkpoint.c:
Log handler receives const strings now
storage/maria/ma_checksum.c:
unneeded cast
storage/maria/ma_commit.c:
Log handler receives const strings now
storage/maria/ma_create.c:
Log handler receives const strings now
storage/maria/ma_dbug.c:
fixing warning of gcc -Wcast-qual
storage/maria/ma_delete.c:
Log handler receives const strings now
storage/maria/ma_delete_all.c:
Log handler receives const strings now
storage/maria/ma_delete_table.c:
Log handler receives const strings now
storage/maria/ma_dynrec.c:
fixing some warnings of gcc -Wcast-qual. Unneeded casts removed.
Comment about function which lies.
storage/maria/ma_ft_parser.c:
fix for warnings of gcc -Wcast-qual, removing unneeded casts
storage/maria/ma_ft_update.c:
less casts, comment
storage/maria/ma_key.c:
less casts, stay const (warnings of gcc -Wcast-qual)
storage/maria/ma_key_recover.c:
Log handler receives const strings now
storage/maria/ma_loghandler.c:
Log handler receives const strings now
storage/maria/ma_loghandler.h:
Log handler receives const strings now
storage/maria/ma_loghandler_lsn.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
storage/maria/ma_page.c:
Log handler receives const strings now; more const
storage/maria/ma_recovery.c:
Log handler receives const strings now
storage/maria/ma_rename.c:
Log handler receives const strings now
storage/maria/ma_rt_index.c:
more const, to emphasize that functions don't change pointed content.
best_key= NULL was forgotten during merge from MyISAM a few days ago,
was causing a Valgrind warning
storage/maria/ma_rt_index.h:
new proto
storage/maria/ma_rt_key.c:
more const
storage/maria/ma_rt_key.h:
new proto
storage/maria/ma_rt_mbr.c:
more const for functions which deserve it
storage/maria/ma_rt_mbr.h:
new prototype
storage/maria/ma_rt_split.c:
make const what is not changed.
storage/maria/ma_search.c:
un-needed casts, more const
storage/maria/ma_sp_key.c:
more const
storage/maria/ma_unique.c:
un-needed casts.
storage/maria/ma_write.c:
Log handler receives const strings now
storage/maria/maria_def.h:
some more const
storage/maria/unittest/ma_test_loghandler-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Log handler receives const strings now
2008-04-03 15:40:25 +02:00
|
|
|
LEX_CUSTRING log_array[TRANSLOG_INTERNAL_PARTS + 4];
|
2008-01-11 00:47:52 +01:00
|
|
|
translog_size_t total_rec_length= 0;
|
First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 18:08:06 +01:00
|
|
|
uint k;
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
LSN lsn;
|
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
2007-06-26 16:49:23 +02:00
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 1].length= 1 + 2 + 2 +
|
2008-01-10 20:21:36 +01:00
|
|
|
(uint) kfile_size_before_extension;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
/* we are needing maybe 64 kB, so don't use the stack */
|
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
2007-06-26 16:49:23 +02:00
|
|
|
log_data= my_malloc(log_array[TRANSLOG_INTERNAL_PARTS + 1].length, MYF(0));
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
if ((log_data == NULL) ||
|
2011-07-13 21:10:18 +02:00
|
|
|
mysql_file_pread(file, 1 + 2 + 2 + log_data,
|
2008-01-10 20:21:36 +01:00
|
|
|
(size_t) kfile_size_before_extension, 0, MYF(MY_NABP)))
|
2007-06-28 14:01:57 +02:00
|
|
|
goto err;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
/*
|
|
|
|
remember if the data file was created or not, to know if Recovery can
|
|
|
|
do it or not, in the future
|
|
|
|
*/
|
2014-02-19 11:05:15 +01:00
|
|
|
log_data[0]= MY_TEST(flags & HA_DONT_TOUCH_DATA);
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
int2store(log_data + 1, kfile_size_before_extension);
|
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
2007-06-26 16:49:23 +02:00
|
|
|
int2store(log_data + 1 + 2, share.base.keystart);
|
2008-08-06 16:03:27 +02:00
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 0].str= (uchar *)name;
|
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records
found in a Maria log (see file's revision comment)
- minor, misc fixes
storage/maria/Makefile.am:
new program maria_read_log
storage/maria/ha_maria.cc:
create control file if missing
storage/maria/ma_blockrec.c:
0 -> LSN_IMPOSSIBLE; comments
storage/maria/ma_checkpoint.h:
preparations for Checkpoint module
storage/maria/ma_close.c:
comment
storage/maria/ma_control_file.c:
renaming constants.
Possibility to say "open control file but don't create it if it's
missing" (used by maria_read_log which does not want to create
anything)
storage/maria/ma_control_file.h:
renaming constants
storage/maria/ma_create.c:
I had duplicated "linkname" and "linkname_ptr", now I see it's not
needed, reverting. Indeed those variables don't contain interesting
information; fixing log record accordingly (the links are in
ci->data/index_file_name). Storing keystart in log record is needed,
to know at which size we must extend the file if we replay
LOGREC_CREATE_TABLE.
storage/maria/ma_loghandler.c:
some structures need to be known to maria_read_log.c, taking
them to ma_loghandler.h
storage/maria/ma_loghandler.h:
we have page_store, adding page_korr.
translog_lock() made public, because Checkpoint will need it (to
write to control file).
Some structures moved from ma_loghandler.c because maria_read_log.c
needs them (needs to know the execute-in-REDO-phase hooks of each
record).
storage/maria/ma_loghandler_lsn.h:
constants defined in ma_control_file.h serve everywhere,
and they relate to LSNs, so putting them in ma_loghandler_lsn.h.
Stronger constraints in LSN_VALID().
storage/maria/ma_pagecache.c:
renaming constants
storage/maria/ma_recovery.h:
copyright
storage/maria/ma_test1.c:
new prototype
storage/maria/ma_test2.c:
new prototype
storage/maria/trnman_public.h:
double-inclusion safe
storage/maria/unittest/ma_control_file-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
constants renamed, new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
constants renamed, new prototype
storage/myisam/mi_close.c:
comment
storage/maria/maria_read_log.c:
program to read and print log records from a Maria transaction log,
and optionally apply them to tables. Very basic, early version.
Should serve as a base for Recovery's code. Designed to be idempotent.
Create a log by running maria.test, then cd to var/master-data
and run "maria_read_log --only-display" to see info about records;
run "maria_read_log --display-and-apply" to also apply the records
to tables (it's more interesting if you first wipe out the
tables in var/master-data/test, to see how they get re-created).
Only a few records are handled by now: LONG_TRANSACTION_ID,
COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for
REDO_INSERT_ROW_HEAD where I could use Monty's help (search for
"Monty" in the file). Note: changes to the index pages, index's header
and bitmap pages are not properly logged yet, so don't expect
the program to work with that.
2007-06-26 16:49:23 +02:00
|
|
|
/* we store the end-zero, for Recovery to just pass it to my_create() */
|
2008-08-06 16:03:27 +02:00
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 0].length= strlen(name) + 1;
|
Injecting more "const" declarations into code which does not change
pointed data.
I ran gcc -Wcast-qual on storage/maria, this identified un-needed casts,
a couple of functions which said they had a const parameter though
they changed the pointed content! This is fixed here. Some suspicious
places receive a comment.
The original intention of running -Wcast-qual was to find what code
changes R-tree keys: I added const words, but hidden casts
like those of int2store (casts target to (uint16*)) removed const
checking; -Wcast-qual helped find those hidden casts.
Log handler does not change the content pointed by LEX_STRING::str it
receives, so we now use a struct which has a const inside, to emphasize
this and be able to pass "const uchar*" buffers to log handler
without fear of their content being changed by it.
One-line fix for a merge glitch (when merging from MyISAM).
include/m_string.h:
As Maria's log handler uses LEX_STRING but never changes the content
pointed by LEX_STRING::str, and assigns uchar* into this member most
of the time, we introduce a new struct LEX_CUSTRING
(C const U unsigned) for the log handler.
include/my_global.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
include/my_handler.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
ha_find_null() does not change *a.
include/my_sys.h:
insert_dynamic() does not change *element.
include/myisampack.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
mysys/array.c:
insert_dynamic() does not change *element
mysys/my_handler.c:
ha_find_null() does not change *a
storage/maria/ma_bitmap.c:
Log handler receives const strings now
storage/maria/ma_blockrec.c:
Log handler receives const strings now.
_ma_apply_undo_row_delete/update() do change *header.
storage/maria/ma_blockrec.h:
correct prototype
storage/maria/ma_check.c:
Log handler receives const strings now. Un-needed casts
storage/maria/ma_checkpoint.c:
Log handler receives const strings now
storage/maria/ma_checksum.c:
unneeded cast
storage/maria/ma_commit.c:
Log handler receives const strings now
storage/maria/ma_create.c:
Log handler receives const strings now
storage/maria/ma_dbug.c:
fixing warning of gcc -Wcast-qual
storage/maria/ma_delete.c:
Log handler receives const strings now
storage/maria/ma_delete_all.c:
Log handler receives const strings now
storage/maria/ma_delete_table.c:
Log handler receives const strings now
storage/maria/ma_dynrec.c:
fixing some warnings of gcc -Wcast-qual. Unneeded casts removed.
Comment about function which lies.
storage/maria/ma_ft_parser.c:
fix for warnings of gcc -Wcast-qual, removing unneeded casts
storage/maria/ma_ft_update.c:
less casts, comment
storage/maria/ma_key.c:
less casts, stay const (warnings of gcc -Wcast-qual)
storage/maria/ma_key_recover.c:
Log handler receives const strings now
storage/maria/ma_loghandler.c:
Log handler receives const strings now
storage/maria/ma_loghandler.h:
Log handler receives const strings now
storage/maria/ma_loghandler_lsn.h:
In macros which read pointed content: use const pointers so that
gcc -Wcast-qual does not warn about casting a const pointer to non-const.
storage/maria/ma_page.c:
Log handler receives const strings now; more const
storage/maria/ma_recovery.c:
Log handler receives const strings now
storage/maria/ma_rename.c:
Log handler receives const strings now
storage/maria/ma_rt_index.c:
more const, to emphasize that functions don't change pointed content.
best_key= NULL was forgotten during merge from MyISAM a few days ago,
was causing a Valgrind warning
storage/maria/ma_rt_index.h:
new proto
storage/maria/ma_rt_key.c:
more const
storage/maria/ma_rt_key.h:
new proto
storage/maria/ma_rt_mbr.c:
more const for functions which deserve it
storage/maria/ma_rt_mbr.h:
new prototype
storage/maria/ma_rt_split.c:
make const what is not changed.
storage/maria/ma_search.c:
un-needed casts, more const
storage/maria/ma_sp_key.c:
more const
storage/maria/ma_unique.c:
un-needed casts.
storage/maria/ma_write.c:
Log handler receives const strings now
storage/maria/maria_def.h:
some more const
storage/maria/unittest/ma_test_loghandler-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Log handler receives const strings now
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Log handler receives const strings now
2008-04-03 15:40:25 +02:00
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 1].str= log_data;
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
/* symlink description is also needed for re-creation by Recovery: */
|
2008-08-06 16:03:27 +02:00
|
|
|
{
|
|
|
|
const char *s= ci->data_file_name ? ci->data_file_name : empty_string;
|
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 2].str= (uchar*)s;
|
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 2].length= strlen(s) + 1;
|
|
|
|
s= ci->index_file_name ? ci->index_file_name : empty_string;
|
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 3].str= (uchar*)s;
|
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 3].length= strlen(s) + 1;
|
|
|
|
}
|
First part of redo/undo for key pages
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
For transactional tables, shift record number in keys up with 1 bit to have place to indicate if transid follows
Checksum for MyISAM now ignores NULL and not used part of VARCHAR
Renamed some variables that caused shadow compiler warnings
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
Fixed crashing bugs when using Maria TEMPORARY tables with TRUNCATE. Removed 'hack' code in sql directory to go around this bug.
pagecache_unlock_by_ulink() now has extra argument to say if page was changed.
Give error message if we fail to open control file
Mark page cache variables as not flushable
include/maria.h:
Made min page cache larger (needed for pinning key page)
Added key_nr to st_maria_keydef for faster keyinfo->keynr conversion
Added write_comp_flag to move some runtime code to maria_open()
include/my_base.h:
Added new error message to be used when handler initialization failed
include/my_global.h:
Renamed dummy to swap_dummy to avoid conflicts with local 'dummy' variables
include/my_handler.h:
Added const to some parameters
mysys/array.c:
More DBUG
mysys/my_error.c:
Fixed indentation
mysys/my_handler.c:
Added const to some parameters
Added missing error messages
sql/field.h:
Renamed variables to avoid variable shadowing
sql/handler.h:
Renamed parameter to avoid variable name conflict
sql/item.h:
Renamed variables to avoid variable shadowing
sql/log_event_old.h:
Renamed variables to avoid variable shadowing
sql/set_var.h:
Renamed variables to avoid variable shadowing
sql/sql_delete.cc:
Removed maria hack for temporary tables
Fixed indentation
sql/sql_table.cc:
Moved extra() call when waiting for tables to not be used to after tables are removed from cache.
This was needed to ensure we don't do a PREPARE_FOR_DROP or similar call while the table is still in use.
sql/table.cc:
Copy page_checksum from share
Removed Maria hack
storage/maria/Makefile.am:
Added new files
storage/maria/ha_maria.cc:
Renamed records -> record_count and info -> create_info to avoid variable name conflicts
Mark page cache variables as not flushable
storage/maria/ma_blockrec.c:
Moved _ma_unpin_all_pages() to ma_key_recover.c
Moved init of info->pinned_pages to ma_open.c
Moved _ma_finalize_row() to maria_key_recover.h
Renamed some variables to avoid variable name conflicts
Mark page_link.changed for blocks we change directly
Simplify handling of undo link when writing LOGREC_UNDO_ROW_INSERT (old code crashed when having redo for index)
storage/maria/ma_blockrec.h:
Removed extra empty line
storage/maria/ma_checkpoint.c:
Remove not needed trnman.h
storage/maria/ma_close.c:
Free pinned pages (which are now always allocated)
storage/maria/ma_control_file.c:
Give error message if we fail to open control file
storage/maria/ma_delete.c:
Changes for redo logging (first part, logging of underflow not yet done)
- Log undo-key-delete
- Log delete of key
- Updated arguments to _ma_fetch_keypage(), _ma_dispose(), _ma_write_keypage(), _ma_insert()
- Added new arguments to some functions to be able to write redo information
- Mark key pages as changed when we write with PAGECACHE_LOCK_LEFT_WRITELOCKED
Remove one not needed _ma_write_keypage() in d_search() when upper level will do the write anyway
Changed 2 bmove_upp() to bmove() as this made code easer to understand
More function comments
Indentation fixes
storage/maria/ma_ft_update.c:
New arguments to _ma_write_keypage()
storage/maria/ma_loghandler.c:
Fixed some DBUG_PRINT messages
Simplify code
Added new log entrys for key page redo
Renamed some variables to avoid variable name shadowing
storage/maria/ma_loghandler.h:
Moved some defines here
Added define for storing key number on key pages
Added new translog record types
Added enum for type of operations in LOGREC_REDO_INDEX
storage/maria/ma_open.c:
Always allocate info.pinned_pages (we need now also for normal key page usage)
Update keyinfo->key_nr
Added virtual functions to convert record position o number to be stored on key pages
Update keyinfo->write_comp_flag to value of search flag to be used when writing key
storage/maria/ma_page.c:
Added redo for key pages
- Extended _ma_fetch_keypage() with type of lock to put on page and address to used MARIA_PINNED_PAGE
- _ma_fetch_keypage() now pin's pages if needed
- Extended _ma_write_keypage() with type of locks to be used
- ma_dispose() now locks info->s->state.key_del from other threads
- ma_dispose() writes redo log record
- ma_new() locks info->s->state.key_del from other threads if it was used
- ma_new() now pins read page
Other things:
- Removed some not needed arguments from _ma_new() and _ma_dispose)
- Added some new variables to simplify code
- If EXTRA_DEBUG is used, do crc on full page to catch not unitialized bytes
storage/maria/ma_pagecache.h:
Applied patch from Sanja to add extra argument to pagecache_unlock_by_ulink() to mark if page was changed
Added some defines for pagecache priority levels that one can use
storage/maria/ma_range.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_recovery.c:
- Added hooks for new translog types:
REDO_INDEX, REDO_INDEX_NEW_PAGE, REDO_INDEX_FREE_PAGE, UNDO_KEY_INSERT, UNDO_KEY_DELETE and
UNDO_KEY_DELETE_WITH_ROOT.
- Moved variable declarations to start of function (portability fixes)
- Removed some not needed initializations
- Set only relevant state changes for each redo/undo entry
storage/maria/lockman.c:
Removed end space
storage/maria/ma_check.c:
Removed end space
storage/maria/ma_create.c:
Removed end space
storage/maria/ma_locking.c:
Removed end space
storage/maria/ma_packrec.c:
Removed end space
storage/maria/ma_pagecache.c:
Removed end space
storage/maria/ma_panic.c:
Removed end space
storage/maria/ma_rt_index.c:
Added new arguments for call to _ma_fetch_keypage(), _ma_write_keypage(), _ma_dispose() and _ma_new()
Fixed indentation
storage/maria/ma_rt_key.c:
Added new arguments for call to _ma_fetch_keypage()
storage/maria/ma_rt_split.c:
Added new arguments for call to _ma_new()
Use new keypage header
Added new arguments for call to _ma_write_keypage()
storage/maria/ma_search.c:
Updated comments & indentation
Added new arguments for call to _ma_fetch_keypage()
Made some variables and arguments const
Added virtual functions for converting row position to number to be stored in key
use MARIA_RECORD_POS of record position instead of my_off_t
Record in MARIA_KEY_PARAM how page was changed one key insert (needed for REDO)
storage/maria/ma_sort.c:
Removed end space
storage/maria/ma_statrec.c:
Updated arguments for call to _ma_rec_pos()
storage/maria/ma_test1.c:
Fixed too small buffer to init_pagecache()
Fixed bug when using insert_count and test_flag
storage/maria/ma_test2.c:
Use more resonable pagecache size
Remove not used code
Reset blob_length to fix wrong output message
storage/maria/ma_test_all.sh:
Fixed wrong test
storage/maria/ma_write.c:
Lots of new code to handle REDO of key pages
No logic changes because of REDO code, mostly adding new arguments and adding new code for logging
Added new arguments for calls to _ma_fetch_keypage(), _ma_write_keypage() and similar functions
Move setting of comp_flag in ma_ck_wrte_btree() from runtime to maria_open()
Zerofill new used pages for:
- To remove possible sensitive data left in buffer
- To get idenitical data on pages after running redo
- Better compression of pages if archived
storage/maria/maria_chk.c:
Added information if table is crash safe
storage/maria/maria_def.h:
New virtual function to convert between record position on key and normal record position
Aded mutex and extra variables to handle locking of share->state.key_del
Moved some structure variables to get things more aligned
Added extra arguments to MARIA_KEY_PARAM to be able to remember what was changed on key page on key insert
Added argument to MARIA_PINNED_PAGE to indicate if page was changed
Updated prototypes for functions
Added some structures for signaling changes in REDO handling
storage/maria/unittest/ma_pagecache_single.c:
Updated arguments for changed function calls
storage/myisam/mi_check.c:
Made calc_check_checksum virtual
storage/myisam/mi_checksum.c:
Update checksums to ignore null columns
storage/myisam/mi_create.c:
Mark if table has null column (to know when we have to use mi_checksum())
storage/myisam/mi_open.c:
Added virtual function for calculating checksum to be able to easily ignore NULL fields
storage/myisam/mi_test2.c:
Fixed bug
storage/myisam/myisamdef.h:
Added virtual function for calculating checksum during check table
Removed ha_key_cmp() as this is in handler.h
storage/maria/ma_key_recover.c:
New BitKeeper file ``storage/maria/ma_key_recover.c''
storage/maria/ma_key_recover.h:
New BitKeeper file ``storage/maria/ma_key_recover.h''
storage/maria/ma_key_redo.c:
New BitKeeper file ``storage/maria/ma_key_redo.c''
2007-11-14 18:08:06 +01:00
|
|
|
for (k= TRANSLOG_INTERNAL_PARTS;
|
|
|
|
k < (sizeof(log_array)/sizeof(log_array[0])); k++)
|
2008-01-11 18:39:43 +01:00
|
|
|
total_rec_length+= (translog_size_t) log_array[k].length;
|
WL#3072 - Maria recovery
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.
sql/handler.cc:
typo
storage/maria/Makefile.am:
Adding ma_test_recovery (which ma_test_all invokes, and which can
also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
comments
storage/maria/ma_bitmap.c:
fixing comments. 2 -> sizeof(maria_bitmap_marker).
Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
Now putting the "bm" signature when creating the first bitmap page
(it used to happen only at next open, but that
caused an annoying difference when testing Recovery if the original
run didn't open the table, and it looks more
logical like this: it goes to disk only with its signature correct);
see the "QQ" comment towards the _ma_initialize_data_file() call
in ma_create.c for more).
When reading a bitmap page, verify its signature (happens when normally
using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
* no need to sync the data file if table is not transactional
* Comments, code cleanup (log-related data moved to log-related code
block, int5store->page_store).
* Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
do for other records (though this record will soon be replaced
with a CLR).
* If "page" is 1 it means the page which extends from byte
page*block_size+1 to (page+1)*block_size (byte number 1 being
the first byte of the file). The last byte of the file is
data_file_length (same convention).
A new page needs to be created if the last byte of the page is
beyond the last byte of the file, i.e.
(page+1)*block_size+1 > data_file_length, so we correct the test
(bug found when testing log applying for ma_test1 -M -T --skip-update).
* update the page's LSN when removing a row from it during
execution of a REDO_PURGE_ROW record (bug found when testing log
applying for ma_test1 -M -T --skip-update).
* applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
we can always flush the table's state when closing the last instance
of the table. And it is needed for maria_read_log (as it does
not use maria_lock_database()).
storage/maria/ma_control_file.c:
when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
double-inclusion safe
storage/maria/ma_create.c:
during recovery, don't log records. Comments.
Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
during recovery, don't log records. Log the end-zero of the dropped
table's name, so that recovery can use the string in place without
extending it to fit an end zero.
storage/maria/ma_loghandler.c:
* inwrite_rec_hook also needs access to the MARIA_SHARE, like
prewrite_rec_hook. This will be needed to update
share->records_diff (in the upcoming patch "recovery of the state").
* LOG_DESC::record_ends_group changed to an enum.
* LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
corrected
* Sanja please see the @todo LOG BUG
* avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
- log write hooks called while the log's lock is held (inwrite_rec_hook)
now need the MARIA_SHARE, like prewrite_rec_hook already had
- instead of a bool saying if this record's type ends groups or not,
we refine: it may not end a group, it may end a group, or it may
be a group in itself. Imagine that we had a physical write failure
to a table before we log the UNDO, we still end up in
external_lock(F_UNLCK) and then we log a COMMIT: we don't want
to consider this COMMIT as ending the group of REDOs (don't want
to execute those REDOs during Recovery), that's why we say "COMMIT
is a group in itself, it aborts any previous group". This also
gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
New Recovery code, replacing the old pseudocode.
Most of maria_read_log moved here.
Call-able from ha_maria, but not enabled yet.
Compared to the previous version of maria_read_log, some bugs have
been fixed, debugging output can go to stdout or a disk file (for now
it's useful for me, later it can be changed), execution of
REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
has been factored into functions. We abort an unfinished group
of records if we see a record which is a group in itself (like COMMIT).
No need for maria_panic() after a bug (which caused tables to not
be closed) was fixed; if there is yet another bug I prefer to see it.
When opening a table for Recovery, set data_file_length
and key_file_length to their real physical value (these are the
easiest state members to restore :). Warn us if the last page
was truncated (but Recovery handles it).
MARIA_SHARE::state::state::records is now partly recovered (not
idempotent, but works if recreating tables from scracth).
When applying a REDO to a page, stamp it with the UNDO's LSN
(current_group_end_lsn), not with the REDO's LSN; it makes
the table more identical to the original table (easier to compare
the two tables in the end).
Big thing missing: some types of REDOs are not handled,
and the UNDO phase does not exist (missing functions to execute UNDOs
to actually rollback). So for now tests are only inserting/deleting
a few 100 rows, closing the table and seeing if the log is applied ok;
it works. UPDATE not handled.
storage/maria/ma_recovery.h:
new functions: ma_recover() for recovery from inside ha_maria;
_ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
don't write log records during recovery
storage/maria/ma_test2.c:
- fail if maria_info() or other subtests find some wrong information
- new option -g to skip updates.
- init the translog before creating the table, so that log applying
can work.
- in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
cleanup files. Test log applying.
storage/maria/maria_read_log.c:
most of the logic moves to ma_recovery.c to be shared between
maria_read_log and recovery-from-inside-mysqld.
See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
unit test for Recovery. Tests insert and delete,
REDO_UPDATE not yet coded.
Script is called from ma_test_all. Can run standalone.
2007-07-26 11:56:21 +02:00
|
|
|
/**
|
|
|
|
For this record to be of any use for Recovery, we need the upper
|
|
|
|
MySQL layer to be crash-safe, which it is not now (that would require
|
|
|
|
work using the ddl_log of sql/sql_table.cc); when it is, we should
|
|
|
|
reconsider the moment of writing this log record (before or after op,
|
|
|
|
under THR_LOCK_maria or not...), how to use it in Recovery.
|
|
|
|
For now this record can serve when we apply logs to a backup,
|
|
|
|
so we sync it. This happens before the data file is created. If the
|
|
|
|
data file was created before, and we crashed before writing the log
|
|
|
|
record, at restart the table may be used, so we would not have a
|
|
|
|
trustable history in the log (impossible to apply this log to a
|
|
|
|
backup). The way we do it, if we crash before writing the log record
|
|
|
|
then there is no data file and the table cannot be used.
|
|
|
|
@todo Note that in case of TRUNCATE TABLE we also come here; for
|
|
|
|
Recovery to be able to finish TRUNCATE TABLE, instead of leaving a
|
|
|
|
half-truncated table, we should log the record at start of
|
|
|
|
maria_create(); for that we shouldn't write to the index file but to a
|
|
|
|
buffer (DYNAMIC_STRING), put the buffer into the record, then put the
|
|
|
|
buffer into the index file (so, change _ma_keydef_write() etc). That
|
|
|
|
would also enable Recovery to finish a CREATE TABLE. The final result
|
|
|
|
would be that we would be able to finish what the SQL layer has asked
|
|
|
|
for: it would be atomic.
|
|
|
|
When in CREATE/TRUNCATE (or DROP or RENAME or REPAIR) we have not
|
|
|
|
called external_lock(), so have no TRN. It does not matter, as all
|
|
|
|
these operations are non-transactional and sync their files.
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
*/
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
if (unlikely(translog_write_record(&lsn,
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
LOGREC_REDO_CREATE_TABLE,
|
|
|
|
&dummy_transaction_object, NULL,
|
|
|
|
total_rec_length,
|
|
|
|
sizeof(log_array)/sizeof(log_array[0]),
|
WL#3072 - Maria recovery.
* Recovery of the table's live checksum (CREATE TABLE ... CHECKSUM=1)
is achieved in this patch. The table's live checksum
(info->s->state.state.checksum) is updated in inwrite_rec_hook's
under the log mutex when writing UNDO_ROW_INSERT|UPDATE|DELETE
and REDO_DELETE_ALL. The checksum variation caused by the operation
is stored in these UNDOs, so that the REDO phase, when it sees such
UNDOs, can update the live checksum if it is older (state.is_of_lsn is
lower) than the record. It is also used, as a nice add-on with no
cost, to do less row checksum computation during the UNDO phase
(as we have it in the record already).
Doing this work, it became pressing to move in-write hooks
(write_hook_for_redo() et al) to ma_blockrec.c.
The 'parts' argument of inwrite_rec_hook is unpredictable (it comes
mangled at this stage, for example by LSN compression) so it is
replaced by a 'void* hook_arg', which is used to pass down information,
currently only to write_hook_for_clr_end() (previous undo_lsn and
type of undone record).
* If from ha_maria, we print to stderr how many seconds (with one
fractional digit) the REDO phase took, same for UNDO phase and for
final table close. Just to give an indication for debugging and maybe
also for Support.
storage/maria/ha_maria.cc:
question for Monty
storage/maria/ma_blockrec.c:
* log in-write hooks (write_hook_for_redo() etc) move from
ma_loghandler.c to here; this is natural: the hooks are coupled
to their callers (functions in ma_blockrec.c).
* translog_write_record() now has a new argument "hook_arg";
using it to pass down to write_hook_for_clr_end() the transaction's
previous_undo_lsn and the type of the being undone record, and also
to pass down to all UNDOs the live checksum variation caused by the
operation.
* If table has live checksum, store in UNDO_ROW_INSERT|UPDATE|DELETE
and in CLR_END the checksum variation ("delta") caused by the
operation. For example if a DELETE caused the table's live checksum
to change from 123 to 456, we store in the UNDO_ROW_DELETE, in 4 bytes,
the value 333 (456-123).
* Instead of hard-coded "1" as length of the place where we store
the undone record's type in CLR_END, use a symbol CLR_TYPE_STORE_SIZE;
use macros clr_type_store and clr_type_korr.
* write_block_record() has a new parameter 'old_record_checksum'
which is the pre-computed checksum of old_record; that value is used
to update the table's live checksum when writing UNDO_ROW_UPDATE|CLR_END.
* In allocate_write_block_record(), if we are executing UNDO_ROW_DELETE
the row's checksum is already computed.
* _ma_update_block_record2() now expect the new row's checksum into
cur_row.checksum (was already true) and the old row's checksum into
new_row.checksum (that's new). Its two callers, maria_update() and
_ma_apply_undo_row_update(), honour this.
* When executing an UNDO_ROW_INSERT|UPDATE|DELETE in UNDO phase, pick
up the checksum delta from the log record. It is then used to update
the table's live checksum when writing CLR_END, and saves us a
computation of record.
storage/maria/ma_blockrec.h:
in-write hooks move from ma_loghandler.c
storage/maria/ma_check.c:
more straightforward size of buffer
storage/maria/ma_checkpoint.c:
<= is enough
storage/maria/ma_commit.c:
new prototype of translog_write_record()
storage/maria/ma_create.c:
new prototype of translog_write_record()
storage/maria/ma_delete.c:
The row's checksum must be computed before calling(*delete_record)(),
not after, because it must be known inside _ma_delete_block_record()
(to update the table's live checksum when writing UNDO_ROW_DELETE).
If deleting from a transactional table, live checksum was already updated
when writing UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
@todo is now done (in ma_loghandler.c)
storage/maria/ma_delete_table.c:
new prototype of translog_write_record()
storage/maria/ma_loghandler.c:
* in-write hooks move to ma_blockrec.c.
* translog_write_record() gets a new argument 'hook_arg' which is
passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
for those hooks, because when those hooks are called, 'parts' has
possibly been mangled (like with LSN compression) and is so
unpredictable.
* fix for compiler warning (unused buffer_start when compiling without
debug support)
* Because checksum delta is stored into UNDO_ROW_INSERT|UPDATE|DELETE
and CLR_END, but only if the table has live checksum, these records
are not PSEUDOFIXEDLENGTH anymore, they are now VARIABLE_LENGTH (their
length is X if no live checksum and X+4 otherwise).
* add an inwrite_rec_hook for UNDO_ROW_UPDATE, which updates the
table's live checksum. Update it also in hooks of UNDO_ROW_INSERT|
DELETE and REDO_DELETE_ALL and CLR_END.
* Bugfix: when reading a record in translog_read_record(), it happened
that "length" became negative, because the function assumed that
the record extended beyond the page's end, whereas it may be shorter.
storage/maria/ma_loghandler.h:
* Instead of hard-coded "1" and "4", use symbols and macros
to store/retrieve the type of record which the CLR_END corresponds
to, and the checksum variation caused by the operation which logs the
record
* translog_write_record() gets a new argument 'hook_arg' which is
passed down to pre|inwrite_rec_hook. It is more useful that 'parts'
for those hooks, because when those hooks are called, 'parts' has
possibly been mangled (like with LSN compression) and is so
unpredictable.
storage/maria/ma_open.c:
fix for "empty body in if() statement" (when compiling without safemutex)
storage/maria/ma_pagecache.c:
<= is enough
storage/maria/ma_recovery.c:
* print the time that each recovery phase (REDO/UNDO/flush) took;
this is enabled only when recovering from ha_maria. Is it printed
n seconds with a fractional part of one digit (like 123.4 seconds).
* In the REDO phase, update the table's live checksum by using
the checksum delta stored in UNDO_ROW_INSERT|DELETE|UPDATE and CLR_END.
Update it too when seeing REDO_DELETE_ALL.
* In the UNDO phase, when executing UNDO_ROW_INSERT, if the table does
not have live checksum then reading the record's header (as done by
the master loop of run_undo_phase()) is enough; otherwise we
do a translog_read_record() to have the checksum delta ready
for _ma_apply_undo_row_insert().
* When at the end of the REDO phase we notice that there is an unfinished
group of REDOs, don't assert in debug binaries, as I verified that it
can happen in real life (with kill -9)
* removing ' in #error as it confuses gcc3
storage/maria/ma_rename.c:
new prototype of translog_write_record()
storage/maria/ma_test_recovery.expected:
Change in output of ma_test_recovery: now all live checksums of
original tables equal those of tables recreated by the REDO phase
and those of tables fixed by the UNDO phase. I.e. recovery of
the live checksum looks like working (which was after all the only
goal of this changeset).
I checked by hand that it's not just all live checksums which are
now 0 and that's why they match. They are the old values like
3757530372. maria.test has hard-coded checksum values in its result
file so checks this too.
storage/maria/ma_update.c:
* It's useless to put up HA_STATE_CHANGED in 'key_changed',
as we put up HA_STATE_CHANGED in info->update anyway.
* We need to compute the old and new rows' checksum before calling
(*update_record)(), as checksum delta must be known when logging
UNDO_ROW_UPDATE which is done by _ma_update_block_record(). Note that
some functions change the 'newrec' record (at least _ma_check_unique()
does) so we cannot move the checksum computation too early in the
function.
storage/maria/ma_write.c:
If inserting into a transactional table, live's checksum was
already updated when writing UNDO_ROW_INSERT. The multiplication
is a trick to save an if().
storage/maria/unittest/ma_test_loghandler-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype of translog_write_record()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
new prototype of translog_write_record()
storage/myisam/sort.c:
fix for compiler warnings in pushbuild (write_merge_key* functions
didn't have their declaration match MARIA_HA::write_key).
2007-10-02 18:02:09 +02:00
|
|
|
log_array, NULL, NULL) ||
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
translog_flush(lsn)))
|
2007-06-28 14:01:57 +02:00
|
|
|
goto err;
|
2009-01-16 22:00:39 +01:00
|
|
|
share.kfile.file= file;
|
|
|
|
DBUG_EXECUTE_IF("maria_flush_whole_log",
|
|
|
|
{
|
|
|
|
DBUG_PRINT("maria_flush_whole_log", ("now"));
|
|
|
|
translog_flush(translog_get_horizon());
|
|
|
|
});
|
|
|
|
DBUG_EXECUTE_IF("maria_crash_create_table",
|
|
|
|
{
|
|
|
|
DBUG_PRINT("maria_crash_create_table", ("now"));
|
2017-07-30 10:45:49 +02:00
|
|
|
DBUG_SUICIDE();
|
2009-01-16 22:00:39 +01:00
|
|
|
});
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
/*
|
|
|
|
store LSN into file, needed for Recovery to not be confused if a
|
|
|
|
DROP+CREATE happened (applying REDOs to the wrong table).
|
|
|
|
*/
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
if (_ma_update_state_lsns_sub(&share, lsn, trnman_get_min_safe_trid(),
|
|
|
|
FALSE, TRUE))
|
2007-06-28 14:01:57 +02:00
|
|
|
goto err;
|
2011-04-25 17:22:25 +02:00
|
|
|
my_free(log_data);
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
|
|
|
|
2007-06-28 14:01:57 +02:00
|
|
|
if (!(flags & HA_DONT_TOUCH_DATA))
|
|
|
|
{
|
|
|
|
if (ci->data_file_name)
|
|
|
|
{
|
|
|
|
char *dext= strrchr(ci->data_file_name, '.');
|
|
|
|
int have_dext= dext && !strcmp(dext, MARIA_NAME_DEXT);
|
|
|
|
|
|
|
|
if (tmp_table)
|
|
|
|
{
|
|
|
|
char *path;
|
|
|
|
/* chop off the table name, tempory tables use generated name */
|
|
|
|
if ((path= strrchr(ci->data_file_name, FN_LIBCHAR)))
|
|
|
|
*path= '\0';
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(dfilename, name, ci->data_file_name, MARIA_NAME_DEXT,
|
2007-06-28 14:01:57 +02:00
|
|
|
MY_REPLACE_DIR | MY_UNPACK_FILENAME | MY_APPEND_EXT);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(dfilename, ci->data_file_name, "", MARIA_NAME_DEXT,
|
2007-06-28 14:01:57 +02:00
|
|
|
MY_UNPACK_FILENAME |
|
|
|
|
(have_dext ? MY_REPLACE_EXT : MY_APPEND_EXT));
|
|
|
|
}
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(dlinkname, name, "",MARIA_NAME_DEXT,
|
2007-06-28 14:01:57 +02:00
|
|
|
MY_UNPACK_FILENAME | MY_APPEND_EXT);
|
2017-02-20 22:40:47 +01:00
|
|
|
dlinkname_ptr= dlinkname;
|
2007-06-28 14:01:57 +02:00
|
|
|
create_flag=0;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2017-02-20 22:40:47 +01:00
|
|
|
fn_format(dfilename,name,"", MARIA_NAME_DEXT,
|
2007-06-28 14:01:57 +02:00
|
|
|
MY_UNPACK_FILENAME | MY_APPEND_EXT);
|
2007-10-04 19:33:42 +02:00
|
|
|
create_flag= (flags & HA_CREATE_KEEP_FILES) ? 0 : MY_DELETE_OLD;
|
2007-06-28 14:01:57 +02:00
|
|
|
}
|
|
|
|
if ((dfile=
|
2017-02-20 22:40:47 +01:00
|
|
|
mysql_file_create_with_symlink(key_file_dfile, dlinkname_ptr,
|
|
|
|
dfilename, 0, create_mode,
|
2011-07-13 21:10:18 +02:00
|
|
|
MYF(MY_WME | create_flag | sync_dir))) < 0)
|
2007-06-28 14:01:57 +02:00
|
|
|
goto err;
|
|
|
|
errpos=3;
|
|
|
|
|
2007-07-01 15:48:57 +02:00
|
|
|
if (_ma_initialize_data_file(&share, dfile))
|
2007-06-28 14:01:57 +02:00
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
/* Enlarge files */
|
2007-01-18 20:38:14 +01:00
|
|
|
DBUG_PRINT("info", ("enlarge to keystart: %lu",
|
|
|
|
(ulong) share.base.keystart));
|
2011-07-13 21:10:18 +02:00
|
|
|
if (mysql_file_chsize(file,(ulong) share.base.keystart,0,MYF(0)))
|
2006-04-11 15:45:10 +02:00
|
|
|
goto err;
|
|
|
|
|
2017-06-30 16:56:58 +02:00
|
|
|
if (!internal_table && sync_dir && mysql_file_sync(file, MYF(0)))
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
goto err;
|
|
|
|
|
2006-04-11 15:45:10 +02:00
|
|
|
if (! (flags & HA_DONT_TOUCH_DATA))
|
|
|
|
{
|
|
|
|
#ifdef USE_RELOC
|
2011-07-13 21:10:18 +02:00
|
|
|
if (mysql_file_chsize(key_file_dfile, dfile,
|
|
|
|
share.base.min_pack_length*ci->reloc_rows,0,MYF(0)))
|
2006-04-11 15:45:10 +02:00
|
|
|
goto err;
|
|
|
|
#endif
|
2017-06-30 16:56:58 +02:00
|
|
|
if (!internal_table && sync_dir && mysql_file_sync(dfile, MYF(0)))
|
WL#3071 Maria checkpoint
Finally this is the real checkpoint code.
It however exhibits unstabilities when a checkpoint runs concurrently
with data-modifying clients (table corruption, transaction log's
assertions) so for now a checkpoint is taken only at startup after
recovery and at shutdown, i.e. not in concurrent situations. Later
we will let it run periodically, as well as flush dirty pages
periodically (almost all needed code is there already, only pagecache
code is written but not committed).
WL#3072 Maria recovery
* replacing UNDO_ROW_PURGE with CLR_END; testing of those CLR_END via
ma_test2 which has INSERTs failing with duplicate keys.
* replaying of REDO_RENAME_TABLE
Now, off to test Recovery in ha_maria :)
BitKeeper/deleted/.del-ma_least_recently_dirtied.c:
Delete: storage/maria/ma_least_recently_dirtied.c
BitKeeper/deleted/.del-ma_least_recently_dirtied.h:
Delete: storage/maria/ma_least_recently_dirtied.h
storage/maria/Makefile.am:
compile Checkpoint module
storage/maria/ha_maria.cc:
When ha_maria starts, do a recovery from last checkpoint.
Take a checkpoint when that recovery has ended and when ha_maria
shuts down cleanly.
storage/maria/ma_blockrec.c:
* even if my_sync() fails we have to my_close() (otherwise we leak
a descriptor)
* UNDO_ROW_PURGE is replaced by a simple CLR_END for UNDO_ROW_INSERT,
as promised in the old comment; it gives us skipping during the
UNDO phase.
storage/maria/ma_check.c:
All REDOs before create_rename_lsn are ignored by Recovery. So
create_rename_lsn must be set only after all data/index has been
flushed and forced to disk. We thus move write_log_record_for_repair()
to after _ma_flush_tables_files_after_repair().
storage/maria/ma_checkpoint.c:
Checkpoint module.
storage/maria/ma_checkpoint.h:
optional argument if caller wants a thread to periodically take
checkpoints and flush dirty pages.
storage/maria/ma_create.c:
* no need to init some vars as the initial bzero(share) takes care of this.
* update to new function's name
* even if we fail in my_sync() we have to my_close()
storage/maria/ma_extra.c:
Checkpoint reads share->last_version under intern_lock, so we make
maria_extra() update it under intern_lock. THR_LOCK_maria still needed
because of _ma_test_if_reopen().
storage/maria/ma_init.c:
destroy checkpoint module when Maria shuts down.
storage/maria/ma_loghandler.c:
* UNDO_ROW_PURGE gone (see ma_blockrec.c)
* we need to remember the LSN of the LOGREC_FILE_ID for a share,
because this LSN is needed into the checkpoint record (Recovery wants
to know the validity domain of an id->name mapping)
* translog_get_horizon_no_lock() needed for Checkpoint
* comment about failing assertion (Sanja knows)
* translog_init_reader_data() thought that translog_read_record_header_scan()
returns 0 in case of error, but 0 just means "0-length header".
* translog_assign_id_to_share() now needs the MARIA_HA because
LOGREC_FILE_ID uses a log-write hook.
* Verify that (de)assignment of share->id happens only under intern_lock,
as Checkpoint reads this id with intern_lock.
* translog_purge() can accept TRANSLOG_ADDRESS, not necessarily
a real LSN.
storage/maria/ma_loghandler.h:
prototype updates
storage/maria/ma_open.c:
no need to initialize "res"
storage/maria/ma_pagecache.c:
When taking a checkpoint, we don't need to know the maximum rec_lsn
of dirty pages; this LSN was intended to be used in the two-checkpoint
rule, but last_checkpoint_lsn is as good.
4 bytes for stored_list_size is enough as PAGECACHE::blocks (number
of blocks which the pagecache can contain) is int.
storage/maria/ma_pagecache.h:
new prototype
storage/maria/ma_recovery.c:
* added replaying of REDO_RENAME_TABLE
* UNDO_ROW_PURGE gone (see ma_blockrec.c), replaced by CLR_END
* Recovery from the last checkpoint record now possible
* In new_table() we skip the table if the id->name mapping is older than
create_rename_lsn (mapping dates from lsn_of_file_id).
* in get_MARIA_HA_from_REDO_record() we skip the record
if the id->name mapping is newer than the record (can happen if processing
a record which is before the checkpoint record).
* parse_checkpoint_record() has to return a LSN, that's what caller expects
storage/maria/ma_rename.c:
new function's name; log end zeroes of tables' names (ease recovery)
storage/maria/ma_test2.c:
* equivalent of ma_test1's --test-undo added (named -u here).
* -t=1 now stops right after creating the table, so that
we can test undoing of INSERTs with duplicate keys (which tests the
CLR_END logged by _ma_write_abort_block_record()).
storage/maria/ma_test_recovery.expected:
Result of testing undoing of INSERTs with duplicate keys; there are
some differences in maria_chk -dvv but they are normal (removing
records does not shrink data/index file, does not put back the
"analyzed, optimized keys"(etc) index state.
storage/maria/ma_test_recovery:
Test undoing of INSERTs with duplicate keys, using ma_test2;
when such INSERT happens, it logs REDO_INSERT, UNDO_INSERT, REDO_DELETE,
CLR_END; we abort after that, and test that CLR_END causes recovery
to jump over UNDO_INSERT.
storage/maria/ma_write.c:
comment
storage/maria/maria_chk.c:
comment
storage/maria/maria_def.h:
* a new bit in MARIA_SHARE::in_checkpoint, used to build a list
of unique shares during Checkpoint.
* MARIA_SHARE::lsn_of_file_id added: the LSN of the last LOGREC_FILE_ID
for this share; needed to know to which LSN domain the mappings
found in the Checkpoint record apply (new mappings should not apply
to old REDOs).
storage/maria/trnman.c:
* small changes to how trnman_collect_transactions() fills its buffer;
it also uses a non-dummy lsn_read_non_atomic() found in ma_checkpoint.h
2007-09-12 11:27:34 +02:00
|
|
|
goto err;
|
2011-07-13 21:10:18 +02:00
|
|
|
if (mysql_file_close(dfile,MYF(0)))
|
2006-04-11 15:45:10 +02:00
|
|
|
goto err;
|
|
|
|
}
|
2013-03-25 23:03:13 +01:00
|
|
|
if (!internal_table)
|
|
|
|
mysql_mutex_unlock(&THR_LOCK_maria);
|
2007-04-05 13:38:05 +02:00
|
|
|
res= 0;
|
2011-04-25 17:22:25 +02:00
|
|
|
my_free((char*) rec_per_key_part);
|
2014-12-22 15:53:17 +01:00
|
|
|
ma_crypt_free(&share);
|
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria"
- similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and
DELETE no_WHERE_clause (== the DELETE which just truncates the files)
- create_rename_lsn added to MARIA_SHARE's state
- all these operations (except DROP TABLE) also update the table's
create_rename_lsn, which is needed for the correctness of
Recovery (see function comment of _ma_repair_write_log_record()
in ma_check.c)
- write a COMMIT record when transaction commits.
- don't log REDOs/UNDOs if this is an internal temporary table
like inside ALTER TABLE (I expect this to be a big win). There was
already no logging for user-created "CREATE TEMPORARY" tables.
- don't fsync files/directories if the table is not transactional
- in translog_write_record(), autogenerate a 2-byte-id for the table
and log the "id->name" pair (LOGREC_FILE_ID); log
LOGREC_LONG_TRANSACTION_ID; automatically store
the table's 2-byte-id in any log record.
- preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint
when some dirty pages are unknown; capturing trn->rec_lsn,
trn->first_undo_lsn for Checkpoint and log's low-water-mark computing.
- assertions, comments.
storage/maria/Makefile.am:
more files to build
storage/maria/ha_maria.cc:
- logging a REPAIR log record if REPAIR/OPTIMIZE was successful.
- ha_maria::data_file_type does not have to be set in every info()
call, just do it once in open().
- if caller said that transactionality can be disabled (like if
caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we
temporarily disable transactionality of the table in external_lock();
that will ensure that no REDOs/UNDOs are logged for this possibly
massive write operation (they are not needed, as if any write fails,
the table will be dropped). We re-enable in external_lock(F_UNLCK),
which in ALTER TABLE happens before the tmp table replaces the original
one (which is good, as thus the final table will have a REDO RENAME
and a correct create_rename_lsn).
- when we commit we also have to write a log record, so
trnman_commit_trn() calls become ma_commit() calls
- at end of engine's initialization, we are potentially entering a
multi-threaded dangerous world (clients are going to be accepted)
and so some assertions of mutex-owning become enforceable, for that
we set maria_multi_threaded=TRUE (see ma_control_file.c)
storage/maria/ha_maria.h:
new member ha_maria::save_transactional (see also ha_maria.cc)
storage/maria/ma_blockrec.c:
- fixing comments according to discussion with Monty
- if a table is transactional but temporarily non-transactional
(like in ALTER TABLE), we need to give a sensible LSN to the pages
(and, if we give 0, pagecache asserts).
- translog_write_record() now takes care of storing the share's
2-byte-id in the log record
storage/maria/ma_blockrec.h:
fixing comment according to discussion with Monty
storage/maria/ma_check.c:
When REPAIR/OPTIMIZE modify the data/index file, if this is a
transactional table, they must sync it; if they remove files or rename
files, they must sync the directory, so that everything is durable.
This is just applying to REPAIR/OPTIMIZE the logic already implemented
in CREATE/DROP/RENAME a few months ago.
Adding a function to write a LOGREC_REPAIR_TABLE at end of
REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and
to update the table's create_rename_lsn.
storage/maria/ma_close.c:
fix for a future bug
storage/maria/ma_control_file.c:
ensuring that if Maria is running in multi-threaded mode, anybody
wanting to write to the control file and update
last_checkpoint_lsn/last_logno owns the log's lock.
storage/maria/ma_control_file.h:
see ma_control_file.c
storage/maria/ma_create.c:
when creating a table:
- sync it and its directory only if this is a transactional table
and there is a log (no point in syncing in maria_chk)
- decouple the two uses of linkname/linkname_ptr (for index file and
for data file) into more variables, as we need to know all links
until the moment we write the LOGREC_CREATE_TABLE.
- set share.data_file_type early so that _ma_initialize_data_file()
knows it (Monty's bugfix so that a table always has at least a bitmap
page when it is created; so data-file is not 0 bytes anymore).
- log a LOGREC_CREATE_TABLE; it contains the bytes which we have
just written to the index file's header. Update table's
create_rename_lsn.
- syncing of kfile had been bugified in a previous merge, correcting
- syncing of dfile is now needed as it's not empty anymore
- in _ma_initialize_data_file(), use share's block_size and not the
global one. This is a gratuitous change, both variables are equal,
just that I find it more future-proof to use share-bound variable
rather than global one.
storage/maria/ma_delete_all.c:
log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows();
update create_rename_lsn then.
storage/maria/ma_delete_table.c:
- logging LOGREC_DROP_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- we need to sync directories only if the table is transactional
storage/maria/ma_extra.c:
questions
storage/maria/ma_init.c:
when maria_end() is called, engine is not multithreaded
storage/maria/ma_loghandler.c:
- translog_inited has to be visible to ma_create() (see how it is used
in ma_create())
- checkpoint record will be a single record, not three
- no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will
log a REDO_CREATE)
- adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by
truncating the files), REPAIR.
- MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk
- in translog_write_record(), if MARIA_SHARE does not yet have a
2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically
store this short id into log records.
- in translog_write_record(), if transaction has not logged its
long trid, log LOGREC_LONG_TRANSACTION_ID.
- For Checkpoint, we need to know the current end-of-log: adding
translog_get_horizon().
- For Control File, adding an assertion that the thread owns the
log's lock (control file is protected by this lock)
storage/maria/ma_loghandler.h:
Changes in log records (see ma_loghandler.c).
new prototypes, new functions.
storage/maria/ma_loghandler_lsn.h:
adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn,
where the most significant byte is used for flags.
storage/maria/ma_open.c:
storing the create_rename_lsn in the index file's header (in the
state, precisely) and retrieving it from there.
storage/maria/ma_pagecache.c:
- my set_if_bigger was wrong, correcting it
- if the first_in_switch list is not empty, it means that
changed_blocks misses some dirty pages, so Checkpoint cannot run and
needs to wait. A variable missing_blocks_in_changed_list is added to
tell that (should it be named missing_blocks_in_changed_blocks?)
- pagecache_collect_changed_blocks_with_lsn() now also tells the
minimum rec_lsn (needed for low-water mark computation).
storage/maria/ma_pagecache.h:
see ma_pagecache.c
storage/maria/ma_panic.c:
comment
storage/maria/ma_range.c:
comment
storage/maria/ma_rename.c:
- logging LOGREC_RENAME_TABLE; knowing if this is needed, requires
knowing if the table is transactional, which requires opening the
table.
- update create_rename_lsn
- we need to sync directories only if the table is transactional
storage/maria/ma_static.c:
comment
storage/maria/ma_test_all.sh:
- tip for Valgrind-ing ma_test_all
- do "export maria_path=somepath" before calling ma_test_all,
if you want to run ma_test_all out of storage/maria (useful
to have parallel runs, like one normal and one Valgrind, they
must not use the same tables so need to run in different directories)
storage/maria/maria_def.h:
- state now contains, in memory and on disk, the create_rename_lsn
- share now contains a 2-byte-id
storage/maria/trnman.c:
preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn;
minimum first_undo_lsn needed to know log's low-water-mark
storage/maria/trnman.h:
using most significant byte of first_undo_lsn to hold miscellaneous
flags, for now TRANSACTION_LOGGED_LONG_ID.
dummy_transaction_object is already declared in ma_static.c.
storage/maria/trnman_public.h:
dummy_transaction_object was declared in all files including
trnman_public.h, while in fact it's a single object.
new prototype
storage/maria/unittest/ma_test_loghandler-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
update for new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
update for new prototype
storage/maria/ma_commit.c:
function which wraps:
- writing a LOGREC_COMMIT record (==commit on disk)
- calling trnman_commit_trn() (=commit in memory)
storage/maria/ma_commit.h:
new header file
.tree-is-private:
this file is now needed to keep our tree private (don't push it
to public trees). When 5.1 is merged into mysql-maria, we can abandon
our maria-specific post-commit trigger; .tree_is_private will take
care of keeping commit mails private. Don't push this file to public
trees.
2007-06-22 14:49:37 +02:00
|
|
|
errpos=0;
|
2011-07-13 21:10:18 +02:00
|
|
|
if (mysql_file_close(file,MYF(0)))
|
2007-04-05 13:38:05 +02:00
|
|
|
res= my_errno;
|
|
|
|
DBUG_RETURN(res);
|
2006-04-11 15:45:10 +02:00
|
|
|
|
|
|
|
err:
|
2013-03-25 23:03:13 +01:00
|
|
|
if (!internal_table)
|
|
|
|
mysql_mutex_unlock(&THR_LOCK_maria);
|
2007-04-05 13:38:05 +02:00
|
|
|
|
|
|
|
err_no_lock:
|
2006-04-11 15:45:10 +02:00
|
|
|
save_errno=my_errno;
|
|
|
|
switch (errpos) {
|
|
|
|
case 3:
|
2011-07-13 21:10:18 +02:00
|
|
|
mysql_file_close(dfile, MYF(0));
|
2017-02-20 22:40:47 +01:00
|
|
|
if (! (flags & HA_DONT_TOUCH_DATA))
|
|
|
|
{
|
|
|
|
mysql_file_delete(key_file_dfile, dfilename, MYF(sync_dir));
|
|
|
|
if (dlinkname_ptr)
|
|
|
|
mysql_file_delete(key_file_dfile, dlinkname_ptr, MYF(sync_dir));
|
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
/* fall through */
|
|
|
|
case 1:
|
2011-07-13 21:10:18 +02:00
|
|
|
mysql_file_close(file, MYF(0));
|
2006-04-11 15:45:10 +02:00
|
|
|
if (! (flags & HA_DONT_TOUCH_DATA))
|
2017-02-20 22:40:47 +01:00
|
|
|
{
|
|
|
|
mysql_file_delete(key_file_kfile, kfilename, MYF(sync_dir));
|
|
|
|
if (klinkname_ptr)
|
|
|
|
mysql_file_delete(key_file_kfile, klinkname_ptr, MYF(sync_dir));
|
|
|
|
}
|
2006-04-11 15:45:10 +02:00
|
|
|
}
|
2014-12-22 15:53:17 +01:00
|
|
|
ma_crypt_free(&share);
|
2011-04-25 17:22:25 +02:00
|
|
|
my_free(log_data);
|
|
|
|
my_free(rec_per_key_part);
|
2006-04-11 15:45:10 +02:00
|
|
|
DBUG_RETURN(my_errno=save_errno); /* return the fatal errno */
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
uint maria_get_pointer_length(ulonglong file_length, uint def)
|
|
|
|
{
|
|
|
|
DBUG_ASSERT(def >= 2 && def <= 7);
|
|
|
|
if (file_length) /* If not default */
|
|
|
|
{
|
|
|
|
#ifdef NOT_YET_READY_FOR_8_BYTE_POINTERS
|
2013-04-07 14:00:16 +02:00
|
|
|
if (file_length >= (1ULL << 56))
|
2006-04-11 15:45:10 +02:00
|
|
|
def=8;
|
2007-03-01 18:23:58 +01:00
|
|
|
else
|
2006-04-11 15:45:10 +02:00
|
|
|
#endif
|
2013-04-07 14:00:16 +02:00
|
|
|
if (file_length >= (1ULL << 48))
|
2006-04-11 15:45:10 +02:00
|
|
|
def=7;
|
2013-04-07 14:00:16 +02:00
|
|
|
else if (file_length >= (1ULL << 40))
|
2006-04-11 15:45:10 +02:00
|
|
|
def=6;
|
2013-04-07 14:00:16 +02:00
|
|
|
else if (file_length >= (1ULL << 32))
|
2006-04-11 15:45:10 +02:00
|
|
|
def=5;
|
2013-04-07 14:00:16 +02:00
|
|
|
else if (file_length >= (1ULL << 24))
|
2006-04-11 15:45:10 +02:00
|
|
|
def=4;
|
2013-04-07 14:00:16 +02:00
|
|
|
else if (file_length >= (1ULL << 16))
|
2006-04-11 15:45:10 +02:00
|
|
|
def=3;
|
|
|
|
else
|
|
|
|
def=2;
|
|
|
|
}
|
|
|
|
return def;
|
|
|
|
}
|
2007-01-18 20:38:14 +01:00
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
Sort columns for records-in-block
|
|
|
|
|
|
|
|
IMPLEMENTATION
|
|
|
|
Sort columns in following order:
|
|
|
|
|
|
|
|
Fixed size, not null columns
|
|
|
|
Fixed length, null fields
|
2008-01-07 17:54:41 +01:00
|
|
|
Numbers (zero fill fields)
|
|
|
|
Variable length fields (CHAR, VARCHAR) according to length
|
2007-01-18 20:38:14 +01:00
|
|
|
Blobs
|
|
|
|
|
|
|
|
For same kind of fields, keep fields in original order
|
|
|
|
*/
|
|
|
|
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
static inline int sign(long a)
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
|
|
|
return a < 0 ? -1 : (a > 0 ? 1 : 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2007-04-12 11:05:30 +02:00
|
|
|
static int compare_columns(MARIA_COLUMNDEF **a_ptr, MARIA_COLUMNDEF **b_ptr)
|
2007-01-18 20:38:14 +01:00
|
|
|
{
|
|
|
|
MARIA_COLUMNDEF *a= *a_ptr, *b= *b_ptr;
|
|
|
|
enum en_fieldtype a_type, b_type;
|
|
|
|
|
2008-01-07 17:54:41 +01:00
|
|
|
a_type= (a->type == FIELD_CHECK) ? FIELD_NORMAL : a->type;
|
|
|
|
b_type= (b->type == FIELD_CHECK) ? FIELD_NORMAL : b->type;
|
2007-01-18 20:38:14 +01:00
|
|
|
|
|
|
|
if (a_type == FIELD_NORMAL && !a->null_bit)
|
|
|
|
{
|
|
|
|
if (b_type != FIELD_NORMAL || b->null_bit)
|
|
|
|
return -1;
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
return sign((long) a->offset - (long) b->offset);
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
|
|
|
if (b_type == FIELD_NORMAL && !b->null_bit)
|
|
|
|
return 1;
|
|
|
|
if (a_type == b_type)
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
return sign((long) a->offset - (long) b->offset);
|
2007-01-18 20:38:14 +01:00
|
|
|
if (a_type == FIELD_NORMAL)
|
|
|
|
return -1;
|
|
|
|
if (b_type == FIELD_NORMAL)
|
|
|
|
return 1;
|
2008-01-07 17:54:41 +01:00
|
|
|
if (a_type == FIELD_SKIP_ZERO)
|
|
|
|
return -1;
|
|
|
|
if (b_type == FIELD_SKIP_ZERO)
|
|
|
|
return 1;
|
|
|
|
if (a->type != FIELD_BLOB && b->type != FIELD_BLOB)
|
|
|
|
if (a->length != b->length)
|
|
|
|
return sign((long) a->length - (long) b->length);
|
2007-01-18 20:38:14 +01:00
|
|
|
if (a_type == FIELD_BLOB)
|
|
|
|
return 1;
|
|
|
|
if (b_type == FIELD_BLOB)
|
|
|
|
return -1;
|
Fixed bug in undo_key_delete; Caused crashed key files in recovery
Maria is now used for internal temporary tables in MySQL
Better usage of VARCHAR and long strings in temporary tables
Use packed fields if BLOCK_RECORD is used
null_bytes are not anymore stored in a separate field
New interface to remember and restore scan position
Fixed bugs in unique handling
Don't sync Maria temporary tables
Lock control file while it's used to stop several processes from using it
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/my_sys.h:
Changed value of MA_DONT_OVERWRITE_FILE as it collided with MY_SYNC_DIR
Split MY_DONT_WAIT into MY_NO_WAIT and MY_SHORT_WAIT (for my_lock())
Added MY_FORCE_LOCK
include/myisam.h:
Make MyISAM columndef compile time compatible with Maria
mysql-test/lib/mtr_process.pl:
Removed confusing warning (It's common that there is a lot of other files than pid files)
mysql-test/mysql-test-run.pl:
Added --sync-frm to speed up tests
mysql-test/r/maria-recovery.result:
Updated results from wrong push
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
Marked test as --big
mysys/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
mysys/my_thr_init.c:
Fix that we don't give name to thread before it's properly initied
sql/handler.cc:
Added myisam.h
sql/handler.h:
Changes to use Maria for internal temporary tables
Removed not needed argument to restart_rnd_next()
Added function remember_rnd_pos()
sql/my_lock.c:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/mysql_priv.h:
Added maria_hton
sql/sql_class.h:
Changes to use Maria for internal temporary tables
sql/sql_select.cc:
Changes to use Maria for internal temporary tables
Temporary tables didn't properly switch to dynamic row format if long strings was used
Better usage of VARCHAR in temporary tables
Use new interface to restart scan in duplicate removal
sql/sql_select.h:
Changes to use Maria for internal temporary tables
sql/sql_show.cc:
Changes to use Maria for internal temporary tables
Removed all end space
sql/sql_table.cc:
Set HA_OPTION_PACK_RECORD if we are not using default or static record
sql/sql_union.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
sql/sql_update.cc:
If MY_FORCE_LOCK is given, use locking even if my_disable_locking is given
If MY_NO_WAIT is given, return at once if lock is occupied
If MY_SHORT_WAIT is given, wait some time for lock before returning (This was called MY_DONT_WAIT before)
storage/maria/ha_maria.cc:
Use packed fields
null_bytes are not anymore stored in a separate field
Changes to use Maria for internal temporary tables
Give warning if we try to do an ALTER TABLE to a unusable row format
storage/maria/ha_maria.h:
Allow Maria with block format to restart scanning at given position
storage/maria/ma_blockrec.c:
Added functions to remember and restore scan position
Allocate cur_row.extents so that we don't have to do a malloc on first read
Fixed bug when using packed row without packed strings
Removed unneeded calls to free_full_pages()
Fixed unlikely bug when using old bitmap to read head page and head page had gone away
Remember row position when doing undo of delete and update row (needed for undo of key delete)
storage/maria/ma_blockrec.h:
Added functions to remember and restore scan position
storage/maria/ma_close.c:
Don't sync temporary tables
storage/maria/ma_control_file.c:
Lock control file while it's used to stop several processes from using it
storage/maria/ma_create.c:
Fixed bug when using FIELD_NORMAL that was longer than FULL_PAGE_SIZE
Fixed bug that casued fields to not be ordered according to offset
Fixed bug in unique creation
storage/maria/ma_delete.c:
Don't write record reference when deleting key.
(Rowid is likely to be different when we undo this)
storage/maria/ma_dynrec.c:
Fixed core dump when comparing records (happended in unique handling)
storage/maria/ma_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
Removed TODO comment. (Was not relevant as all other instances are guranteed to be closed when we the code is excecuted)
Added DBUG_ASSERT() to prove above.
storage/maria/ma_key_recover.c:
CLR's for UNDO_ROW_DELETE and UNDO_ROW_UPDATE now include rowid for the row.
This was needed for undo_key_delete to work, as undo of delete row is likely to put row in a new position.
undo_delete_key now doesn't include row position
storage/maria/ma_open.c:
Added virtual functions for remembering and restoring scan position
Fixed wrong key search method when using multi-byte character sets (Bug#32705)
Store original column number in index file
NOTE: Index files are now incompatible with previous versions!
(Ok as we haven't yet made a public Maria release)
storage/maria/ma_recovery.c:
Set info->cur_row.lastpos when reading CLR's for UNDO_ROW_DELETE or UNDO_ROW_UPDATE
storage/maria/ma_scan.c:
Added default function to remember and restore scan position
storage/maria/maria_def.h:
Added virtual functions & variables to remember and restore scan position
Added MARIA_MAX_CONTROL_FILE_LOCK_RETRY
storage/myisam/ha_myisam.cc:
Fixed compiler errors as columdef->type is now an enum, not an integer
Added functions to remember and restore scan position
storage/myisam/ha_myisam.h:
Added functions to remember and restore scan position
storage/myisam/mi_check.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_extra.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/mi_open.c:
MY_DONT_WAIT -> MY_SHORT_WAIT
storage/myisam/myisamdef.h:
MY_DONT_WAIT -> MY_SHORT_WAIT
2007-12-17 00:17:37 +01:00
|
|
|
return sign((long) a->offset - (long) b->offset);
|
2007-01-18 20:38:14 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
2007-12-18 02:21:32 +01:00
|
|
|
/**
|
|
|
|
@brief Initialize data file
|
|
|
|
|
|
|
|
@note
|
|
|
|
In BLOCK_RECORD, a freshly created datafile is one page long; while in
|
|
|
|
other formats it is 0-byte long.
|
|
|
|
*/
|
2007-01-18 20:38:14 +01:00
|
|
|
|
2007-07-01 15:20:57 +02:00
|
|
|
int _ma_initialize_data_file(MARIA_SHARE *share, File dfile)
|
2007-04-12 11:05:30 +02:00
|
|
|
{
|
|
|
|
if (share->data_file_type == BLOCK_RECORD)
|
|
|
|
{
|
WL#3072 - Maria recovery
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.
sql/handler.cc:
typo
storage/maria/Makefile.am:
Adding ma_test_recovery (which ma_test_all invokes, and which can
also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
comments
storage/maria/ma_bitmap.c:
fixing comments. 2 -> sizeof(maria_bitmap_marker).
Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
Now putting the "bm" signature when creating the first bitmap page
(it used to happen only at next open, but that
caused an annoying difference when testing Recovery if the original
run didn't open the table, and it looks more
logical like this: it goes to disk only with its signature correct);
see the "QQ" comment towards the _ma_initialize_data_file() call
in ma_create.c for more).
When reading a bitmap page, verify its signature (happens when normally
using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
* no need to sync the data file if table is not transactional
* Comments, code cleanup (log-related data moved to log-related code
block, int5store->page_store).
* Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
do for other records (though this record will soon be replaced
with a CLR).
* If "page" is 1 it means the page which extends from byte
page*block_size+1 to (page+1)*block_size (byte number 1 being
the first byte of the file). The last byte of the file is
data_file_length (same convention).
A new page needs to be created if the last byte of the page is
beyond the last byte of the file, i.e.
(page+1)*block_size+1 > data_file_length, so we correct the test
(bug found when testing log applying for ma_test1 -M -T --skip-update).
* update the page's LSN when removing a row from it during
execution of a REDO_PURGE_ROW record (bug found when testing log
applying for ma_test1 -M -T --skip-update).
* applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
we can always flush the table's state when closing the last instance
of the table. And it is needed for maria_read_log (as it does
not use maria_lock_database()).
storage/maria/ma_control_file.c:
when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
double-inclusion safe
storage/maria/ma_create.c:
during recovery, don't log records. Comments.
Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
during recovery, don't log records. Log the end-zero of the dropped
table's name, so that recovery can use the string in place without
extending it to fit an end zero.
storage/maria/ma_loghandler.c:
* inwrite_rec_hook also needs access to the MARIA_SHARE, like
prewrite_rec_hook. This will be needed to update
share->records_diff (in the upcoming patch "recovery of the state").
* LOG_DESC::record_ends_group changed to an enum.
* LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
corrected
* Sanja please see the @todo LOG BUG
* avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
- log write hooks called while the log's lock is held (inwrite_rec_hook)
now need the MARIA_SHARE, like prewrite_rec_hook already had
- instead of a bool saying if this record's type ends groups or not,
we refine: it may not end a group, it may end a group, or it may
be a group in itself. Imagine that we had a physical write failure
to a table before we log the UNDO, we still end up in
external_lock(F_UNLCK) and then we log a COMMIT: we don't want
to consider this COMMIT as ending the group of REDOs (don't want
to execute those REDOs during Recovery), that's why we say "COMMIT
is a group in itself, it aborts any previous group". This also
gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
New Recovery code, replacing the old pseudocode.
Most of maria_read_log moved here.
Call-able from ha_maria, but not enabled yet.
Compared to the previous version of maria_read_log, some bugs have
been fixed, debugging output can go to stdout or a disk file (for now
it's useful for me, later it can be changed), execution of
REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
has been factored into functions. We abort an unfinished group
of records if we see a record which is a group in itself (like COMMIT).
No need for maria_panic() after a bug (which caused tables to not
be closed) was fixed; if there is yet another bug I prefer to see it.
When opening a table for Recovery, set data_file_length
and key_file_length to their real physical value (these are the
easiest state members to restore :). Warn us if the last page
was truncated (but Recovery handles it).
MARIA_SHARE::state::state::records is now partly recovered (not
idempotent, but works if recreating tables from scracth).
When applying a REDO to a page, stamp it with the UNDO's LSN
(current_group_end_lsn), not with the REDO's LSN; it makes
the table more identical to the original table (easier to compare
the two tables in the end).
Big thing missing: some types of REDOs are not handled,
and the UNDO phase does not exist (missing functions to execute UNDOs
to actually rollback). So for now tests are only inserting/deleting
a few 100 rows, closing the table and seeing if the log is applied ok;
it works. UPDATE not handled.
storage/maria/ma_recovery.h:
new functions: ma_recover() for recovery from inside ha_maria;
_ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
don't write log records during recovery
storage/maria/ma_test2.c:
- fail if maria_info() or other subtests find some wrong information
- new option -g to skip updates.
- init the translog before creating the table, so that log applying
can work.
- in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
cleanup files. Test log applying.
storage/maria/maria_read_log.c:
most of the logic moves to ma_recovery.c to be shared between
maria_read_log and recovery-from-inside-mysqld.
See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
unit test for Recovery. Tests insert and delete,
REDO_UPDATE not yet coded.
Script is called from ma_test_all. Can run standalone.
2007-07-26 11:56:21 +02:00
|
|
|
share->bitmap.block_size= share->base.block_size;
|
|
|
|
share->bitmap.file.file = dfile;
|
|
|
|
return _ma_bitmap_create_first(share);
|
2007-04-12 11:05:30 +02:00
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
2007-06-27 12:58:08 +02:00
|
|
|
|
|
|
|
|
|
|
|
/**
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
@brief Writes create_rename_lsn, skip_redo_lsn and is_of_horizon to disk,
|
|
|
|
can force.
|
2007-06-27 12:58:08 +02:00
|
|
|
|
|
|
|
This is for special cases where:
|
|
|
|
- we don't want to write the full state to disk (so, not call
|
|
|
|
_ma_state_info_write()) because some parts of the state may be
|
|
|
|
currently inconsistent, or because it would be overkill
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
- we must sync these LSNs immediately for correctness.
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
It acquires intern_lock to protect the LSNs and state write.
|
2007-06-27 12:58:08 +02:00
|
|
|
|
|
|
|
@param share table's share
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
@param lsn LSN to write to log files
|
|
|
|
@param create_trid Trid to be used as state.create_trid
|
2007-06-27 12:58:08 +02:00
|
|
|
@param do_sync if the write should be forced to disk
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
@param update_create_rename_lsn if this LSN should be updated or not
|
2007-06-27 12:58:08 +02:00
|
|
|
|
|
|
|
@return Operation status
|
|
|
|
@retval 0 ok
|
|
|
|
@retval 1 error (disk problem)
|
|
|
|
*/
|
|
|
|
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
int _ma_update_state_lsns(MARIA_SHARE *share, LSN lsn, TrID create_trid,
|
|
|
|
my_bool do_sync, my_bool update_create_rename_lsn)
|
2007-06-27 12:58:08 +02:00
|
|
|
{
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
int res;
|
2012-03-23 17:11:29 +01:00
|
|
|
DBUG_ENTER("_ma_update_state_lsns");
|
2011-07-13 21:10:18 +02:00
|
|
|
mysql_mutex_lock(&share->intern_lock);
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
res= _ma_update_state_lsns_sub(share, lsn, create_trid, do_sync,
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
update_create_rename_lsn);
|
2011-07-13 21:10:18 +02:00
|
|
|
mysql_mutex_unlock(&share->intern_lock);
|
2012-03-23 17:11:29 +01:00
|
|
|
DBUG_RETURN(res);
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/**
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
@brief Writes create_rename_lsn, skip_redo_lsn and is_of_horizon to disk,
|
|
|
|
can force.
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
Shortcut of _ma_update_state_lsns() when we know that intern_lock is not
|
|
|
|
needed (when creating a table or opening it for the first time).
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
|
|
|
|
@param share table's share
|
Fix for BUG#37876 "Importing Maria table from other server via binary copy does not work":
- after auto-zerofill (ha_maria::check_and_repair()) kepts its state's LSNs unchanged, which could
be the same as the create_rename_lsn of another pre-existing table, which would break versioning as this LSN
serves as unique identifier in the versioning code (in maria_open()). Even the state pieces which
maria_zerofill() did change were lost (because they didn't go to disk).
- after this fix, if two tables were auto-zerofilled at the same time (by _ma_mark_changed())
they could receive the same create_rename_lsn, which would break versioning again. Fix is to write a log
record each time a table is imported.
- Print state's LSNs (create_rename_lsn, is_of_horizon, skip_redo_lsn) and UUID in maria_chk -dvv.
mysql-test/r/maria-autozerofill.result:
result
mysql-test/t/maria-autozerofill.test:
Test for auto-zerofilling
storage/maria/ha_maria.cc:
The state changes done by auto-zerofilling never reached disk.
storage/maria/ma_check.c:
When zerofilling a table, including its pages' LSNs, new state LSNs are needed next time the table
is imported into a Maria instance.
storage/maria/ma_create.c:
Write LOGREC_IMPORTED_TABLE when importing a table. This is informative and ensures
that the table gets a unique create_rename_lsn even though multiple tables
are imported by concurrent threads (it advances the log's end LSN).
storage/maria/ma_key_recover.c:
comment
storage/maria/ma_locking.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_loghandler.c:
New type of log record
storage/maria/ma_loghandler.h:
New type of log record
storage/maria/ma_loghandler_lsn.h:
New name for constant as can be used not only by maria_chk but auto-zerofill now too.
storage/maria/ma_open.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_recovery.c:
print content of LOGREC_IMPORTED_TABLE in maria_read_log.
storage/maria/maria_chk.c:
print info about LSNs of the table's state, and UUID, when maria_chk -dvv
storage/maria/maria_pack.c:
new name for constant
storage/maria/unittest/ma_test_recovery.pl:
Now that maria_chk -dvv shows state LSNs and UUID those need to be filtered out,
as maria_read_log -a does not use the same as at original run.
2008-07-09 11:02:27 +02:00
|
|
|
@param lsn LSN to write to state; if LSN_IMPOSSIBLE, write
|
|
|
|
a LOGREC_IMPORTED_TABLE and use its LSN as lsn.
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
@param create_trid Trid to be used as state.create_trid
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
@param do_sync if the write should be forced to disk
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
@param update_create_rename_lsn if this LSN should be updated or not
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
|
|
|
|
@return Operation status
|
|
|
|
@retval 0 ok
|
|
|
|
@retval 1 error (disk problem)
|
|
|
|
*/
|
|
|
|
|
2018-04-30 14:21:52 +02:00
|
|
|
#if defined(_MSC_VER) && (_MSC_VER == 1310)
|
2008-01-10 19:07:17 +01:00
|
|
|
/*
|
|
|
|
Visual Studio 2003 compiler produces internal compiler error
|
|
|
|
in this function. Disable optimizations to workaround.
|
|
|
|
*/
|
|
|
|
#pragma optimize("",off)
|
|
|
|
#endif
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
int _ma_update_state_lsns_sub(MARIA_SHARE *share, LSN lsn, TrID create_trid,
|
|
|
|
my_bool do_sync,
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
my_bool update_create_rename_lsn)
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
{
|
2008-02-07 21:46:32 +01:00
|
|
|
uchar buf[LSN_STORE_SIZE * 3], *ptr;
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
uchar trid_buff[8];
|
2007-06-27 12:58:08 +02:00
|
|
|
File file= share->kfile.file;
|
|
|
|
DBUG_ASSERT(file >= 0);
|
Fix for BUG#37876 "Importing Maria table from other server via binary copy does not work":
- after auto-zerofill (ha_maria::check_and_repair()) kepts its state's LSNs unchanged, which could
be the same as the create_rename_lsn of another pre-existing table, which would break versioning as this LSN
serves as unique identifier in the versioning code (in maria_open()). Even the state pieces which
maria_zerofill() did change were lost (because they didn't go to disk).
- after this fix, if two tables were auto-zerofilled at the same time (by _ma_mark_changed())
they could receive the same create_rename_lsn, which would break versioning again. Fix is to write a log
record each time a table is imported.
- Print state's LSNs (create_rename_lsn, is_of_horizon, skip_redo_lsn) and UUID in maria_chk -dvv.
mysql-test/r/maria-autozerofill.result:
result
mysql-test/t/maria-autozerofill.test:
Test for auto-zerofilling
storage/maria/ha_maria.cc:
The state changes done by auto-zerofilling never reached disk.
storage/maria/ma_check.c:
When zerofilling a table, including its pages' LSNs, new state LSNs are needed next time the table
is imported into a Maria instance.
storage/maria/ma_create.c:
Write LOGREC_IMPORTED_TABLE when importing a table. This is informative and ensures
that the table gets a unique create_rename_lsn even though multiple tables
are imported by concurrent threads (it advances the log's end LSN).
storage/maria/ma_key_recover.c:
comment
storage/maria/ma_locking.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_loghandler.c:
New type of log record
storage/maria/ma_loghandler.h:
New type of log record
storage/maria/ma_loghandler_lsn.h:
New name for constant as can be used not only by maria_chk but auto-zerofill now too.
storage/maria/ma_open.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_recovery.c:
print content of LOGREC_IMPORTED_TABLE in maria_read_log.
storage/maria/maria_chk.c:
print info about LSNs of the table's state, and UUID, when maria_chk -dvv
storage/maria/maria_pack.c:
new name for constant
storage/maria/unittest/ma_test_recovery.pl:
Now that maria_chk -dvv shows state LSNs and UUID those need to be filtered out,
as maria_read_log -a does not use the same as at original run.
2008-07-09 11:02:27 +02:00
|
|
|
|
|
|
|
if (lsn == LSN_IMPOSSIBLE)
|
|
|
|
{
|
|
|
|
int res;
|
|
|
|
LEX_CUSTRING log_array[TRANSLOG_INTERNAL_PARTS + 1];
|
|
|
|
/* table name is logged only for information */
|
2008-10-15 14:44:31 +02:00
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 0].str=
|
|
|
|
(uchar *)(share->open_file_name.str);
|
Fix for BUG#37876 "Importing Maria table from other server via binary copy does not work":
- after auto-zerofill (ha_maria::check_and_repair()) kepts its state's LSNs unchanged, which could
be the same as the create_rename_lsn of another pre-existing table, which would break versioning as this LSN
serves as unique identifier in the versioning code (in maria_open()). Even the state pieces which
maria_zerofill() did change were lost (because they didn't go to disk).
- after this fix, if two tables were auto-zerofilled at the same time (by _ma_mark_changed())
they could receive the same create_rename_lsn, which would break versioning again. Fix is to write a log
record each time a table is imported.
- Print state's LSNs (create_rename_lsn, is_of_horizon, skip_redo_lsn) and UUID in maria_chk -dvv.
mysql-test/r/maria-autozerofill.result:
result
mysql-test/t/maria-autozerofill.test:
Test for auto-zerofilling
storage/maria/ha_maria.cc:
The state changes done by auto-zerofilling never reached disk.
storage/maria/ma_check.c:
When zerofilling a table, including its pages' LSNs, new state LSNs are needed next time the table
is imported into a Maria instance.
storage/maria/ma_create.c:
Write LOGREC_IMPORTED_TABLE when importing a table. This is informative and ensures
that the table gets a unique create_rename_lsn even though multiple tables
are imported by concurrent threads (it advances the log's end LSN).
storage/maria/ma_key_recover.c:
comment
storage/maria/ma_locking.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_loghandler.c:
New type of log record
storage/maria/ma_loghandler.h:
New type of log record
storage/maria/ma_loghandler_lsn.h:
New name for constant as can be used not only by maria_chk but auto-zerofill now too.
storage/maria/ma_open.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_recovery.c:
print content of LOGREC_IMPORTED_TABLE in maria_read_log.
storage/maria/maria_chk.c:
print info about LSNs of the table's state, and UUID, when maria_chk -dvv
storage/maria/maria_pack.c:
new name for constant
storage/maria/unittest/ma_test_recovery.pl:
Now that maria_chk -dvv shows state LSNs and UUID those need to be filtered out,
as maria_read_log -a does not use the same as at original run.
2008-07-09 11:02:27 +02:00
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS + 0].length=
|
2008-08-25 13:49:47 +02:00
|
|
|
share->open_file_name.length + 1;
|
Fix for BUG#37876 "Importing Maria table from other server via binary copy does not work":
- after auto-zerofill (ha_maria::check_and_repair()) kepts its state's LSNs unchanged, which could
be the same as the create_rename_lsn of another pre-existing table, which would break versioning as this LSN
serves as unique identifier in the versioning code (in maria_open()). Even the state pieces which
maria_zerofill() did change were lost (because they didn't go to disk).
- after this fix, if two tables were auto-zerofilled at the same time (by _ma_mark_changed())
they could receive the same create_rename_lsn, which would break versioning again. Fix is to write a log
record each time a table is imported.
- Print state's LSNs (create_rename_lsn, is_of_horizon, skip_redo_lsn) and UUID in maria_chk -dvv.
mysql-test/r/maria-autozerofill.result:
result
mysql-test/t/maria-autozerofill.test:
Test for auto-zerofilling
storage/maria/ha_maria.cc:
The state changes done by auto-zerofilling never reached disk.
storage/maria/ma_check.c:
When zerofilling a table, including its pages' LSNs, new state LSNs are needed next time the table
is imported into a Maria instance.
storage/maria/ma_create.c:
Write LOGREC_IMPORTED_TABLE when importing a table. This is informative and ensures
that the table gets a unique create_rename_lsn even though multiple tables
are imported by concurrent threads (it advances the log's end LSN).
storage/maria/ma_key_recover.c:
comment
storage/maria/ma_locking.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_loghandler.c:
New type of log record
storage/maria/ma_loghandler.h:
New type of log record
storage/maria/ma_loghandler_lsn.h:
New name for constant as can be used not only by maria_chk but auto-zerofill now too.
storage/maria/ma_open.c:
instead of using translog_get_horizon() for state's LSNs of imported table,
use the LSN of to-be-written LOGREC_IMPORTED_TABLE.
storage/maria/ma_recovery.c:
print content of LOGREC_IMPORTED_TABLE in maria_read_log.
storage/maria/maria_chk.c:
print info about LSNs of the table's state, and UUID, when maria_chk -dvv
storage/maria/maria_pack.c:
new name for constant
storage/maria/unittest/ma_test_recovery.pl:
Now that maria_chk -dvv shows state LSNs and UUID those need to be filtered out,
as maria_read_log -a does not use the same as at original run.
2008-07-09 11:02:27 +02:00
|
|
|
if ((res= translog_write_record(&lsn, LOGREC_IMPORTED_TABLE,
|
|
|
|
&dummy_transaction_object, NULL,
|
|
|
|
(translog_size_t)
|
|
|
|
log_array[TRANSLOG_INTERNAL_PARTS +
|
|
|
|
0].length,
|
|
|
|
sizeof(log_array)/sizeof(log_array[0]),
|
|
|
|
log_array, NULL, NULL)))
|
|
|
|
return res;
|
|
|
|
}
|
|
|
|
|
- WL#3072 Maria Recovery:
Recovery of state.records (the count of records which is stored into
the header of the index file). For that, state.is_of_lsn is introduced;
logic is explained in ma_recovery.c (look for "Recovery of the state").
The net gain is that in case of crash, we now recover state.records,
and it is idempotent (ma_test_recovery tests it).
state.checksum is not recovered yet, mail sent for discussion.
- WL#3071 Maria Checkpoint: preparation for it, by protecting
all modifications of the state in memory or on disk with intern_lock
(with the exception of the really-often-modified state.records,
which is now protected with the log's lock, see ma_recovery.c
(look for "Recovery of the state"). Also, if maria_close() sees that
Checkpoint is looking at this table it will not my_free() the share.
- don't compute row's checksum twice in case of UPDATE (correction
to a bugfix I made yesterday).
storage/maria/ha_maria.cc:
protect state write with intern_lock (against Checkpoint)
storage/maria/ma_blockrec.c:
* don't reset trn->rec_lsn in _ma_unpin_all_pages(), because it
should wait until we have corrected the allocation in the bitmap
(as the REDO can serve to correct the allocation during Recovery);
introducing _ma_finalize_row() for that.
* In a changeset yesterday I moved computation of the checksum
into write_block_record(), to fix a bug in UPDATE. Now I notice
that maria_update() already computes the checksum, it's just that
it puts it into info->cur_row while _ma_update_block_record()
uses info->new_row; so, removing the checksum computation from
write_block_record(), putting it back into allocate_and_write_block_record()
(which is called only by INSERT and UNDO_DELETE), and copying
cur_row->checksum into new_row->checksum in _ma_update_block_record().
storage/maria/ma_check.c:
new prototypes, they will take intern_lock when writing the state;
also take intern_lock when changing share->kfile. In both cases
this is to protect against Checkpoint reading/writing the state or reading
kfile at the same time.
Not updating create_rename_lsn directly at end of write_log_record_for_repair()
as it wouldn't have intern_lock.
storage/maria/ma_close.c:
Checkpoint builds a list of shares (under THR_LOCK_maria), then it
handles each such share (under intern_lock) (doing flushing etc);
if maria_close() freed this share between the two, Checkpoint
would see a bad pointer. To avoid this, when building the list Checkpoint
marks each share, so that maria_close() knows it should not free it
and Checkpoint will free it itself.
Extending the zone covered by intern_lock to protect against
Checkpoint reading kfile, writing state.
storage/maria/ma_create.c:
When we update create_rename_lsn, we also update is_of_lsn to
the same value: it is logical, and allows us to test in maria_open()
that the former is not bigger than the latter (the contrary is a sign
of index header corruption, or severe logging bug which hinders
Recovery, table needs a repair).
_ma_update_create_rename_lsn_on_disk() also writes is_of_lsn;
it now operates under intern_lock (protect against Checkpoint),
a shortcut function is available for cases where acquiring
intern_lock is not needed (table's creation or first open).
storage/maria/ma_delete.c:
if table is transactional, "records" is already decremented
when logging UNDO_ROW_DELETE.
storage/maria/ma_delete_all.c:
comments
storage/maria/ma_extra.c:
Protect modifications of the state, in memory and/or on disk,
with intern_lock, against a concurrent Checkpoint.
When state goes to disk, update it's is_of_lsn (by calling
the new _ma_state_info_write()).
In HA_EXTRA_FORCE_REOPEN, don't set share->changed to 0 (undoing
a change I made a few days ago) and ASK_MONTY
storage/maria/ma_locking.c:
no real code change here.
storage/maria/ma_loghandler.c:
Log-write-hooks for updating "state.records" under log's mutex
when writing/updating/deleting a row or deleting all rows.
storage/maria/ma_loghandler_lsn.h:
merge (make LSN_ERROR and LSN_REPAIRED_BY_MARIA_CHK different)
storage/maria/ma_open.c:
When opening a table verify that is_of_lsn >= create_rename_lsn; if
false the header must be corrupted.
_ma_state_info_write() is split in two: _ma_state_info_write_sub()
which is the old _ma_state_info_write(), and _ma_state_info_write()
which additionally takes intern_lock if requested (to protect
against Checkpoint) and updates is_of_lsn.
_ma_open_keyfile() should change kfile.file under intern_lock
to protect Checkpoint from reading a wrong kfile.file.
storage/maria/ma_recovery.c:
Recovery of state.records: when the REDO phase sees UNDO_ROW_INSERT
which has a LSN > state.is_of_lsn it increments state.records.
Same for UNDO_ROW_DELETE and UNDO_ROW_PURGE.
When closing a table during Recovery, we know its state is at least
as new as the current log record we are looking at, so increase
is_of_lsn to the LSN of the current log record.
storage/maria/ma_rename.c:
update for new behaviour of _ma_update_create_rename_lsn_on_disk().
storage/maria/ma_test1.c:
update to new prototype
storage/maria/ma_test2.c:
update to new prototype (actually prototype was changed days ago,
but compiler does not complain about the extra argument??)
storage/maria/ma_test_recovery.expected:
new result file of ma_test_recovery. Improvements: record
count read from index's header is now always correct.
storage/maria/ma_test_recovery:
"rm" fails if file does not exist. Redirect stderr of script.
storage/maria/ma_write.c:
if table is transactional, "records" is already incremented when
logging UNDO_ROW_INSERT. Comments.
storage/maria/maria_chk.c:
update is_of_lsn too
storage/maria/maria_def.h:
- MARIA_STATE_INFO::is_of_lsn which is used by Recovery. It is stored
into the index file's header.
- Checkpoint can now mark a table as "don't free this", and maria_close()
can reply "ok then you will free it".
- new functions
storage/maria/maria_pack.c:
update for new name
2007-09-07 15:02:30 +02:00
|
|
|
for (ptr= buf; ptr < (buf + sizeof(buf)); ptr+= LSN_STORE_SIZE)
|
|
|
|
lsn_store(ptr, lsn);
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
share->state.skip_redo_lsn= share->state.is_of_horizon= lsn;
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
share->state.create_trid= create_trid;
|
|
|
|
mi_int8store(trid_buff, create_trid);
|
2011-01-24 14:19:40 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
Update create_rename_lsn if update was requested or if the old one had an
|
|
|
|
impossible value.
|
|
|
|
*/
|
|
|
|
if (update_create_rename_lsn ||
|
|
|
|
(share->state.create_rename_lsn > lsn && lsn != LSN_IMPOSSIBLE))
|
WL#3071 Maria checkpoint
Finally this is the real checkpoint code.
It however exhibits unstabilities when a checkpoint runs concurrently
with data-modifying clients (table corruption, transaction log's
assertions) so for now a checkpoint is taken only at startup after
recovery and at shutdown, i.e. not in concurrent situations. Later
we will let it run periodically, as well as flush dirty pages
periodically (almost all needed code is there already, only pagecache
code is written but not committed).
WL#3072 Maria recovery
* replacing UNDO_ROW_PURGE with CLR_END; testing of those CLR_END via
ma_test2 which has INSERTs failing with duplicate keys.
* replaying of REDO_RENAME_TABLE
Now, off to test Recovery in ha_maria :)
BitKeeper/deleted/.del-ma_least_recently_dirtied.c:
Delete: storage/maria/ma_least_recently_dirtied.c
BitKeeper/deleted/.del-ma_least_recently_dirtied.h:
Delete: storage/maria/ma_least_recently_dirtied.h
storage/maria/Makefile.am:
compile Checkpoint module
storage/maria/ha_maria.cc:
When ha_maria starts, do a recovery from last checkpoint.
Take a checkpoint when that recovery has ended and when ha_maria
shuts down cleanly.
storage/maria/ma_blockrec.c:
* even if my_sync() fails we have to my_close() (otherwise we leak
a descriptor)
* UNDO_ROW_PURGE is replaced by a simple CLR_END for UNDO_ROW_INSERT,
as promised in the old comment; it gives us skipping during the
UNDO phase.
storage/maria/ma_check.c:
All REDOs before create_rename_lsn are ignored by Recovery. So
create_rename_lsn must be set only after all data/index has been
flushed and forced to disk. We thus move write_log_record_for_repair()
to after _ma_flush_tables_files_after_repair().
storage/maria/ma_checkpoint.c:
Checkpoint module.
storage/maria/ma_checkpoint.h:
optional argument if caller wants a thread to periodically take
checkpoints and flush dirty pages.
storage/maria/ma_create.c:
* no need to init some vars as the initial bzero(share) takes care of this.
* update to new function's name
* even if we fail in my_sync() we have to my_close()
storage/maria/ma_extra.c:
Checkpoint reads share->last_version under intern_lock, so we make
maria_extra() update it under intern_lock. THR_LOCK_maria still needed
because of _ma_test_if_reopen().
storage/maria/ma_init.c:
destroy checkpoint module when Maria shuts down.
storage/maria/ma_loghandler.c:
* UNDO_ROW_PURGE gone (see ma_blockrec.c)
* we need to remember the LSN of the LOGREC_FILE_ID for a share,
because this LSN is needed into the checkpoint record (Recovery wants
to know the validity domain of an id->name mapping)
* translog_get_horizon_no_lock() needed for Checkpoint
* comment about failing assertion (Sanja knows)
* translog_init_reader_data() thought that translog_read_record_header_scan()
returns 0 in case of error, but 0 just means "0-length header".
* translog_assign_id_to_share() now needs the MARIA_HA because
LOGREC_FILE_ID uses a log-write hook.
* Verify that (de)assignment of share->id happens only under intern_lock,
as Checkpoint reads this id with intern_lock.
* translog_purge() can accept TRANSLOG_ADDRESS, not necessarily
a real LSN.
storage/maria/ma_loghandler.h:
prototype updates
storage/maria/ma_open.c:
no need to initialize "res"
storage/maria/ma_pagecache.c:
When taking a checkpoint, we don't need to know the maximum rec_lsn
of dirty pages; this LSN was intended to be used in the two-checkpoint
rule, but last_checkpoint_lsn is as good.
4 bytes for stored_list_size is enough as PAGECACHE::blocks (number
of blocks which the pagecache can contain) is int.
storage/maria/ma_pagecache.h:
new prototype
storage/maria/ma_recovery.c:
* added replaying of REDO_RENAME_TABLE
* UNDO_ROW_PURGE gone (see ma_blockrec.c), replaced by CLR_END
* Recovery from the last checkpoint record now possible
* In new_table() we skip the table if the id->name mapping is older than
create_rename_lsn (mapping dates from lsn_of_file_id).
* in get_MARIA_HA_from_REDO_record() we skip the record
if the id->name mapping is newer than the record (can happen if processing
a record which is before the checkpoint record).
* parse_checkpoint_record() has to return a LSN, that's what caller expects
storage/maria/ma_rename.c:
new function's name; log end zeroes of tables' names (ease recovery)
storage/maria/ma_test2.c:
* equivalent of ma_test1's --test-undo added (named -u here).
* -t=1 now stops right after creating the table, so that
we can test undoing of INSERTs with duplicate keys (which tests the
CLR_END logged by _ma_write_abort_block_record()).
storage/maria/ma_test_recovery.expected:
Result of testing undoing of INSERTs with duplicate keys; there are
some differences in maria_chk -dvv but they are normal (removing
records does not shrink data/index file, does not put back the
"analyzed, optimized keys"(etc) index state.
storage/maria/ma_test_recovery:
Test undoing of INSERTs with duplicate keys, using ma_test2;
when such INSERT happens, it logs REDO_INSERT, UNDO_INSERT, REDO_DELETE,
CLR_END; we abort after that, and test that CLR_END causes recovery
to jump over UNDO_INSERT.
storage/maria/ma_write.c:
comment
storage/maria/maria_chk.c:
comment
storage/maria/maria_def.h:
* a new bit in MARIA_SHARE::in_checkpoint, used to build a list
of unique shares during Checkpoint.
* MARIA_SHARE::lsn_of_file_id added: the LSN of the last LOGREC_FILE_ID
for this share; needed to know to which LSN domain the mappings
found in the Checkpoint record apply (new mappings should not apply
to old REDOs).
storage/maria/trnman.c:
* small changes to how trnman_collect_transactions() fills its buffer;
it also uses a non-dummy lsn_read_non_atomic() found in ma_checkpoint.h
2007-09-12 11:27:34 +02:00
|
|
|
{
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
share->state.create_rename_lsn= lsn;
|
|
|
|
if (share->id != 0)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
If OP is the operation which is calling us, if table is later written,
|
|
|
|
we could see in the log:
|
|
|
|
FILE_ID ... REDO_OP ... REDO_INSERT.
|
|
|
|
(that can happen in real life at least with OP=REPAIR).
|
|
|
|
As FILE_ID will be ignored by Recovery because it is <
|
|
|
|
create_rename_lsn, REDO_INSERT would be ignored too, wrongly.
|
|
|
|
To avoid that, we force a LOGREC_FILE_ID to be logged at next write:
|
|
|
|
*/
|
|
|
|
translog_deassign_id_from_share(share);
|
|
|
|
}
|
WL#3071 Maria checkpoint
Finally this is the real checkpoint code.
It however exhibits unstabilities when a checkpoint runs concurrently
with data-modifying clients (table corruption, transaction log's
assertions) so for now a checkpoint is taken only at startup after
recovery and at shutdown, i.e. not in concurrent situations. Later
we will let it run periodically, as well as flush dirty pages
periodically (almost all needed code is there already, only pagecache
code is written but not committed).
WL#3072 Maria recovery
* replacing UNDO_ROW_PURGE with CLR_END; testing of those CLR_END via
ma_test2 which has INSERTs failing with duplicate keys.
* replaying of REDO_RENAME_TABLE
Now, off to test Recovery in ha_maria :)
BitKeeper/deleted/.del-ma_least_recently_dirtied.c:
Delete: storage/maria/ma_least_recently_dirtied.c
BitKeeper/deleted/.del-ma_least_recently_dirtied.h:
Delete: storage/maria/ma_least_recently_dirtied.h
storage/maria/Makefile.am:
compile Checkpoint module
storage/maria/ha_maria.cc:
When ha_maria starts, do a recovery from last checkpoint.
Take a checkpoint when that recovery has ended and when ha_maria
shuts down cleanly.
storage/maria/ma_blockrec.c:
* even if my_sync() fails we have to my_close() (otherwise we leak
a descriptor)
* UNDO_ROW_PURGE is replaced by a simple CLR_END for UNDO_ROW_INSERT,
as promised in the old comment; it gives us skipping during the
UNDO phase.
storage/maria/ma_check.c:
All REDOs before create_rename_lsn are ignored by Recovery. So
create_rename_lsn must be set only after all data/index has been
flushed and forced to disk. We thus move write_log_record_for_repair()
to after _ma_flush_tables_files_after_repair().
storage/maria/ma_checkpoint.c:
Checkpoint module.
storage/maria/ma_checkpoint.h:
optional argument if caller wants a thread to periodically take
checkpoints and flush dirty pages.
storage/maria/ma_create.c:
* no need to init some vars as the initial bzero(share) takes care of this.
* update to new function's name
* even if we fail in my_sync() we have to my_close()
storage/maria/ma_extra.c:
Checkpoint reads share->last_version under intern_lock, so we make
maria_extra() update it under intern_lock. THR_LOCK_maria still needed
because of _ma_test_if_reopen().
storage/maria/ma_init.c:
destroy checkpoint module when Maria shuts down.
storage/maria/ma_loghandler.c:
* UNDO_ROW_PURGE gone (see ma_blockrec.c)
* we need to remember the LSN of the LOGREC_FILE_ID for a share,
because this LSN is needed into the checkpoint record (Recovery wants
to know the validity domain of an id->name mapping)
* translog_get_horizon_no_lock() needed for Checkpoint
* comment about failing assertion (Sanja knows)
* translog_init_reader_data() thought that translog_read_record_header_scan()
returns 0 in case of error, but 0 just means "0-length header".
* translog_assign_id_to_share() now needs the MARIA_HA because
LOGREC_FILE_ID uses a log-write hook.
* Verify that (de)assignment of share->id happens only under intern_lock,
as Checkpoint reads this id with intern_lock.
* translog_purge() can accept TRANSLOG_ADDRESS, not necessarily
a real LSN.
storage/maria/ma_loghandler.h:
prototype updates
storage/maria/ma_open.c:
no need to initialize "res"
storage/maria/ma_pagecache.c:
When taking a checkpoint, we don't need to know the maximum rec_lsn
of dirty pages; this LSN was intended to be used in the two-checkpoint
rule, but last_checkpoint_lsn is as good.
4 bytes for stored_list_size is enough as PAGECACHE::blocks (number
of blocks which the pagecache can contain) is int.
storage/maria/ma_pagecache.h:
new prototype
storage/maria/ma_recovery.c:
* added replaying of REDO_RENAME_TABLE
* UNDO_ROW_PURGE gone (see ma_blockrec.c), replaced by CLR_END
* Recovery from the last checkpoint record now possible
* In new_table() we skip the table if the id->name mapping is older than
create_rename_lsn (mapping dates from lsn_of_file_id).
* in get_MARIA_HA_from_REDO_record() we skip the record
if the id->name mapping is newer than the record (can happen if processing
a record which is before the checkpoint record).
* parse_checkpoint_record() has to return a LSN, that's what caller expects
storage/maria/ma_rename.c:
new function's name; log end zeroes of tables' names (ease recovery)
storage/maria/ma_test2.c:
* equivalent of ma_test1's --test-undo added (named -u here).
* -t=1 now stops right after creating the table, so that
we can test undoing of INSERTs with duplicate keys (which tests the
CLR_END logged by _ma_write_abort_block_record()).
storage/maria/ma_test_recovery.expected:
Result of testing undoing of INSERTs with duplicate keys; there are
some differences in maria_chk -dvv but they are normal (removing
records does not shrink data/index file, does not put back the
"analyzed, optimized keys"(etc) index state.
storage/maria/ma_test_recovery:
Test undoing of INSERTs with duplicate keys, using ma_test2;
when such INSERT happens, it logs REDO_INSERT, UNDO_INSERT, REDO_DELETE,
CLR_END; we abort after that, and test that CLR_END causes recovery
to jump over UNDO_INSERT.
storage/maria/ma_write.c:
comment
storage/maria/maria_chk.c:
comment
storage/maria/maria_def.h:
* a new bit in MARIA_SHARE::in_checkpoint, used to build a list
of unique shares during Checkpoint.
* MARIA_SHARE::lsn_of_file_id added: the LSN of the last LOGREC_FILE_ID
for this share; needed to know to which LSN domain the mappings
found in the Checkpoint record apply (new mappings should not apply
to old REDOs).
storage/maria/trnman.c:
* small changes to how trnman_collect_transactions() fills its buffer;
it also uses a non-dummy lsn_read_non_atomic() found in ma_checkpoint.h
2007-09-12 11:27:34 +02:00
|
|
|
}
|
WL#3072 - Maria Recovery
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will
truncate files; this is an optimization and a bugfix (table was left
half-repaired by crash).
Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start,
this is easier for recovery (tells it to skip old REDOs or even UNDO
phase) and user (tells it to repair) in case of crash, sync files
in the end.
Recovery skips missing or corrupted table and moves to next record
(in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase.
Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery)
and maria_enable_indexes().
Create missing bitmaps when needed (there can be more than one to create,
in rare cases), log a record for this.
include/myisamchk.h:
new flag: bulk insert repair mustn't bump create_rename_lsn
mysql-test/lib/mtr_report.pl:
skip normal warning in maria-recovery.test
mysql-test/r/maria-recovery.result:
result: crash before bulk insert is committed, causes proper rollback,
and crash right after OPTIMIZE replaces index file with new index file
leads to table marked corrupted and recovery not failing.
mysql-test/t/maria-recovery.test:
- can't check the table or it would commit the transaction,
but check is made after recovery.
- test of crash before bulk-insert-with-repair is committed
(to see if it is rolled back), and of crash after OPTIMIZE has replaced
index file but not finished all operations (to see if recovery fails -
it used to assert when trying to execute an old REDO on the new
index).
storage/maria/CMakeLists.txt:
new file
storage/maria/Makefile.am:
new file
storage/maria/ha_maria.cc:
- If bulk insert on a transactional table using an index repair:
table is initially empty, so don't log REDO/UNDO for data rows
(optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR
which will, if executed, empty the data and index file. Re-enable
logging in end_bulk_insert().
- write log record for repair operation only after it's fully done,
index sort including (maria_repair*() used to write the log record).
- Adding back file->trn=NULL which was removed by mistake earlier.
storage/maria/ha_maria.h:
new member (see ha_maria.cc)
storage/maria/ma_bitmap.c:
Functions to create missing bitmaps:
- one function which creates missing bitmaps in page cache, except
the missing one with max offset which it does not put into page cache
as it will be modified very soon.
- one function which the one above calls, and creates bitmaps in page
cache
- one function to execute REDO_BITMAP_NEW_PAGE which uses the second
one above.
storage/maria/ma_blockrec.c:
- when logging REDO_DELETE_ALL, not only 'records' and 'checksum'
has to be reset under log's mutex.
- execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list
- execution of UNDO_BULK_INSERT_WITH_REPAIR
storage/maria/ma_blockrec.h:
new functions
storage/maria/ma_check.c:
- table-flush-before-repair is moved to a separate function reused
by maria_sort_index(); syncing is added
- maria_repair() is allowed to re-enable logging only if it is the one
which disabled it.
- "_ma_flush_table_files_after_repair" was a bad name, it's not after
repair now, and it should not sync as we do more changes to the files
shortly after (sync is postponed to when writing the log record)
- REDO_REPAIR record should be written only after all repair
operations (in particular after sorting index in ha_mara::repair())
- close to the end of repair by sort, flushing of pages must happen
also in the non-quick case, to prepare for the sync at end.
- in parallel repair, some page flushes are not needed as done
by initialize_variables_for_repair().
storage/maria/ma_create.c:
Update skip_redo_lsn, create_rename_lsn optionally.
storage/maria/ma_delete_all.c:
Need to sync files at end of maria_delete_all_rows(), if transactional.
storage/maria/ma_extra.c:
During repair, we sometimes call _ma_flush_table_files() (via
_ma_flush_table_files_before_swap()) while there is a WRITE_CACHE.
storage/maria/ma_key_recover.c:
- when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable
indices.
- fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT
wrongly, leading to recovery failure
storage/maria/ma_key_recover.h:
new prototype
storage/maria/ma_locking.c:
DBUG_VOID_RETURN missing
storage/maria/ma_loghandler.c:
UNDO for bulk insert with repair, and REDO for creating bitmaps.
LOGREC_FIRST_FREE to not have to change the for() every time we
add a new record type.
storage/maria/ma_loghandler.h:
new UNDO and REDO
storage/maria/ma_open.c:
Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns()
can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK
was not corrected on disk by maria_open().
Store skip_redo_lsn in index' header.
maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty
file has one page, not 0 bytes.
storage/maria/ma_recovery.c:
- Skip a corrupted, missing, or repaired-with-maria_chk, table in
recovery: don't fail, just go to next REDO or UNDO; but if an UNDO
is skipped in UNDO phase we issue warnings.
- Skip REDO|UNDO in REDO phase if <skip_redo_lsn.
- If UNDO phase fails, delete transactions to not make trnman
assert.
- Update skip_redo_lsn when playing REDO_CREATE_TABLE
- Don't record UNDOs for old transactions which we don't know (long_trid==0)
- Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c)
- Execution of UNDO_BULK_INSERT_WITH_REPAIR
- Don't try to find a page number in REDO_DELETE_ALL
- Pieces moved to ma_recovery_util.c
storage/maria/ma_rename.c:
name change
storage/maria/ma_static.c:
I modified layout of the index' header (inserted skip_redo_lsn in its middle)
storage/maria/ma_test2.c:
allow breaking the test towards the end, tests execution of
UNDO_KEY_DELETE_WITH_ROOT
storage/maria/ma_test_recovery.expected:
6 as testflag instead of 4
storage/maria/ma_test_recovery:
Increase the amount of rollback work to do when testing recovery
with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug.
storage/maria/maria_chk.c:
skip_redo_lsn should be updated too, for consistency.
Write a REDO_REPAIR after all operations (including sort-records)
have been done.
No reason to flush blocks after maria_chk_data_link() and
maria_sort_records(), there is maria_close() in the end.
write_log_record() is a function, to not clutter maria_chk().
storage/maria/maria_def.h:
New member skip_redo_lsn in the state, and comments
storage/maria/maria_pack.c:
skip_redo_lsn should be updated too, for consistency
storage/maria/ma_recovery_util.c:
_ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed
by ma_blockrec.c; this causes link issues, resolved by putting
_ma_redo_not_needed_for_page() into a new file (so that it is not
in the same file as repair-related objects of ma_recovery.c).
storage/maria/ma_recovery_util.h:
new file
2008-01-17 23:59:32 +01:00
|
|
|
else
|
|
|
|
lsn_store(buf, share->state.create_rename_lsn);
|
Added versioning of Maria index
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid
Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it
.bzrignore:
Added missing temporary directory
extra/Makefile.am:
comp_err is now deleted on make distclean
include/maria.h:
Added structure MARIA_KEY, which is used for intern key objects in Maria.
Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
Changed some functions that always return true or false to my_bool.
Added virtual function make_key() to avoid if in _ma_make_key()
Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
Added SEARCH_INSERT to be used when inserting new keys
Added SEARCH_PART_KEY for partial searches
Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
Read and restore current database; Don't assume we are using mysqltest.
Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
New tests
mysql-test/r/maria-purge.result:
New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
New tests
mysql-test/r/variables-big.result:
Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
New tests
mysql-test/t/maria-purge.test:
Remove logs to make test results predictable
mysql-test/t/maria.test:
New tests for some possible problems
mysql-test/t/variables-big.test:
Don't log id as it's not predictable
mysys/my_handler.c:
Updated function comment to reflect old code
Changed nextflag to be uint32 to ensure we can have flags > 16 bit
Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
Use 'file->start_state' as default state for transactional tables without versioning
At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
Updated comment
storage/maria/ma_check.c:
Compact key pages (removes transid) when doing --zerofill
Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
Removed not needed argument from get_record_for_key()
Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
Use info->lastkey_buff instead of info->lastkey
_ma_dpos() -> _ma_row_pos_from_key()
_ma_make_key() -> keyinfo->make_key()
_ma_print_key() -> _ma_print_keydata()
_ma_move_key() -> ma_copy_copy()
Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Ensure that data on page doesn't overwrite page checksum position
Use DBUG_DUMP_KEY instead of DBUG_DUMP
Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
Fixed check if rowid points outside of BLOCK_RECORD data file
Use info->lastkey_buff instead of key on stack in some safe places
Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
Removed assert that is already checked in maria_init()
Force transactinal tables to be of type BLOCK_RECORD
Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
info->lastkey2-> info->lastkey_buff2
Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
Use new interface for get_key(), _ma_get_last_key() and others
_ma_dpos() -> ma_row_pos_from_key()
Simplify setting of prev_key in del()
Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
Reset history state if maria_delete_all_rows() are called
Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
Check if blocksize is legal
(Moved test here from ma_open())
storage/maria/ma_key.c:
Added functions for storing/reading of transid
Modified functions to use MARIA_KEY instead of key pointer and key length
Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
Add transid to keys if table is versioned
Added _ma_copy_key()
storage/maria/ma_key_recover.c:
Add logging of page_flag (holds information if there are keys with transid on page)
Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
Added lots of DBUG_PRINT()
Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
Fixed all logging of LSN to look similar in DBUG log
Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
Added new index page operations
storage/maria/ma_open.c:
Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
Changed info->lastkey to MARIA_KEY type
Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Check on open that state.create_trid is correct
Extend share->base.max_key_length in case of transactional table so that it can hold transid
Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
Initialize all new virtual function pointers
Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
Added comment to describe key page structure
Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
Added handling of pages with transid
storage/maria/ma_recovery.c:
Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Don't use USE_WHOLE_KEY, use real length of key
Use share->row_is_visible() to test if row is visible
Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
Updated comment
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Use better variable names
Removed not needed casts
_ma_dpos() -> _ma_row_pos_from_key()
Use info->last_rtree_keypos to save position to key instead of info->int_keypos
Simplify err: condition
Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
Changed type of 'nextflag' to uint32
Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
Fixed wrong comment
Reset recinfo to avoid valgrind varnings
Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added support of keys with optional trid
Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
_ma_dpos() -> _ma_row_pos_from_key()
If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
Combine similar code at end of _ma_get_binary_pack_key()
Removed not used function _ma_move_key()
In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
Update info->cur_row.trid with trid for found key
Removed some not needed casts
Added _ma_trid_from_key()
Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
_ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
Don't test sizeof(double), test against 8 as we are using float8store()
Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
maria_versioning() now only calls _ma_block_get_status() if table supports versioning
Added _ma_row_visible_xxx() functions for different occasions
When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
Indentation changes
storage/maria/ma_statrec.c:
Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
Modified functions to use MARIA_KEY instead of key pointer and key length
Simplify code by using local variable keyinfo instead if share->keyinfo[i]
In _ma_enlarge_root(), mark in page_flag if new key has transid
_ma_dpos() -> _ma_row_pos_from_key()
Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
Added error if trying to sort with HA_BINARY_PACK_KEY
Use new interface to get_key() and _ma_dpointer()
_ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
Modified functions to use MARIA_KEY instead of key pointer and key length
Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
Added create_trid to MARIA_SHARE
Removed not used state variables 'process' and 'unique'
Added defines for handling TRID's in index pages
Changed to use MARIA_SHARE instead of MARIA_HA for some functions
Added 'have_versioning' flag if table supports versioning
Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
Changed last_key to be of type MARIA_KEY. Removed lastkey_length
lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
lastkey -> lastkey_buff
storage/maria/trnman.c:
Fixed wrong initialization of min_read_from and max_commit_trid
Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
Added --start-from
storage/myisam/mi_check.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
Updated comment
storage/myisam/mi_write.c:
Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
Fixed wrong parameter to rtree_get_req() which could cause crash
2008-06-26 07:18:28 +02:00
|
|
|
return (my_pwrite(file, buf, sizeof(buf),
|
|
|
|
sizeof(share->state.header) +
|
|
|
|
MARIA_FILE_CREATE_RENAME_LSN_OFFSET, MYF(MY_NABP)) ||
|
|
|
|
my_pwrite(file, trid_buff, sizeof(trid_buff),
|
|
|
|
sizeof(share->state.header) +
|
|
|
|
MARIA_FILE_CREATE_TRID_OFFSET, MYF(MY_NABP)) ||
|
2011-07-13 21:10:18 +02:00
|
|
|
(do_sync && mysql_file_sync(file, MYF(0))));
|
2007-06-27 12:58:08 +02:00
|
|
|
}
|
2018-04-30 14:21:52 +02:00
|
|
|
#if defined(_MSC_VER) && (_MSC_VER == 1310)
|
2008-01-10 19:07:17 +01:00
|
|
|
#pragma optimize("",on)
|
|
|
|
#endif /*VS2003 compiler bug workaround*/
|