Commit graph

48370 commits

Author SHA1 Message Date
unknown
711a820133 Hackish fix for at-shutdown assertions in
./mtr --mysqld=--default-storage-engine=maria --mem --skip-ndb temp_table truncate --force
The root cause of the problem has been reported as BUG#30309
"mysql_truncate() does not inform engine that the recreated table is temporary"
The temporary fix used here is that when mysql_truncate() identifies
a temp table of engine "Maria" it declares it non-transactional,
thus the table is re-created by ha_maria::create() as non-transactional.


sql/mysqld.cc:
  porting fix of BUG#29133 (mysqld takes too much time to shutdown
  without this fix, I can't wait for the next merge)
sql/sql_delete.cc:
  a hack to work around BUG#30309 "mysql_truncate() does not inform
  engine that the recreated table is temporary"
2007-08-08 17:07:12 +02:00
unknown
62b30a1922 propagation to maria_repair_by_sort() and maria_repair_parallel()
of bugfix made to maria_repair() yesterday.
Fail "bk delta" (and thus "bk citool") if an added or modified line
of a C/C++ file has white space at end of line


BitKeeper/triggers/pre-delta:
  detection gave false alarm on added newline
storage/maria/ma_check.c:
  propagation to maria_repair_by_sort() and maria_repair_parallel()
  of bugfix made to maria_repair() yesterday. No effect now as
  those two repair variants are never used with BLOCK_RECORD.
2007-08-08 11:36:41 +02:00
unknown
1ad3a05dd7 Fix for errors during:
"./mtr --mysqld=--default-storage-engine=maria mysqldump".
First problem was use of INSERT DELAYED and MERGE tables without
specifying that the tables to create should always be MyISAM.
After fixing this, no rows were returned by the final SELECT of the
"BUG 19025" portion of the test. Simplified problem was:
LOCK TABLES `t1` WRITE;
/*!40000 ALTER TABLE `t1` DISABLE KEYS */;
INSERT INTO `t1` VALUES ('bla',1000),('bla',1001),('bla',1002);
/*!40000 ALTER TABLE `t1` ENABLE KEYS */;
UNLOCK TABLES;
select * from t1;
The SELECT would find no rows. Reason: ENABLE KEYS does a maria_repair();
but data pages are still in the page cache and not on disk (because
they were not flushed because maria_lock_database(F_UNLCK) was
not called at the end of INSERT because under LOCK TABLES).
At start of maria_repair(), sort_info.filelength is set to the
physical size of the data file (=> too small because pages are in
cache and not on disk).
Then in sort_get_next_record(), when seeing end-of-file, this is done:
sort_param->max_pos= sort_info->filelength;
Further in maria_repair(), this is done:
info->state->data_file_length= sort_param.max_pos;
and so data_file_length is smaller (0) than reality (16384).
This makes SELECT think EOF is where it is not, and thus find
no rows.
This is fixed by flushing all data pages at the start of maria_repair()
(no performance problem is introduced as in common cases where
ALTER TABLE is not under LOCK TABLES, the previous statement did
this flush anyway).
Another reason to do this flush is that, if not doing it, old cached
pages might go down onto the repaired data file at a later point
and thus corrupt it (assume a REPAIR non-QUICK).
A similar bug is fixed:
LOCK TABLES WRITE; INSERT; CHECK TABLE;
reports "Size of datafile is: 0         Should be: 16384"
again because the physical size was read without a preliminary
page cache flush.


mysql-test/r/maria.result:
  result update
mysql-test/r/mysqldump.result:
  result update
mysql-test/t/maria.test:
  adding test for fixed bug in LOCK TABLES + CHECK TABLE + block format.
  Disabling portion which hits "incorrect key file" but still
  letting it make the test fail (Monty to fix).
mysql-test/t/mysqldump.test:
  in places where test expects engine to support INSERT DELAYED and
  be includable in a MERGE table, i.e. be MyISAM, we explicitely
  ask for MyISAM.
storage/maria/ma_check.c:
  Before reading the data file's physical size with my_seek(MY_SEEK_END)
  during maria_chk_size() and maria_repair(), we must flush this
  data file, otherwise physical size is misleading and leads to
  - CHECK TABLE finding the table corrupted ("size of datafile should be"
  error)
  - ALTER TABLE ENABLE KEYS losing rows (maria_repair()
  setting data_file_length to a too small value => later SELECT does
  not find rows though they are in the data file).
  This fixes the "mysqldump.test" failure.
  sort_info.filelength contains the physical size, re-using it.
2007-08-07 18:23:49 +02:00
unknown
72c3c369e4 Fix for three bugs:
number 1: "./mtr --mysqld=--default-storage-engine=maria backup"
restored no rows (forgot to flush data pages before my_copy(),
and also the maria_repair() used by ha_maria::restore() needed
a correct data_file_length to not miss rows). [note that BACKUP
TABLE will be removed anyway in 5.2]
number 2: "./mtr --mysqld=--default-storage-engine=maria bootstrap"
caused segfault (uninitialized variable)
number 3: "./mtr --mysqld=--default-storage-engine=maria check"
showed warning in CHECK TABLE (maria_create() created a non-empty
data file with data_file_length==0).


storage/maria/ha_maria.cc:
  in ha_maria::backup, need to flush the data file before copying it,
  otherwise data misses from the copy (bug 1)
storage/maria/ma_bitmap.c:
  when allocating data at the end of the bitmap, best_data is at "end",
  should not be left to 0 (bug 2)
storage/maria/ma_check.c:
  _ma_scan_block_record() is used in QUICK repair. It relies on
  data_file_length. RESTORE TABLE mixes the MAI of an empty table
  (so, data_file_length==0) with an non-empty MAD, and does a 
  QUICK repair; that got fooled (thought it had hit EOF immediately,
  so found no records) (bug 1)
storage/maria/ma_create.c:
  At the end of maria_create() we have, in the index file,
  data_file_length==0, while the data file has a bitmap page (8192).
  This inconsistency makes CHECK TABLE rightly complain.
  Fixed by not creating a first bitmap page during maria_create()
  (also saves disk space) (bug 3) Question for Monty.
storage/maria/ma_extra.c:
  A function to flush the data and index files before one can
  use OS syscalls (reads, writes) on those files. For example,
  ha_maria::backup() does a my_copy() of the data file and so
  all cached pieces of this file must be sent to the OS (bug 1)
  This function will have to be used elsewhere in Maria, several places
  have not been updated when we added pagecache-ing of the data file
  (they still only flush the index file), they are probable bugs.
storage/maria/maria_def.h:
  new function. Needs to be visible from ha_maria::backup.
2007-08-07 16:06:42 +02:00
unknown
1b47e3dabd Make test does not fail if run with
--mysqld=--default-storage-engine=X where X does not support
INSERT DELAYED (like Maria)


mysql-test/r/alter_table.result:
  result update
mysql-test/t/alter_table.test:
  so that test does not fail if run with
  --mysqld=--default-storage-engine=X where X does not support
  INSERT DELAYED (like Maria)
2007-08-06 20:17:38 +02:00
unknown
ec547ae830 fixes of bad merge (probably). Less duplication in ma_test_recovery.
storage/maria/ma_range.c:
  old code, which is wrong now (key_len is now a local variable,
  not initialized). Fixes some problems in running ma_test2.
storage/maria/ma_test2.c:
  keypart_map should be used instead of length (fix of bad merge
  probably)
storage/maria/ma_test_recovery:
  less duplication in this script (one loop instead).
2007-08-06 16:13:42 +02:00
unknown
3d0f42a94c Log handler's unit test were broken by me on June 22, fixing this:
don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests,
as they read their log and expect to find only the records they asked
for, and so not LOGREC_LONG_TRANSACTION_ID. All log handler's unit
tests pass. A way to run them faster: when their LONG_BUFFER_SIZE is
too big (1GB) I divide it by ten and then I can run them on /dev/shm
- they are then "instant".
By the way, pushbuild was not displaying anymore "unit: failed" in
the list of all pushes, which participates to the reasons why I didn't
notice the breakage earlier. The other reason being that I was too lazy
to run log handler unit tests on my machine as they took long (hadn't
yet thought about the /dev/shm idea) and so I relied on pushbuild;
Danny has now quickly fixed pushbuild - thanks :)


storage/maria/unittest/ma_test_loghandler-t.c:
  don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests,
  as they read their log and expect to find only the records they asked
  for
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests,
  as they read their log and expect to find only the records they asked
  for
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests,
  as they read their log and expect to find only the records they asked
  for
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests,
  as they read their log and expect to find only the records they asked
  for
2007-08-03 15:58:50 +02:00
unknown
91f03a4d31 WL#3072 Maria checkpoint
Preparation: for the two-checkpoint rule we will flush all pages
which have block->rec_lsn <= last_checkpoint_lsn. Pages with
rec_lsn not yet set (i.e. ==0) were dirtied very recently and so should
not be flushed (otherwise it's not efficient: we should try to let
a page be modified several times in memory before we flush it to disk).
To make this easy, "block->rec_lsn not yet set" is now expressed
with block->rec_lsn==LSN_MAX, not block->rec_lsn==0 anymore.
It is easier this way because LSN_MAX>last_checkpoint_lsn whereas
0<=last_checkpoint_lsn.


storage/maria/ma_blockrec.c:
  typo
storage/maria/ma_loghandler.c:
  typo
storage/maria/ma_loghandler_lsn.h:
  LSN_MAX
storage/maria/ma_pagecache.c:
  "block->rec_lsn not yet set" is now expressed by block->rec_lsn==LSN_MAX,
  not block_rec_lsn==0.
2007-08-02 12:42:49 +02:00
unknown
372b4ed49a WL#3072 Maria recovery:
preparation to enable recoverability of the table's state, more exactly
info->state->checksum. This will require that info->state->checksum
be updated in a inwrite_hook when writing an UNDO record, thus
info->cur_row.checksum needs to be accessible to inwrite_hook, so we
make translog_write_record() accept a MARIA_HA* (info) instead of
MARIA_SHARE* (info->s); with this, we will be able to access
info->cur_row.checksum. Old code which needed the MARIA_SHARE
can derive it from MARIA_HA. Fix for typos and compiler warnings.


storage/maria/ma_blockrec.c:
  fix for new loghandler API. Removing strange lines (how could gcc
  accept that?)
storage/maria/ma_check.c:
  fix for new loghandler API
storage/maria/ma_delete_all.c:
  fix for new loghandler API
storage/maria/ma_loghandler.c:
  functions now take a MARIA_HA in argument, this is more powerful
  than a MARIA_SHARE (MARIA_SHARE can be derived from MARIA_HA, not
  the other way around). MARIA_HA will be needed to allow recoverability
  of the table's state.
  Fixing wrong DBUG_PRINT ('i' is not the id).
  When writing the LOGREC_FILE_ID, we don't have a MARIA_HA around,
  so we cannot ask translog_write_record() to store the id for us;
  we thus store the file's id by ourselves. Alternative would have been
  to pass MARIA_HA to translog_assign_id_to_share() but I didn't like it.
storage/maria/ma_loghandler.h:
  new loghandler API
storage/maria/tablockman.c:
  fix for compiler warning (intptr is int on my machine)
2007-08-01 15:52:57 +02:00
unknown
8dcd83d0da testcase for a bug not yet fixed. Once fixed, the test should fail
in EXPLAIN's output.


mysql-test/t/maria.test:
  adding a test for "incorrect key file" error in UPDATE with
  row_format=dynamic (does not happen with row_format=page)
2007-07-31 19:05:19 +02:00
unknown
562923e85f updating result file of ps_maria, it is now identical to
the result of ps_2myisam.result. Thanks Serg for the help.


mysql-test/r/ps_maria.result:
  Result file was probably out-of-date; it thus becomes identical to
  the result of ps_2myisam.result
2007-07-31 19:01:23 +02:00
unknown
2cccfcd8dd Applying Sanja's patch which makes the log handler not issue
errors when reading a log record which has a 0-length header
(like LOGREC_REDO_DROP_TABLE).


storage/maria/ma_loghandler.c:
  Functions reading record's header now don't use 0 to indicate error,
  as some valid records have a 0-length header (like REDO_DROP_TABLE).
  Instead, negative values are used for EOF and error.
storage/maria/ma_loghandler.h:
  functions to read record's header now return an int
  (either the length of this header (>=0) or some negative values
  for EOF or error).
storage/maria/ma_recovery.c:
  update to the new log handler's behaviour. Note the @todo.
storage/maria/maria_read_log.c:
  inform when program failed
storage/maria/unittest/ma_test_loghandler-t.c:
  update to new log handler's API
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  update to new log handler's API
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  update to new log handler's API
2007-07-30 15:05:43 +02:00
unknown
9554b40df2 Maria: it is allowed to change unknown type to any legal type of page
in the pagecache: fixing wrong assertion and a test case.
maria.test and ps_maria.test still fail.


mysql-test/r/maria.result:
  result update
mysql-test/t/maria.test:
  added test for incorrect assert on page's type
storage/maria/ma_pagecache.c:
  It is allowed to change unknown type to any legal type of page in the
  pagecache.
2007-07-30 12:01:52 +02:00
unknown
9b1e83dba7 porting Serg's fix for BUG#30094 to Maria. Now ma_test_all passes.
maria.test and ps_maria.test still fail.


mysys/mf_keycache.c:
  split string annoys some compilers
storage/maria/ha_maria.cc:
  fix for compiler warnings
storage/maria/ma_test1.c:
  porting Serg's fix for BUG#30094 to Maria
storage/maria/ma_test2.c:
  porting Serg's fix for BUG#30094 to Maria
storage/maria/ma_test3.c:
  porting Serg's fix for BUG#30094 to Maria
storage/maria/ma_test_recovery:
  don't print ma_test1's messages if no problem
2007-07-27 16:11:40 +02:00
unknown
a3d2ae4648 merging MyISAM changes into Maria (not done in 5.1->maria merge of
Jul 7th). "maria.test" and "ps_maria.test" still fail;
"ma_test_all" starts failing (MyISAM has the same issue see BUG#30094).


include/maria.h:
  merging MyISAM changes into Maria
mysys/mf_keycache.c:
  mi_test_all showed "floating point exception", this was already
  fixed in the latest 5.1, importing fix.
sql/item_xmlfunc.cc:
  compiler warning (already fixed in latest 5.1)
storage/maria/ha_maria.cc:
  merging MyISAM changes into Maria. See #ifdef ASK_MONTY.
storage/maria/ha_maria.h:
  merging MyISAM changes into Maria
storage/maria/ma_cache.c:
  merging MyISAM changes into Maria
storage/maria/ma_check.c:
  merging MyISAM changes into Maria
storage/maria/ma_create.c:
  merging MyISAM changes into Maria
storage/maria/ma_dynrec.c:
  merging MyISAM changes into Maria
storage/maria/ma_extra.c:
  merging MyISAM changes into Maria
storage/maria/ma_ft_boolean_search.c:
  merging MyISAM changes into Maria
storage/maria/ma_ft_nlq_search.c:
  merging MyISAM changes into Maria
storage/maria/ma_info.c:
  merging MyISAM changes into Maria
storage/maria/ma_key.c:
  merging MyISAM changes into Maria
storage/maria/ma_loghandler.c:
  compiler warning (part->length is size_t)
storage/maria/ma_open.c:
  merging MyISAM changes into Maria
storage/maria/ma_preload.c:
  merging MyISAM changes into Maria
storage/maria/ma_range.c:
  merging MyISAM changes into Maria
storage/maria/ma_rkey.c:
  merging MyISAM changes into Maria
storage/maria/ma_rt_index.c:
  merging MyISAM changes into Maria
storage/maria/ma_rt_key.c:
  merging MyISAM changes into Maria
storage/maria/ma_rt_split.c:
  merging MyISAM changes into Maria
storage/maria/ma_search.c:
  merging MyISAM changes into Maria
storage/maria/ma_sort.c:
  merging MyISAM changes into Maria
storage/maria/maria_def.h:
  merging MyISAM changes into Maria
2007-07-27 12:06:39 +02:00
unknown
662002fc8f post-merge fixes, and fixes for some of the 16 compiler warnings
found in pushbuild on sapsrv1. Some not fixed as not repeatable
on my machine (32/64 bit issue?).
Fixes for some test failures:
- "maria-connect" now passes;
- "maria": after fixing the obvious reasons for failures, the test
went further and hit a more complex issues: difference in the output
of EXPLAIN output; not fixed;
- "ps_maria" still crashes in assertion
mysqld: ha_maria.cc:1627: virtual int ha_maria::index_read(uchar*, const uchar*, uint, ha_rkey_function): Ass
ertion `inited == INDEX' failed, as already observable in pushbuild.
All this might just be due to an incomplete merge of MyISAM changes
into Maria when 5.1 was last merged to mysql-maria.


include/my_global.h:
  temporary fix until next merge of 5.1; without this it does not build
mysql-test/r/maria-connect.result:
  position changed
mysql-test/t/maria-connect.test:
  If one wants to use the binlog it has to ask for it.
  1582 is not used for dup entry error anymore (it was in older 5.1).
  Size of first event in binlog was increased by 4 (when the new type
  of event "gap" was added).
mysql-test/t/maria.test:
  1582 not used anymore in this case
storage/maria/ha_maria.cc:
  engine now has to say what binlogging it supports
storage/maria/ma_blockrec.c:
  fix for compiler warnings ("comparison is always true" or "always
  false")
storage/maria/ma_loghandler.c:
  fix for compiler warnings (comparing char* to uchar*)
storage/maria/ma_packrec.c:
  fix for compiler warning (fix simply merged from MyISAM)
storage/maria/ma_pagecache.c:
  info_check_pin() was not used so gave a compiler warning.
storage/maria/ma_pagecache.h:
  fixing typo from the last 5.1->maria merge.
storage/maria/ma_recovery.c:
  my_free() has a void* argument, so why cast. byte->uchar.
storage/maria/ma_search.c:
  fix for compiler warning (fix simply merged from MyISAM)
storage/maria/maria_read_log.c:
  gptr->uchar*
storage/maria/trnman.c:
  probable fix for warning found in pushbuild (but not on my machine):
  storage/maria/trnman.c: 142
   passing argument 6 of \u2018lf_hash_init\u2019 from incompatible pointer type
  on sapsrv1.
2007-07-26 17:51:49 +02:00
unknown
79f7cb4e95 merge
sql/handler.cc:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ma_blockrec.c:
  Auto merged
storage/maria/ma_close.c:
  Auto merged
storage/maria/ma_create.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/ma_loghandler.h:
  Auto merged
storage/maria/ma_test2.c:
  Auto merged
storage/maria/maria_read_log.c:
  will merge by hand
2007-07-26 12:15:47 +02:00
unknown
6bbca54d7d WL#3072 - Maria recovery
Unit test for recovery: runs ma_test1 and ma_test2 (both only with
INSERTs and DELETEs; UPDATEs disabled as not handled by recovery)
then moves the tables elswhere; recreates tables from the log, and
compares and fails if there is a difference. Passes now.
Most of maria_read_log.c moved to ma_recovery.c, as it will be re-used
for recovery-from-ha_maria.
Bugfixes of applying of REDO_INSERT, REDO_PURGE_ROW.
Applying of REDO_PURGE_BLOCKS, REDO_DELETE_ALL, REDO_DROP_TABLE,
UNDO_ROW_INSERT (in REDO phase only, i.e. just doing records++),
UNDO_ROW_DELETE, UNDO_ROW_PURGE.
Code cleanups.
Monty: please look for "QQ". Sanja: please look for "Sanja".
Future tasks: recovery of the bitmap (easy), recovery of the state
(make it idempotent), more REDOs (Monty to work on
REDO_UPDATE?), UNDO phase...
Pushing this cset as it looks safe, contains test and bugfixes which
will help Monty implement applying of REDO_UPDATE.


sql/handler.cc:
  typo
storage/maria/Makefile.am:
  Adding ma_test_recovery (which ma_test_all invokes, and which can
  also be run alone). Most of maria_read_log.c moved to ma_recovery.c
storage/maria/ha_maria.cc:
  comments
storage/maria/ma_bitmap.c:
  fixing comments. 2 -> sizeof(maria_bitmap_marker).
  Bitmap-related part of _ma_initialize_datafile() moves in bitmap module.
  Now putting the "bm" signature when creating the first bitmap page
  (it used to happen only at next open, but that
  caused an annoying difference when testing Recovery if the original
  run didn't open the table, and it looks more
  logical like this: it goes to disk only with its signature correct);
  see the "QQ" comment towards the _ma_initialize_data_file() call
  in ma_create.c for more).
  When reading a bitmap page, verify its signature (happens when normally
  using the table or when CHECKing it; not when REPAIRing it).
storage/maria/ma_blockrec.c:
  * no need to sync the data file if table is not transactional
  * Comments, code cleanup (log-related data moved to log-related code
  block, int5store->page_store).
  * Store the table's short id into LOGREC_UNDO_ROW_PURGE, like we
  do for other records (though this record will soon be replaced
  with a CLR).
  * If "page" is 1 it means the page which extends from byte
  page*block_size+1 to (page+1)*block_size (byte number 1 being
  the first byte of the file). The last byte of the file is
  data_file_length (same convention).
  A new page needs to be created if the last byte of the page is
  beyond the last byte of the file, i.e.
   (page+1)*block_size+1 > data_file_length, so we correct the test
  (bug found when testing log applying for ma_test1 -M -T --skip-update).
  * update the page's LSN when removing a row from it during
  execution of a REDO_PURGE_ROW record (bug found when testing log
  applying for ma_test1 -M -T --skip-update).
  * applying of REDO_PURGE_BLOCKs (limited to a one-page range for now).
storage/maria/ma_blockrec.h:
  new functions. maria_bitmap_marker does not need to be exported.
storage/maria/ma_close.c:
  we can always flush the table's state when closing the last instance
  of the table. And it is needed for maria_read_log (as it does
  not use maria_lock_database()).
storage/maria/ma_control_file.c:
  when in Recovery, some assertions should not be used.
storage/maria/ma_control_file.h:
  double-inclusion safe
storage/maria/ma_create.c:
  during recovery, don't log records. Comments.
  Moving the creation of the first bitmap page to ma_bitmap.c
storage/maria/ma_delete_table.c:
  during recovery, don't log records. Log the end-zero of the dropped
  table's name, so that recovery can use the string in place without
  extending it to fit an end zero.
storage/maria/ma_loghandler.c:
  * inwrite_rec_hook also needs access to the MARIA_SHARE, like
  prewrite_rec_hook. This will be needed to update
  share->records_diff (in the upcoming patch "recovery of the state").
  * LOG_DESC::record_ends_group changed to an enum.
  * LOG_DESC for LOGREC_REDO_PURGE_BLOCKS and LOGREC_UNDO_ROW_PURGE
  corrected
  * Sanja please see the @todo LOG BUG
  * avoiding DBUG_RETURN(func()) as it gives confusing debug traces.
storage/maria/ma_loghandler.h:
  - log write hooks called while the log's lock is held (inwrite_rec_hook)
  now need the MARIA_SHARE, like prewrite_rec_hook already had
  - instead of a bool saying if this record's type ends groups or not,
  we refine: it may not end a group, it may end a group, or it may
  be a group in itself. Imagine that we had a physical write failure
  to a table before we log the UNDO, we still end up in
  external_lock(F_UNLCK) and then we log a COMMIT: we don't want
  to consider this COMMIT as ending the group of REDOs (don't want
  to execute those REDOs during Recovery), that's why we say "COMMIT
  is a group in itself, it aborts any previous group". This also
  gives one more sanity check in maria_read_log.
storage/maria/ma_recovery.c:
  New Recovery code, replacing the old pseudocode.
  Most of maria_read_log moved here.
  Call-able from ha_maria, but not enabled yet.
  Compared to the previous version of maria_read_log, some bugs have
  been fixed, debugging output can go to stdout or a disk file (for now
  it's useful for me, later it can be changed), execution of
  REDO_DROP_TABLE, REDO_DELETE_ALL, REDO_PURGE_BLOCKS has been added. Duplicate code
  has been factored into functions. We abort an unfinished group
  of records if we see a record which is a group in itself (like COMMIT).
  No need for maria_panic() after a bug (which caused tables to not
  be closed) was fixed; if there is yet another bug I prefer to see it.
  When opening a table for Recovery, set data_file_length
  and key_file_length to their real physical value (these are the
  easiest state members to restore :). Warn us if the last page
  was truncated (but Recovery handles it).
  MARIA_SHARE::state::state::records is now partly recovered (not
  idempotent, but works if recreating tables from scracth).
  When applying a REDO to a page, stamp it with the UNDO's LSN
  (current_group_end_lsn), not with the REDO's LSN; it makes
  the table more identical to the original table (easier to compare
  the two tables in the end).
  Big thing missing: some types of REDOs are not handled,
  and the UNDO phase does not exist (missing functions to execute UNDOs
  to actually rollback). So for now tests are only inserting/deleting
  a few 100 rows, closing the table and seeing if the log is applied ok;
  it works. UPDATE not handled.
storage/maria/ma_recovery.h:
  new functions: ma_recover() for recovery from inside ha_maria;
  _ma_apply_log() for maria_read_log (ma_recover() calls _ma_apply_log()).
  Btw, we need to not use the word "recover" for REPAIR/maria_chk anymore.
storage/maria/ma_rename.c:
  don't write log records during recovery
storage/maria/ma_test2.c:
  - fail if maria_info() or other subtests find some wrong information
  - new option -g to skip updates.
  - init the translog before creating the table, so that log applying
  can work.
  - in "#if 0" you'll see some fixed bugs (will be removed).
storage/maria/ma_test_all.sh:
  cleanup files. Test log applying.
storage/maria/maria_read_log.c:
  most of the logic moves to ma_recovery.c to be shared between
  maria_read_log and recovery-from-inside-mysqld.
  See ma_recovery.c for additional changes made to the moved code.
storage/maria/ma_test_recovery:
  unit test for Recovery. Tests insert and delete,
  REDO_UPDATE not yet coded.
  Script is called from ma_test_all. Can run standalone.
2007-07-26 11:56:21 +02:00
unknown
85510db991 Merged from main 5.1 2007-07-07 23:17:35 +03:00
unknown
81e32ad00b Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-5.1
into  labbari.dsl.inet.fi:/home/my/bk/mysql-maria.prod


configure.in:
  Auto merged
include/my_base.h:
  Auto merged
mysql-test/lib/mtr_report.pl:
  Auto merged
mysql-test/t/disabled.def:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/share/errmsg.txt:
  Auto merged
storage/csv/ha_tina.cc:
  Auto merged
storage/myisam/ft_parser.c:
  Auto merged
2007-07-07 22:13:36 +03:00
unknown
8c35eec8af Merge acurtis@bk-internal.mysql.com:/home/bk/mysql-5.1-engines
into  anubis.xiphis.org:/usr/home/antony/work/mysql-5.1-engines.merge
2007-07-07 09:45:43 -07:00
unknown
fe00f78d80 disable federated_innodb test until bug29522 is fixed 2007-07-07 09:28:35 -07:00
unknown
ebd53ca2e6 Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-5.1
into  labbari.dsl.inet.fi:/home/my/bk/mysql-maria.prod


BitKeeper/etc/ignore:
  auto-union
client/mysqldump.c:
  Auto merged
include/my_base.h:
  Auto merged
include/my_sys.h:
  Auto merged
mysql-test/lib/mtr_cases.pl:
  Auto merged
mysql-test/lib/mtr_report.pl:
  Auto merged
mysql-test/r/ps_2myisam.result:
  Auto merged
mysql-test/r/ps_3innodb.result:
  Auto merged
mysql-test/r/ps_4heap.result:
  Auto merged
mysql-test/r/ps_5merge.result:
  Auto merged
mysql-test/r/ps_7ndb.result:
  Auto merged
mysql-test/r/view.result:
  Auto merged
mysql-test/t/disabled.def:
  Auto merged
mysql-test/t/view.test:
  Auto merged
sql/Makefile.am:
  Auto merged
sql/ha_ndbcluster.cc:
  Auto merged
sql/handler.cc:
  Auto merged
sql/handler.h:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_show.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
sql/share/errmsg.txt:
  Auto merged
sql/table.cc:
  Auto merged
sql/table.h:
  Auto merged
storage/myisam/mi_key.c:
  Auto merged
storage/myisam/mi_open.c:
  Merged with main 5.1.
2007-07-07 18:33:43 +03:00
unknown
1ea806b174 Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-maria
into  labbari.dsl.inet.fi:/home/my/bk/mysql-maria.prod


mysys/thr_lock.c:
  Auto merged
storage/csv/ha_tina.cc:
  Auto merged
storage/csv/ha_tina.h:
  Auto merged
storage/maria/ma_blockrec.c:
  Auto merged
storage/maria/ma_check.c:
  Auto merged
storage/maria/ma_loghandler.c:
  Auto merged
storage/maria/trnman.c:
  Auto merged
storage/myisam/mi_locking.c:
  Auto merged
sql/lock.cc:
  SCCS merged
2007-07-07 18:03:07 +03:00
unknown
efbcbad4df Merge ymer.(none):/usr/local/mysql/mysql-5.0-engines
into  ymer.(none):/usr/local/mysql/mysql-5.1-engines


sql/stacktrace.c:
  Auto merged
BUILD/compile-pentium-gcov:
  Manual merge.
2007-07-07 07:48:43 +02:00
unknown
87e1a302c3 Bug #29543 GCov information not written in case of crash.
For GCov builds, if the server crashes, the normal exit handler for writing
coverage information is not executed due to the abnormal termination.

Fix this by explicitly calling the __gcov_flush function in our crash handler.
2007-07-07 07:46:17 +02:00
unknown
107d13e7c7 Merge ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.0-engines.merge
into  ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.1-engines.merge
2007-07-06 14:52:18 -07:00
unknown
84b9715d18 Merge acurtis@bk-internal.mysql.com:/home/bk/mysql-5.1
into  ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.1-engines.merge


libmysql/libmysql.c:
  Auto merged
2007-07-06 14:33:14 -07:00
unknown
b1dce946cc Merge acurtis@bk-internal.mysql.com:/home/bk/mysql-5.0
into  ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.0-engines.merge
2007-07-06 14:29:58 -07:00
unknown
6096d90ec1 Merge ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.0-engines.merge
into  ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.1-engines.merge


mysql-test/t/fulltext2.test:
  Auto merged
BitKeeper/deleted/.del-fulltext3.result:
  Delete: mysql-test/r/fulltext3.result
BitKeeper/deleted/.del-fulltext3.test:
  Delete: mysql-test/t/fulltext3.test
mysql-test/r/fulltext2.result:
  SCCS merged
2007-07-06 11:39:55 -07:00
unknown
a077186b70 fix test for bug29299 2007-07-06 11:35:10 -07:00
unknown
db9a06b7ca Merge ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.0-engines.merge
into  ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.1-engines.merge


sql/sql_insert.cc:
  Auto merged
2007-07-06 09:07:08 -07:00
unknown
09ee3a5ae1 Merge anubis.xiphis.org:/usr/home/antony/work/mysql-5.1-engines
into  anubis.xiphis.org:/usr/home/antony/work/mysql-5.1-engines.merge


include/my_base.h:
  Auto merged
mysql-test/lib/mtr_report.pl:
  Auto merged
mysql-test/r/events_bugs.result:
  Auto merged
mysql-test/r/rpl_sp.result:
  Auto merged
mysql-test/r/show_check.result:
  Auto merged
mysql-test/r/sp.result:
  Auto merged
mysql-test/t/disabled.def:
  Auto merged
mysql-test/t/ndb_single_user.test:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/share/errmsg.txt:
  Auto merged
sql/sql_partition.cc:
  Auto merged
2007-07-06 09:03:50 -07:00
unknown
b0f65c166c Merge ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-4.1-engines.merge
into  ppcg5.local:/private/Network/Servers/anubis.xiphis.org/home/antony/work/mysql-5.0-engines.merge
2007-07-06 09:02:36 -07:00
unknown
fec6d3f134 Merge anubis.xiphis.org:/usr/home/antony/work/mysql-5.0-engines
into  anubis.xiphis.org:/usr/home/antony/work/mysql-5.0-engines.merge


sql/sql_insert.cc:
  Auto merged
2007-07-06 09:00:40 -07:00
unknown
46943ab628 Merge anubis.xiphis.org:/usr/home/antony/work/mysql-4.1-engines
into  anubis.xiphis.org:/usr/home/antony/work/mysql-4.1-engines.merge
2007-07-06 08:57:18 -07:00
unknown
d4e6d4b2c1 Merge labbari.dsl.inet.fi:/home/my/bk/mysql-5.0-marvel
into  labbari.dsl.inet.fi:/home/my/bk/mysql-5.1-marvel


BitKeeper/etc/ignore:
  auto-union
libmysql/libmysql.c:
  Auto merged
mysql-test/r/innodb_mysql.result:
  Auto merged
mysql-test/include/mix1.inc:
  Manual merge from 5.0
storage/innobase/handler/ha_innodb.cc:
  Manual merge from 5.0
2007-07-06 13:49:15 +03:00
unknown
0b24005753 Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-5.0
into  labbari.dsl.inet.fi:/home/my/bk/mysql-5.0-marvel


libmysql/libmysql.c:
  Auto merged
mysql-test/r/innodb_mysql.result:
  Manual merge from main 5.1 to 5.1-marvel.
mysql-test/t/innodb_mysql.test:
  Manual merge from main 5.1 to 5.1-marvel.
2007-07-06 09:51:02 +03:00
unknown
bd087a1756 Merge chilla.local:/home/mydev/mysql-5.1-ateam
into  chilla.local:/home/mydev/mysql-5.1-bug26827
2007-07-05 11:46:40 +02:00
unknown
26af0b96a8 Bug#26827 - table->read_set is set incorrectly,
causing update of a different column

Post-pushbuild fix.

bitmap_set_bit() is an inline function in DEBUG builds and
a macro in non-DEBUG builds. The latter evaluates its 'bit'
argument twice. So one must not use increment/decrement operators
on this argument.

Moved increment of pointer out of bitmap_set_bit() call.


include/my_bitmap.h:
  Bug#26827 - table->read_set is set incorrectly,
              causing update of a different column
  Added a warning comment.
sql/sql_partition.cc:
  Bug#26827 - table->read_set is set incorrectly,
              causing update of a different column
  Moved increment of pointer out of bitmap_set_bit() call.
2007-07-05 11:31:03 +02:00
unknown
23d10db8a3 Merge bk-internal.mysql.com:/home/bk/mysql-maria
into  janus.mylan:/usr/home/serg/Abk/mysql-maria


storage/maria/ma_locking.c:
  Auto merged
2007-07-05 10:35:56 +02:00
unknown
e4ee2c5c08 BUG#27564 - Valgrind: UDF does not cleanup correctly
Enabling rpl_udf test.


mysql-test/t/disabled.def:
  Enabling rpl_udf test.
2007-07-05 12:48:11 +05:00
unknown
4beb79a46e Merge mysql.com:/home/svoj/devel/mysql/BUG27564/mysql-5.0-engines
into  mysql.com:/home/svoj/devel/mysql/BUG27564/mysql-5.1-engines


mysys/hash.c:
  Auto merged
2007-07-05 11:47:53 +05:00
unknown
dc308c5147 Merge mysql.com:/home/svoj/devel/mysql/BUG27564/mysql-4.1-engines
into  mysql.com:/home/svoj/devel/mysql/BUG27564/mysql-5.0-engines


mysys/hash.c:
  Auto merged
2007-07-05 11:45:58 +05:00
unknown
b7bf972585 BUG#27564 - Valgrind: UDF does not cleanup correctly
Dropping an user defined function may cause server crash in
case this function is still in use by another thread.

The problem was that our hash implementation didn't update
hash link list properly when hash_update() was called.


mysys/hash.c:
  The following requirement wasn't met by hash_update() function
  causing corruption of hash links list:
  
  After a record was unlinked from the old chain during update, it
  holds random position. By the chance this position is equal to
  position for the first element in the new chain. That means
  updated record is the only record in the new chain.
2007-07-05 11:45:14 +05:00
unknown
fcdc76c289 in mysql_unlock_tables(), do thr_unlock() AFTER external_unlock().
it means, {update,restore}_status() should be called in external_lock, 
not in thr_unlock. Only affects storage engines that support
TL_WRITE_CONCURRENT.
2007-07-04 22:27:58 +02:00
unknown
435df1859f Bug#26827 - table->read_set is set incorrectly,
causing update of a different column

For efficiency some storage engines do not read a complete record
for update, but only the columns required for selecting the rows.

When updating a row of a partitioned table, modifying a column
that is part of the partition or subpartition expression, then
the row may need to move from one [sub]partition to another one.
This is done by inserting the new row into the target
[sub]partition and deleting the old row from the originating one.
For the insert we need a complete record.

If an above mentioned engine was used for a partitioned table, we
did not have a complete record in update_row(). The implicitly
executed write_row() got an incomplete record.

This is solved by instructing the engine to read a complete record
if one of the columns of the partition or subpartiton is to be
updated.

No testcase. This can be reproduced with Falcon only. The engines
contained in standard 5.1 do always return complete records on
update.


sql/ha_partition.cc:
  Bug#26827 - table->read_set is set incorrectly,
              causing update of a different column
  Setting partition field bits in read_set if a writing
  operation is going on. This replaces the old function
  include_partition_fields_in_used_fields().
  Setting all bits in read_set if write_set contains a column
  used in a partition or subpartition expression.
  Removed include_partition_fields_in_used_fields().
sql/ha_partition.h:
  Bug#26827 - table->read_set is set incorrectly,
              causing update of a different column
  Removed declaration of
  include_partition_fields_in_used_fields().
sql/partition_info.h:
  Bug#26827 - table->read_set is set incorrectly,
              causing update of a different column
  Added a bitmap to partition_info for a quick check of
  columns used in a partition or subpartition expression.
sql/sql_partition.cc:
  Bug#26827 - table->read_set is set incorrectly,
              causing update of a different column
  Initializing the new bitmap with all columns used in a
  partition or subpartition expression.
2007-07-04 21:55:26 +02:00
unknown
3a1c7c914c Maria: correcting assertions (should be enforced only in multi-threaded
mode) so ma_test_all works again; more error detection in ma_test_all;
maria_control renamed to maria_log_control (Monty's suggestion,
so that a "rm maria_log*" removes all log-related files). Disabling
failing wrong assertion.


storage/maria/ma_blockrec.c:
  disabling assertion which fails because cur_block is a
  local variable not initialized
storage/maria/ma_check.c:
  comment
storage/maria/ma_control_file.h:
  control file renamed
storage/maria/ma_loghandler.c:
  assertions needed only in multi-threaded mode (ma_test1 and ma_test2
  are single-threaded, it's ok for them to use dummy_transaction_object
  with transactional tables: trn->rec_lsn can be set without interfering
  with other threads).
storage/maria/ma_test_all.sh:
  got caught by failures in some ma_test1 runs, which I didn't see
  because ma_test_all returned 0 and I didn't scroll up in the window;
  now using "set -e" to avoid that. Also testing that
  we get the errors and warnings we expect.
storage/maria/unittest/Makefile.am:
  maria_control renamed
2007-07-04 16:01:00 +02:00
unknown
6ed46be70e Fixed bytes to uchar and gptr to uchar* 2007-07-04 12:39:19 +03:00
unknown
8ba21c616e Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-maria
into  a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-maria.prod


BitKeeper/etc/ignore:
  auto-union
include/maria.h:
  Auto merged
include/my_base.h:
  Auto merged
mysql-test/include/ps_conv.inc:
  Auto merged
mysql-test/r/ps_2myisam.result:
  Auto merged
mysql-test/r/ps_3innodb.result:
  Auto merged
mysql-test/r/ps_4heap.result:
  Auto merged
mysql-test/r/ps_5merge.result:
  Auto merged
mysql-test/r/ps_7ndb.result:
  Auto merged
mysys/Makefile.am:
  Auto merged
mysys/mf_iocache.c:
  Auto merged
mysys/mf_tempfile.c:
  Auto merged
mysys/my_error.c:
  Auto merged
mysys/my_init.c:
  Auto merged
mysys/my_open.c:
  Auto merged
sql/handler.cc:
  Auto merged
sql/handler.h:
  Auto merged
sql/lex.h:
  Auto merged
sql/sql_show.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
sql/table.cc:
  Auto merged
sql/table.h:
  Auto merged
storage/maria/ha_maria.cc:
  Auto merged
storage/maria/ha_maria.h:
  Auto merged
storage/maria/ma_bitmap.c:
  Auto merged
storage/maria/ma_blockrec.h:
  Auto merged
storage/maria/ma_create.c:
  Auto merged
storage/maria/ma_dynrec.c:
  Auto merged
storage/maria/ma_loghandler.h:
  Auto merged
storage/maria/ma_loghandler_lsn.h:
  Auto merged
storage/maria/ma_open.c:
  Auto merged
storage/maria/ma_pagecache.c:
  Auto merged
storage/maria/ma_pagecache.h:
  Auto merged
storage/maria/ma_test1.c:
  Auto merged
storage/maria/ma_test2.c:
  Auto merged
storage/maria/ma_update.c:
  Auto merged
storage/maria/maria_chk.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Auto merged
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Auto merged
storage/myisam/ft_stopwords.c:
  Auto merged
storage/myisam/mi_close.c:
  Auto merged
support-files/compiler_warnings.supp:
  Auto merged
mysys/mf_keycache.c:
  Pulled from mysql-maria tree, manual merge.
mysys/my_handler.c:
  Pulled from mysql-maria tree, manual merge.
mysys/my_symlink2.c:
  Pulled from mysql-maria tree, manual merge.
sql/mysqld.cc:
  Pulled from mysql-maria tree, manual merge.
sql/sql_table.cc:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_blockrec.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_check.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_close.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/ma_loghandler.c:
  Pulled from mysql-maria tree, manual merge.
storage/maria/maria_def.h:
  Pulled from mysql-maria tree, manual merge.
2007-07-04 12:20:53 +03:00