Commit graph

153 commits

Author SHA1 Message Date
Michael Widenius
acba7d2f9f Fixed MDEV-306 / LP:1007967 - Assertion `table->file->stats.records > 0 || error' failed join_read_const_table on concurrent SELECT and DROP/ADD INDEX
sql/sql_table.cc:
  Added comment
storage/maria/ma_close.c:
  Don't store history if it's visible to all.
  This fixed the MDEV-306 bug
storage/maria/ma_delete_table.c:
  Removed old comment
  Delete history state for deleted tables
storage/maria/ma_info.c:
  More DBUG_PRINT
storage/maria/ma_open.c:
  More DBUG_PRINT
2012-06-15 12:52:58 +03:00
Michael Widenius
d513153f77 Merge of compatibility fixes
Fixed failing tests in sys_vars as we have now stricter checking of setting of variables.

mysql-test/suite/sys_vars/r/innodb_adaptive_flushing_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_adaptive_hash_index_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_large_prefix_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_random_read_ahead_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_stats_on_metadata_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_strict_mode_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_support_xa_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/innodb_table_locks_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/rpl_semi_sync_master_enabled_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/r/rpl_semi_sync_slave_enabled_basic.result:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_adaptive_flushing_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_adaptive_hash_index_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_large_prefix_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_random_read_ahead_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_stats_on_metadata_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_strict_mode_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_support_xa_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysql-test/suite/sys_vars/t/innodb_table_locks_basic.test:
  One can now only assign 0 or 1 to boolean variables
mysys/my_getsystime.c:
  Merge + fixed bug that __NR_clock_gettime didn't work in 5.5
2012-04-02 13:33:16 +03:00
Michael Widenius
bb6cc52528 Automatic merge 2012-03-28 13:58:14 +03:00
Michael Widenius
3bc932ec17 Merge with 5.1 2012-03-28 13:49:07 +03:00
Michael Widenius
74b0649332 Fixed lp:944422 "mysql_upgrade destroys Maria tables?"
The issue was that check/optimize/anaylze did not zerofill the table before they started to work on it.
Added one more element to not often used function handler::auto_repair() to allow handler to decide when to auto repair.


mysql-test/suite/maria/r/maria-autozerofill.result:
  Test case for lp:944422
mysql-test/suite/maria/t/maria-autozerofill.test:
  Test case for lp:944422
sql/ha_partition.cc:
  Added argument to auto_repair()
sql/ha_partition.h:
  Added argument to auto_repair()
sql/handler.h:
  Added argument to auto_repair()
sql/table.cc:
  Let auto_repair() decide which errors to trigger auto-repair
storage/archive/ha_archive.h:
  Added argument to auto_repair()
storage/csv/ha_tina.h:
  Added argument to auto_repair()
storage/maria/ha_maria.cc:
  Give better error & warning messages for auto-repaired tables.
storage/maria/ha_maria.h:
  Added argument to auto_repair()
  Always auto-repair in case of moved table.
storage/maria/ma_open.c:
  Remove special handling of HA_ERR_OLD_FILE (this is now handled in auto_repair())
storage/myisam/ha_myisam.h:
  Added argument to auto_repair()
2012-03-28 13:22:21 +03:00
Michael Widenius
a579adea66 Cleanups:
- Don't use SAFEMALLOC on valgrind builds (slows things down)
- Added back lost option from 5.3: debug-mutex-deadlock-detector
- Flush pages before taking lock mutex (speeds up closing of Aria tables).

BUILD/SETUP.sh:
  - Don't use SAFEMALLOC on valgrind builds (slows things down)
sql/lock.cc:
  Make default argument explicit (improves readability)
sql/mysqld.cc:
  Removed compiler warnings
  Sorted debug options alphabetically
  Added back lost option from 5.3: debug-mutex-deadlock-detector
storage/maria/ma_close.c:
  Flush pages before taking lock mutex (speeds up closing of Aria tables).
storage/maria/ma_open.c:
  More DBUG_PRINT
storage/maria/maria_def.h:
  Better DBUG_PRINT
storage/maria/trnman.c:
  Better DBUG_PRINT
2012-03-20 15:23:56 +02:00
Michael Widenius
4ab7a33e1e Automatic merge 2012-03-05 22:00:24 +02:00
Igor Babaev
8b469eb515 Merge 5.3->5.5. 2012-03-01 14:22:22 -08:00
Michael Widenius
e4e85cd2ef Fixed lp:925377 "Querying myisam table metadata while 'alter table..enable keys' is running may corrupt the table"
Fixed wrong mutex order bug in Aria when flush_log_for_bitmap() was called when table is not yet marked for change.

include/my_base.h:
  Added flag that table is opened only for status
mysql-test/r/myisam-big.result:
  Test case for lp:925377
mysql-test/t/myisam-big.test:
  Test case for lp:925377
sql/sql_base.cc:
  If thd->version == 0 (happens only when we are opening a table that is flushed under  MYSQL_LOCK_IGNORE_FLUSH), open the table in HA_OPEN_FOR_STATUS mode
storage/maria/ma_bitmap.c:
  Fixed wrong mutex order bug in Aria when flush_log_for_bitmap() was called when table is not yet marked for change.
storage/maria/ma_dbug.c:
  Ignore last_version <= 1 as these are either flushed or only opened for status
storage/maria/ma_open.c:
  Use last_version=1 as a marker that table was opened with HA_OPEN_FOR_STATUS.
  In this case we just open a new version of the table in read only mode.
storage/myisam/mi_create.c:
  Update prototype
storage/myisam/mi_dbug.c:
  Ignore last_version <= 1 as these are either flushed or only opened for status
storage/myisam/mi_open.c:
  Use last_version=1 as a marker that table was opened with HA_OPEN_FOR_STATUS.
  If HA_OPEN_FOR_STATUS is used, we will not assert if there is an old not-to-be-used version of the table existing.
  In this case we just open a new version of the table in read only mode.
storage/myisam/myisamdef.h:
  Updated prototype
2012-02-28 23:18:52 +02:00
Sergey Petrunya
2256ecdd7c Merge 5.2->5.3 2012-02-24 20:07:12 +04:00
Michael Widenius
0919edf32d Automatic merge 2012-02-24 17:21:44 +02:00
Michael Widenius
60aad31422 Fix for lp:909635: MariaDB crashes on a select with long varchar and blob fields
Problem was a crash in internal temporary (Maria) files when row length exceeded 65535

mysql-test/suite/maria/r/maria3.result:
  Added test case
mysql-test/suite/maria/t/maria3.test:
  Added test case
storage/maria/ma_open.c:
  Added support for row length > 65535.
  This fixes crash when using tables with longer row lengths.
2012-02-24 17:01:47 +02:00
Sergei Golubchik
4f435bddfd 5.3 merge 2012-01-13 15:50:02 +01:00
Michael Widenius
6d4224a31c Merge with 5.2.
no_error handling for select (used by INSERT ... SELECT) still needs to be fixed, but I will do that in a separate commit
2011-12-11 11:34:44 +02:00
Michael Widenius
9d0ea0ab4b Fixed bug where automaticly zerofilled table was not part of recovery if crash happended before next checkpoint.
mysql-test/suite/maria/r/maria-autozerofill.result:
  Updated test case
storage/maria/ha_maria.cc:
  Write create_rename_lsn for auto_zerofilled tables.
storage/maria/ma_delete.c:
  Added DBUG_ASSERT() to find errors when deleting pages.
storage/maria/ma_locking.c:
  Fixed typo
storage/maria/ma_open.c:
  Don't regard file as movable if create_rename_lsn is not LSN_NEEDS_NEW_STATE_LSNS
2011-12-02 17:22:17 +02:00
Sergei Golubchik
d2755a2c9c 5.3->5.5 merge 2011-11-22 18:04:38 +01:00
Sergei Golubchik
557f0d3ad0 5.2->5.3 merge 2011-11-12 18:08:12 +01:00
unknown
dd564da398 5.1->5.2 merge (gcc 4.6 warnings and apple hwaddress fixes). 2011-10-27 17:51:30 +03:00
unknown
28e2777991 Fix gcc 4.6 warnings about assigned but not used variables.
Fixed my_gethwaddr.c to allow compilation on Mac OS X.
2011-10-27 15:22:52 +03:00
Michael Widenius
fa36a7426b Fixed lp:879939 "assertion in ha_maria::enable_indexes with derived_with_keys=on"
Honor unique/not unique when creating keys for internal tempory tables.
Added new variables to be used to limit how keys are created for internal temporary tables.


include/maria.h:
  Added maria_max_key_length() and maria_max_key_segments()
include/myisam.h:
  Added myisam_max_key_length() and myisam_max_key_segments()
mysql-test/r/mysql.result:
  Drop all used tables
mysql-test/r/subselect4.result:
  Added test case for lp:879939
mysql-test/t/mysql.test:
  Drop all used tables
mysql-test/t/subselect4.test:
  Added test case for lp:879939
sql/mysql_priv.h:
  Added internal_tmp_table_max_key_length and internal_tmp_table_max_key_segments to be used to limit how keys for derived tables are created.
sql/mysqld.cc:
  Added internal_tmp_table_max_key_length and internal_tmp_table_max_key_segments to be used to limit how keys for derived tables are created.
sql/share/errmsg.txt:
  Added new error message for internal errors
sql/sql_select.cc:
  Give error if we try to create a wrong key (this error should never happen)
  Honor unique/not unique when creating keys for internal tempory tables.
storage/maria/ha_maria.cc:
  Added change_table_ptr() to ensure that external_ref points always to the correct table.
  (Not having this caused an assert in the included test)
storage/maria/ha_maria.h:
  Added change_table_ptr() to ensure that external_ref points always to the correct table.
storage/maria/ma_check.c:
  Fixed bug in Duplicate key error printing (now row position is printed correctly)
storage/maria/ma_create.c:
  maria_max_key_length() -> _ma_max_key_length()
storage/maria/ma_info.c:
  Added extern function maria_max_key_length() to calculate the max key length based on current block size.
storage/maria/ma_open.c:
  maria_max_key_length() -> _ma_max_key_length()
storage/maria/maria_def.h:
  maria_max_key_length() -> _ma_max_key_length()
storage/myisam/ha_myisam.cc:
  Added change_table_ptr() to ensure that external_ref points always to the correct table.
  (Not having this caused an assert in the included test)
storage/myisam/ha_myisam.h:
  Added change_table_ptr() to ensure that external_ref points always to the correct table.
2011-10-26 20:25:18 +03:00
Sergei Golubchik
76f0b94bb0 merge with 5.3
sql/sql_insert.cc:
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
  ******
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
  small cleanup
  ******
  small cleanup
2011-10-19 21:45:18 +02:00
unknown
255c04aed8 Compiler warning about assigned but not used variables fixed. 2011-10-18 13:44:12 +03:00
Michael Widenius
fb135f7ee5 Automatic merge with 5.2 2011-08-15 22:14:08 +03:00
Michael Widenius
b95d2778aa Fixes bugs found by testcase for lp:815022 and lp:726374 "ma_blockrec.c:3000: write_block_record: Assertion `cur_block[1].page_count == 0' failed with a multi-index Aria workload"
The issues was:
- For some tables with a lot of not packed fields, we didn't allocate enough memory in head page which caused DBUG_ASSERT's
- Removed wrong DBUG_ASSERT()
- Fixed a problem with underflow() where it generates a key page where all keys didn't fit.
- Max key length is now limited by block_size/3  (was block_size /2).  This is required for underflow() to work with packed keys.




mysql-test/lib/v1/mysql-test-run.pl:
  Remove --alignment=8 as this doesn't work on 64 bit systems
mysql-test/suite/maria/r/small_blocksize.result:
  Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize-master.opt:
  Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize.test:
  Test case for Aria bug
storage/maria/ha_maria.cc:
  Fixed comment
storage/maria/ma_bitmap.c:
  Fixed wrong variable usage in find_where_to_split_row() where we allocated too little memory for head page.
  We did not take into account space for head extents (long VARCHAR) when trying to split row on head page. This caused us to allocate too little space from bitmap which lead to ASSERT failures later.
storage/maria/ma_blockrec.c:
  Made some argument const (to ensure they was not accidently changed)
  Removed wrong DBUG_ASSERT()
storage/maria/ma_blockrec.h:
  Removed not used variable
storage/maria/ma_delete.c:
  Added my_afree() in case of error
  More comments and DBUG_ASSERT() for underflow()
storage/maria/ma_open.c:
  Make keyinfo->underflow_block_length smaller for packed keys. This has to be done as for long packed keys, underflow() otherwise generates a key page where all keys didn't fit.
storage/maria/ma_page.c:
  New DBUG_ASSERT()
storage/maria/ma_write.c:
  Fixed comment
storage/maria/maria_def.h:
  We have to have space for at least 3 keys on a key page.
  (Otherwise the underflow() code doesn't work for packed keys, even when we have an underflow() for an empty key page)
2011-08-15 16:39:53 +03:00
Michael Widenius
cfd2725d61 Optimize mutex usage.
storage/maria/ma_blockrec.c:
  Unlock bitmaps earlier (no reason to have them unlocked over _ma_write_clr())
storage/maria/ma_extra.c:
  Don't lock THR_LOCK_maria for HA_EXTRA_PREPARE_FOR_RENAME (upper level ensures that we are not opening the same table during this call)
  We don't need to have share->intern_lock locked over _ma_flush_table_files()
storage/maria/ma_open.c:
  Update comment
2011-08-08 14:53:52 +03:00
unknown
efcb48f23e Automerge 5.2->5.3 2011-07-31 22:59:55 +02:00
Michael Widenius
1d61f71384 Merge with 5.1
Fixed lp:814237: Wrong mutex usage in Aria


storage/maria/ma_bitmap.c:
  Added call to _ma_bitmap_mark_file_changed() when flushing bitmap. This fixes lp:814237
2011-07-25 21:52:15 -07:00
Michael Widenius
10065404ea Fixes lp:805930 Sysbench breaks on multiple table test with MariaDB 5.2.7 + Aria
The bug happens when one uses MAX_ROWS=# with Aria & row_format=page and one insert more than # rows.


mysql-test/mysql-test-run.pl:
  Ignore table is full error messages
mysql-test/suite/maria/r/max_length.result:
  Test case for 'Table is full'
mysql-test/suite/maria/t/max_length.test:
  Test case for 'Table is full'
storage/maria/ma_bitmap.c:
  Ensure that we don't allocate bits outside of max_data_file_size.
  Adjust max_data_file_size based on bitmap alignments.
  Backport fix to adjust wrong first_bitmap_with_space.
storage/maria/ma_blockrec.c:
  Calculate value of max_data_file_length
storage/maria/ma_blockrec.h:
  Updated prototype for _ma_bitmap_init()
storage/maria/ma_check.c:
  Give warnings if file sizes are above max file sizes.
  Give more warnings in case of errors.
  Have maria_chk write if table is recreated.
storage/maria/ma_create.c:
  Better calculation of max_data_file_length and thus data pointer length.
  Fixes some wrong pointer lengths when using MAX_ROWS=#
storage/maria/ma_open.c:
  Removed duplicate assigment.
  Use block size from file instead of global variable.
storage/maria/maria_chk.c:
  Remove -1 from printed file length
storage/maria/maria_def.h:
  Update struct st_maria_file_bitmap
2011-07-24 01:25:28 -07:00
Sergei Golubchik
a844635637 use PSI wrappers in aria and other non-MySQL code 2011-07-13 21:10:18 +02:00
Michael Widenius
c9e969b62d Automatic merge 2011-07-04 17:27:46 +03:00
Michael Widenius
7199ac5916 Aria fixes:
- Fixed multi-user problem with one thread doing inserts and another doing scans that gave error 175
- Fixed bug that caused assert in move_to_next_bitmap() & _ma_read_bitmap_page()
- Much more DBUG_ASSERT(!maria_assert_if_crashed_table) to detect errors early
- EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere

storage/maria/ma_bitmap.c:
  More secure handling of first_bitmap_with_space (now we also take care of wrong values)
  Don't set page to -1; This fixed unlikely bug that caused assert in move_to_next_bitmap() & _ma_read_bitmap_page()
storage/maria/ma_blockrec.c:
  More DBUG_ASSERT()'s
  Fixed multi-user problem with one thread doing inserts and another doing scans that gave error 175
  (We should use data_file_length from start of scan, not new value as new bitmap page may not yet be in page cache)
storage/maria/ma_check.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere)
storage/maria/ma_checkpoint.c:
  Made maria_checkpoint_min_activity static so that one can change it in debugger while testing.
  Fixed long standing performance problem that caused write of state info at checkpoint for any file that was ever changed since open.
storage/maria/ma_create.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_dynrec.c:
  Added missing MARIA_EXTERNAL_LOCKING (minor performance improvement)
storage/maria/ma_locking.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_open.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_pagecache.c:
  Added assert to detect reads outside of data file
storage/maria/maria_def.h:
  Added checkpoint_state to cache state writes in checkpoint
2011-07-04 04:32:53 +03:00
Sergei Golubchik
b4a0b2c2f8 post-merge fixes.
most tests pass.
5.3 merge is next
2011-07-02 22:12:12 +02:00
Sergei Golubchik
0accbd0364 lots of post-merge changes 2011-04-25 17:22:25 +02:00
Michael Widenius
139a2b64bf Merge with 5.2 2011-03-09 15:47:59 +02:00
Michael Widenius
4ad6d6b6c9 Merge in new handler and handlersocket code into 5.3 main 2011-02-22 11:15:47 +02:00
Michael Widenius
6db663d614 Fixes for Aria storage engine:
- Fixed bug lp:624099 ma_close.c:75: maria_close: Assertion `share->in_trans == 0' failed on UNLOCK TABLES
- Fixed bug that caused table to be marked as not closed (crashed) during recovery testing.
- Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
- Added checks to ensure that bitmap and internal mutex are always taken in right order.
- For transactional tables, only mark the table as changed before page for table is to be written to disk (and thus the log is flushed).
  This speeds up things a bit and fixes a problem where open_count was incremented on disk but there was no log entry to fix it during recovery -> table was crashed.
- Fixed a bug in repair() where table was not automaticly repaired.
- Ensure that state->global_changed, share->changed and share->state.open_count are set and reset properly.
- Added option --ignore-control-file to maria_chk to be able to run maria_chk even if the control file is locked.


mysql-test/suite/maria/r/maria-recover.result:
  Test result changed as we now force checkpoint before copying table, which results in pagecache getting flushed and we have more rows to recover.
mysql-test/suite/maria/r/maria.result:
  Added new tests
mysql-test/suite/maria/t/maria-recover.test:
  Force checkpoint before copying table.
  This is needed as now the open-count is increased first when first page is flushed.
mysql-test/suite/maria/t/maria.test:
  Added tests to verify fix for lp:624099
storage/maria/ha_maria.cc:
  Use table->in_use instead of current_thd (trivial optimization)
  Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
  More DBUG_ASSERT()
  Reset locked tables count after locked tables have been moved to new transaction. This fixed lp:624099
storage/maria/ma_bitmap.c:
  Temporarly unlock bitmap mutex when calling _ma_mark_file_changed() and pagecache_delete_pages() to ensure right mutex lock order.
  Call _ma_bitmap_unpin_all() when bitmap->non_flusable is set to 0. This fixed a case when bitmap was not proparly unpinned.
  More comments
  Added DBUG_ASSERT() for detecting wrong share->bitmap usage
storage/maria/ma_blockrec.c:
  More DBUG_ASSERT()
  Moved code around in _ma_apply_redo_insert_row_head_or_tail() to make things safer on error conditions.
storage/maria/ma_check.c:
  Changed parameter for _ma_set_uuid()
  Corrected test for detecting if we lost many rows. This fixed some cases where auto-recovery failed.
  share->changed need to be set if state.open_count is changed.
  Removed setting of share->changed= 0 as called function sets it.
storage/maria/ma_close.c:
  - Added code to properly decrement open_count and have it written by _ma_state_info_write() for transactional tables.
    (This is more correct and also saves us one extra write by _ma_decrement_open_count() at close.
  - Added DBUG_ASSERT() to detect if open_count is wrong at maria_close().
storage/maria/ma_delete.c:
  Updated argument to _ma_mark_file_changed()
storage/maria/ma_delete_all.c:
  Updated argument to _ma_mark_file_changed()
  For transactional tables, call _ma_mark_file_changed() after log entry has been written (to allow recover to fix open_count)
  Reset more needed variables in _ma_reset_status()
storage/maria/ma_delete_table.c:
  Moved deletion of Aria files to maria_delete_table_files().
  Remove RAID usage (old not working code)
storage/maria/ma_extra.c:
  Set share->changed=1 when state needs to be updated on disk.
  Don't reset share->changed after call to _ma_state_info_write() as this calls sets share->changed.
  Set share->state.open_count to 1 to force table to be auto repaired if drop fails.
  Set share->global_changed before call to _ma_flush_table_files() to ensure that we don't try to mark the table changed during flush.
  Added DBUG_ENTER
storage/maria/ma_locking.c:
  Split _ma_mark_file_changed() into two functions to delay marking transactional tables as changed on disk until first disk write.
  Added argument to _ma_decrement_open_count() to tell if we should call ma_lock_database() or not.
  Don't decrement open count for transactional tables during _ma_decrement_open_count(). This will be done during close.
  Changed parameter for _ma_set_uuid()
storage/maria/ma_open.c:
  Set share->open_count_not_zero_on_open if state.open_count is not zero.
  This is needed for DBUG_ASSERT() in maria_close() that is there to enforce that open_count is always 0 at close.
  This test doesn't however work for tables that had open_count != 0 already on disk (ie, crashed tables).
  Enforce right mutex order for share->intern_lock and share->bitmap.bitmap_lock
  Don't set share->changed to 0 if share->state.open_count != 0, as state needs to be be written at close
storage/maria/ma_pagecache.c:
  Moved a bit of code in find_block() to avoid one if.
  More DBUG_ASSERT()
  (I left a comment in the code for Sanja to look at;  The code probably works but we need to check if it's optimal)
storage/maria/ma_pagecrc.c:
  For transactional tables, just before first write to disk, but after log is flushed, mark the file changed.
  This fixes some cases where recovery() did not detect that table was marked as changed and could thus not recover the marker.
storage/maria/ma_recovery.c:
  Set share->changed when share->global_changed is set.
storage/maria/ma_update.c:
  Updated parameter for _ma_mark_file_changed()
storage/maria/ma_write.c:
  Updated parameter for _ma_mark_file_changed()
storage/maria/maria_chk.c:
  Added option --ignore-control-file to be able to run maria_chk even if the control file is locked.
storage/maria/maria_def.h:
  Updated function prototypes.
  Added open_count_not_zero_on_open to MARIA_SHARE.
storage/myisam/ha_myisam.cc:
  current_thd -> table->in_use
2011-02-10 20:33:51 +02:00
Michael Widenius
26565ae1d6 Aria issues:
- Fix for LP#700623 "Aria recovery: ma_blockrec.c:3930: _ma_update_at_original_place: Assertion `block->org_bitmap_value == _ma_bitmap_get_page_bits(info, &info->s->bitmap, page)' failed"
  - Issue was that when deleting a tail page where all index entries where full, the page was marked wrongly in the bitmap.
- If debug_assert_if_crashed_table is set, we now crash when we find Aria corrupted.
- Write more information if we find something wrong with the bitmap.
- Fixed that REPAIR also can fix wrong create_rename_lsn issues (a very unlikely event)
- Define STATE_CRASHED_FLAGS as set of all CRASHED flags (to simplify code)


storage/maria/ha_maria.cc:
  Mark the normal page cache (not the page cache for the logs) so that we can request extra debugging for it.
  Copy the value of debug_assert_if_crashed_table to maria_assert_if_crashed_table so that we can request a crash at exactly the point where we find Aria corrupted.
  Use STATE_CRASHED_FLAGS
storage/maria/ma_bitmap.c:
  Made bits_to_txt extern so that we can use this in maria_chk
  Added extra information to the log files to be able to easier find bitmap failures in recovery. (When compiling with -DEXTRA_DEBUG_BITMAP)
  Added _ma_get_bitmap_description() to request a clear text description of the bitmap.
  Simplify _ma_check_bitmap_data(), as we know the bitmap pattern in the caller.
storage/maria/ma_blockrec.c:
  In delete_head_or_tail(), fixed a bug where we sent wrong information to _ma_bitmap_set() if the directory was full for a page that should be freed.
  This fixed LP#700623 (failure in bitmap found during recovery)
storage/maria/ma_blockrec.h:
  Added definitions for _ma_get_bitmap_description() and bits_to_txt
storage/maria/ma_check.c:
  Simplify call to _ma_check_bitmap_data().
  Write more information if we find something wrong with the bitmap.
  Moved getting clear text information about the bitmap to ma_bitmap.c::_ma_get_bitmap_description()
storage/maria/ma_checkpoint.c:
  More asserts
storage/maria/ma_create.c:
  Fix wrong create_rename_lsn during repair.
  (Create_rename_lsn can be too big if someone restores an old maria_log_file after an Aria file was created)
storage/maria/ma_delete.c:
  Call _ma_set_fatal_error() in case of crashed file
  Remove not needed test of save_errno == HA_ERR_KEY_NOT_FOUND. (Handled by other code
storage/maria/ma_extra.c:
  Call _ma_set_fatal_error() in case of crashed file
  Reset share->bitmap.changed_not_flushed to not cause new ASSERTS to trigger.
  Added _ma_file_callback_to_id() for writing share->id to log file in case of DEBUG logging.
storage/maria/ma_init.c:
  Destroy also translog if it's readonly (as when called by maria_read_log -d)
storage/maria/ma_key.c:
  Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_key_recover.c:
  STATE_CRASHED -> STATE_CRASHED_FLAGS
storage/maria/ma_keycache.c:
  Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_locking.c:
  Call _ma_set_fatal_error() in case of crashed file.
  Added _ma_set_fatal_error()
storage/maria/ma_open.c:
  Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_page.c:
  Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_pagecache.c:
  Added extra information to log file to simply debugging of bitmap errors.
storage/maria/ma_pagecache.h:
  Added extra_debug flag to allow marking of row and index cache for extra logging (for debugging).
storage/maria/ma_panic.c:
  Flush both data and index blocks in case of HA_PANIC_CLOSE
  Fixed wrong position of 'break'. (Not critical for MariaDB as MariaDB never uses this code)
storage/maria/ma_recovery_util.c:
  Avoid writing extra not needed \n to DBUG log.
storage/maria/ma_rkey.c:
  Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_search.c:
  Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_static.c:
  Define maria_assert_if_crashed_table
storage/maria/ma_update.c:
  Call _ma_set_fatal_error() in case of crashed file.
  The new code also avoids a problem where we before would print the error twice.
storage/maria/ma_write.c:
  Call _ma_set_fatal_error() in case of crashed file
storage/maria/maria_chk.c:
  STATE_CRASHED -> STATE_CRASHED_FLAGS
storage/maria/maria_def.h:
  Added STATE_CRASHED_PRINTED to avoid giving error message about crash twice.
  Added STATE_CRASHED_FLAGS to be able to easily detect and set all CRASHED related flags.
  Added prototypes for new functions.
storage/myisam/mi_panic.c:
  Fixed wrong position of 'break'. (Not critical for MariaDB as MariaDB never uses this code)
2011-01-24 15:19:40 +02:00
Sergei Golubchik
6c9076256f issue an informative error message for a common Aria problem when opening a table
(incorrect block size)
2011-01-14 21:51:55 +01:00
Sergei Golubchik
418df96d51 use bulk insert and repair by sort for unique keys in
Aria and MyISAM in create_internal_tmp_table_from_heap()
(safe, as duplicates are impossible).
This gives a HUGE speed boost!

sql/opt_subselect.cc:
  Fixed problem with wrong recinfo in create_duplicate_weedout_tmp_tabl()
  Tagged the table with 'no_rows' so that when we create the table on disk,
  we only store the index data. This gave us a major speedup!
sql/sql_select.cc:
  create_tmp_table_from_heap() now uses bulk_insert + repair_by_sort
  when creating Aria/MyISAM tables from HEAP tables.
  This gives a HUGE speed boost!
storage/maria/ha_maria.cc:
  Extended bulk_insert() to recreate UNIQUE keys for
  internal temporary tables
storage/maria/ma_open.c:
  Initialize m_info->lock.type properly for temporarly tables
  (needed for start_bulk_insert())
storage/maria/ma_write.c:
  Don't check uniques that are disabled
storage/myisam/ha_myisam.cc:
  Extended bulk_insert() to recreate UNIQUE keys for
  internal temporary tables.
2011-01-14 12:03:41 +01:00
Sergei Golubchik
f5ea301acc Added support for NO_RECORD record format (don't store any row data) for Aria.
This makes the keys smaller (no row pointer) and gives us proper errors if we
use the table wrongly.

sql/sql_select.cc:
  Use NO_RECORD for tables that doesn't need row data.
storage/maria/Makefile.am:
  Added ma_norec.c
storage/maria/ma_check.c:
  Added support for NO_RECORD record format (don't store any row data)
storage/maria/ma_norec.c:
  Added support for NO_RECORD record format
storage/maria/ma_open.c:
  Added support for NO_RECORD record format
storage/maria/ma_search.c:
  Added support for 0 size row pointers (used with NO_RECORD)
storage/maria/ma_test1.c:
  Added testing of NO_RECORD record format.
storage/maria/maria_chk.c:
  Added support for NO_RECORD
storage/maria/maria_def.h:
  Added support for NO_RECORD
storage/maria/unittest/ma_test_all-t:
  Added testing of NO_RECORD record format
2011-01-14 11:54:39 +01:00
Michael Widenius
050c004f5e Merge with 5.1
Fixes to get Aria handler tests to work.
Fixes LP#697597 "HANDLER + Aria asserts in maria-5.3-handler"
2011-01-11 15:36:41 +02:00
Michael Widenius
505c663a1e - Fixed that Aria works with HANDLER commands
- Added test case for Aria
- Tested HANDLER with HEAP (changes to HEAP code will be pushed in 5.3)
- Moved all HANDLER test to suite/handler.


mysql-test/Makefile.am:
  Added suite/handler
mysql-test/mysql-test-run.pl:
  Added suite/handler
mysql-test/r/lock_multi.result:
  Remove test that is already in handler test suite
mysql-test/suite/handler/aria.result:
  Test for HANDLER with Aria storage engine
mysql-test/suite/handler/aria.test:
  Test for HANDLER with Aria storage engine
mysql-test/suite/handler/handler.inc:
  Extended the general handler test
  Moved interface testing to 'interface.test'
mysql-test/suite/handler/init.inc:
  Common init for handler tests.
mysql-test/suite/handler/innodb.result:
  New results
mysql-test/suite/handler/innodb.test:
  Update to use new include files
mysql-test/suite/handler/interface.result:
  Test of HANDLER interface (not storage engine dependent parts)
mysql-test/suite/handler/interface.test:
  Test of HANDLER interface (not storage engine dependent parts)
mysql-test/suite/handler/myisam.result:
  New results
mysql-test/suite/handler/myisam.test:
  Update to use new include files
mysql-test/t/lock_multi.test:
  Remove test that is already in handler test suite
mysys/tree.c:
  Added missing handling of read previous (showed up in HEAP testing)
sql/handler.cc:
  Don't marka 'HA_ERR_RECORD_CHANGED' as fatal (can be used with HANDLER READ, especially with MEMORY ENGINE)
sql/handler.h:
  Added prototype for can_continue_handler_scan()
sql/sql_handler.cc:
  Re-initialize search if we switch from key to table search.
  Check if handler can continue searching between calls (via can_continue_handler_scan())
  Don't write common not fatal errors to log
storage/maria/ma_extra.c:
  Don't set index 0 as default. This forces call to ma_check_index() to set up index variables.
storage/maria/ma_ft_boolean_search.c:
  Ensure that info->last_key.keyinfo is set
storage/maria/ma_open.c:
  Don't set index 0 as default. This forces call to ma_check_index() to set up index variables.
storage/maria/ma_rkey.c:
  Trivial optimization
storage/maria/ma_rnext.c:
  Added missing code from mi_rnext.c to ensure that handler next/prev works.
storage/maria/ma_rsame.c:
  Simple optimizations
storage/maria/ma_search.c:
  Initialize info->last_key once and for all when we change keys.
storage/maria/ma_unique.c:
  Ensure that info->last_key.keyinfo is up to date.
2011-01-10 23:22:40 +02:00
Michael Widenius
7c56b08216 Added TRASH() to table->record[0] to find out if we access not initialzed data.
- Changed Cached_item_field not copy data for fields with NULL value
- In key_copy() and key_restore() don't copy data for fields with NULL value

Fixed code to avoid valgrind warnings
- Use c_ptr_safe instead of c_ptr()

Removed "QQ" from comments (QQ was ment to be used for internal comments that should be removed before pushing)
Fixed wrong alias used (from previous patch)


sql/event_db_repository.cc:
  Update testing if event table is valid (to avoid valgrind errors)
sql/ha_partition.cc:
  m_ordered_scan_ongoing was not initialized
  Reset null bits in record to avoid valgrind errors
sql/handler.h:
  Added flag if storage engine will write row verbatim and the row contains varchar or null fields
  (in which case we must clear the row to avoid valgrind warnings)
sql/item_buff.cc:
  Changed Cached_item_field not copy data for fields with NULL value
  (Optimization and avoids valgrind warnings)
sql/item_func.cc:
  c_ptr() -> c_ptr_safe()
sql/key.cc:
  In key_copy() and key_restore() don't copy data for fields with NULL value
sql/opt_range.cc:
  c_ptr() -> c_ptr_safe()
sql/sql_base.cc:
  Added TRASH() to table->record[0] to find out if we access not initialzed data.
  Initialize null_bytes to:
  - Get consistent tests
  - Ensure we don't get valgrind warnings for null fields (as we may only update a couple of bits in a byte)
sql/sql_class.cc:
  Removed "QQ" from comments
sql/sql_insert.cc:
  Initialize row to default values if we are using valgrind and row will be copied verbatim to disk in storage engine.
sql/sql_load.cc:
  QQ -> TODO
sql/sql_parse.cc:
  Removed old not used code marked QQ and withing "#ifdef REMOVED"
sql/sql_select.cc:
  QQ -> TODO
  Initialize some variables that was used uninitialized
  Added DBUG_ASSERT() to find out if thd was not properly initialized for sub queries
sql/sql_test.cc:
  Fixed format for printing to DBUG file
  Fixed wrong alias used (from previous patch)
sql/sql_trigger.h:
  QQ -> TODO
sql/table.cc:
  QQ -> TODO
storage/maria/ha_maria.cc:
  Mark table with HA_RECORD_MUST_BE_CLEAN_ON_WRITE, if row is written verbatim to disk and contains varchar or null fields.
storage/maria/ma_open.c:
  Added flags if table has varchar or null fields
storage/maria/maria_def.h:
  Added flags if table has varchar or null fields
storage/myisam/ha_myisam.cc:
  Mark table with HA_RECORD_MUST_BE_CLEAN_ON_WRITE, if row is written verbatim to disk and contains varchar or null fields.
storage/myisam/mi_open.c:
  Fixed memory overrun bug when using fulltext keys
storage/xtradb/row/row0sel.c:
  Removed initialization of null bits. (not needed anymore)
2010-11-27 17:29:52 +02:00
Michael Widenius
20acfbf30d Fix for: LP #634955: Assert in _ma_update_at_original_place()
Added locking of lock mutex when updating status in external_unlock() for Aria and MyISAM tables.
Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards. (Allows one to run --debug with mysql-test-run scripts that uses @debug)
Fixed several core dump, deadlock and crashed table bugs in handling of LOCK TABLE with MERGE tables:
- Added priority of locks to avoid crashes with MERGE tables.
- Added thr_lock_merge() to allow one to merge two results of thr_lock().
Fixed 'not found row' bug in REPLACE with Aria tables.
Mark MyISAM tables that are part of MERGE with HA_OPEN_MERGE_TABLE and set the locks to have priority THR_LOCK_MERGE_PRIV.
- By sorting MERGE tables last in thr_multi_unlock() it's safer to release and relock them many times (can happen when TRIGGERS are created)
Avoid printing (null) in debug file (to easier find out wrong NULL pointer usage with %s).



client/mysql.cc:
  Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
client/mysqltest.cc:
  Don't send NULL to fn_format(). (Can cause crash on Solaris when using --debug)
dbug/dbug.c:
  DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards.
include/my_base.h:
  Added flag to signal if one opens a MERGE table.
  Added extra() command to signal that one is not part of a MERGE table anymore.
include/thr_lock.h:
  Added priority for locks (needed to fix bug in thr_lock when using MERGE tables)
  Added option to thr_unlock() if get_status() should be called.
  Added prototype for thr_merge_locks().
mysql-test/mysql-test-run.pl:
  Ignore crashed table warnings for tables named 'crashed'.
mysql-test/r/merge.result:
  Renamed triggers to make debugging easier.
  Added some CHECK TABLES to catch errors earlier.
  Additional tests.
mysql-test/r/merge_debug.result:
  Test of error handling when reopening MERGE tables.
mysql-test/r/udf_query_cache.result:
  Added missing flush status
mysql-test/suite/parts/r/partition_repair_myisam.result:
  Update results
mysql-test/t/merge.test:
  Renamed triggers to make debugging easier.
  Added some CHECK TABLES to catch errors earlier.
  Additional tests.
mysql-test/t/merge_debug.test:
  Test of error handling when reopening MERGE tables.
mysql-test/t/udf_query_cache.test:
  Added missing flush status
mysys/my_getopt.c:
  Removed not used variable
mysys/my_symlink2.c:
  Changed (null) to (NULL) to make it easier to find NULL arguments to DBUG_PRINT() functions.
  (On linux, NULL to sprintf is printed 'null')
mysys/thr_lock.c:
  Added priority of locks to avoid crashes with MERGE tables.
  Added thr_lock_merge() to allow one to merge two results of thr_lock().
  - This is needed for MyISAM as all locked table must share the same status. If not, you will not see newly inserted rows in other instances of the table.
  If calling thr_unlock() with THR_UNLOCK_UPDATE_STATUS, call update_status() and restore_status() for the locks. This is needed in some rare cases where we call thr_unlock() followed by thr_lock() without calling external_unlock/external_lock in between.
  Simplify loop in thr_multi_lock().
  Added 'start_trans', which is called at end of thr_multi_lock() when all locks are taken.
  - This was needed by Aria to ensure that transaction is started when we got all locks, not at get_status(). Without this, some rows could not be visible when we lock two tables at the same time, causing REPLACE using two tables to fail unexpectedly.
sql/handler.cc:
  Add an assert() in handler::print_error() for "impossible errors" (like table is crashed) when --debug-assert-if-crashed-table is used.
sql/lock.cc:
  Simplify mysql_lock_tables() code if get_lock_data() returns 0 locks.
  Added new parameter to thr_multi_unlock()
  In mysql_unlock_read_tables(), call first externa_unlock(), then thr_multi_unlock();  This is same order as we do in mysql_unlock_tables().
  Don't abort locks in mysql_lock_abort() for merged tables when a MERGE table is deleted; Would cause a spin lock.
  Added call to thr_merge_locks() in mysql_lock_merge() to ensure consistency in thr_locks().
  - New locks of same type and table is stored after the old lock to ensure that we get the status from the original lock.
sql/mysql_priv.h:
  Added debug_assert_if_crashed_table
sql/mysqld.cc:
  Added --debug-assert-if-crashed-table
sql/parse_file.cc:
  Don't print '(null)' in DBUG_PRINT of no dir given
sql/set_var.cc:
  Increase default size of buffer for @debug variable.
sql/sql_base.cc:
  In case of error from reopen_table() in reopen_tables(), call unlock_open_table() and restart loop.
  - This fixed bug when we twice deleted same table from open_cache.
  Don't take name lock for already name locked table in open_unireg_entry().
  - Fixed bug when doing repair in reopen_table().
  - In detach_merge_children(), always detach if 'clear_refs' is given. We can't trust parent->children_attached as this function can be called twice, first time with clear_refs set to 0.
sql/sql_class.cc:
  Changed printing of (null) to "" in set_thd_proc_info()
sql/sql_parse.cc:
  Added DBUG
sql/sql_trigger.cc:
  Don't call unlink_open_table() if reopen_table() fails as the table may already be freed.
storage/maria/ma_bitmap.c:
  Fixed DBUG_ASSERT() in allocate_tail()
storage/maria/ma_blockrec.c:
  Fixed wrong calculation of row length for very small rows in undo_row_update().
  - Fixes ASSERT() when doing undo.
storage/maria/ma_blockrec.h:
  Added _ma_block_start_trans() and _ma_block_start_trans_no_versioning()
storage/maria/ma_locking.c:
  Call _ma_update_status_with_lock() when releasing write locks.
  - Fixes potential problem with updating status without the proper lock.
storage/maria/ma_open.c:
  Changed to use start_trans() instead of get_status() to ensure that we see all rows in all locked tables when we got the locks.
  - Fixed 'not found row' bug in REPLACE with Aria tables.
storage/maria/ma_state.c:
  Added _ma_update_status_with_lock() and _ma_block_start_trans().
  This is to ensure that we see all rows in all locked tables when we got the locks.
storage/maria/ma_state.h:
  Added _ma_update_status_with_lock()
storage/maria/ma_write.c:
  More DBUG_PRINT
storage/myisam/mi_check.c:
  Fixed error message
storage/myisam/mi_extra.c:
  Added HA_EXTRA_DETACH_CHILD:
  - Detach MyISAM table to not be part of MERGE table (remove flag & lock priority).
storage/myisam/mi_locking.c:
  Call mi_update_status_with_lock() when releasing write locks.
  - Fixes potential problem with updating status without the proper lock.
  Change to use new HA_OPEN_MERGE_TABLE flag to test if MERGE table.
  Added mi_fix_status(), called by thr_merge().
storage/myisam/mi_open.c:
  Added marker if part of MERGE table.
  Call mi_fix_status() in thr_lock() for transactional tables.
storage/myisam/myisamdef.h:
  Change my_once_flag to uint, as it stored different values than just 0/1
  Added 'open_flag' to store state given to mi_open()
storage/myisammrg/ha_myisammrg.cc:
  Add THR_LOCK_MERGE_PRIV to THR_LOCK_DATA to get MERGE locks sorted after other types of locks.
storage/myisammrg/myrg_locking.c:
  Remove windows specific code.
storage/myisammrg/myrg_open.c:
  Use HA_OPEN_MERGE_TABLE to mi_open().
  Set HA_OPEN_MERGE_TABLE for linked MyISAM tables.
storage/xtradb/buf/buf0buf.c:
  Fixed compiler warning
storage/xtradb/buf/buf0lru.c:
  Initialize variable that could be used not initialized.
2010-11-02 17:22:57 +02:00
Michael Widenius
b9890b0544 Fix for LP#634943 "marked as crashed", "zerofilling" and "wrong data in bitmap" when recovering Aria tables
This was an interaction of several bugs:
- Tables marked as opened was not properly unmarked on recovery if there was not changes since checkpoint
- zerofill of tables put wrong data in bitmap if directory for page was full
- Tables was thought as 'moved' during recovery if they had a create_lsn bigger than the lsn in the control file.


storage/maria/ha_maria.cc:
  If table is moved and crashed, threat it as crashed.
  (Not a related to this bug, but still good to have fixed)
storage/maria/ma_blockrec.c:
  Make enough_free_entries_on_page() global
storage/maria/ma_blockrec.h:
  Make enough_free_entries_on_page() global
storage/maria/ma_check.c:
  If directory is full, mark page full. Fixes bug in zerofill
storage/maria/ma_open.c:
  Don't marke files as MOVED during recovery if create_trid > trnman_max_trid, as this fails for tables created after checkpoint.
storage/maria/ma_recovery.c:
  Reset open_count in file that was open during crash and was part of checkpoint.
  Fixed wrong warning of 'open count' after recovery of files that was not touched since checkpoint.
storage/maria/maria_chk.c:
  Changed not documented option --log-dir to --logdir
  Document more of the options.
  Clean up output for --help
storage/maria/trnman.c:
  Added DBUG_PRINT
2010-09-10 23:27:26 +03:00
Michael Widenius
0f3d4b2f76 Fixed bug that 'maria_read_log -a' didn't set max_trid when reparing tables.
Fixed bug in Aria when replacing short keys with long keys and a key tree both overflow and underflow at same time.
Fixed several bugs when generating recovery logs when using RGQ with replacing long keys with short keys and vice versa.
Lots of new DBUG_ASSERT()'s
Added more information to recovery log to make it easier to know from where log entry orginated.
Introduced MARIA_PAGE->org_size that tells what the size of the page was in last log entry. This allows us to find out if all key changes for index page was logged.
Small code cleanups:
- Introduced _ma_log_key_changes() to log crc of key page changes
- Added share->max_index_block_size as max size of data one can put in key block (block_size - KEYPAGE_CHECKSUM_SIZE)
  This will later simplify adding a directory to index pages.
- Write page number instead of page postition to DBUG log



mysql-test/lib/v1/mysql-test-run.pl:
  Use --general-log instead of --log to disable warning when using RQG
sql/mysqld.cc:
  If we have already sent ok to client when we get an error, log this to stderr
  Don't disable option --log-output if CSV engine is not supported.
storage/maria/ha_maria.cc:
  Log queries to recovery log also in LOCK TABLES
storage/maria/ma_check.c:
  If param->max_trid is set, use this value instead of max_trid_in_system().
  This is used by recovery to set max_trid to max seen trid so far.
  keyinfo->block_length - KEYPAGE_CHECKSUM_SIZE -> max_index_block_size (Style optimization)
storage/maria/ma_delete.c:
  Mark tables crashed early
  Write page number instead of page position to debug log.
  Added parameter to ma_log_delete() and ma_log_prefix() that is logged so that we can find where wrong log entries where generated.
  Fixed bug where a page was not proplerly written when same key tree had both an overflow and underflow when deleting a key.
  keyinfo->block_length - KEYPAGE_CHECKSUM_SIZE => max_index_block_size (Style optimization)
  ma_log_delete() now has extra parameter of how many bytes from end of page should be appended to log for page (for page overflows)
storage/maria/ma_key_recover.c:
  Added extra parameter to ma_log_prefix() to indicate what caused log entry.
  Update MARIA_PAGE->org_size when logging info about page.
  Much more DBUG_ASSERT()'s.
  Fix some bugs in maria_log_add() to handle page overflows.
  Added _ma_log_key_changes() to log crc of key page changes.
  If EXTRA_STORE_FULL_PAGE_IN_KEY_CHANGES is defines, log the resulting pages to log so one can trivally
  see how the resulting page should have looked like (for errors in CRC values)
storage/maria/ma_key_recover.h:
  Added _ma_log_key_changes() which is only called if EXTRA_DEBUG_KEY_CHANGES is defined.
  Updated function prototypes.
storage/maria/ma_loghandler.h:
  Added more values to en_key_debug, to get more exact location where things went wrong when logging to recovery log.
storage/maria/ma_open.c:
  Initialize share->max_index_block_size
storage/maria/ma_page.c:
  Added updating and testing of MARIA_PAGE->org_size
  Write page number instead of page postition to DBUG log
  Generate error if we read page with wrong data.
  Removed wrong assert: key_del_current != share->state.key_del.
  Simplify _ma_log_compact_keypage()
storage/maria/ma_recovery.c:
  Set param.max_trid to max seen trid before running repair table (used for alter table to create index)
storage/maria/ma_rt_key.c:
  Update call to _ma_log_delete()
storage/maria/ma_rt_split.c:
  Use _ma_log_key_changes()
  Update MARIA_PAGE->org_size
storage/maria/ma_unique.c:
  Remove casts
storage/maria/ma_write.c:
  keyinfo->block_length - KEYPAGE_CHECKSUM_SIZE => share->max_index_block_length.
  Updated calls to _ma_log_prefix()
  Changed code to use _ma_log_key_changes()
  Update ma_page->org_size
  Fixed bug in _ma_log_split() for pages that overflow
  Added KEY_OP_DEBUG logging to functions
  Log KEYPAGE_FLAG in all log entries
storage/maria/maria_def.h:
  Added SHARE->max_index_block_size
  Added MARIA_PAGE->org_size
storage/maria/trnman.c:
  Reset flags for new transaction.
2010-09-06 02:25:44 +03:00
Michael Widenius
d2f8b7d045 Fix for LP#602604: RQG: ma_blockrec.c:6187: _ma_apply_redo_insert_row_head_or_tail: Assertion `0' failed on Maria engine recovery
More DBUG_PRINT (to simplify future debugging)
Aria: Added STATE_IN_REPAIR, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
Aria: Some trivial speedup optimization
Aria: Better warning if table was marked crashed by unfinnished repair


mysql-test/lib/v1/mysql-test-run.pl:
  Fix so one can run RQG
mysql-test/suite/maria/r/maria-recovery2.result:
  Update for new error message.
mysys/stacktrace.c:
  Fixed compiler warning
storage/maria/ha_maria.cc:
  More DBUG_PRINT
  Added STATE_IN_REPAIR flag, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
  Don't log query for dropping temporary table.
storage/maria/ha_maria.h:
  Added prototype for drop_table()
storage/maria/ma_blockrec.c:
  More DBUG_PRINT
  Make read_long_data() inline for most cases. (Trivial speedup optimization)
storage/maria/ma_check.c:
  Better warning if table was marked crashed by unfinnished repair
storage/maria/ma_open.c:
  More DBUG_PRINT
storage/maria/ma_recovery.c:
  Give warning if found crashed table.
  Changed warning for tables that can't be opened.
storage/maria/ma_recovery_util.c:
  Write warnings to DBUG file
storage/maria/maria_chk.c:
  Added STATE_IN_REPAIR flag, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
storage/maria/maria_def.h:
  Added maria_mark_in_repair(x)
storage/maria/maria_read_log.c:
  Added option: --character-sets-dir
storage/maria/trnman.c:
  By default set min_read_from to max value.
  This allows us to remove TRN:s from rows during recovery to get more space.
  This fixes bug LP#602604: RQG: ma_blockrec.c:6187: _ma_apply_redo_insert_row_head_or_tail: Assertion `0' failed on Maria engine recovery
2010-07-30 10:45:27 +03:00
Michael Widenius
ec281a3c34 Fixed some bugs in the Maria storage engine
- Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
- Fixed a rase condition when two threads calls external_lock and thr_lock() in different order. When this happend the transaction that called external lock first
  and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
- Fixed that one can run maria_chk on an automatcally recovered tables without warnings about too small transaction id
- Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
- Fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.

client/mysqldump.c:
  Add "" around error message to make it more readable
client/mysqltest.cc:
  Free environment variables
mysql-test/r/mysqldump.result:
  Updated results
mysql-test/r/openssl_1.result:
  Updated results
mysql-test/suite/maria/r/maria-recover.result:
  Updated results
mysql-test/suite/maria/r/maria3.result:
  Updated results
mysql-test/suite/maria/t/maria3.test:
  Added more test of temporary tables
storage/maria/ha_maria.cc:
  Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
  Start transaction in ma_block_get_status() instead of in ha_maria::external_lock().
  - This fixes a rase condition when two threads calls external lock and thr_lock() in different order. When this happend the transaction that called external lock first and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
  Store latest transaction id in controll file if recovery was done.
  - This allows one to run maria_chk on an automatcally recovered tables without warnings about too small transaction id
storage/maria/ha_maria.h:
  Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
storage/maria/ma_blockrec.h:
  Added new function "_ma_block_get_status_no_versioning()"
storage/maria/ma_init.c:
  Added hook to create trn in ma_block_get_status() if we are using MariaDB
storage/maria/ma_open.c:
  Ensure we call _ma_block_get_status_no_versioning() for transactional tables without versioning (like tables with fulltext)
storage/maria/ma_pagecache.c:
  Allow one to flush blocks that are pinned for read.
  This fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.
storage/maria/ma_recovery.c:
  Set maria_recovery_changed_data to 1 if recover changed something.
  Set max_trid_in_control_file to max found trn if we found a bigger trn.
  The allows will ensure that the control file is up to date after recovery which allows one to run maria_chk on the tables without warnings about too big trn
storage/maria/ma_state.c:
  Call maria_create_trn_hook() in _ma_setup_live_state() instead of ha_maria::external_lock()
  This ensures that 'state' and trn are in sync and thus fixes the race condition mentioned for ha_maria.cc
storage/maria/ma_static.c:
  Added maria_create_trn_hook() and maria_recovery_changed_data
storage/maria/maria_def.h:
  Added MARIA_HANDLER->external_ptr, which is used to hold MariaDB thd.
  Added some new external variables
  Removed reference to non existing function: maria_concurrent_inserts()
2010-06-14 01:13:32 +03:00
Michael Widenius
f1183fc160 Changed default thread stack to 288K to get better memory missalignment between stacks of different threads (should speed up things) and to get a bit extra safety.
In maria_open(), don't allocate big arrays on stack as this may lead to stack overflow.
This fixes a valgrind warning detected by buildbot

include/my_pthread.h:
  Changed default thread stack to 288K to get better memory missalignment between stacks of different threads (should speed up things) and to get a bit extra safety.
storage/maria/ma_open.c:
  In maria_open(), don't allocate big arrays on stack as this may lead to stack overflow.
2009-06-30 15:01:29 +03:00
Michael Widenius
945fa0d913 Merge with mysql-maria tree
mysql-test/t/variables.test:
  Reset delay_key_write, otherwise maria.maria test may fail
sql/set_var.cc:
  Reset ha_open_options if one resets the delay_key_write variable.
  Before there was no way to reset it without restarting mysqld, which caused some tests to fail
2009-02-19 11:01:25 +02:00