Reason for the problem was that the hash of changed files in the key cache was too small (was 128). Fixed by making the hash size larger and changeable.
- Introduced key-cache-file-hash-size (default 512) for MyISAM and aria_pagecache_file_hash_size (default 512) for Aria.
- Added new status variable "Feature_delay_key_write" which counts number of tables opened that are using delay_key_write
mysql-test/r/features.result:
Added test of Feature_delay_key_write
mysql-test/r/key_cache.result:
Updated tests as the number of blocks has changed
mysql-test/r/mysqld--help.result:
Updated result
mysql-test/suite/maria/maria3.result:
Updated result
mysql-test/suite/sys_vars/r/key_cache_file_hash_size_basic.result:
Test new variable
mysql-test/suite/sys_vars/t/aria_pagecache_file_hash_size_basic.test:
Test new variable
mysql-test/suite/sys_vars/t/key_cache_file_hash_size_basic.test:
Test new variable
mysql-test/t/features.test:
Added test of Feature_delay_key_write
mysql-test/t/key_cache.test:
Updated tests as the number of blocks has changed
mysys/mf_keycache.c:
Made CHANGED_BLOCKS_HASH dynamic
sql/handler.cc:
Updated call to init_key_cache()
sql/mysqld.cc:
Added "Feature_delay_key_write"
Added support for key-cache-file-hash-size
sql/mysqld.h:
Added support for key-cache-file-hash-size
sql/sql_class.h:
Added feature_files_opened_with_delayed_keys
sql/sys_vars.cc:
Added key_cache_file_hash_size
storage/maria/ha_maria.cc:
Added pagecache_file_hash_size
Added counting of files with delay_key_write
storage/maria/ma_checkpoint.c:
Fixed compiler warning
storage/maria/ma_pagecache.c:
Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_pagecache.h:
Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_rt_test.c:
Updated parameters for init_pagecache()
storage/maria/ma_test1.c:
Updated parameters for init_pagecache()
storage/maria/ma_test2.c:
Updated parameters for init_pagecache()
storage/maria/ma_test3.c:
Updated parameters for init_pagecache()
storage/maria/maria_chk.c:
Updated parameters for init_pagecache()
storage/maria/maria_ftdump.c:
Updated parameters for init_pagecache()
storage/maria/maria_pack.c:
Updated parameters for init_pagecache()
storage/maria/maria_read_log.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_consist.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist2.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_single.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Updated parameters for init_pagecache()
storage/myisam/ha_myisam.cc:
Added counting of files with delay_key_write
storage/myisam/mi_check.c:
Updated call to init_key_cache()
storage/myisam/mi_test1.c:
Updated call to init_key_cache()
storage/myisam/mi_test2.c:
Updated call to init_key_cache()
storage/myisam/mi_test3.c:
Updated call to init_key_cache()
storage/myisam/mi_test_all.sh:
Fixed broken test
storage/myisam/myisam_ftdump.c:
Updated call to init_key_cache()
storage/myisam/myisamchk.c:
Updated call to init_key_cache()
storage/myisam/myisamlog.c:
Updated call to init_key_cache()
This is port of fix for MySQL BUG#17647863.
revno: 5572
revision-id: jon.hauglid@oracle.com-20131030232243-b0pw98oy72uka2sj
committer: Jon Olav Hauglid <jon.hauglid@oracle.com>
timestamp: Thu 2013-10-31 00:22:43 +0100
message:
Bug#17647863: MYSQL DOES NOT COMPILE ON OSX 10.9 GM
Rename test() macro to MY_TEST() to avoid conflict with libc++.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
- Replaced old DBUG_ASSERT with a new correct one + a comment.
storage/maria/ma_pagecache.c:
Replaced old DBUG_ASSERT with a new correct one + a comment.
- Fixed multi-user problem with one thread doing inserts and another doing scans that gave error 175
- Fixed bug that caused assert in move_to_next_bitmap() & _ma_read_bitmap_page()
- Much more DBUG_ASSERT(!maria_assert_if_crashed_table) to detect errors early
- EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_bitmap.c:
More secure handling of first_bitmap_with_space (now we also take care of wrong values)
Don't set page to -1; This fixed unlikely bug that caused assert in move_to_next_bitmap() & _ma_read_bitmap_page()
storage/maria/ma_blockrec.c:
More DBUG_ASSERT()'s
Fixed multi-user problem with one thread doing inserts and another doing scans that gave error 175
(We should use data_file_length from start of scan, not new value as new bitmap page may not yet be in page cache)
storage/maria/ma_check.c:
EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere)
storage/maria/ma_checkpoint.c:
Made maria_checkpoint_min_activity static so that one can change it in debugger while testing.
Fixed long standing performance problem that caused write of state info at checkpoint for any file that was ever changed since open.
storage/maria/ma_create.c:
EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_dynrec.c:
Added missing MARIA_EXTERNAL_LOCKING (minor performance improvement)
storage/maria/ma_locking.c:
EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_open.c:
EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_pagecache.c:
Added assert to detect reads outside of data file
storage/maria/maria_def.h:
Added checkpoint_state to cache state writes in checkpoint
- Fixed performance bug in alter table with Aria; Aria didn't use disable keys + enable keys
- Fixed wrong warning about 'Wrong CRC on datapage' from REPAIR TABLE with aria block tables.
- Fixed bug in aria_chk that disabled performance counters.
- Added --translog_buffer_size to maria_read_log.
storage/maria/ha_maria.cc:
Fixed performance bug in alter table with Aria; Aria didn't use disable keys + enable keys
storage/maria/ma_bitmap.c:
Added some DBUG_ASSERT()'s and made code more uniform
storage/maria/ma_check.c:
Fixed wrong warning about 'Wrong CRC on datapage' from REPAIR TABLE with aria block tables.
storage/maria/ma_pagecache.c:
Copy read pages to buffer even if there was an error (to be able to detect zero filled pages)
storage/maria/maria_chk.c:
Fixed bug in aria_chk that disabled performance counters.
storage/maria/maria_read_log.c:
Added option to set translog_buffer_size
- Fixed assert in transaction log handler when aria_check was run on block-record table that was much bigger than expected.
- Fixed warnings about wrong mutex order between bitmap and intern_lock
- Fixed error in bitmap that could cause two rows to use same block for a block record.
- Fixed wrong test that could cause error if last page for a bitmap was used by a blob.
- Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks (very unlikely case).
mysql-test/suite/maria/r/maria-recovery3.result:
Updated results
sql/mysqld.cc:
Allow mi_check() to send information messages for log file
storage/maria/ma_bitmap.c:
Fixed problem with wrong mutex order when bitmap was the first page that was flushed out of page cache
- Fixed by introducing _ma_bitmap_mark_file_changed() that marks file changed without a bitmap lock.
- Fixed one case in _ma_change_bitmap_page() where we didn't mark the bitmap changed. This could cause to rows to reuse same block if this was the only change to the bitmap.
- Split _ma_bitmap_get_page_bits() in two parts to not take a bitmap lock when we already have it
- Fixed bug in _ma_bitmap_set_full_page_bits() that caused an error if last page for a bitmap was used by a blob
storage/maria/ma_check.c:
Better handling of wrong file length.
Fixed bug when we tried to write to transaction log when it was not opened (happened when block record file was bigger than expected)
storage/maria/ma_pagecache.c:
Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks:
- In link_block() mark a block given to another thread with PCBLOCK_REASSIGNED to ensure that no other threads can start re-using the block
before the thread that requsted a block.
- In free_block(), don't reset status for a block that is in re-assign by link_block() (we don't want to loose the PCBLOCK_REASSIGNED flag).
- Added call to wait_for_flush() when we got a new block in find_block() to ensure that we don't use a block that is beeing flushed by another thread.
- Moved setting of hits_left and last_hit_time in find_block() to where we assign the block.
Code cleanup and making code uniform:
- Changed a lot of KEYCACHE_DBUG_PRINT to use DBUG_PRINT
- Streamlined all reporting of 'signal' and 'wait' between threads to be identical.
- Use thread name instead of thread number (for each match against --debug)
- Added more DBUG_ENTER, DBUG_PRINT and DBUG_ASSERT()
- Added more comments
storage/myisam/ha_myisam.cc:
Only print information about that we make a backup if we are really making a backup.
storage/myisam/mi_check.c:
Inform mysqld that we are creating a backup of the data file (for inclusion in error log).
Added option to aria_read_log to crash recovery at certain points in the recovery process.
Fixed bug that caused future recovery attempts to fail if we got a crash/got killed during closing of tables at end of recovery process.
mysql-test/mysql-test-run.pl:
Don't abort if 'var' points to stale place; Just remove it.
mysql-test/suite/maria/r/maria.result:
Fixed wrong indentation
mysql-test/t/partition_error.test:
Fixed portability problem with partiton_error.test
storage/maria/ma_close.c:
More DBUG_PRINT info
storage/maria/ma_pagecache.c:
Copy flush_log_callback when writing to page cache. This fixes problem in recovery when switching from mode of file
storage/maria/ma_recovery.c:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/ma_recovery.h:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/maria_chk.c:
Align aria_chk -d output
Don't write warning Aria table '...' is usable but should be fixed if the table was before marked as crashed but now is ok
storage/maria/maria_read_log.c:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
- Fixed bug lp:624099 ma_close.c:75: maria_close: Assertion `share->in_trans == 0' failed on UNLOCK TABLES
- Fixed bug that caused table to be marked as not closed (crashed) during recovery testing.
- Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
- Added checks to ensure that bitmap and internal mutex are always taken in right order.
- For transactional tables, only mark the table as changed before page for table is to be written to disk (and thus the log is flushed).
This speeds up things a bit and fixes a problem where open_count was incremented on disk but there was no log entry to fix it during recovery -> table was crashed.
- Fixed a bug in repair() where table was not automaticly repaired.
- Ensure that state->global_changed, share->changed and share->state.open_count are set and reset properly.
- Added option --ignore-control-file to maria_chk to be able to run maria_chk even if the control file is locked.
mysql-test/suite/maria/r/maria-recover.result:
Test result changed as we now force checkpoint before copying table, which results in pagecache getting flushed and we have more rows to recover.
mysql-test/suite/maria/r/maria.result:
Added new tests
mysql-test/suite/maria/t/maria-recover.test:
Force checkpoint before copying table.
This is needed as now the open-count is increased first when first page is flushed.
mysql-test/suite/maria/t/maria.test:
Added tests to verify fix for lp:624099
storage/maria/ha_maria.cc:
Use table->in_use instead of current_thd (trivial optimization)
Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
More DBUG_ASSERT()
Reset locked tables count after locked tables have been moved to new transaction. This fixed lp:624099
storage/maria/ma_bitmap.c:
Temporarly unlock bitmap mutex when calling _ma_mark_file_changed() and pagecache_delete_pages() to ensure right mutex lock order.
Call _ma_bitmap_unpin_all() when bitmap->non_flusable is set to 0. This fixed a case when bitmap was not proparly unpinned.
More comments
Added DBUG_ASSERT() for detecting wrong share->bitmap usage
storage/maria/ma_blockrec.c:
More DBUG_ASSERT()
Moved code around in _ma_apply_redo_insert_row_head_or_tail() to make things safer on error conditions.
storage/maria/ma_check.c:
Changed parameter for _ma_set_uuid()
Corrected test for detecting if we lost many rows. This fixed some cases where auto-recovery failed.
share->changed need to be set if state.open_count is changed.
Removed setting of share->changed= 0 as called function sets it.
storage/maria/ma_close.c:
- Added code to properly decrement open_count and have it written by _ma_state_info_write() for transactional tables.
(This is more correct and also saves us one extra write by _ma_decrement_open_count() at close.
- Added DBUG_ASSERT() to detect if open_count is wrong at maria_close().
storage/maria/ma_delete.c:
Updated argument to _ma_mark_file_changed()
storage/maria/ma_delete_all.c:
Updated argument to _ma_mark_file_changed()
For transactional tables, call _ma_mark_file_changed() after log entry has been written (to allow recover to fix open_count)
Reset more needed variables in _ma_reset_status()
storage/maria/ma_delete_table.c:
Moved deletion of Aria files to maria_delete_table_files().
Remove RAID usage (old not working code)
storage/maria/ma_extra.c:
Set share->changed=1 when state needs to be updated on disk.
Don't reset share->changed after call to _ma_state_info_write() as this calls sets share->changed.
Set share->state.open_count to 1 to force table to be auto repaired if drop fails.
Set share->global_changed before call to _ma_flush_table_files() to ensure that we don't try to mark the table changed during flush.
Added DBUG_ENTER
storage/maria/ma_locking.c:
Split _ma_mark_file_changed() into two functions to delay marking transactional tables as changed on disk until first disk write.
Added argument to _ma_decrement_open_count() to tell if we should call ma_lock_database() or not.
Don't decrement open count for transactional tables during _ma_decrement_open_count(). This will be done during close.
Changed parameter for _ma_set_uuid()
storage/maria/ma_open.c:
Set share->open_count_not_zero_on_open if state.open_count is not zero.
This is needed for DBUG_ASSERT() in maria_close() that is there to enforce that open_count is always 0 at close.
This test doesn't however work for tables that had open_count != 0 already on disk (ie, crashed tables).
Enforce right mutex order for share->intern_lock and share->bitmap.bitmap_lock
Don't set share->changed to 0 if share->state.open_count != 0, as state needs to be be written at close
storage/maria/ma_pagecache.c:
Moved a bit of code in find_block() to avoid one if.
More DBUG_ASSERT()
(I left a comment in the code for Sanja to look at; The code probably works but we need to check if it's optimal)
storage/maria/ma_pagecrc.c:
For transactional tables, just before first write to disk, but after log is flushed, mark the file changed.
This fixes some cases where recovery() did not detect that table was marked as changed and could thus not recover the marker.
storage/maria/ma_recovery.c:
Set share->changed when share->global_changed is set.
storage/maria/ma_update.c:
Updated parameter for _ma_mark_file_changed()
storage/maria/ma_write.c:
Updated parameter for _ma_mark_file_changed()
storage/maria/maria_chk.c:
Added option --ignore-control-file to be able to run maria_chk even if the control file is locked.
storage/maria/maria_def.h:
Updated function prototypes.
Added open_count_not_zero_on_open to MARIA_SHARE.
storage/myisam/ha_myisam.cc:
current_thd -> table->in_use
Fixed error in Maria unittest
Fixes other issues found by test case for lp:700623 "Aria recovery: ma_blockrec.c:3930: _ma_update_at_original_place: Assertion `block->org_bitmap_value == .."
Fixes lp:670356 "Aria table "is marked as crashed and should be repaired"
mysql-test/lib/v1/mysql-test-run.pl:
Don't delete directory mysql-test/var if we use mysql-test-run --vardir=
mysql-test/mysql-test-run.pl:
Don't delete directory mysql-test/var if we use mysql-test-run --vardir=
(Needed to be able to run multiple concurrent mysql-test-run's/randgen on the same server)
Added extra supression
storage/maria/ma_pagecache.c:
Removed duplicate DBUG_PRINT information
Added KEYCACHE_PRINT for printing information that was already printed to DBUG log (to get rid of some duplicated output)
More comments
Fixed bug that caused page block to be used by two treads at the same time (with different page information)
Mark block->status with PCBLOCK_DIRECT_W independent of the block is changed or not. (Safety fix)
storage/maria/maria_chk.c:
Better --help
storage/maria/unittest/ma_test_recovery.pl:
Ignore differences in 'recover time'.
Fixed error in Maria unittest
- Fixed a bug where we didn't signal a thread waiting for bitmap flush that it's now time to continue which caused a deadlock in Aria.
- Fix for lp:700623 "Aria recovery: ma_blockrec.c:3930: _ma_update_at_original_place: Assertion `block->org_bitmap_value == _ma_bitmap_get_page_bits(info, &info->s->bitmap, page)' failed"
- Fixed a bug in pagecache where a block could change while it was in use.
- In maria_chk set --update-state to on by default so that open_count is cleared if table was ok during check.
storage/maria/ma_bitmap.c:
Fixed a bug where we didn't signal a thread waiting for bitmap flush that it's now time to continue.
This fix adds counters for the different conditions that may be waited upon and signals if there is a waiter when the condition changes.
storage/maria/ma_blockrec.c:
Check if directory if full when calculating what should be in the bitmap.
Fixes lp:700623 "Aria recovery: ma_blockrec.c:3930: _ma_update_at_original_place: Assertion `block->org_bitmap_value == _ma_bitmap_get_page_bits(info, &info->s->bitmap, page)' failed"
storage/maria/ma_pagecache.c:
Added more comments
Removed some duplicated DBUG_PRINT and DBUG_ASSERT()
find_block() now waits for block to be usable if we are not copying it directly. This fixes a bug where a block changed information while we where using it. Fixed by adding an extra parameter to find_block()
Simplified code in make_lock_and_pin() as block can never be == 0 here.
storage/maria/ma_recovery.c:
Reset open_count for tables that are closed in middle of recovery.
storage/maria/maria_chk.c:
Set --update-state to on by default so that open_count is cleared if table was ok during check.
Update time when table was recovered/checked if --update-state was used.
Updated --help message with missing information.
storage/maria/maria_def.h:
Added wait counters
- Fix for LP#700623 "Aria recovery: ma_blockrec.c:3930: _ma_update_at_original_place: Assertion `block->org_bitmap_value == _ma_bitmap_get_page_bits(info, &info->s->bitmap, page)' failed"
- Issue was that when deleting a tail page where all index entries where full, the page was marked wrongly in the bitmap.
- If debug_assert_if_crashed_table is set, we now crash when we find Aria corrupted.
- Write more information if we find something wrong with the bitmap.
- Fixed that REPAIR also can fix wrong create_rename_lsn issues (a very unlikely event)
- Define STATE_CRASHED_FLAGS as set of all CRASHED flags (to simplify code)
storage/maria/ha_maria.cc:
Mark the normal page cache (not the page cache for the logs) so that we can request extra debugging for it.
Copy the value of debug_assert_if_crashed_table to maria_assert_if_crashed_table so that we can request a crash at exactly the point where we find Aria corrupted.
Use STATE_CRASHED_FLAGS
storage/maria/ma_bitmap.c:
Made bits_to_txt extern so that we can use this in maria_chk
Added extra information to the log files to be able to easier find bitmap failures in recovery. (When compiling with -DEXTRA_DEBUG_BITMAP)
Added _ma_get_bitmap_description() to request a clear text description of the bitmap.
Simplify _ma_check_bitmap_data(), as we know the bitmap pattern in the caller.
storage/maria/ma_blockrec.c:
In delete_head_or_tail(), fixed a bug where we sent wrong information to _ma_bitmap_set() if the directory was full for a page that should be freed.
This fixed LP#700623 (failure in bitmap found during recovery)
storage/maria/ma_blockrec.h:
Added definitions for _ma_get_bitmap_description() and bits_to_txt
storage/maria/ma_check.c:
Simplify call to _ma_check_bitmap_data().
Write more information if we find something wrong with the bitmap.
Moved getting clear text information about the bitmap to ma_bitmap.c::_ma_get_bitmap_description()
storage/maria/ma_checkpoint.c:
More asserts
storage/maria/ma_create.c:
Fix wrong create_rename_lsn during repair.
(Create_rename_lsn can be too big if someone restores an old maria_log_file after an Aria file was created)
storage/maria/ma_delete.c:
Call _ma_set_fatal_error() in case of crashed file
Remove not needed test of save_errno == HA_ERR_KEY_NOT_FOUND. (Handled by other code
storage/maria/ma_extra.c:
Call _ma_set_fatal_error() in case of crashed file
Reset share->bitmap.changed_not_flushed to not cause new ASSERTS to trigger.
Added _ma_file_callback_to_id() for writing share->id to log file in case of DEBUG logging.
storage/maria/ma_init.c:
Destroy also translog if it's readonly (as when called by maria_read_log -d)
storage/maria/ma_key.c:
Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_key_recover.c:
STATE_CRASHED -> STATE_CRASHED_FLAGS
storage/maria/ma_keycache.c:
Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_locking.c:
Call _ma_set_fatal_error() in case of crashed file.
Added _ma_set_fatal_error()
storage/maria/ma_open.c:
Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_page.c:
Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_pagecache.c:
Added extra information to log file to simply debugging of bitmap errors.
storage/maria/ma_pagecache.h:
Added extra_debug flag to allow marking of row and index cache for extra logging (for debugging).
storage/maria/ma_panic.c:
Flush both data and index blocks in case of HA_PANIC_CLOSE
Fixed wrong position of 'break'. (Not critical for MariaDB as MariaDB never uses this code)
storage/maria/ma_recovery_util.c:
Avoid writing extra not needed \n to DBUG log.
storage/maria/ma_rkey.c:
Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_search.c:
Call _ma_set_fatal_error() in case of crashed file
storage/maria/ma_static.c:
Define maria_assert_if_crashed_table
storage/maria/ma_update.c:
Call _ma_set_fatal_error() in case of crashed file.
The new code also avoids a problem where we before would print the error twice.
storage/maria/ma_write.c:
Call _ma_set_fatal_error() in case of crashed file
storage/maria/maria_chk.c:
STATE_CRASHED -> STATE_CRASHED_FLAGS
storage/maria/maria_def.h:
Added STATE_CRASHED_PRINTED to avoid giving error message about crash twice.
Added STATE_CRASHED_FLAGS to be able to easily detect and set all CRASHED related flags.
Added prototypes for new functions.
storage/myisam/mi_panic.c:
Fixed wrong position of 'break'. (Not critical for MariaDB as MariaDB never uses this code)
mysql-test/suite/handler/innodb.result:
Added test case
mysql-test/suite/handler/innodb.test:
Added test case
sql/handler.h:
Move setting/resetting of active_index to ha_index_init()/ha_index_end() to simplify handler functions index_init()/index_end()
Fixed that get_index() returns MAX_KEY if index is not inited (this fixed LP#697610)
storage/federated/ha_federated.cc:
Settting of active_index is not needed anymore
storage/maria/ma_pagecache.c:
Added error message if we have too little memory for Maria page cache
storage/maria/ma_pagecache.c:
- Ensure that we write also the last buffer in flush if buffer overflows.
- Don't set PCBLOCK_IN_FLUSH if we don't flush the page
- Added some new safety asserts.
- Don't delete pages without flushing that has had a tail or head information in pagecache_delete()
This fixes a case where REPAIR could find old deleted rows.
storage/maria/ha_maria.cc:
Remove calls to depricated function ha_statistic_increment
storage/maria/ma_blockrec.c:
Don't delete pages without flushing that has had a tail or head information in pagecache_delete()
storage/maria/ma_pagecache.c:
Added possibility to mark pages to not be deleted by pagecache_delete() without beeing flushed.
storage/maria/ma_pagecache.h:
Added new prototype
Fixed timing test failures.
Fixed a failure in the Aria engines page cache and log handler (found with maria.maria-big test)
- This could cause a core dump when deleting big blobs.
- Added test to end_pagecache() to verify that page cache was correctly used.
- inc_counter_for_resize_op and dec_counter_for_resize_op are called same number of times.
- All page cache blocks was properly deallocated (empty)
mysql-test/suite/innodb/t/innodb_bug38231.test:
Fixed timing issue (code comment says it all)
mysql-test/suite/innodb_plugin/t/innodb_bug38231.test:
Fixed timing issue (code comment says it all)
sql/debug_sync.cc:
Fixed compiler warning
storage/maria/ma_loghandler.c:
Fixed bug found by maria.maria-big test:
- Fixed race condition between update thread logging a very big blog and checkpoint thread.
storage/maria/ma_pagecache.c:
Added assert to ensure mutex was properly locked.
Added test to end_pagecache() to verify that page cache was correctly used.
- inc_counter_for_resize_op and dec_counter_for_resize_op are called same number of times.
- All page cache blocks was properly deallocated (empty)
In pagecache_delete_internal(), properly reset counters and pins if functions aborts.
Added missing inc_counter_for_resize_op() to pagecache_wait_lock().
Added missing dec_counter_for_resize_op() to pagecache_delete()
- Fixed bug in pagecache_delete_internal() when deleting block that was flushed by another thread (fixed bug when block->next_used was unexpectedly null)
Fixed some using uninitialized memory warnings found by valgrind.
mysql-test/t/information_schema_all_engines-master.opt:
Added options to make slow test run faster
sql/sp.cc:
Fixed valgrind warning.
sql/sql_show.cc:
Fixed valgrind warning.
storage/maria/ma_bitmap.c:
Fixed wrong call parameter to pagecache_unlock_by_link() that caused assert crash in pagecache
storage/maria/ma_pagecache.c:
Fixed possible error in pagecache_wait_lock() when hash_link was not set. (We never hit this issue, but could be possible when two threads updates the same page.
Fixed bug in pagecache_delete_internal() when deleting block that was flushed by another thread (fixed bug when block->next_used was unexpectedly null)
Cleanup: moved pagecache_pthread_mutex_unlock() over comments and asserts to be just before pagecache_fwrite()
storage/maria/unittest/ma_pagecache_single.c:
Check if we can flush write locked page for normal flush
Check that we can't flush read locked page with FLUSH_KEEP_LAZY (used by checkpoint)
- Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
- Fixed a rase condition when two threads calls external_lock and thr_lock() in different order. When this happend the transaction that called external lock first
and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
- Fixed that one can run maria_chk on an automatcally recovered tables without warnings about too small transaction id
- Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
- Fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.
client/mysqldump.c:
Add "" around error message to make it more readable
client/mysqltest.cc:
Free environment variables
mysql-test/r/mysqldump.result:
Updated results
mysql-test/r/openssl_1.result:
Updated results
mysql-test/suite/maria/r/maria-recover.result:
Updated results
mysql-test/suite/maria/r/maria3.result:
Updated results
mysql-test/suite/maria/t/maria3.test:
Added more test of temporary tables
storage/maria/ha_maria.cc:
Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
Start transaction in ma_block_get_status() instead of in ha_maria::external_lock().
- This fixes a rase condition when two threads calls external lock and thr_lock() in different order. When this happend the transaction that called external lock first and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
Store latest transaction id in controll file if recovery was done.
- This allows one to run maria_chk on an automatcally recovered tables without warnings about too small transaction id
storage/maria/ha_maria.h:
Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
storage/maria/ma_blockrec.h:
Added new function "_ma_block_get_status_no_versioning()"
storage/maria/ma_init.c:
Added hook to create trn in ma_block_get_status() if we are using MariaDB
storage/maria/ma_open.c:
Ensure we call _ma_block_get_status_no_versioning() for transactional tables without versioning (like tables with fulltext)
storage/maria/ma_pagecache.c:
Allow one to flush blocks that are pinned for read.
This fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.
storage/maria/ma_recovery.c:
Set maria_recovery_changed_data to 1 if recover changed something.
Set max_trid_in_control_file to max found trn if we found a bigger trn.
The allows will ensure that the control file is up to date after recovery which allows one to run maria_chk on the tables without warnings about too big trn
storage/maria/ma_state.c:
Call maria_create_trn_hook() in _ma_setup_live_state() instead of ha_maria::external_lock()
This ensures that 'state' and trn are in sync and thus fixes the race condition mentioned for ha_maria.cc
storage/maria/ma_static.c:
Added maria_create_trn_hook() and maria_recovery_changed_data
storage/maria/maria_def.h:
Added MARIA_HANDLER->external_ptr, which is used to hold MariaDB thd.
Added some new external variables
Removed reference to non existing function: maria_concurrent_inserts()
mysql-test/t/variables.test:
Reset delay_key_write, otherwise maria.maria test may fail
sql/set_var.cc:
Reset ha_open_options if one resets the delay_key_write variable.
Before there was no way to reset it without restarting mysqld, which caused some tests to fail
- Removed not needed casts
- Indentation fixes
- Changed some buffer variables to uchar, to be able to remove casts
storage/maria/ma_bitmap.c:
Removed not needed casts
storage/maria/ma_blockrec.c:
Removed not needed casts
storage/maria/ma_check.c:
Removed not needed casts.
Changed type of buffers to be able to remove casts
storage/maria/ma_create.c:
Removed not needed casts
storage/maria/ma_delete.c:
Removed not needed casts
storage/maria/ma_dynrec.c:
Removed not needed casts
storage/maria/ma_extra.c:
Removed not needed casts
storage/maria/ma_ft_boolean_search.c:
Removed not needed casts
storage/maria/ma_ft_nlq_search.c:
Removed not needed casts
storage/maria/ma_ft_parser.c:
Removed not needed casts
storage/maria/ma_loghandler.c:
Removed not needed casts
storage/maria/ma_open.c:
Removed not needed casts
storage/maria/ma_packrec.c:
Removed not needed casts
storage/maria/ma_pagecache.c:
Removed not needed casts
storage/maria/ma_preload.c:
Removed not needed casts
Removed not needed initialization
storage/maria/ma_rnext_same.c:
Removed not needed casts
storage/maria/ma_rt_index.c:
Removed not needed casts
storage/maria/ma_search.c:
Removed not needed casts
storage/maria/ma_sort.c:
Removed not needed casts
Removed not needed if
Fixed indentation
storage/maria/ma_statrec.c:
Removed not needed casts
storage/maria/ma_test2.c:
Removed not needed casts
storage/maria/ma_write.c:
Removed not needed casts
Indentation fixes
storage/maria/maria_chk.c:
Removed not needed casts
storage/maria/maria_pack.c:
Removed not needed casts
Added code to detect and give error when doing an insert into a view where we accessed fields in a not yet read table
Disabled test in subselect.test as the CHECK_OPTION for views doesn't work for insert.
This needs to be fixed properly later.
The problem with views are described in Bug #41760 Inserting into multiple-table views is not working
mysql-test/r/insert.result:
Fixed wrong usage of insert into view.
mysql-test/r/subselect.result:
Disabled wrong test (temporary)
mysql-test/suite/maria/r/maria.result:
Added test of size of table
mysql-test/suite/maria/t/maria.test:
Added test of size of table
mysql-test/t/insert.test:
Fixed wrong usage of insert into view
The bug is that during insert/update we currently don't read any of the referenced tables of the view.
This means that we can't get a value from another table to use as part of the update.
mysql-test/t/subselect.test:
Disabled not working test until someone has time to fix insert into view properly
Here we where refering to last used value in t2, which is wrong.
sql/sql_insert.cc:
Detect if we are trying to update one table in a view based on value in another, not yet read, table.
This fixes the problem discovered in insert.test
storage/maria/ma_blockrec.c:
Don't ignore not critical changes to the last page in the table.
We need to write the last page as otherwise we can during aborting of a row with a duplicate key get
state.data_file_length and the real length of file out of sync
storage/maria/ma_check.c:
Flush the page cache even if we got an error during zerofill.
(This fixes a call to assert() in case of a too short data file)
storage/maria/ma_pagecache.c:
Mark page as read when we do a write of a full page.
This fixes a bug when we got an error during read and then used direct write to page to update it
storage/maria/ma_state.c:
Restore info->lock.type after call to maria_versioning.
Fixed crash in maria_recover.test
storage/maria/maria_read_log.c:
Don't write thread id in debug log. (Not needed as maria_read_log is a single treaded program)
Scenario of the BUG#40731 ("Maria: hang (probably in page cache) under concurrency"):
T1: Disable logging for the table
T1: Start inserting into the table
T2: Tries to lock the table so waits.
T2: Tries unlock and relock during the process see that the table has disabled logging and reenables it
T1: Got DBUG_ASSERT because suddenly start use table with transaction switched on which is not expected during bulk insert
storage/maria/ma_pagecache.c:
Page type print added for debugging purposes.
storage/maria/ma_recovery.c:
Check that it was this thred which switched off logging (transactional mode).
storage/maria/maria_def.h:
Flag for controling which thread switched off transactiona mode for the table added.
storage/maria/ma_pagecache.c:
Fixed ability to read without read lock acquiring.
storage/maria/unittest/CMakeLists.txt:
New unit test which tests simple read and prolonged writes consistency added.
storage/maria/unittest/Makefile.am:
New unit test which tests simple read and prolonged writes consistency added.
storage/maria/unittest/ma_pagecache_rwconsist2.c:
New unit test which tests simple read and prolonged writes consistency added.
in write_changed_bitmap(), and page cache forbids that. Here we make the page
cache more relaxed. Original patch by Sanja, simplified by me as limited to
not-locked. See comment of ma_bitmap.c.
With that, maria_stress.yy runs until hitting BUG 39665.
storage/maria/ma_bitmap.c:
A thread which unpins bitmap pages in _ma_bitmap_unpin_all() sometimes
hit an assertion in the page cache (info!=0 in remove_pin()) which states
that you can unpin/unlock only what *you* have pinned/locked.
Fixed by setting the new last parameter of pagecache_unlock_by_link()
to TRUE in _ma_bitmap_unpin_all().
storage/maria/ma_blockrec.c:
new prototype and splitting assertion in three (3rd one fires: BUG 39665)
storage/maria/ma_check.c:
new prototype
storage/maria/ma_key_recover.c:
new prototype
storage/maria/ma_loghandler.c:
new prototype
storage/maria/ma_pagecache.c:
Allow a thread to unpin, with pagecache_unlock_by_link(), a non-locked page pinned by others.
This is a hack for _ma_bitmap_unpin_all() which needs to unpin pages which were
pinned by other threads in write_changed_bitmap().
storage/maria/ma_pagecache.h:
new prototype
storage/maria/ma_preload.c:
new prototype
storage/maria/unittest/ma_pagecache_rwconsist.c:
new prototype
storage/maria/unittest/ma_pagecache_single.c:
new prototype
Fixed compiler warning
Ensure that pagecache returns correct error number
storage/maria/ma_pagecache.c:
Ensure that block->error contain error code from my_errno (not return value from pread/pwrite)
Went through usage of my_pread/my_pwrite and ensured that result was threated correctly
Fixed some wrong usage of my_bool
Ensure that pagecache gives correct error number even if error for block happend
mysys/my_pread.c:
Indentation fix
storage/maria/ha_maria.cc:
filenames changed to be of type LEX_STRING
storage/maria/ma_check.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_checkpoint.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_create.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_dbug.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_delete.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_info.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_keycache.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_locking.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_loghandler.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_open.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_pagecache.c:
Store error number for last failed operation in the page block
This should fix some asserts() when errno was not properly set after failure to read block in another thread
storage/maria/ma_recovery.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_update.c:
filenames changed to be of type LEX_STRING
storage/maria/ma_write.c:
filenames changed to be of type LEX_STRING
storage/maria/maria_def.h:
filenames changed to be of type LEX_STRING
storage/maria/maria_ftdump.c:
filenames changed to be of type LEX_STRING
storage/maria/maria_pack.c:
filenames changed to be of type LEX_STRING
Recovery of R-tree and fulltext indices.
Fix for BUG#35551 "Maria: crash in REPAIR TABLE/ENABLE KEYS if using
repair-with-keycache method".
Fix for bug (see ma_rt_index.c) where we could have a wrong
page_link pointer causing wrong memory access during some R-tree
index insert/delete.
Making ma_rt_test work again (it had been neglected over time) and
adding options (record type etc) to prepare it for integration into
ma_test_all-t (but there is BUG#36321 about "ma_rt_test -M" crash)
mysql-test/r/maria.result:
correct result
mysql-test/t/maria.test:
now we get no error
storage/maria/ma_blockrec.c:
delete_dir_entry() and delete_head_or_tail() don't use info->keyread_buff.
ma_get_length() does not change **packet, marking it with 'const' to
remove some casts in callers of this function. The
(const uchar**)&header casts will be removed when Monty changes 'header'
to const uchar*.
_ma_apply_redo_purge_row_head_or_tail() sets 'buff' from pagecache_read()
so its initialization was superfluous.
storage/maria/ma_check.c:
Fix for BUG#35551 "Maria: crash in REPAIR TABLE/ENABLE KEYS if using repair-with-keycache method"
(see comment in code)
storage/maria/ma_create.c:
FULLTEXT and SPATIAL indices have logging now, they are recoverable.
storage/maria/ma_delete.c:
Logging done by _ma_ck_delete() is moved to a function
(_ma_write_undo_key_delete()), for reusal by R-tree logging.
_ma_log_delete() is made non-static for same
reason, and some of its parameters are made pointers to const.
Removed wrong comment ("Note that for delete key" etc, contradicted by
code and comment "Log also position to row" a few lines above)
storage/maria/ma_ft_update.c:
unneeded cast, comment for future
storage/maria/ma_key_recover.c:
Comment about possible deadlock.
Write bad page to DBUG trace if KEY_OP_CHECK founds bad CRC.
Support operation KEY_OP_MULTI_COPY.
When we execute, in UNDO phase, UNDO_KEY_DELETE|INSERT, we must call
the proper key insertion|deletion depending on if this is R-tree
or B-tree.
Explanation of of _ma_[un]lock_key_del() work, maybe useful for
mortals like me.
storage/maria/ma_key_recover.h:
change of prototypes
storage/maria/ma_loghandler.h:
New operation which can be stored in REDO_INDEX log records: KEY_OP_MULTI_COPY
storage/maria/ma_page.c:
Comments
storage/maria/ma_pagecache.c:
typo
storage/maria/ma_rt_index.c:
Fix for bug: the page_link pointer in maria_rtree_insert_req()
could be wrong when we set its 'changed' member; for the solution
see ma_key_recover.h. It is needed only in cases when we manipulate
several pages.
Logging of changes done to pages by key insert/delete.
maria_rtree_delete()'s main work is moved to a new function
maria_rtree_real_delete(), which is used by maria_rtree_delete()
and by applying of UNDO_KEY_INSERT.
storage/maria/ma_rt_index.h:
new prototypes and macros for ma_rt_index.c
storage/maria/ma_rt_key.c:
Logging of maria_rtree_add_key() and maria_rtree_delete_key().
When inserting, split is necessary if there is not enough room for key:
take checksum's occupied space in this calculation.
storage/maria/ma_rt_key.h:
new prototypes (those functions need to know the page's id
because they do logging)
storage/maria/ma_rt_mbr.c:
Comments about what the functions change.
storage/maria/ma_rt_split.c:
maria_rtree_split_page() needs to know the page's id, because
it does logging.
Logging of what a split operation does to the split page (see
comment of _ma_log_rt_split(): moves of keys inside the page,
sometimes insertion of the new key, and shrinking of the page)
and to the new page (receives some keys from split page, and
sometimes the new key).
storage/maria/ma_rt_test.c:
ma_rt_test had been forgotten when maria_rkey() was changed some months ago
(0->HA_WHOLE_KEY change), and when calls to maria_rnd(,,HA_OFFSET_ERROR)
were rewritten to maria_scan() calls (which implies maria_scan_init()).
The 'max_i' change is to adapt to the fact that maria_scan() does
not return deleted records for BLOCK_RECORD but does so for other formats;
the initial code assumed a certain number of deleted records would be
returned, we change it to rather count only non-deleted ones.
We also add more features to this test, like ma_test1 (the plan
is to run ma_rt_test in ma_test_all-t):
options to choose records' format, table checksum, transactions,
checkpoints, end at specific stages, abort without committing,
and debug trace.
storage/maria/ma_test1.c:
MY_INIT() does my_init().
storage/maria/ma_write.c:
Logging done by _ma_ck_write_btree_with_log() is moved to a function
(_ma_write_undo_key_insert()), for reusal by R-tree logging.
_ma_log_new() and _ma_log_change() are made non-static for same
reason. Some parameters of logging functions are made pointers to const.
If EXTRA_DEBUG_KEY_CHANGES, we now log CRC in _ma_log_change() too
(better checks, bigger record).
storage/maria/maria_read_log.c:
Program takes no arguments, bail out if any, instead of silently discarding them
storage/myisam/rt_test.c:
rt_test had been forgotten when mi_rkey() was changed some months ago
(0->HA_WHOLE_KEY change).
The 'max_i' change is to make it symmetric with ma_rt_test.c
mysql-test/r/maria-gis-rtree-dynamic.result:
correct result
mysql-test/r/maria-gis-rtree-trans.result:
correct result
mysql-test/r/maria-recovery-rtree-ft.result:
almost correct result (hitting BUG# in the end)
mysql-test/t/maria-gis-rtree-dynamic.test:
test R-tree & dynamic row format
mysql-test/t/maria-gis-rtree-trans.test:
Test R-tree and page row format and transactional
mysql-test/t/maria-recovery-rtree-ft-master.opt:
usual options for recovery testing
mysql-test/t/maria-recovery-rtree-ft.test:
test of recovery of R-tree and fulltext indices.
