The issues was:
- For some tables with a lot of not packed fields, we didn't allocate enough memory in head page which caused DBUG_ASSERT's
- Removed wrong DBUG_ASSERT()
- Fixed a problem with underflow() where it generates a key page where all keys didn't fit.
- Max key length is now limited by block_size/3 (was block_size /2). This is required for underflow() to work with packed keys.
mysql-test/lib/v1/mysql-test-run.pl:
Remove --alignment=8 as this doesn't work on 64 bit systems
mysql-test/suite/maria/r/small_blocksize.result:
Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize-master.opt:
Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize.test:
Test case for Aria bug
storage/maria/ha_maria.cc:
Fixed comment
storage/maria/ma_bitmap.c:
Fixed wrong variable usage in find_where_to_split_row() where we allocated too little memory for head page.
We did not take into account space for head extents (long VARCHAR) when trying to split row on head page. This caused us to allocate too little space from bitmap which lead to ASSERT failures later.
storage/maria/ma_blockrec.c:
Made some argument const (to ensure they was not accidently changed)
Removed wrong DBUG_ASSERT()
storage/maria/ma_blockrec.h:
Removed not used variable
storage/maria/ma_delete.c:
Added my_afree() in case of error
More comments and DBUG_ASSERT() for underflow()
storage/maria/ma_open.c:
Make keyinfo->underflow_block_length smaller for packed keys. This has to be done as for long packed keys, underflow() otherwise generates a key page where all keys didn't fit.
storage/maria/ma_page.c:
New DBUG_ASSERT()
storage/maria/ma_write.c:
Fixed comment
storage/maria/maria_def.h:
We have to have space for at least 3 keys on a key page.
(Otherwise the underflow() code doesn't work for packed keys, even when we have an underflow() for an empty key page)
Fixed lp:814237: Wrong mutex usage in Aria
storage/maria/ma_bitmap.c:
Added call to _ma_bitmap_mark_file_changed() when flushing bitmap. This fixes lp:814237
The bug happens when one uses MAX_ROWS=# with Aria & row_format=page and one insert more than # rows.
mysql-test/mysql-test-run.pl:
Ignore table is full error messages
mysql-test/suite/maria/r/max_length.result:
Test case for 'Table is full'
mysql-test/suite/maria/t/max_length.test:
Test case for 'Table is full'
storage/maria/ma_bitmap.c:
Ensure that we don't allocate bits outside of max_data_file_size.
Adjust max_data_file_size based on bitmap alignments.
Backport fix to adjust wrong first_bitmap_with_space.
storage/maria/ma_blockrec.c:
Calculate value of max_data_file_length
storage/maria/ma_blockrec.h:
Updated prototype for _ma_bitmap_init()
storage/maria/ma_check.c:
Give warnings if file sizes are above max file sizes.
Give more warnings in case of errors.
Have maria_chk write if table is recreated.
storage/maria/ma_create.c:
Better calculation of max_data_file_length and thus data pointer length.
Fixes some wrong pointer lengths when using MAX_ROWS=#
storage/maria/ma_open.c:
Removed duplicate assigment.
Use block size from file instead of global variable.
storage/maria/maria_chk.c:
Remove -1 from printed file length
storage/maria/maria_def.h:
Update struct st_maria_file_bitmap
- Fixed assert in transaction log handler when aria_check was run on block-record table that was much bigger than expected.
- Fixed warnings about wrong mutex order between bitmap and intern_lock
- Fixed error in bitmap that could cause two rows to use same block for a block record.
- Fixed wrong test that could cause error if last page for a bitmap was used by a blob.
- Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks (very unlikely case).
mysql-test/suite/maria/r/maria-recovery3.result:
Updated results
sql/mysqld.cc:
Allow mi_check() to send information messages for log file
storage/maria/ma_bitmap.c:
Fixed problem with wrong mutex order when bitmap was the first page that was flushed out of page cache
- Fixed by introducing _ma_bitmap_mark_file_changed() that marks file changed without a bitmap lock.
- Fixed one case in _ma_change_bitmap_page() where we didn't mark the bitmap changed. This could cause to rows to reuse same block if this was the only change to the bitmap.
- Split _ma_bitmap_get_page_bits() in two parts to not take a bitmap lock when we already have it
- Fixed bug in _ma_bitmap_set_full_page_bits() that caused an error if last page for a bitmap was used by a blob
storage/maria/ma_check.c:
Better handling of wrong file length.
Fixed bug when we tried to write to transaction log when it was not opened (happened when block record file was bigger than expected)
storage/maria/ma_pagecache.c:
Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks:
- In link_block() mark a block given to another thread with PCBLOCK_REASSIGNED to ensure that no other threads can start re-using the block
before the thread that requsted a block.
- In free_block(), don't reset status for a block that is in re-assign by link_block() (we don't want to loose the PCBLOCK_REASSIGNED flag).
- Added call to wait_for_flush() when we got a new block in find_block() to ensure that we don't use a block that is beeing flushed by another thread.
- Moved setting of hits_left and last_hit_time in find_block() to where we assign the block.
Code cleanup and making code uniform:
- Changed a lot of KEYCACHE_DBUG_PRINT to use DBUG_PRINT
- Streamlined all reporting of 'signal' and 'wait' between threads to be identical.
- Use thread name instead of thread number (for each match against --debug)
- Added more DBUG_ENTER, DBUG_PRINT and DBUG_ASSERT()
- Added more comments
storage/myisam/ha_myisam.cc:
Only print information about that we make a backup if we are really making a backup.
storage/myisam/mi_check.c:
Inform mysqld that we are creating a backup of the data file (for inclusion in error log).
Added option to aria_read_log to crash recovery at certain points in the recovery process.
Fixed bug that caused future recovery attempts to fail if we got a crash/got killed during closing of tables at end of recovery process.
mysql-test/mysql-test-run.pl:
Don't abort if 'var' points to stale place; Just remove it.
mysql-test/suite/maria/r/maria.result:
Fixed wrong indentation
mysql-test/t/partition_error.test:
Fixed portability problem with partiton_error.test
storage/maria/ma_close.c:
More DBUG_PRINT info
storage/maria/ma_pagecache.c:
Copy flush_log_callback when writing to page cache. This fixes problem in recovery when switching from mode of file
storage/maria/ma_recovery.c:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/ma_recovery.h:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/maria_chk.c:
Align aria_chk -d output
Don't write warning Aria table '...' is usable but should be fixed if the table was before marked as crashed but now is ok
storage/maria/maria_read_log.c:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
mysql-test/suite/maria/t/maria3.test:
Fix when compiling with safemalloc
storage/maria/ha_maria.cc:
Fix that MAX_FILE_SIZE is not limited on 32 bit machines.
A lot of small fixes and new test cases.
client/mysqlbinlog.cc:
Cast removed
client/mysqltest.cc:
Added missing DBUG_RETURN
include/my_pthread.h:
set_timespec_time_nsec() now only takes one argument
mysql-test/t/date_formats.test:
Remove --disable_ps_protocl as now also ps supports microseconds
mysys/my_uuid.c:
Changed to use my_interval_timer() instead of my_getsystime()
mysys/waiting_threads.c:
Changed to use my_hrtime()
sql/field.h:
Added bool special_const_compare() for fields that may convert values before compare (like year)
sql/field_conv.cc:
Added test to get optimal copying of identical temporal values.
sql/item.cc:
Return that item_int is equal if it's positive, even if unsigned flag is different.
Fixed Item_cache_str::save_in_field() to have identical null check as other similar functions
Added proper NULL check to Item_cache_int::save_in_field()
sql/item_cmpfunc.cc:
Don't call convert_constant_item() if there is nothing that is worth converting.
Simplified test when years should be converted
sql/item_sum.cc:
Mark cache values in Item_sum_hybrid as not constants to ensure they are not replaced by other cache values in compare_datetime()
sql/item_timefunc.cc:
Changed sec_to_time() to take a my_decimal argument to ensure we don't loose any sub seconds.
Added Item_temporal_func::get_time() (This simplifies some things)
sql/mysql_priv.h:
Added Lazy_string_decimal()
sql/mysqld.cc:
Added my_decimal constants max_seconds_for_time_type, time_second_part_factor
sql/table.cc:
Changed expr_arena to be of type CONVENTIONAL_EXECUTION to ensure that we don't loose any items that are created by fix_fields()
sql/tztime.cc:
TIME_to_gmt_sec() now sets *in_dst_time_gap in case of errors
This is needed to be able to detect if timestamp is 0
storage/maria/lockman.c:
Changed from my_getsystime() to set_timespec_time_nsec()
storage/maria/ma_loghandler.c:
Changed from my_getsystime() to my_hrtime()
storage/maria/ma_recovery.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/maria/unittest/trnman-t.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/xtradb/handler/ha_innodb.cc:
Added support for new time,datetime and timestamp
unittest/mysys/thr_template.c:
my_getsystime() -> my_interval_timer()
unittest/mysys/waiting_threads-t.c:
my_getsystime() -> my_interval_timer()
- Fixed bug lp:624099 ma_close.c:75: maria_close: Assertion `share->in_trans == 0' failed on UNLOCK TABLES
- Fixed bug that caused table to be marked as not closed (crashed) during recovery testing.
- Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
- Added checks to ensure that bitmap and internal mutex are always taken in right order.
- For transactional tables, only mark the table as changed before page for table is to be written to disk (and thus the log is flushed).
This speeds up things a bit and fixes a problem where open_count was incremented on disk but there was no log entry to fix it during recovery -> table was crashed.
- Fixed a bug in repair() where table was not automaticly repaired.
- Ensure that state->global_changed, share->changed and share->state.open_count are set and reset properly.
- Added option --ignore-control-file to maria_chk to be able to run maria_chk even if the control file is locked.
mysql-test/suite/maria/r/maria-recover.result:
Test result changed as we now force checkpoint before copying table, which results in pagecache getting flushed and we have more rows to recover.
mysql-test/suite/maria/r/maria.result:
Added new tests
mysql-test/suite/maria/t/maria-recover.test:
Force checkpoint before copying table.
This is needed as now the open-count is increased first when first page is flushed.
mysql-test/suite/maria/t/maria.test:
Added tests to verify fix for lp:624099
storage/maria/ha_maria.cc:
Use table->in_use instead of current_thd (trivial optimization)
Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
More DBUG_ASSERT()
Reset locked tables count after locked tables have been moved to new transaction. This fixed lp:624099
storage/maria/ma_bitmap.c:
Temporarly unlock bitmap mutex when calling _ma_mark_file_changed() and pagecache_delete_pages() to ensure right mutex lock order.
Call _ma_bitmap_unpin_all() when bitmap->non_flusable is set to 0. This fixed a case when bitmap was not proparly unpinned.
More comments
Added DBUG_ASSERT() for detecting wrong share->bitmap usage
storage/maria/ma_blockrec.c:
More DBUG_ASSERT()
Moved code around in _ma_apply_redo_insert_row_head_or_tail() to make things safer on error conditions.
storage/maria/ma_check.c:
Changed parameter for _ma_set_uuid()
Corrected test for detecting if we lost many rows. This fixed some cases where auto-recovery failed.
share->changed need to be set if state.open_count is changed.
Removed setting of share->changed= 0 as called function sets it.
storage/maria/ma_close.c:
- Added code to properly decrement open_count and have it written by _ma_state_info_write() for transactional tables.
(This is more correct and also saves us one extra write by _ma_decrement_open_count() at close.
- Added DBUG_ASSERT() to detect if open_count is wrong at maria_close().
storage/maria/ma_delete.c:
Updated argument to _ma_mark_file_changed()
storage/maria/ma_delete_all.c:
Updated argument to _ma_mark_file_changed()
For transactional tables, call _ma_mark_file_changed() after log entry has been written (to allow recover to fix open_count)
Reset more needed variables in _ma_reset_status()
storage/maria/ma_delete_table.c:
Moved deletion of Aria files to maria_delete_table_files().
Remove RAID usage (old not working code)
storage/maria/ma_extra.c:
Set share->changed=1 when state needs to be updated on disk.
Don't reset share->changed after call to _ma_state_info_write() as this calls sets share->changed.
Set share->state.open_count to 1 to force table to be auto repaired if drop fails.
Set share->global_changed before call to _ma_flush_table_files() to ensure that we don't try to mark the table changed during flush.
Added DBUG_ENTER
storage/maria/ma_locking.c:
Split _ma_mark_file_changed() into two functions to delay marking transactional tables as changed on disk until first disk write.
Added argument to _ma_decrement_open_count() to tell if we should call ma_lock_database() or not.
Don't decrement open count for transactional tables during _ma_decrement_open_count(). This will be done during close.
Changed parameter for _ma_set_uuid()
storage/maria/ma_open.c:
Set share->open_count_not_zero_on_open if state.open_count is not zero.
This is needed for DBUG_ASSERT() in maria_close() that is there to enforce that open_count is always 0 at close.
This test doesn't however work for tables that had open_count != 0 already on disk (ie, crashed tables).
Enforce right mutex order for share->intern_lock and share->bitmap.bitmap_lock
Don't set share->changed to 0 if share->state.open_count != 0, as state needs to be be written at close
storage/maria/ma_pagecache.c:
Moved a bit of code in find_block() to avoid one if.
More DBUG_ASSERT()
(I left a comment in the code for Sanja to look at; The code probably works but we need to check if it's optimal)
storage/maria/ma_pagecrc.c:
For transactional tables, just before first write to disk, but after log is flushed, mark the file changed.
This fixes some cases where recovery() did not detect that table was marked as changed and could thus not recover the marker.
storage/maria/ma_recovery.c:
Set share->changed when share->global_changed is set.
storage/maria/ma_update.c:
Updated parameter for _ma_mark_file_changed()
storage/maria/ma_write.c:
Updated parameter for _ma_mark_file_changed()
storage/maria/maria_chk.c:
Added option --ignore-control-file to be able to run maria_chk even if the control file is locked.
storage/maria/maria_def.h:
Updated function prototypes.
Added open_count_not_zero_on_open to MARIA_SHARE.
storage/myisam/ha_myisam.cc:
current_thd -> table->in_use
- Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables.
- Fixed compiler warnings
- Added a name for each thr_lock to get better error messages (This is needed to find out why 'archive.test' sometimes fails)
BUILD/SETUP.sh:
Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables.
BUILD/build_mccge.sh:
Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables.
client/mysqltest.cc:
Fixed bug in remove_files_wildcards (the orignal code never removed anything)
extra/libevent/devpoll.c:
Fixed compiler warning
include/thr_lock.h:
Added a name for each thr_lock to get better error messages.
mysql-test/suite/maria/t/maria3.test:
Speed up test.
mysys/thr_lock.c:
Added a name for each thr_lock to get better error messages.
Added a second 'check_locks' to find if something goes wrong in 'wake_up_waiters'.
sql/lock.cc:
Added a name for each thr_lock to get better error messages.
storage/xtradb/fil/fil0fil.c:
Fixed compiler warning
Added logging of all possible fatal table errors if --log-warnings set to > 1
mysql-test/extra/rpl_tests/rpl_EE_err.test:
Safety fix
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/r/archive.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/r/csv.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/maria/r/maria-autozerofill.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/maria/t/maria-autozerofill.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/maria/t/maria-recover.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/parts/t/partition_recover_myisam.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/r/rpl_bug38694.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/r/rpl_idempotency.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/r/rpl_ignore_table.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/r/rpl_row_conflicts.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/t/rpl_bug38694.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/t/rpl_idempotency.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/t/rpl_ignore_table.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/t/rpl_row_basic_11bugs.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/t/rpl_row_conflicts.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/t/archive.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
mysql-test/t/csv.test:
Added suppression of possible error message (so that one can run test with --log-warnings=2)
sql/handler.cc:
If running with --assert-of-crashed-table or --log-warnings > 1 then print engine error to log
sql/sql_select.cc:
Disable not initialized warning from gcc
strings/Makefile.am:
Fixed compiler error on Solaris 10 (duplicate strmov() function)
This patch adds options to annotate the binlog (and the mysqlbinlog
output) with the original SQL query for queries that are logged
using row-based replication.
The patch also fixes a race in rpl_stop_slave.test.
On machines with lots of CPU and memory, something like `mtr --parallel=10`
can speed up the test suite enormously. However, we have a few test cases
that run for long (several minutes), and if we are unlucky and happen to
schedule those towards the end of the test suite, we end up with most
workers idle while waiting for the last slow test to end, significantly
delaying the finish of the entire suite.
Improve this by marking the offending tests as taking "long", and trying
to schedule those tests early. This reduces the time towards the end of
the test suite run where some workers are waiting with nothing to do for
the remaining workers each to finish their last test.
Also, the rpl_stop_slave test had a race which could cause it to take
a 300 seconds debug_sync timeout; this is fixed.
Testing on a 4-core 8GB machine, this patch speeds up the test suite with
around 30% for --parallel=10 (debug build), allowing to run the entire
suite in 5 minutes.
BUILD/compile-pentium64:
Added --with-zlib-dir=bundled when doing static build.
mysql-test/suite/maria/r/maria.result:
Added test case
mysql-test/suite/maria/t/maria.test:
Added test case
storage/maria/ma_blockrec.c:
We need to flush blob pages to disk to ensure they overwrite any reused head/tail pages.
If not, REPAIR will find rows that was previously deleted.
- Fixed problem with oqgraph and 'make dist'
Note that after this merge we have a problem show in join_outer where we examine too many rows in one specific case (related to BUG#57024).
This will be fixed when mwl#128 is merged into 5.3.
maria_chk & maria_read_log now reads block size from control file.
mysql-test/suite/maria/r/maria.result:
Updated results after trivial change of maria_chk's output
storage/maria/ma_bitmap.c:
More DBUG_PRINT
storage/maria/ma_blockrec.c:
Fixed bug that we didn't mark page full in bitmap if directory is full
storage/maria/ma_check.c:
Write out if directory is full for errors in bitmap
storage/maria/ma_control_file.c:
Don't give error for wrong block size if block size is 0
storage/maria/maria_chk.c:
Read block size from control file
In case of -dvv, write also out bitmap information (good for debugging)
storage/maria/maria_read_log.c:
Read block size from control file
Fixed that maria_read_log works with different page size than TRANSLOG_PAGE_SIZE
Fixed compiler warnings
Disabled some tests that doesn't work on windows (uses shell tools or strange characters)
client/mysqlshow.c:
Fixed compiler warnings
client/mysqlslap.c:
Fixed compiler warnings
mysql-test/mysql-test-run.pl:
Use an error file instead of /dev/null
(Fixes problem on Windows and the output may be usefull)
mysql-test/suite/maria/r/maria.result:
Test case for bug LP#613408 (not complete)
mysql-test/suite/maria/t/maria.test:
Test case for bug LP#613408 (not complete)
mysql-test/suite/percona/percona_log_slow_slave_statements-and-use_global_long_query_time.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_log_slow_slave_statements.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-control_global_slow.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-log_slow_filter.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-log_slow_verbosity.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-long_query_time.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-microseconds_in_slow_query_log.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-min_examined_row_limit.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-use_global_long_query_time.test:
This test doesn't work one windows (needs shell tools)
mysql-test/t/ctype_filesystem.test:
This test doesn't work one windows (problem with character sets)
mysql-test/t/events_time_zone.test:
Use longer times to get predictable tests
mysql-test/t/show_check-master.opt:
set long query time to get more predictable tests
storage/maria/ma_check.c:
Restore info->s->lock_key_trees after repair.
Disable logging to temp tables for all repair cases (safety fix)
storage/maria/ma_state.c:
Ensurethat info->state_start doesn't point to freed memory. (Could happen after running an internal repair to fast create indexes)
storage/maria/trnman.c:
Added longer comment
Fixed compiler warnings
client/mysqlslap.c:
Fixed compiler warnings
mysql-test/suite/maria/r/maria.result:
Test case for LP#614265
mysql-test/suite/maria/t/maria.test:
Test case for LP#614265
mysql-test/suite/pbxt/t/skip_name_resolve-master.opt:
Ensure that we get restart before test (as test uses show processlist)
sql/handler.cc:
Added cloned marker if clone was called (for safety checks & debugging)
sql/handler.h:
Added cloned marker if clone was called (for safety checks & debugging)
storage/maria/ha_maria.cc:
In clone call, set file->trn if cloned file had this set. This is needed as maria_create_trn_for_mysql() and thus file->trn is never set for cloned table.
Ensure that file->trn is properly reset after calls to repair/check/zerofill.
Increment locked table count if file->trn is set (as we decrement this in the unlock call)
tests/mysql_client_test.c:
Fixed compiler warnings
Added 'Aria' as an alias for the Maria storage engine. Real rename is happening in 5.2
sql/handler.cc:
Added 'Aria' as an alias for the Maria storage engine.
More DBUG_PRINT (to simplify future debugging)
Aria: Added STATE_IN_REPAIR, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
Aria: Some trivial speedup optimization
Aria: Better warning if table was marked crashed by unfinnished repair
mysql-test/lib/v1/mysql-test-run.pl:
Fix so one can run RQG
mysql-test/suite/maria/r/maria-recovery2.result:
Update for new error message.
mysys/stacktrace.c:
Fixed compiler warning
storage/maria/ha_maria.cc:
More DBUG_PRINT
Added STATE_IN_REPAIR flag, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
Don't log query for dropping temporary table.
storage/maria/ha_maria.h:
Added prototype for drop_table()
storage/maria/ma_blockrec.c:
More DBUG_PRINT
Make read_long_data() inline for most cases. (Trivial speedup optimization)
storage/maria/ma_check.c:
Better warning if table was marked crashed by unfinnished repair
storage/maria/ma_open.c:
More DBUG_PRINT
storage/maria/ma_recovery.c:
Give warning if found crashed table.
Changed warning for tables that can't be opened.
storage/maria/ma_recovery_util.c:
Write warnings to DBUG file
storage/maria/maria_chk.c:
Added STATE_IN_REPAIR flag, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
storage/maria/maria_def.h:
Added maria_mark_in_repair(x)
storage/maria/maria_read_log.c:
Added option: --character-sets-dir
storage/maria/trnman.c:
By default set min_read_from to max value.
This allows us to remove TRN:s from rows during recovery to get more space.
This fixes bug LP#602604: RQG: ma_blockrec.c:6187: _ma_apply_redo_insert_row_head_or_tail: Assertion `0' failed on Maria engine recovery
Fixed bug: LP#603026 RQG: pagecache_read: Assertion `pageno < ((1ULL) << 40)' on OPTIMIZE TABLE of a Maria table
client/mysql.cc:
Removed Oracle copyright from stdout, as Oracle doesn't have copyright to all code in this file.
Ignore --comments at start of command line. This allows one to more easily run mysqltest tests trough the command line.
mysql-test/suite/maria/r/optimize.result:
Added test for LP#603026
mysql-test/suite/maria/t/optimize.test:
Added test for LP#603026
sql/net_serv.cc:
Removed DBUG_ASSERT(), as this code can happen during testing.
storage/maria/ma_check.c:
Fixed bug: LP#603026 RQG: pagecache_read: Assertion `pageno < ((1ULL) << 40)' on OPTIMIZE TABLE of a Maria table
The problem was duplicated memory usage with long packed keys.
- Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
- Fixed a rase condition when two threads calls external_lock and thr_lock() in different order. When this happend the transaction that called external lock first
and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
- Fixed that one can run maria_chk on an automatcally recovered tables without warnings about too small transaction id
- Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
- Fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.
client/mysqldump.c:
Add "" around error message to make it more readable
client/mysqltest.cc:
Free environment variables
mysql-test/r/mysqldump.result:
Updated results
mysql-test/r/openssl_1.result:
Updated results
mysql-test/suite/maria/r/maria-recover.result:
Updated results
mysql-test/suite/maria/r/maria3.result:
Updated results
mysql-test/suite/maria/t/maria3.test:
Added more test of temporary tables
storage/maria/ha_maria.cc:
Changed default recovery mode from OFF to NORMAL to get automatic repair of not properly closed tables.
Start transaction in ma_block_get_status() instead of in ha_maria::external_lock().
- This fixes a rase condition when two threads calls external lock and thr_lock() in different order. When this happend the transaction that called external lock first and thr_lock() last did not see see the rows from the other transaction, even if if it had to wait in thr_lock() for other to complete.
Store latest transaction id in controll file if recovery was done.
- This allows one to run maria_chk on an automatcally recovered tables without warnings about too small transaction id
storage/maria/ha_maria.h:
Don't give warning that crashed table could not be repaired if repair was disabled (and thus not run)
storage/maria/ma_blockrec.h:
Added new function "_ma_block_get_status_no_versioning()"
storage/maria/ma_init.c:
Added hook to create trn in ma_block_get_status() if we are using MariaDB
storage/maria/ma_open.c:
Ensure we call _ma_block_get_status_no_versioning() for transactional tables without versioning (like tables with fulltext)
storage/maria/ma_pagecache.c:
Allow one to flush blocks that are pinned for read.
This fixed a error result from flush_key_cache() which caused a DBUG_ASSERT() when one was using concurrent reads on non transactional tables that was updated.
storage/maria/ma_recovery.c:
Set maria_recovery_changed_data to 1 if recover changed something.
Set max_trid_in_control_file to max found trn if we found a bigger trn.
The allows will ensure that the control file is up to date after recovery which allows one to run maria_chk on the tables without warnings about too big trn
storage/maria/ma_state.c:
Call maria_create_trn_hook() in _ma_setup_live_state() instead of ha_maria::external_lock()
This ensures that 'state' and trn are in sync and thus fixes the race condition mentioned for ha_maria.cc
storage/maria/ma_static.c:
Added maria_create_trn_hook() and maria_recovery_changed_data
storage/maria/maria_def.h:
Added MARIA_HANDLER->external_ptr, which is used to hold MariaDB thd.
Added some new external variables
Removed reference to non existing function: maria_concurrent_inserts()
mysql-test/suite/maria/r/maria3.result:
Updated test results
mysql-test/suite/maria/t/maria3.test:
Don't show maria_used_for_temp_tables, as it's value is depending on configure options
mysql-test/suite/maria/r/group_commit.result:
Test of gruoup commit.
mysql-test/suite/maria/r/maria3.result:
The new variables added.
mysql-test/suite/maria/t/group_commit.test:
Test of group commit.
randgen:
Randon query generator tests.
randgen/conf:
Random query generator tests.
randgen/conf/maria_group_commit.yy:
Random query generator test for group commit.
storage/maria/ha_maria.cc:
New variables and support procedures for group commit added.
storage/maria/ma_init.c:
Correct shutdown of group commit service thread and group commit.
storage/maria/ma_loghandler.c:
Group commit added.
Initialization of variables for embeded server edded.
storage/maria/ma_loghandler.h:
Group commit types and routines.
myisam-recover options changed from OFF to 'DEFAULT' to get less change of data loss when using MyISAM.
(The disadvantage is that changed MyISAM tables will be checked at access time; Use --myisam-recover=OFF for old behavior)
Don't call extra(HA_EXTRA_FORCE_REOPEN) in ALTER TABLE if table is locked as this will mark table as crashed!
Added assert to detect if we accidently would use MyISAM versioning in MySQL
include/my_base.h:
Mark NOT_USED as USED, as we now use this as a flag to not call extra()
mysql-test/mysql-test-run.pl:
Don't write all options when there is something wrong with the arguments
mysql-test/r/sp-destruct.result:
Add missing flush of mysql.proc (as the test copied live tables)
mysql-test/r/variables.result:
myisam-recover options changed to 'default'
mysql-test/r/view.result:
Don't show create time in result
mysql-test/suite/maria/t/maria-recovery2-master.opt:
Don't run test with myisam-recover (as this produces extra warnings during simulated death)
mysql-test/t/sp-destruct.test:
Add missing flush of mysql.proc (as the test copied live tables)
mysql-test/t/view.test:
Don't show create time in result
sql/lock.cc:
Added marker if table was deleted to argument list
sql/mysql_priv.h:
Added marker if table was deleted to argument list
sql/mysqld.cc:
myisam-recover options changed from OFF to 'DEFAULT' to get less change of data loss when using MyISAM
Allow one to specify OFF as argument to myisam-recover (was default before but one couldn't specify it)
sql/sql_base.cc:
Mark if table is going to be deleted
sql/sql_delete.cc:
Mark if table is going to be deleted
sql/sql_table.cc:
Mark if table is going to be deleted
Don't call extra(HA_EXTRA_FORCE_REOPEN) in ALTER TABLE if table is locked as this will mark table as crashed!
sql/table.cc:
Signal to handler if table is getting deleted as part of getting droped from table cache.
sql/table.h:
Added marker if table is going to be deleted.
storage/maria/ha_maria.cc:
Don't search for transaction handler if file is not transactional or outside of transaction
(Fixed possible core dump)
storage/maria/ma_blockrec.c:
Don't write changed information if table is going to be deleted.
storage/maria/ma_close.c:
Don't write changed information if table is going to be deleted.
storage/maria/ma_extra.c:
Mark tables that are deleted as crased, to ensure good behavior on restart if we suddenly crash.
storage/maria/ma_locking.c:
Cleanup
storage/maria/ma_recovery.c:
We need trnman to be inited during redo phase (to be able to open tables checked with maria_chk)
storage/maria/maria_def.h:
Added marker if table is going to be deleted.
storage/myisam/mi_close.c:
Don't write changed information if table is going to be deleted.
storage/myisam/mi_extra.c:
Mark tables that are deleted as crased, to ensure good behavior on restart if we suddenly crash.
storage/myisam/mi_open.c:
Added assert to detect if we accidently would use MyISAM versioning in MySQL
storage/myisam/myisamdef.h:
Added marker if table is going to be deleted.
Fix by explicitly setting timezone for a few more tests that need it.
(We avoid setting TZ everywhere by default as this breaks some tests
on windows).
Also add fix of two other windows failures due to non-portable
suppressions, thanks to Alex Budovski.
mysql-test/mysql-test-run.pl:
Don't let --timezone option from one test stray into the next test
(mostly to make results more deterministic rather than depend on
whatever test happens to run just before).
mysql-test/suite/maria/r/maria-recover.result:
Fix suppression pattern to also work with Windows \ path separator.
mysql-test/suite/maria/t/maria-recover.test:
Fix suppression pattern to also work with Windows \ path separator.
mysql-test/suite/parts/inc/partition_timestamp.inc:
Set timezone explicitly for test that needs it.
mysql-test/suite/parts/r/partition_recover_myisam.result:
Fix suppression pattern to also work with Windows \ path separator.
mysql-test/suite/parts/t/partition_recover_myisam.test:
Fix suppression pattern to also work with Windows \ path separator.
mysql-test/t/mysqlbinlog_row-master.opt:
Set timezone explicitly for test that needs it.
mysql-test/t/mysqlbinlog_row_innodb-master.opt:
Set timezone explicitly for test that needs it.
mysql-test/t/mysqlbinlog_row_myisam-master.opt:
Set timezone explicitly for test that needs it.
mysql-test/t/mysqlbinlog_row_trans-master.opt:
Set timezone explicitly for test that needs it.
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
WL#2474 "Multi Range Read: Change the default MRR implementation to implement new MRR interface"
WL#2475 "Batched range read functions for MyISAM/InnoDb"
"Index condition pushdown for MyISAM/InnoDB"
- Adjust test results (checked)
- Code cleanup.
This fixes a race condition in the test system
mysql-test/include/maria_empty_logs.inc:
Added comment
mysql-test/include/wait_for_status_var.inc:
Fixed type in variable name. Add ' around variables for better error message
mysql-test/mysql-test-run.pl:
Remove -skip-falcon (not to be used), add skip-pbxt to bootstrap
mysql-test/suite/maria/t/maria_showlog_error-master.opt:
Force restart before maria_showlog_error to get rid of status from previous connection
The --enable_prepare_warnings allows to not discard warnings from autorepair
of crashed table in --ps-protocol mode.
Use this to properly fix the parts.partition_recover_myisam and
maria.maria-recover tests.
Add a test case for the new feature. This also adds missing test coverage
for the case where the same warning is thrown in both prepare and execute
phase.
client/mysqltest.cc:
Implement new commands --enable-prepare_warnings and --disable_prepare_warnings.
mysql-test/r/mysqltest_ps.result:
Add test case for new --enable_prepare_warning mysqltest command.
mysql-test/suite/maria/t/maria-recover.test:
Better fix of test case using new --enable_prepare_warnings command.
mysql-test/suite/parts/t/partition_recover_myisam.test:
Fix test failure in --ps-protocol mode.
mysql-test/t/mysqltest_ps.test:
Add test case for new --enable_prepare_warning mysqltest command.
after the table has been converted to MyISAM and back. (PAGE_CHECKSUM is
still there)
mysql-test/suite/maria/r/maria3.result:
Update tests results after previous cset. 'TRANSACTIONAL=1' attribute goes
after the table has been converted to MyISAM and back. (PAGE_CHECKSUM is
still there)
- Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
- Extended the maximun key parts for an index from 16 to 32
- Extended MyISAM and Maria engines to support up to 32 parts
Added checks for return value from ha_index_init()
include/my_handler.h:
Extended number of key parts for MyISAM and Maria from 16 to 32
include/my_pthread.h:
Ensure we always have 256M of stack.
(Required to be able to handle the current number of keys and key parts in MyISAM)
mysql-test/r/create.result:
Extended to test for 32 key parts
mysql-test/r/myisam.result:
Test that we can create 32 but not 33 key parts
mysql-test/r/ps_1general.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_2myisam.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_3innodb.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_4heap.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/r/ps_5merge.result:
Length of ref is now 2048 as we can have more key parts
mysql-test/suite/maria/r/maria.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/maria3.result:
Max key length is now 1208 bytes
mysql-test/suite/maria/r/ps_maria.result:
Max key length is now 1208 byte
mysql-test/t/create.test:
Extended to test for 32 key parts
mysql-test/t/myisam.test:
Test that we can create 32 but not 33 key parts
sql/handler.cc:
Check return value from ha_index_init()
sql/handler.h:
Added a handler call (prepare_index_scan()) to inform storage engines that an index scan is about to take place.
sql/sql_select.cc:
Checks all return values from ha_index_init()
Call prepare_index_scan()) to inform storage engines that an index scan is about to take place.
Fixed indentation
sql/table.cc:
Fixed wrong types for key_length (rest of code assumed this was 32 bit)
sql/unireg.h:
Extended the maximun key parts for an index from 16 to 32
storage/maria/ha_maria.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/ha_myisam.cc:
Don't allocate HA_CHECK on the stack in functions where we call repair() as HA_CHECK is HUGE and will overflow stack
storage/myisam/mi_check.c:
Fixed wrong check if value overflow
tests/mysql_client_test.c:
Added fflush() to fix output in case of error
Fixed wrong check of 'ref' length in EXPLAIN
Use MY_MUTEX_INIT_FAST for pool mutex
mysql-test/mysql-test-run.pl:
Added option --staging-run
Added information about --parallell=# to help message
mysql-test/suite/federated/federated_server.test:
Slow test, don't run with --staging-run
mysql-test/suite/maria/t/maria-preload.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_optimize.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_relayrotate.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_001.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_mysqlbinlog.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_row_sp003.test:
Slow test, don't run with --staging-run
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
Slow test, don't run with --staging-run
mysql-test/t/compress.test:
Slow test, don't run with --staging-run
mysql-test/t/count_distinct3.test:
Slow test, don't run with --staging-run
mysql-test/t/index_merge_innodb.test:
Slow test, don't run with --staging-run
mysql-test/t/information_schema_all_engines.test:
Slow test, don't run with --staging-run
mysql-test/t/innodb_mysql.test:
Slow test, don't run with --staging-run
mysql-test/t/pool_of_threads.test:
Slow test, don't run with --staging-run
mysql-test/t/preload.test:
Slow test, don't run with --staging-run
mysql-test/t/ssl.test:
Slow test, don't run with --staging-run
mysql-test/t/ssl_compress.test:
Slow test, don't run with --staging-run
mysql-test/valgrind.supp:
Suppress warnings from SuSE 11.1 on x86
sql/scheduler.cc:
Use MY_MUTEX_INIT_FAST for pool mutex
mysql-test/t/variables.test:
Reset delay_key_write, otherwise maria.maria test may fail
sql/set_var.cc:
Reset ha_open_options if one resets the delay_key_write variable.
Before there was no way to reset it without restarting mysqld, which caused some tests to fail
There were so many changes into mtr (this is the new mtr coming) that I rather
copied mtr from 6.0-main here (at least this one knows how to run Maria tests).
I also fixed suite/maria tests to be accepted by the new mtr.
mysys/thr_mutex.c:
adding DBUG_PRINT here, so that we can locate where the warning is issued.
mysql-test/suite/maria/r/maria-recovery3.result:
result update
mysql-test/suite/maria/t/maria-recovery3.test:
Test for BUG#42112; before the bugfix, recovery would assert like this:
ma_blockrec.c:6051: _ma_apply_redo_insert_row_head_or_tail: Assertion `rownr == 0 && new_page' failed.
storage/maria/ma_create.c:
Fix for BUG#42112; plus some intentional crashes to test the fix. The bug was that if crash happened during
TRUNCATE TABLE, in maria_create(), after the index file's state has been written but before its LSNs
have been updated (so, if crash happened between _ma_state_info_write_sub() and _ma_update_state__lsns_sub()),
then that would leave a table with create_rename_lsn==0. Recovery would then try old pre-TRUNCATE REDOs
on this table, and fail as this table is already partly shortened. Fix is to write create_rename_lsn==LSN_MAX
as soon as TRUNCATE touches the index file, so that Recovery ignores this table. This allows Maria to start;
the table is still corrupted but the user can successfully repeat TRUNCATE TABLE (which required Maria to start).
storage/maria/ma_delete_all.c:
A comment.
include/atomic/generic-msvc.h:
prevent possible compiler warnings
include/lf.h:
comments, better definition for LF_HASH_OVERHEAD
include/maria.h:
define MARIA_CANNOT_ROLLBACK here
include/my_pthread.h:
avoid possible name clash
include/waiting_threads.h:
comments, const, move WT_RESOURCE to waiting_threads.c
mysql-test/suite/maria/r/maria_notembedded.result:
new test
mysql-test/suite/maria/t/maria_notembedded.test:
new test - 5-way deadlock
mysys/lf_hash.c:
better definition for LF_HASH_OVERHEAD
mysys/my_static.c:
comment
mysys/my_thr_init.c:
casts
mysys/waiting_threads.c:
comments, asserts, etc
server-tools/instance-manager/parse.cc:
fix my_init_dynamic_array() to follow new calling conventions
sql/mysqld.cc:
call wt_init after set_proper_floating_point_mode
sql/sql_class.h:
comment
storage/maria/ha_maria.cc:
move MARIA_CANNOT_ROLLBACK to a common header
storage/maria/ma_commit.c:
comment
storage/maria/ma_write.c:
comments, check for HA_ERR_FOUND_DUPP_KEY
storage/maria/trnman.c:
comments, assert
storage/maria/trnman.h:
comments
storage/maria/unittest/trnman-t.c:
be paranoid
unittest/mysys/lf-t.c:
comments
unittest/mysys/waiting_threads-t.c:
comments, safety, memory leak
output related to this.
mysql-test/suite/maria/r/maria-recovery3.result:
result update
mysql-test/suite/maria/t/maria-recovery3.test:
Test for bug; before the fix, the "CHECK TABLE EXTENDED" would mention a bad bitmap, because the
REDO_INSERT_ROW_BLOBS was containing a page number which was actually the one of a tail, so execution of this
record would mark the tail page as full in bitmap (like if it were a blob page), though it wasn't full.
Also, the assertion added around ma_blockrec.c:6580 in the present revision fired.
storage/maria/ma_blockrec.c:
- fix for BUG#41493: if we found out that logging was not needed at this point (blob_length==0 i.e. tail page),
then we forgot to increment tmp_block, so in the second iteration (assuming two BLOB columns), we would log the
page range of the first iteration (i.e. the tail page's number) for this second BLOB, which would cause
Recovery to overwrite the tail page with the second BLOB.
- assert when marking the table corrupted during REDO phase; this catches some problems earlier
otherwise they get caught only when a later record wants to use the table.
- _ma_apply_redo_insert_row_blobs() now fills some synthetic info about the blobs and pages involved
in a REDO_INSERT_ROW_BLOBS record, for inclusion into maria_recovery.trace: number of blobs, of ranges,
first and last page (does not tell about any gaps in the middle, but good enough for now). It also asserts
that it's not overwriting a tail/head page (which happened in the bug).
storage/maria/ma_blockrec.h:
new prototype for _ma_apply_redo_insert_row_blobs
storage/maria/ma_recovery.c:
Print info got from _ma_apply_redo_insert_row_blobs() to maria_recovery.trace (so far this file had mentioned
what head and tail pages a record touched, but not blob pages).
of Maria tests with --big (back-ports from 6.0-maria)
mysql-test/suite/maria/r/maria-big.result:
result update
mysql-test/suite/maria/r/maria-recovery-big.result:
result update
mysql-test/suite/maria/t/maria-big.test:
this test didn't work anymore due to changes in 5.1 around max_allowed_packet allowed settings
(fix back-ported from 6.0-maria)
mysql-test/suite/maria/t/maria-recovery-big-master.opt:
this test didn't work anymore due to changes in 5.1 around max_allowed_packet allowed settings
(fix back-ported from 6.0-maria)
mysql-test/suite/maria/t/maria-recovery-big.test:
this test didn't work anymore due to changes in 5.1 around max_allowed_packet allowed settings
(fix back-ported from 6.0-maria)
mysql-test/suite/maria/t/maria-recovery2-master.opt:
Fix for BUG#42012 "Maria: test maria-recovery2 fails with --embedded": file is updated like
maria-recovery-master.opt had already been in guilhem@mysql.com-20080701204709-pa4megwfvllq3s7g
Bug #41962 Maria: view-related test failures (insert, view, maria, trigger tests)
Added error handling for wrong update of view.
See Bug #41760 Inserting into multiple-table views is not working
mysql-test/r/delayed.result:
Fixed test as we are now testing values before fields.
Added new tests to test all error combinations
mysql-test/suite/maria/r/maria.result:
Added error handling for not supported update of view.
mysql-test/suite/maria/t/maria.test:
Added error handling for not supported update of view.
mysql-test/t/delayed.test:
Fixed test as we are now testing values before fields.
Added new tests to test all error combinations
sql/sql_base.cc:
Fixed warning from valgrind
sql/sql_insert.cc:
Don't test from which table values are in case of INSERT ... SELECT
Run fix_fields() in values before we do it on fields.
This is needed becasue check_view_single_update() are accessing values.
storage/maria/ma_blockrec.c:
Don't call pagecache_delete_pages() if no pages to delete.
This fixes a DBUG_ASSERT() error in maria_test_recovery
Added code to detect and give error when doing an insert into a view where we accessed fields in a not yet read table
Disabled test in subselect.test as the CHECK_OPTION for views doesn't work for insert.
This needs to be fixed properly later.
The problem with views are described in Bug #41760 Inserting into multiple-table views is not working
mysql-test/r/insert.result:
Fixed wrong usage of insert into view.
mysql-test/r/subselect.result:
Disabled wrong test (temporary)
mysql-test/suite/maria/r/maria.result:
Added test of size of table
mysql-test/suite/maria/t/maria.test:
Added test of size of table
mysql-test/t/insert.test:
Fixed wrong usage of insert into view
The bug is that during insert/update we currently don't read any of the referenced tables of the view.
This means that we can't get a value from another table to use as part of the update.
mysql-test/t/subselect.test:
Disabled not working test until someone has time to fix insert into view properly
Here we where refering to last used value in t2, which is wrong.
sql/sql_insert.cc:
Detect if we are trying to update one table in a view based on value in another, not yet read, table.
This fixes the problem discovered in insert.test
storage/maria/ma_blockrec.c:
Don't ignore not critical changes to the last page in the table.
We need to write the last page as otherwise we can during aborting of a row with a duplicate key get
state.data_file_length and the real length of file out of sync
storage/maria/ma_check.c:
Flush the page cache even if we got an error during zerofill.
(This fixes a call to assert() in case of a too short data file)
storage/maria/ma_pagecache.c:
Mark page as read when we do a write of a full page.
This fixes a bug when we got an error during read and then used direct write to page to update it
storage/maria/ma_state.c:
Restore info->lock.type after call to maria_versioning.
Fixed crash in maria_recover.test
storage/maria/maria_read_log.c:
Don't write thread id in debug log. (Not needed as maria_read_log is a single treaded program)
Fixed bug when removing a newly inserted record (in case of duplicate key).
The bug caused a crash for rows with several blobs and the first blob was small enough to fit into the head page.
Don't change state_history if nothing changed (speed optimization that also simplifies logic).
Reset state_history if we added/deleted or updated rows without versioning.
Fixed wrong test in trnman_exists_active_transactions() if state is visible or not.
Other bugs fixed:
Fixed wrong argument to (lock->get_status) when we had to wait for TL_WRITE_CONCURRENT_INSERT.
Item_equal::update_used_tables() didn't calculate const_item_cache properly.
Added assert's to detect if join_read_const_table() was called under wrong assumptions..
Fixed that _ma_setup_live_state() is called from thr_lock() instead of handler::external_lock().
This was needed to get versioning information to be setup correctly.
Fixed error in debug binaries during a call to _ma_check_table_is_closed() when another thread was opening/closing a table.
Fixed wrong test when finding right history_state to use.
mysql-test/suite/maria/r/maria.result:
Added test for Bug#40311 Assert in MARIA_RECORD_POS during pushbuild 2 test
mysql-test/suite/maria/t/maria.test:
Added test for Bug#40311 Assert in MARIA_RECORD_POS during pushbuild 2 test
mysys/thr_lock.c:
Fixed wrong argument to (lock->get_status) when we had to wait for TL_WRITE_CONCURRENT_INSERT
sql/item_cmpfunc.cc:
Item_equal::update_used_tables() didn't calculate const_item_cache properly, which later caused a wrong result for item->const_item()
sql/sql_base.cc:
In debug mode, Initilize record buffer with unexpected data to catch usage of uninitialized memory
sql/sql_select.cc:
Fixed indentation
Added assert's to detect if join_read_const_table() was called under wrong assumptions.
One assert() is disabled for now as Item_equal() doesn't behave as expected.
storage/maria/ha_maria.cc:
Move calling to _ma_setup_live_state() to ma_state.c::_ma_block_get_status()
This was needed as _ma_setup_live_state() needed to know if the table will be used concurrently or not
storage/maria/ma_blockrec.c:
Fixed bug when removing a newly inserted record (in case of duplicate key).
The bug caused a crash for rows with several blobs and the first blob was small enough to fit into the head page.
storage/maria/ma_dbug.c:
Added mutex to protect the open table list during _ma_check_table_is_closed().
Without the protection we could get a error in debug binaries during a call to _ma_check_table_is_closed()
storage/maria/ma_delete_table.c:
Removed not used code
storage/maria/ma_rename.c:
Removed not used code
storage/maria/ma_state.c:
Fixed wrong test when finding right history_state to use
Mark in tables->state_current.no_transid if we are using transid's or not.
Don't change state_history if nothing changed (speed optimization that also simplifies logic)
Reset state_history if we added/deleted or updated rows without versioning.
More DBUG_ASSERT's and more DBUG
Updated maria_versioning() to initialize environment before calling _ma_blok_get_status(). This was needed because of the new logic in _ma_block_get_status()
storage/maria/ma_state.h:
Added flags to detect if table changed and/or if we changed table without versioning
storage/maria/ma_write.c:
Simple cleanups (No logic changes)
storage/maria/trnman.c:
Fixed wrong test in trnman_exists_active_transactions() if state is visible or not.
mysql-test/suite/maria/r/maria-recovery3.result:
result.
mysql-test/suite/maria/t/maria-recovery3-master.opt:
usual recovery test options
mysql-test/suite/maria/t/maria-recovery3.test:
Test for BUG#41037. Without the bugfix, the test would hang because Recovery would fail with "undo_key_insert got error 126": Recovery would believe INSERT of 2 is uncommitted, try to execute its UNDO_KEY_INSERT, fail to find the key to delete because DELETE deleted it. That failure of _ma_ck_real_delete() would mark table as corrupted (see
in d_search()) which is error 126 (HA_ERR_CRASHED).
storage/maria/ma_blockrec.c:
set trn->rec_lsn to LSN of commit record, when writing it.
storage/maria/ma_blockrec.h:
new function
storage/maria/ma_commit.c:
when committing or rolling back, rec_lsn should be LSN_IMPOSSIBLE: assertion is moved here
from trnman_end_trn(), because now ma_commit() can set rec_lsn and so when we reach trnman_end_trn()
it's not LSN_IMPOSSIBLE anymore.
Adding debug possibility to pause between COMMIT record write and trnman_commit_trn(), to be
able to fire checkpoint in between.
storage/maria/ma_loghandler.c:
in-write hook for COMMIT records
storage/maria/ma_recovery.c:
More debugging info in maria_recovery.trace, to see how the starting point of REDO phase is determined
storage/maria/trnman.c:
assertion cannot be here anymore see ma_commit.c
(BUG#41127: Maria: assertion when SHOW ENGINE MARIA LOGS and missing logs)
mysql-test/suite/maria/r/maria_showlog_error.result:
test suite for the BUG#41127
mysql-test/suite/maria/t/maria_showlog_error.test:
test suite for the BUG#41127
storage/maria/ha_maria.cc:
Do not use MY_WME in the stat call which errors we process on high level.
bug is actually a weirdness in test system, so moving test portion into maria_notembedded.test
mysql-test/suite/maria/r/maria.result:
result update
mysql-test/suite/maria/r/maria_notembedded.result:
result update
mysql-test/suite/maria/t/maria.test:
Removing test portion into maria_notembedded.test. Explanation below.
Running maria.test with --mem --embedded --valgrind I got:
==378== Invalid read of size 4
==378== at 0x857C755: _lf_pinbox_real_free (lf_alloc-pin.c:342)
==378== by 0x857C640: _lf_pinbox_free (lf_alloc-pin.c:267)
==378== by 0x857D50C: ldelete (lf_hash.c:226)
==378== by 0x857DBC1: lf_hash_delete (lf_hash.c:429)
==378== by 0x8243D82: unlock_lock_and_free_resource
(waiting_threads.c:676)
==378== by 0x8244A79: wt_thd_release (waiting_threads.c:948)
==378== by 0x83F301D: wt_thd_release_self (trnman.c:98)
==378== by 0x83F3D49: trnman_end_trn (trnman.c:431)
==378== by 0x837C437: ma_commit (ma_commit.c:60)
==378== by 0x835F082: ha_maria::external_lock(THD*, int)
(ha_maria.cc:2387)
==378== by 0x84D1011: handler::ha_external_lock(THD*, int)
(handler.cc:4433)
==378== by 0x856DA92: unlock_external(THD*, st_table**, unsigned)
(lock.cc:786)
==378== by 0x856DD6F: mysql_unlock_tables(THD*, st_mysql_lock*)
(lock.cc:389)
==378== by 0x8274A57: close_thread_tables(THD*) (sql_base.cc:1307)
==378== by 0x82ABEC3: unlock_locked_tables(THD*) (sql_parse.cc:99)
==378== by 0x82B0BB3: mysql_execute_command(THD*)
(sql_parse.cc:3372)
==378== Address 0x5894a0c is 164 bytes inside a block of size 184
free'd
==378== at 0x40233FC: free (vg_replace_malloc.c:323)
==378== by 0x823CF92: my_thread_end (my_thr_init.c:348)
==378== by 0x8200FEE: mysql_thread_end (libmysql.c:250)
==378== by 0x81A9487: send_one_query (mysqltest.c:535)
==378== by 0x4050111: start_thread (in /lib/libpthread-2.5.so)
==378== by 0x41A52ED: clone (in /lib/libc-2.5.so)
(among other errors). The line where the invalid read happens is:
if (available_stack_size(&pinbox, *pins->stack_ends_here) >
alloca_size)
and stack_ends_here was previously set here:
el->stack_ends_here= & my_thread_var->stack_ends_here;
The mysqltest "send" command, in embedded mode, is implemented this way
(mysqltest.c:do_send_query()): a *new OS thread* is created in
mysqltest, and does this:
mysql_thread_init();
VOID(mysql_send_query(&cn->mysql, cn->cur_query,
cn->cur_query_len));
mysql_thread_end();
So, con_d "send insert t1 values (2)" creates a new OS thread, which
gets a thread-specific data (mysql_thread_init()), then sends the
query; in particular, this sets el->stack_ends_here to a pointer inside
the thread-specific data (see my_thread_var above), that is, thd->pins
depends on the thread-specific data.
This sent "insert" blocks as expected, thread-specific data is free()d
(my_thread_end() above), OS thread terminates.
Then "default" connection runs "insert t1 values (3)"
which blocks on the 3 already inserted by the con_d.
This blocking (see waiting_threads.c) creates a WT_RESOURCE, which has,
as owner, con_d's WT_THD.
When con_d calls "unlock tables" (see stack trace), it wants to release
this resource, which involves con_d's thd->pins, so it ends up reading
stack_ends_here which points inside the freed thread-specific data =>
Valgrind error, crashes...
So this is an effect of having one single connection (con_d) for which
two queries:
send insert t1 values (2)
unlock tables
are done by different OS threads. That is indeed weird design of
mysqltest, and it breaks on the dependency of lf_alloc-pin.c on
thread-specific data. It is maybe already explained in those comments
in lf_alloc-pin.c:
" It is assumed that pins belong to a THD and are not transferable
between THD's (LF_PINS::stack_ends_here being a primary reason
for this limitation)."
" It is assumed that pins belong to a thread and are not
transferable
between threads."
In correct usage of libmysqld (no two queries sent for one connection
by two threads), this problem would not happen, so I call it a
deficiency of the test system.
mysql-test/suite/maria/t/maria_notembedded.test:
moving test portion into maria_notembedded.test
Don't fsync() index file when closing Maria table if not transactional.
mysql-test/suite/maria/r/maria.result:
piece moved
mysql-test/suite/maria/r/maria_partition.result:
result
mysql-test/suite/maria/t/maria.test:
- reset default storage engine at end of test, not in the middle
- move piece which requires partitioning, to maria_partition.test, otherwise test fails
on builds without partitioning compiled in
mysql-test/suite/maria/t/maria_partition.test:
new test for those Maria bugs which are specific of partitioning
mysys/my_uuid.c:
compiler warning fix (fix imported from latest 5.1-main)
storage/maria/ma_close.c:
don't fsync() index file when closing table if not transactional
(same test as in _ma_once_end_block_record() when fsync-ing data file)
storage/maria/ma_create.c:
compiler warning fix (char* assigned to uchar*)
storage/maria/ma_loghandler.c:
compiler warning fix (char* assigned to uchar*)
- The problem was that we didn't inform the handler that we are going to close tables that are locked and may have (at least in Maria) be part of an active transaction.
Fix for Bug#39227 Maria: crash with ALTER TABLE PARTITION
Fix for Bug #39987 main.partition_not_windows fails under debug build
Fixed some compiler errors & warnings found by pushbuild
include/my_base.h:
Added HA_EXTRA_PREPARE_FOR_FORCED_CLOSE for signaling the handler that the file will be forced closed
include/my_global.h:
Removed 'register' from 'swap_variables' as this gives a warnings when the variables are structs. Compilers should also now be smart enough to figure out this themselves
mysql-test/r/subselect_debug.result:
Reset value of the debug variable; Without setting this the subselect_innodb test will fail when run after this one
mysql-test/suite/maria/r/maria.result:
Merged test with myisam.test
Added tests for new fixed bugs
mysql-test/suite/maria/t/maria.test:
Merged test with myisam.test
Added tests for new fixed bugs
mysql-test/t/subselect_debug.test:
Reset value of the debug variable; Without setting this the subselect_innodb test will fail when run after this one
mysys/my_uuid.c:
Fixed compiler error on windows
sql/ha_partition.cc:
Added support for the new extra flag: HA_EXTRA_PREPARE_FOR_FORCED_CLOSE (Bug #39226)
Ensure that we call extra() for HA_EXTRA_PREPARE_FOR_DROP (Bug#39227)
sql/mysqld.cc:
Fix for Bug #39987 main.partition_not_windows fails under debug build
The problem was that when compiling for purify/valgrind realpath() is not used, which causes test_if_data_home_dir to fail when it shouldn't
sql/sql_base.cc:
Call HA_EXTRA_PREPARE_FOR_FORCED_CLOSE for tables that are locked but we are going to force close without doing a commit
sql/sql_parse.cc:
More DBUG_PRINT. Fixed comments
storage/maria/ma_extra.c:
If HA_EXTRA_PREPARE_FOR_FORCED_CLOSE is called and the table is part of a transaction, remove the table from beeing part of a transaction.
This is safe as this is only used as part of flush tables or when the table is not part of a transaction
storage/myisam/mi_open.c:
Indentation fix
unittest/mysys/waiting_threads-t.c:
Remove not needed 'volatile' to get rid of compiler warnings on windows
Fixed also some failing tests after merge. The files that was touched outside of the merge has a more explicit comment
include/my_global.h:
Remove 'register' from 'swap_variables' as this gives a warnings when the variables are structs. Compilers should also now be smart enough to figure out this themselves
mysql-test/t/subselect_debug.test:
Reset value of the debug variable; Without setting this the subselect_innodb test will fail when run after this one
sql/mysqld.cc:
Fix for Bug #39987 main.partition_not_windows fails under debug build
The problem was that when compiling for purify/valgrind realpath() is not used, which causes test_if_data_home_dir to fail when it shouldn't
unittest/mysys/waiting_threads-t.c:
Remove not needed 'volatile' to get rid of compiler warnings on windows
It was a forgotten rw_unlock(), due to the deadlock detector feature (so bug was only in 5.1-maria, not
6.0-maria).
mysql-test/suite/maria/r/maria3.result:
result, all fine
mysql-test/suite/maria/t/maria3.test:
Test of BUG#39697: two scenarios (transactional tables, and non-transactional table but dynamic row format so still taking the rwlock) where the hang happened.
t2 added by this test was masked by a temporary table created earlier in the test, which we forgot to drop.
storage/maria/ha_maria.cc:
use new macro
storage/maria/ma_blockrec.c:
use new macro
storage/maria/ma_commit.c:
use new macro
storage/maria/ma_init.c:
putting address of dummy_transaction_object in --debug trace can be useful
storage/maria/ma_open.c:
use new macro
storage/maria/ma_write.c:
if local_lock_tree is true, we have acquired keyinfo->root_lock so need to release it before "goto err".
A pair of assertions so that our usage of TrIDs is kept sensible.
storage/maria/maria_def.h:
A macro so that changes of MARIA_HA::trn can be tracked with --debug. It helped to understand in what cases,
in maria_write(), we could have !(info->dup_key_trid == info->trn->trid) && !share->now_transactional
(answer: ALTER TABLE adding UNIQUE index on transactional table).
mysql-test/mysql-test-run.pl:
The maria suite made default for execution.
mysql-test/suite/maria:
Maria tests moved to separate suite.
mysql-test/suite/maria/r:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-autozerofill.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-big.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-big2.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-connect.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-gis-rtree-dynamic.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-gis-rtree-trans.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-gis-rtree.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-mvcc.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-no-logging.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-page-checksum.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-preload.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-purge.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-recover.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-recovery-big.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-recovery-bitmap.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-recovery-rtree-ft.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-recovery.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria-recovery2.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria2.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria3.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/maria_notembedded.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/r/ps_maria.result:
Maria tests moved to separate suite.
mysql-test/suite/maria/t:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-autozerofill.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-big.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-big2.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-connect.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-gis-rtree-dynamic.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-gis-rtree-trans.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-gis-rtree.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-mvcc.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-no-logging.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-page-checksum.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-preload.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-purge.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recover-master.opt:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recover.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery-big-master.opt:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery-big.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery-bitmap-master.opt:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery-bitmap.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery-master.opt:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery-rtree-ft-master.opt:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery-rtree-ft.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery2-master.opt:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria-recovery2.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria2.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria3.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/maria_notembedded.test:
Maria tests moved to separate suite.
mysql-test/suite/maria/t/ps_maria.test:
Maria tests moved to separate suite.
