Extend remove_function_from_trace.pl to work with many threads (patch from Sergei)
dbug/remove_function_from_trace.pl:
Extend remove_function_from_trace.pl to work with many threads (patch from Sergei)
storage/maria/ma_bitmap.c:
Added marker that table had changed since last checkpoint.
This ensures that we will flush all bitmap pages from cache at checkpoint.
This fixes bug LP#605798 "wrong data in bitmap" after recovery.
storage/maria/ma_check.c:
Cleaned up error output
storage/maria/ma_recovery.c:
Cleaned up error output
storage/maria/maria_def.h:
Added changed_not_flushed
Don't give warning about block_insert if table is crashed.
scripts/mysql_install_db.sh:
Added link to KB.
storage/maria/ma_recovery.c:
Don't give warning about block_insert if table is crashed.
Don't do UNDO or REDO on a crashed table.
More DBUG_PRINT
sql/sql_parse.cc:
Remove display of 'packet' which is not useful
storage/maria/ma_blockrec.c:
More DBUG_PRINT()
storage/maria/ma_key_recover.c:
Write page number instead of page position
storage/maria/ma_recovery.c:
Write message to stderr if recovery is not likely to succeed because we don't log records for batch inserts.
In normal cases this should never be an issue as we would recreate an empty table if MariaDB dies under batch insert.
This warning will be given if you remove all tables and try to recreate them with maria_read_log, which can't be done as
the log doesn't contain all data.
Don't do UNDO or REDO on a crashed table.
storage/maria/ma_write.c:
Write page number instead of page position.
Fixed test failures in buildbot
Don't write errors when failing to send ok packet
mysql-test/suite/pbxt/r/range.result:
Don't write number of rows as it varies.
mysql-test/suite/pbxt/t/range.test:
Don't write number of rows as it varies.
sql/mysqld.cc:
Don't write errors when failing to send ok packet
storage/maria/ma_bitmap.c:
Added DBUG_ASSERT to detect wrong bitmap pages
storage/maria/ma_blockrec.c:
Don't reset BLOCKUSED_USE_ORG_BITMAP flag. This fixed a bug where bitmap could be wrong after UNDO of row with blobs
storage/maria/ha_maria.cc:
Don't use 'trn' variable to avoid warning about not used variable
storage/maria/ma_delete.c:
Added __attribute__((unused))
storage/maria/ma_key_recover.c:
Added __attribute__((unused))
Fixed wrong placement of #endif
storage/maria/ma_key_recover.h:
Fixed typo
Fixed bug in Aria when replacing short keys with long keys and a key tree both overflow and underflow at same time.
Fixed several bugs when generating recovery logs when using RGQ with replacing long keys with short keys and vice versa.
Lots of new DBUG_ASSERT()'s
Added more information to recovery log to make it easier to know from where log entry orginated.
Introduced MARIA_PAGE->org_size that tells what the size of the page was in last log entry. This allows us to find out if all key changes for index page was logged.
Small code cleanups:
- Introduced _ma_log_key_changes() to log crc of key page changes
- Added share->max_index_block_size as max size of data one can put in key block (block_size - KEYPAGE_CHECKSUM_SIZE)
This will later simplify adding a directory to index pages.
- Write page number instead of page postition to DBUG log
mysql-test/lib/v1/mysql-test-run.pl:
Use --general-log instead of --log to disable warning when using RQG
sql/mysqld.cc:
If we have already sent ok to client when we get an error, log this to stderr
Don't disable option --log-output if CSV engine is not supported.
storage/maria/ha_maria.cc:
Log queries to recovery log also in LOCK TABLES
storage/maria/ma_check.c:
If param->max_trid is set, use this value instead of max_trid_in_system().
This is used by recovery to set max_trid to max seen trid so far.
keyinfo->block_length - KEYPAGE_CHECKSUM_SIZE -> max_index_block_size (Style optimization)
storage/maria/ma_delete.c:
Mark tables crashed early
Write page number instead of page position to debug log.
Added parameter to ma_log_delete() and ma_log_prefix() that is logged so that we can find where wrong log entries where generated.
Fixed bug where a page was not proplerly written when same key tree had both an overflow and underflow when deleting a key.
keyinfo->block_length - KEYPAGE_CHECKSUM_SIZE => max_index_block_size (Style optimization)
ma_log_delete() now has extra parameter of how many bytes from end of page should be appended to log for page (for page overflows)
storage/maria/ma_key_recover.c:
Added extra parameter to ma_log_prefix() to indicate what caused log entry.
Update MARIA_PAGE->org_size when logging info about page.
Much more DBUG_ASSERT()'s.
Fix some bugs in maria_log_add() to handle page overflows.
Added _ma_log_key_changes() to log crc of key page changes.
If EXTRA_STORE_FULL_PAGE_IN_KEY_CHANGES is defines, log the resulting pages to log so one can trivally
see how the resulting page should have looked like (for errors in CRC values)
storage/maria/ma_key_recover.h:
Added _ma_log_key_changes() which is only called if EXTRA_DEBUG_KEY_CHANGES is defined.
Updated function prototypes.
storage/maria/ma_loghandler.h:
Added more values to en_key_debug, to get more exact location where things went wrong when logging to recovery log.
storage/maria/ma_open.c:
Initialize share->max_index_block_size
storage/maria/ma_page.c:
Added updating and testing of MARIA_PAGE->org_size
Write page number instead of page postition to DBUG log
Generate error if we read page with wrong data.
Removed wrong assert: key_del_current != share->state.key_del.
Simplify _ma_log_compact_keypage()
storage/maria/ma_recovery.c:
Set param.max_trid to max seen trid before running repair table (used for alter table to create index)
storage/maria/ma_rt_key.c:
Update call to _ma_log_delete()
storage/maria/ma_rt_split.c:
Use _ma_log_key_changes()
Update MARIA_PAGE->org_size
storage/maria/ma_unique.c:
Remove casts
storage/maria/ma_write.c:
keyinfo->block_length - KEYPAGE_CHECKSUM_SIZE => share->max_index_block_length.
Updated calls to _ma_log_prefix()
Changed code to use _ma_log_key_changes()
Update ma_page->org_size
Fixed bug in _ma_log_split() for pages that overflow
Added KEY_OP_DEBUG logging to functions
Log KEYPAGE_FLAG in all log entries
storage/maria/maria_def.h:
Added SHARE->max_index_block_size
Added MARIA_PAGE->org_size
storage/maria/trnman.c:
Reset flags for new transaction.
This will give a smoother experience when using the Aria engine by those that are using default limits
without still causing a notable problem for desktop users.
The bug was based on wrong undo data in recovery file and not enough checking of bad data.
sql/sql_select.h:
Added comment
storage/maria/ma_blockrec.c:
- Removed wrong sanity checks (didn't work for UNDO records)
- More sanity checks and DBUG_ASSERT
- More DBUG_ENTER and DBUG_PRINT
- Removed filler blocks in extent_to_bitmap_blocks() as it caused problems in write_block_record().
This was the main cause of the bug.
(This change can make records generated by UNDO slightly smaller than original record, which we have to fix
by correcting row_pos.length before calling write_block_record())
- Fixed some problems in write_block_record() while doing UNDO.
- Store header_length without TRANSID_SIZE into recovery log (as UNDO entires doesn't have TRANSID_SIZE)
- Mark table crashed if something goes wrong during UNDO
storage/maria/maria_def.h:
Added header_length
- CTRL-C now aborts 'source' commands in mysql client
- Fix that thread id's are removed in convert-debug-for-diff.sh
client/mysql.cc:
CTRL-C now aborts 'source' commands
scripts/convert-debug-for-diff.sh:
Fix expression to remove thread id
storage/maria/ha_maria.cc:
Don't call DBUG_ASSERT() when we kill a query during REPAIR / ALTER TABLE
storage/maria/ma_bitmap.c:
Added DBUG_ASSERT() if we call _ma_bitmap_set_full_page_bits()
storage/maria/ma_key_recover.c:
Don't do an assert if table is marked crashed.
storage/maria/ma_recovery.c:
Added DBUG_ENTER
More DBUG_ASSERT() to discover errors earlier
More checking of BLOCK structures in Aria.
Fixed crashing bug in Aria when doing UPDATE of several records in same block when doing table scan.
storage/federatedx/ha_federatedx.cc:
Added missing delete_dynamic(); This fixes the valgrind warnings about lost memory discovered by buildbot.
storage/maria/ma_blockrec.c:
Added debugging helper function _ma_print_block_info() to print block structure.
Changed arguments to _ma_print_directory() so it can be called by _ma_print_block_info()
check_directory() now also checks that empty_space on block is calculated correctly.
Added some local variables to get more information about what happend when things crash.
Fixed crashing bug when doing UPDATE of several records in same block when doing table scan.
storage/maria/ma_key_recover.c:
Simple optimization (don't call bmove_upp() if not needed); This could happen during page split.
storage/maria/ma_recovery.c:
Fixed compiler warning
storage/maria/ma_test2.c:
Run test without syncing to disk. (Not needed for this test)
storage/maria/ma_write.c:
More comments.
Added DBUG_ASSERT() to find wrong log entires, like the one reported at LP#616344, during log file creation.
storage/maria/unittest/ma_test_recovery.pl:
Added extra -s to maria_chk to get it more silent.
This fixes that ma_test_all.sh works again.
Print out checked file names in maria_check -s (unless you use a second -s)
Some trivial optimizations
storage/maria/ma_bitmap.c:
Trivial optimizations:
- Combine common code (to be able to remove duplicate mutex_lock call)
- Move setting of thread specific variables outside of mutex
storage/maria/ma_check.c:
Fixed wrong argument to printf
storage/maria/maria_chk.c:
Print out checked table names unless -s -s
storage/maria/maria_read_log.c:
Added option --start-from-checkpoint (to help find bugs in checkpoints)
client/mysqlcheck.c:
Added missing casts
client/mysqldump.c:
Added missing casts
client/mysqlimport.c:
Added missing casts
extra/my_print_defaults.c:
Added missing casts
mysql-test/mysql-test-run.pl:
Added suppression for non-critical warning on windows
storage/maria/maria_pack.c:
Added missing casts
storage/xtradb/buf/buf0lru.c:
Added missing casts
storage/xtradb/fil/fil0fil.c:
Added missing casts
storage/xtradb/handler/i_s.cc:
Added extra argument to call store() function for longlong.
storage/xtradb/srv/srv0srv.c:
Added cast to suppress compiler warning
support-files/compiler_warnings.supp:
Added suppression for some non critical compiler warnings on Windows
unittest/mytap/tap.h:
Fixed prototypes to be same as the actual functions
maria_chk & maria_read_log now reads block size from control file.
mysql-test/suite/maria/r/maria.result:
Updated results after trivial change of maria_chk's output
storage/maria/ma_bitmap.c:
More DBUG_PRINT
storage/maria/ma_blockrec.c:
Fixed bug that we didn't mark page full in bitmap if directory is full
storage/maria/ma_check.c:
Write out if directory is full for errors in bitmap
storage/maria/ma_control_file.c:
Don't give error for wrong block size if block size is 0
storage/maria/maria_chk.c:
Read block size from control file
In case of -dvv, write also out bitmap information (good for debugging)
storage/maria/maria_read_log.c:
Read block size from control file
Fixed that maria_read_log works with different page size than TRANSLOG_PAGE_SIZE
Fixed LP#613418 (M)aria recovery failure: ma_key_recover.c:981: _ma_apply_redo_index: Assertion `check_page_length == page_length' failed
include/my_sys.h:
Added my_disable_sync
mysql-test/mysql-test-run.pl:
Added --sync-sys=0 to run test suite faster
mysys/my_static.c:
Added my_disable_sync
mysys/my_sync.c:
Added my_disable_sync
sql/mysqld.cc:
Added -sync-sys=0 option for mysqld to skip sync() calls for faster testing
storage/maria/ma_key_recover.c:
More DBUG_ASSERT()
Added logging of KEY_OP_DEBUG to make examening of logs easier
Fixed testing of page length in recovery to ensure we don't overwrite checksum (previous tests was too relaxed)
Fixed bug in recovery logging of split pages which caused failure during recovery:
- Length was not adjusted properly for pages to be split
- Added KEY_OP_MAX_PAGELENGTH to tell recovery that page is now full length
- This fixed LP#613418
storage/maria/ma_key_recover.h:
Changed prototype for ma_log_change() for KEY_OP_DEBUG
storage/maria/ma_loghandler.h:
Added new enums for better debugging of recovery logs
storage/maria/ma_rt_index.c:
Added debugging information to calls to ma_log_change()
storage/maria/ma_write.c:
Added debugging information to calls to ma_log_change() and ma_log_split()
Fixed compiler warnings
Disabled some tests that doesn't work on windows (uses shell tools or strange characters)
client/mysqlshow.c:
Fixed compiler warnings
client/mysqlslap.c:
Fixed compiler warnings
mysql-test/mysql-test-run.pl:
Use an error file instead of /dev/null
(Fixes problem on Windows and the output may be usefull)
mysql-test/suite/maria/r/maria.result:
Test case for bug LP#613408 (not complete)
mysql-test/suite/maria/t/maria.test:
Test case for bug LP#613408 (not complete)
mysql-test/suite/percona/percona_log_slow_slave_statements-and-use_global_long_query_time.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_log_slow_slave_statements.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-control_global_slow.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-log_slow_filter.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-log_slow_verbosity.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-long_query_time.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-microseconds_in_slow_query_log.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-min_examined_row_limit.test:
This test doesn't work one windows (needs shell tools)
mysql-test/suite/percona/percona_slow_query_log-use_global_long_query_time.test:
This test doesn't work one windows (needs shell tools)
mysql-test/t/ctype_filesystem.test:
This test doesn't work one windows (problem with character sets)
mysql-test/t/events_time_zone.test:
Use longer times to get predictable tests
mysql-test/t/show_check-master.opt:
set long query time to get more predictable tests
storage/maria/ma_check.c:
Restore info->s->lock_key_trees after repair.
Disable logging to temp tables for all repair cases (safety fix)
storage/maria/ma_state.c:
Ensurethat info->state_start doesn't point to freed memory. (Could happen after running an internal repair to fast create indexes)
storage/maria/trnman.c:
Added longer comment
Changed to use longlong10_to_str() instead of longlong2str() when base is 10 or -10 as former is much faster than later
Changed my_vsnprintf() to use longlong2str instead of int2str() to get rid of warnings and to get support for long pointers even when long is 32 bit.
client/mysqltest.cc:
longlong2str() -> longlong10_to_str()
include/m_string.h:
Added extra argument to longlong2str() to make it have same prototype is int2str()
mysys/charset.c:
Fixed compiler warning
mysys/mf_soundex.c:
Fixed compiler warning
mysys/my_getopt.c:
longlong2str() -> longlong10_to_str()
sql/create_options.cc:
Fixed compiler warning
sql/item_strfunc.cc:
Added extra argument to longlong2str
sql/opt_range.cc:
longlong2str() -> longlong10_to_str()
sql/partition_info.cc:
longlong2str() -> longlong10_to_str()
sql/slave.cc:
longlong2str() -> longlong10_to_str()
sql/sql_bitmap.h:
Added extra argument to longlong2str
sql/sql_partition.cc:
Added extra argument to longlong2str
sql/sql_select.cc:
longlong2str() -> longlong10_to_str()
sql/sql_show.cc:
Added extra argument to longlong2str
storage/innodb_plugin/handler/ha_innodb.cc:
Update to new parameters for longlong2str()
storage/maria/ma_dbug.c:
longlong2str() -> longlong10_to_str()
storage/maria/maria_chk.c:
Added extra argument to longlong2str
storage/myisam/mi_dbug.c:
longlong2str() -> longlong10_to_str()
storage/myisam/myisamchk.c:
Added extra argument to longlong2str
storage/xtradb/handler/ha_innodb.cc:
Update to new parameters for longlong2str()
strings/longlong2str.c:
Added extra argument to longlong2str() to make it have same prototype is int2str()
strings/my_vsnprintf.c:
Changed my_vsnprintf() to use longlong2str instead of int2str() to get rid of warnings and to get support for long pointers even when long is 32 bit.
Added cast to get rid of compiler warnings
Fixed compiler warnings
client/mysqlslap.c:
Fixed compiler warnings
mysql-test/suite/maria/r/maria.result:
Test case for LP#614265
mysql-test/suite/maria/t/maria.test:
Test case for LP#614265
mysql-test/suite/pbxt/t/skip_name_resolve-master.opt:
Ensure that we get restart before test (as test uses show processlist)
sql/handler.cc:
Added cloned marker if clone was called (for safety checks & debugging)
sql/handler.h:
Added cloned marker if clone was called (for safety checks & debugging)
storage/maria/ha_maria.cc:
In clone call, set file->trn if cloned file had this set. This is needed as maria_create_trn_for_mysql() and thus file->trn is never set for cloned table.
Ensure that file->trn is properly reset after calls to repair/check/zerofill.
Increment locked table count if file->trn is set (as we decrement this in the unlock call)
tests/mysql_client_test.c:
Fixed compiler warnings
Added option: --warning-for-wrong-transaction-id to maria_check
mysql-test/r/innodb-use-sys-malloc.result:
Remove not used files
mysql-test/t/innodb-use-sys-malloc-master.opt:
Remove not used files
storage/maria/ma_recovery.c:
Added support for --lsn-end
storage/maria/ma_recovery.h:
New prototype
storage/maria/ma_write.c:
Added logging of KEY_OP_CHECK for all possible keyblock splits.
(Helps us find error in log file handling)
storage/maria/maria_chk.c:
Added option: --warning-for-wrong-transaction-id
--update-status now resets open count if check succeds
storage/maria/maria_read_log.c:
Added option --end-lsn for only appling part of log (good for debugging)
Changed options to use '-' instead of '_'
Fixed some wrong test cases
Fixed bug in null handling in XtraDB
extra/comp_err.c:
Fixed compiler warnings
extra/my_print_defaults.c:
Fixed compiler warnings
mysql-test/suite/binlog/t/binlog_killed.test:
Added support for timeouts
mysql-test/suite/funcs_1/r/is_columns_is.result:
Updated results (INNODB_SYS_TABLES had got new column)
scripts/mysql_install_db.sh:
Fixed typo
sql/mysql_priv.h:
Removed not needed argument for compare_record()
sql/sql_insert.cc:
Removed not needed argument for compare_record()
sql/sql_update.cc:
Removed not needed argument for compare_record()
The argument is not needed becasue we copy the full record[0] to record[1] and the comparison should work even if all columns are not read
sql/table.cc:
The comparison of rows is independent of HA_PARTIAL_COLUMN_READ
storage/maria/maria_chk.c:
Fixed compiler warnings
storage/maria/maria_read_log.c:
Fixed compiler warnings
storage/myisam/myisamchk.c:
Fixed compiler warnings
storage/myisam/myisampack.c:
Fixed compiler warnings
storage/xtradb/dict/dict0load.c:
Fixed compiler warnings
storage/xtradb/row/row0sel.c:
Fixed null handling in XtraDB. (See comment)
storage/xtradb/trx/trx0sys.c:
Fixed compiler warnings
support-files/compiler_warnings.supp:
Fixed compiler warnings
Support of virtual columns added to maria engine.
mysql-test/suite/vcol/r/vcol_handler_maria.result:
Basic tests for virtual column and maria engine.
mysql-test/suite/vcol/t/vcol_handler_maria.test:
Basic tests for virtual column and maria engine.
storage/maria/ha_maria.cc:
Support of virtual columns added to maria engine.
storage/maria/ha_maria.h:
Support of virtual columns added to maria engine.
More DBUG_PRINT (to simplify future debugging)
Aria: Added STATE_IN_REPAIR, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
Aria: Some trivial speedup optimization
Aria: Better warning if table was marked crashed by unfinnished repair
mysql-test/lib/v1/mysql-test-run.pl:
Fix so one can run RQG
mysql-test/suite/maria/r/maria-recovery2.result:
Update for new error message.
mysys/stacktrace.c:
Fixed compiler warning
storage/maria/ha_maria.cc:
More DBUG_PRINT
Added STATE_IN_REPAIR flag, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
Don't log query for dropping temporary table.
storage/maria/ha_maria.h:
Added prototype for drop_table()
storage/maria/ma_blockrec.c:
More DBUG_PRINT
Make read_long_data() inline for most cases. (Trivial speedup optimization)
storage/maria/ma_check.c:
Better warning if table was marked crashed by unfinnished repair
storage/maria/ma_open.c:
More DBUG_PRINT
storage/maria/ma_recovery.c:
Give warning if found crashed table.
Changed warning for tables that can't be opened.
storage/maria/ma_recovery_util.c:
Write warnings to DBUG file
storage/maria/maria_chk.c:
Added STATE_IN_REPAIR flag, which is set on start of repair. This allows us to see if 'crashed' flag was set intentionally.
storage/maria/maria_def.h:
Added maria_mark_in_repair(x)
storage/maria/maria_read_log.c:
Added option: --character-sets-dir
storage/maria/trnman.c:
By default set min_read_from to max value.
This allows us to remove TRN:s from rows during recovery to get more space.
This fixes bug LP#602604: RQG: ma_blockrec.c:6187: _ma_apply_redo_insert_row_head_or_tail: Assertion `0' failed on Maria engine recovery
Fixed bug: LP#603026 RQG: pagecache_read: Assertion `pageno < ((1ULL) << 40)' on OPTIMIZE TABLE of a Maria table
client/mysql.cc:
Removed Oracle copyright from stdout, as Oracle doesn't have copyright to all code in this file.
Ignore --comments at start of command line. This allows one to more easily run mysqltest tests trough the command line.
mysql-test/suite/maria/r/optimize.result:
Added test for LP#603026
mysql-test/suite/maria/t/optimize.test:
Added test for LP#603026
sql/net_serv.cc:
Removed DBUG_ASSERT(), as this code can happen during testing.
storage/maria/ma_check.c:
Fixed bug: LP#603026 RQG: pagecache_read: Assertion `pageno < ((1ULL) << 40)' on OPTIMIZE TABLE of a Maria table
The problem was duplicated memory usage with long packed keys.
