mirror of
https://github.com/MariaDB/server.git
synced 2025-01-23 15:24:16 +01:00
51 commits
Author | SHA1 | Message | Date | |
---|---|---|---|---|
Sergei Golubchik
|
a2bcee626d | Merge branch '10.0' into 10.1 | ||
Monty
|
c3018b0ff4 |
Fixes to get all test to run on MacosX Lion 10.7
This includes fixing all utilities to not have any memory leaks, as safemalloc warnings stopped tests from passing on MacOSX. - Ensure that all clients takes character-set-dir, as the libmysqlclient library will use it. - mysql-test-run now passes character-set-dir to all external clients. - Changed dynstr_free() so that it can be called twice (made freeing code easier) - Changed rpl_global_gtid_slave_state to be allocated dynamicly as it includes a mutex that needs to be initizlied/destroyed before my_end() is called. - Removed rpl_slave_state::init() and rpl_slave_stage::deinit() as their job are better handling by constructor and delete. - Print alias instead of table_name in check_duplicate_key as table_name may have been converted to lower case. Other things: - Fixed a case in time_to_datetime_with_warn() where we where using && instead of & in tests |
||
Monty
|
d7d589dc01 | Push for testing of encryption | ||
Michael Widenius
|
f0f2072d1f |
Fixed problem with very slow shutdown when using 100,000 MyISAM tables with delay_key_write
Reason for the problem was that the hash of changed files in the key cache was too small (was 128). Fixed by making the hash size larger and changeable. - Introduced key-cache-file-hash-size (default 512) for MyISAM and aria_pagecache_file_hash_size (default 512) for Aria. - Added new status variable "Feature_delay_key_write" which counts number of tables opened that are using delay_key_write mysql-test/r/features.result: Added test of Feature_delay_key_write mysql-test/r/key_cache.result: Updated tests as the number of blocks has changed mysql-test/r/mysqld--help.result: Updated result mysql-test/suite/maria/maria3.result: Updated result mysql-test/suite/sys_vars/r/key_cache_file_hash_size_basic.result: Test new variable mysql-test/suite/sys_vars/t/aria_pagecache_file_hash_size_basic.test: Test new variable mysql-test/suite/sys_vars/t/key_cache_file_hash_size_basic.test: Test new variable mysql-test/t/features.test: Added test of Feature_delay_key_write mysql-test/t/key_cache.test: Updated tests as the number of blocks has changed mysys/mf_keycache.c: Made CHANGED_BLOCKS_HASH dynamic sql/handler.cc: Updated call to init_key_cache() sql/mysqld.cc: Added "Feature_delay_key_write" Added support for key-cache-file-hash-size sql/mysqld.h: Added support for key-cache-file-hash-size sql/sql_class.h: Added feature_files_opened_with_delayed_keys sql/sys_vars.cc: Added key_cache_file_hash_size storage/maria/ha_maria.cc: Added pagecache_file_hash_size Added counting of files with delay_key_write storage/maria/ma_checkpoint.c: Fixed compiler warning storage/maria/ma_pagecache.c: Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable storage/maria/ma_pagecache.h: Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable storage/maria/ma_rt_test.c: Updated parameters for init_pagecache() storage/maria/ma_test1.c: Updated parameters for init_pagecache() storage/maria/ma_test2.c: Updated parameters for init_pagecache() storage/maria/ma_test3.c: Updated parameters for init_pagecache() storage/maria/maria_chk.c: Updated parameters for init_pagecache() storage/maria/maria_ftdump.c: Updated parameters for init_pagecache() storage/maria/maria_pack.c: Updated parameters for init_pagecache() storage/maria/maria_read_log.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_pagecache_consist.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_pagecache_rwconsist.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_pagecache_rwconsist2.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_pagecache_single.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_multithread-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_noflush-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_nologs-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Updated parameters for init_pagecache() storage/maria/unittest/ma_test_loghandler_purge-t.c: Updated parameters for init_pagecache() storage/myisam/ha_myisam.cc: Added counting of files with delay_key_write storage/myisam/mi_check.c: Updated call to init_key_cache() storage/myisam/mi_test1.c: Updated call to init_key_cache() storage/myisam/mi_test2.c: Updated call to init_key_cache() storage/myisam/mi_test3.c: Updated call to init_key_cache() storage/myisam/mi_test_all.sh: Fixed broken test storage/myisam/myisam_ftdump.c: Updated call to init_key_cache() storage/myisam/myisamchk.c: Updated call to init_key_cache() storage/myisam/myisamlog.c: Updated call to init_key_cache() |
||
Sergei Golubchik
|
aeaa112edb | fixes for windows | ||
Sergei Golubchik
|
76f0b94bb0 |
merge with 5.3
sql/sql_insert.cc: CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. ****** CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. sql/sql_table.cc: small cleanup ****** small cleanup |
||
Michael Widenius
|
5933619ceb |
Made test-unit run in parlallel. This was achived by having all aria tests that uses temporary files to create the temporary file in an unique tempdirectory.
aria_chk now returns 1 if one got any warnings during check and 2 if one got errors. lp:728919 maria_chk should fail on all detected corruptions mysql-test/lib/My/SafeProcess.pm: Removed ^M lines from the file storage/maria/ma_check.c: Fixed indentation storage/maria/maria_chk.c: aria_chk now returns 1 if one got any warnings during check and 2 if one got errors. storage/maria/unittest/ma_control_file-t.c: Use unique sub directory for test results storage/maria/unittest/ma_maria_log_cleanup.c: Added help function create_tmpdir() storage/maria/unittest/ma_pagecache_consist.c: Use unique sub directory for test results storage/maria/unittest/ma_pagecache_rwconsist.c: Use unique sub directory for test results storage/maria/unittest/ma_pagecache_rwconsist2.c: Use unique sub directory for test results storage/maria/unittest/ma_pagecache_single.c: Use unique sub directory for test results storage/maria/unittest/ma_test_all-t: Use unique sub directory for test results storage/maria/unittest/ma_test_loghandler-t.c: Use unique sub directory for test results storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Use unique sub directory for test results remove delete of files that are already deleted by maria_log_remove() storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Use unique sub directory for test results storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Use unique sub directory for test results storage/maria/unittest/ma_test_loghandler_multithread-t.c: Use unique sub directory for test results storage/maria/unittest/ma_test_loghandler_noflush-t.c: Use unique sub directory for test results remove delete of files that are already deleted by maria_log_remove() storage/maria/unittest/ma_test_loghandler_nologs-t.c: Use unique sub directory for test results storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Use unique sub directory for test results remove delete of files that are already deleted by maria_log_remove() storage/maria/unittest/ma_test_loghandler_purge-t.c: Use unique sub directory for test results support-files/compiler_warnings.supp: Supress some warnings unittest/unit.pl: Run 4 unittests in parallel |
||
Sergei Golubchik
|
0accbd0364 | lots of post-merge changes | ||
Michael Widenius
|
dfefc15db4 | Automatic merge with 5.1-release | ||
unknown
|
6120fbace4 |
Aria unit tests fixed to have correct pagecache shutdown.
The Aria multithread test unlocked. storage/maria/unittest/ma_pagecache_consist.c: Fixed pagecache shutdown. storage/maria/unittest/ma_pagecache_rwconsist.c: Fixed pagecache shutdown. storage/maria/unittest/ma_test_loghandler_multithread-t.c: The Aria multithread test unlocked. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Fixed pagecache shutdown. |
||
Sergei Golubchik
|
e246077bcf | rename maria to aria | ||
Sergei Golubchik
|
7ca3fc4ff0 | compiler warnings | ||
unknown
|
126c1228f5 |
Added versioning of row data
Will in future changeset (soon) av versioning of status variables (number of rows) and index Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings Removed some not needed variables (as noticed by Guilhem) include/maria.h: Added prototypes for maria_chk_init_for_check(), maria_versioning() and maria_ignore_trids() include/my_base.h: Add new error HA_ERR_ROW_NOT_VISIBLE include/myisamchk.h: Added variables for checking visibility of rows during maria_chk include/thr_lock.h: Changed argument type from int to my_bool for get_status Added variable allow_multiple_concurrent_insert, to signal if table supports multiple concurrent inserts mysql-test/r/maria-page-checksum.result: Added missing drop table mysql-test/t/maria-page-checksum.test: Added missing drop table mysys/my_handler.c: Added new error messages mysys/thr_lock.c: Added support for multiple concurrent inserts, if table handler supports it sql/sql_yacc.yy: Added LOCK TABLE table_name WRITE CONCURRENT This was added (temporarly?) to be able to check versioning with Maria storage/csv/ha_tina.cc: Updated parameter for get_status storage/maria/ha_maria.cc: Added calls to maria_chk_init_status() Fixed call to ma_control_file_open() storage/maria/ma_blockrec.c: Changed some LEX_STRING to LEX_CUSTRING to avoid casts and warnings Changed back some 'header' parameters to const char* Removed some casts Added support for versioning: - If info->row_flag & ROW_FLAG_TRANSID is set, store transaction id together with the row - When reading rows, check if rows are visible. Give error if not - When scanning table, ignore not visible rows - Added function parameters to some functions, to be able to call _ma_compact_block_page() with different parameters depending of if the page is a HEAD or TAIL page - _ma_compact_block_page() deletes transaction id's that are visible by all running transactions - Added functions for thr_lock() to enable multiple concurrent inserts - Added helper function 'mysql_versioning()' to enable/disable versioning - Added helper function maria_ignore_trids(), used by maria_chk and maria_pack to see all rows. storage/maria/ma_blockrec.h: Updated parameters for some functions. Added new functions to read/store state with thr_lock storage/maria/ma_check.c: Enable handling of transaction id's in rows Give a readable error if a table contains a transation id that makes rows not visible storage/maria/ma_control_file.c: Added option to not give warning if control file doesn't exists. storage/maria/ma_control_file.h: Updated parameter lists for ma_control_file_open() storage/maria/ma_delete.c: Removed not used variable (suggestion by Guilhem) storage/maria/ma_locking.c: Changed type of argument from int -> my_bool storage/maria/ma_open.c: Removed not used variables 'key_write_undo_lsn' and 'key_delete_undo_lsn' Added new thr_lock interface functions for BLOCK_RECORD to enable multiple concurrent insert storage/maria/ma_test1.c: Added option --versioning (-C) to check versioning storage/maria/ma_test2.c: Added option -C to check versioning storage/maria/ma_test_recovery: Forward argumetns to ma_test_recovery.pl storage/maria/ma_write.c: Removed not used variable key_write_undo_lsn storage/maria/maria_chk.c: Always read control file (if exist) at start Initialize checking of tables by calling maria_chk_init_for_check() In verbose mode and in case of error, print max found transaction id storage/maria/maria_def.h: Added Trid to MARIA_ROW to be able to check transaction id for found row Moved 'base_length' from MARIA_ROW to MARIA_HA to be able to handle different base length (with and without TRANSID) without if's Added default row_flag to MARIA_HA for the same reason Changed LEX_STRING -> LEX_CUSTRING to avoid casts in ma_blockrec.c Removed not needed variables key_write_undo_lsn and key_delete_undo_lsn Added prototypes for new functions and fixed those that had changed storage/maria/maria_pack.c: Ensure we can read all rows from the file, independent of the used transaction id storage/maria/maria_read_log.c: Updated arguments to ma_control_file_open() storage/maria/trnman.c: If we have only one transaction, fixed that min_read_from contains current transaction Fixed that trnman_can_read_from() returns that row is readable if it was written by current transaction storage/maria/unittest/ma_control_file-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_all-t: Added test of versioning Removed printing of one extra space storage/maria/unittest/ma_test_loghandler-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_multithread-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_noflush-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_nologs-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_loghandler_purge-t.c: Updated arguments to ma_control_file_open() storage/maria/unittest/ma_test_recovery.expected: Updated file with result from new tests storage/maria/unittest/ma_test_recovery.pl: Added options --abort-on-error and --verbose In case of --verbose, print all excuted shell commands Added test of versioning storage/myisam/mi_locking.c: Updated type of parameter storage/myisam/myisamdef.h: Updated type of parameter mysql-test/r/maria-mvcc.result: New BitKeeper file ``mysql-test/r/maria-mvcc.result'' mysql-test/t/maria-mvcc.test: New BitKeeper file ``mysql-test/t/maria-mvcc.test'' |
||
unknown
|
722a8ebe5b |
Store maximum transaction id into control file at clean shutdown.
This can serve to maria_chk to check that trids found in rows and keys are not too big. Also used by Recovery when logs are lost. Options --require-control-file, --datadir, --log-dir (yes, the dashes are inconsistent but I imitated mysqld --datadir and --maria-log-dir) for maria_chk. Lock control file _before_ reading its content. storage/maria/ha_maria.cc: new prototype storage/maria/ma_check.c: A function to find the max trid in the system (consults transaction manager and control file), to check tables. storage/maria/ma_checkpoint.c: new prototype storage/maria/ma_control_file.c: Store max trid into control file, in a backward-compatible way (can still read old control files). Parameter to ma_control_file_open(), to not create the log if it's missing (maria_chk needs that). Lock control file _before_ reading its content. Fix for a segfault when reading an old control file (bzero() with a negative second argument) storage/maria/ma_control_file.h: changes to the control file module's API storage/maria/ma_init.c: When Maria shuts down cleanly, store max trid into control file. storage/maria/ma_loghandler.c: new prototype storage/maria/ma_recovery.c: During recovery, consult max trid stored in control file, in case it is bigger than what we found in log (case of logs manually removed by user). storage/maria/ma_test1.c: new prototype storage/maria/ma_test2.c: new prototype storage/maria/maria_chk.c: New option --require-control-file (abort if control file not found), --datadir (path for control file (and for logs if --log-dir not specified)), --log-dir (path for logs). Try to open control file when maria_chk starts. storage/maria/maria_read_log.c: new prototype storage/maria/trnman.c: A new function to know max trid in transaction manager storage/maria/trnman_public.h: New function storage/maria/unittest/ma_control_file-t.c: new prototypes. Testing storing and retrieving the max trid to/from control file storage/maria/unittest/ma_test_loghandler-t.c: new prototype storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: new prototype storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: new prototype storage/maria/unittest/ma_test_loghandler_noflush-t.c: new prototype storage/maria/unittest/ma_test_loghandler_nologs-t.c: new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: new prototype storage/maria/unittest/ma_test_loghandler_purge-t.c: new prototype |
||
unknown
|
f094eff1d9 |
Injecting more "const" declarations into code which does not change
pointed data. I ran gcc -Wcast-qual on storage/maria, this identified un-needed casts, a couple of functions which said they had a const parameter though they changed the pointed content! This is fixed here. Some suspicious places receive a comment. The original intention of running -Wcast-qual was to find what code changes R-tree keys: I added const words, but hidden casts like those of int2store (casts target to (uint16*)) removed const checking; -Wcast-qual helped find those hidden casts. Log handler does not change the content pointed by LEX_STRING::str it receives, so we now use a struct which has a const inside, to emphasize this and be able to pass "const uchar*" buffers to log handler without fear of their content being changed by it. One-line fix for a merge glitch (when merging from MyISAM). include/m_string.h: As Maria's log handler uses LEX_STRING but never changes the content pointed by LEX_STRING::str, and assigns uchar* into this member most of the time, we introduce a new struct LEX_CUSTRING (C const U unsigned) for the log handler. include/my_global.h: In macros which read pointed content: use const pointers so that gcc -Wcast-qual does not warn about casting a const pointer to non-const. include/my_handler.h: In macros which read pointed content: use const pointers so that gcc -Wcast-qual does not warn about casting a const pointer to non-const. ha_find_null() does not change *a. include/my_sys.h: insert_dynamic() does not change *element. include/myisampack.h: In macros which read pointed content: use const pointers so that gcc -Wcast-qual does not warn about casting a const pointer to non-const. mysys/array.c: insert_dynamic() does not change *element mysys/my_handler.c: ha_find_null() does not change *a storage/maria/ma_bitmap.c: Log handler receives const strings now storage/maria/ma_blockrec.c: Log handler receives const strings now. _ma_apply_undo_row_delete/update() do change *header. storage/maria/ma_blockrec.h: correct prototype storage/maria/ma_check.c: Log handler receives const strings now. Un-needed casts storage/maria/ma_checkpoint.c: Log handler receives const strings now storage/maria/ma_checksum.c: unneeded cast storage/maria/ma_commit.c: Log handler receives const strings now storage/maria/ma_create.c: Log handler receives const strings now storage/maria/ma_dbug.c: fixing warning of gcc -Wcast-qual storage/maria/ma_delete.c: Log handler receives const strings now storage/maria/ma_delete_all.c: Log handler receives const strings now storage/maria/ma_delete_table.c: Log handler receives const strings now storage/maria/ma_dynrec.c: fixing some warnings of gcc -Wcast-qual. Unneeded casts removed. Comment about function which lies. storage/maria/ma_ft_parser.c: fix for warnings of gcc -Wcast-qual, removing unneeded casts storage/maria/ma_ft_update.c: less casts, comment storage/maria/ma_key.c: less casts, stay const (warnings of gcc -Wcast-qual) storage/maria/ma_key_recover.c: Log handler receives const strings now storage/maria/ma_loghandler.c: Log handler receives const strings now storage/maria/ma_loghandler.h: Log handler receives const strings now storage/maria/ma_loghandler_lsn.h: In macros which read pointed content: use const pointers so that gcc -Wcast-qual does not warn about casting a const pointer to non-const. storage/maria/ma_page.c: Log handler receives const strings now; more const storage/maria/ma_recovery.c: Log handler receives const strings now storage/maria/ma_rename.c: Log handler receives const strings now storage/maria/ma_rt_index.c: more const, to emphasize that functions don't change pointed content. best_key= NULL was forgotten during merge from MyISAM a few days ago, was causing a Valgrind warning storage/maria/ma_rt_index.h: new proto storage/maria/ma_rt_key.c: more const storage/maria/ma_rt_key.h: new proto storage/maria/ma_rt_mbr.c: more const for functions which deserve it storage/maria/ma_rt_mbr.h: new prototype storage/maria/ma_rt_split.c: make const what is not changed. storage/maria/ma_search.c: un-needed casts, more const storage/maria/ma_sp_key.c: more const storage/maria/ma_unique.c: un-needed casts. storage/maria/ma_write.c: Log handler receives const strings now storage/maria/maria_def.h: some more const storage/maria/unittest/ma_test_loghandler-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_multithread-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_noflush-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_nologs-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Log handler receives const strings now storage/maria/unittest/ma_test_loghandler_purge-t.c: Log handler receives const strings now |
||
unknown
|
ea3ae3c470 |
copyright and license info
storage/maria/Makefile.am: copyright storage/maria/ma_pagecache.c: copyright storage/maria/ma_pagecache.h: copyright storage/maria/ma_pagecaches.c: copyright storage/maria/ma_pagecrc.c: copyright storage/maria/unittest/Makefile.am: copyright storage/maria/unittest/lockman-t.c: copyright storage/maria/unittest/lockman1-t.c: copyright storage/maria/unittest/lockman2-t.c: copyright storage/maria/unittest/ma_control_file-t.c: copyright storage/maria/unittest/ma_loghandler_examples.c: copyright storage/maria/unittest/ma_maria_log_cleanup.c: copyright storage/maria/unittest/ma_pagecache_consist.c: copyright storage/maria/unittest/ma_pagecache_rwconsist.c: copyright storage/maria/unittest/ma_pagecache_single.c: copyright storage/maria/unittest/ma_test_loghandler-t.c: copyright storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: copyright storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: copyright storage/maria/unittest/ma_test_loghandler_multigroup-t.c: copyright storage/maria/unittest/ma_test_loghandler_multithread-t.c: copyright storage/maria/unittest/ma_test_loghandler_noflush-t.c: copyright storage/maria/unittest/ma_test_loghandler_nologs-t.c: copyright storage/maria/unittest/ma_test_loghandler_pagecache-t.c: copyright storage/maria/unittest/ma_test_loghandler_purge-t.c: copyright storage/maria/unittest/test_file.c: copyright storage/maria/unittest/test_file.h: copyright storage/maria/unittest/trnman-t.c: copyright |
||
unknown
|
3057378698 |
Fix for failures of ma_control_file-t, ma_test_loghandler_purge-t,
ma_test_loghandler_pagecache-t on Windows. storage/maria/ma_control_file.c: stat() is unreliable on Windows (does not reflect process' own writes) storage/maria/ma_loghandler.c: translog_set_lsn_for_files() didn't close its file descriptor; it was a real problem as non-closed files could not be purged. Same for translog_truncate_log() in case of error. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: stat() is unreliable on Windows (does not reflect process' own writes) |
||
unknown
|
311d8e898f |
Mac compiler warnings fixed.
include/maria.h: mac compiler warnings fixed. storage/maria/ma_bitmap.c: mac compiler warnings fixed. storage/maria/ma_blockrec.c: mac compiler warnings fixed. storage/maria/ma_check.c: mac compiler warnings fixed. storage/maria/ma_control_file.c: mac compiler warnings fixed. storage/maria/ma_create.c: mac compiler warnings fixed. storage/maria/ma_delete.c: mac compiler warnings fixed. storage/maria/ma_ft_boolean_search.c: mac compiler warnings fixed. storage/maria/ma_page.c: mac compiler warnings fixed. storage/maria/ma_pagecache.c: mac compiler warnings fixed. storage/maria/ma_recovery.c: mac compiler warning fixed. storage/maria/ma_rt_test.c: mac compiler warnings fixed. storage/maria/ma_search.c: mac compiler warning fixed. storage/maria/ma_write.c: mac compiler warnings fixed. storage/maria/unittest/ma_control_file-t.c: mac compiler warnings fixed. storage/maria/unittest/ma_pagecache_consist.c: mac compiler warnings fixed. storage/maria/unittest/ma_pagecache_single.c: mac compiler warnings fixed. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: mac compiler warning fixed. storage/maria/unittest/test_file.c: mac compiler warning fixed. |
||
unknown
|
dc36524546 |
Compiler warnings fixed.
storage/maria/ma_loghandler.c: Compiler warning fixed. storage/maria/ma_test1.c: Compiler warning fixed. storage/maria/ma_test2.c: Compiler warning fixed. storage/maria/maria_chk.c: Compiler warning fixed. storage/maria/unittest/ma_control_file-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_multithread-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_noflush-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_nologs-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Compiler warning fixed. storage/maria/unittest/ma_test_loghandler_purge-t.c: Compiler warning fixed. |
||
unknown
|
c719e1fd33 |
Added wrapper for chmod(): my_chmod()
Automaticly disable ma_test_recovery if not compiled with debugging. This fixes that make test works in Maria. Fixed wrong merge of ma_init.c from 5.1 Portability fixes: - Use my_chmod() instead of my_chmod() - Use my_access() instead of my_stat() to test if file exists - Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable - No reason to test if file exists before we delete it include/my_sys.h: Added my_chmod include/mysys_err.h: Added error for my_chmod mysys/Makefile.am: Added my_chmod mysys/errors.c: Added error for my_chmod mysys/my_init.c: Syncronize with 5.1 to fix setting of QueryPerformanceFrequency() storage/maria/ma_test1.c: Changed short option of --skip-delete and --skip-update to be more logical storage/maria/ma_test_recovery.expected: Updated results after adding more tests storage/maria/ma_test_recovery: Abort test nicely if we are runnning without debugging Added more tests Changed temporary file names so that one can run maria_chk on them Removed some old comments storage/maria/maria_read_log.c: Added note if maria_read_log will not be able to create byte-to-byte identical tables compared to normal execution storage/maria/unittest/ma_pagecache_consist.c: Removed wrong setting of buff that caused memory overwrite Use my_chmod() instead of chmod() Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable storage/maria/unittest/ma_pagecache_single.c: Use my_chmod() instead of chmod() Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: No reason to test if file exists before we delete it storage/maria/unittest/ma_test_loghandler_multithread-t.c: Don't test result value of pthread_mutex_lock() and pthread_mutex_unlock() as this is not portable storage/maria/unittest/ma_test_loghandler_noflush-t.c: No reason to test if file exists before we delete it storage/maria/unittest/ma_test_loghandler_nologs-t.c: Use my_access() instead of my_stat() to test if file exists storage/maria/unittest/ma_test_loghandler_pagecache-t.c: No reason to test if file exists before we delete it chmod -> my_chmod mysys/my_chmod.c: Added wrapper for chmod() |
||
unknown
|
f8b3e118ba |
Disable logging of index pages during repair
Fixed failure in unittest/ma_test_loghandler_pagecache-t Initialize pagecache callbacks explictily, not with pagecache_init(). This is to make things more readable and for the future to make more choices with callbacks storage/maria/ha_maria.cc: Disable logging of index pages during repair storage/maria/ma_bitmap.c: Initialize callbacks explictily, not with pagecache_init(), to make things more readable and for future to have more choices with callbacks Use new interface to flush logs from pagecache storage/maria/ma_check.c: Fixed test for wrong keyblocks Use default functions to setup callbacks for pagecache storage/maria/ma_loghandler.c: Use dummy functions for log flush callback (NULL doesn't work anymore) storage/maria/ma_open.c: Initialize callbacks explictily, not with pagecache_init(), to make things more readable and for future to have more choices with callbacks Prefix external functions with _ma_ storage/maria/ma_pagecache.c: Use new simpler interface to flush logs if needed storage/maria/ma_pagecache.h: Changed interface to a faster, simpler one to flush logs. Now we have a function that takes care of flushing logs, instead of a function to get lsn address storage/maria/ma_pagecrc.c: Add functions for flushing logs storage/maria/ma_recovery.c: Rename functions storage/maria/maria_chk.c: Use default functions to setup callbacks for pagecache storage/maria/maria_def.h: Prefixd global functions with _ma_ storage/maria/unittest/ma_pagecache_consist.c: Use dummy functions for log flush callback (NULL doesn't work anymore) storage/maria/unittest/ma_pagecache_single.c: Use dummy functions for log flush callback (NULL doesn't work anymore) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Use maria_flush_log_for_page to flush log pages. Fixes failure in unittest |
||
unknown
|
4902e80471 | merge | ||
unknown
|
18bc7b695a |
WL#3072 - Maria Recovery
* to honour WAL we now force the whole log when flushing a bitmap page. * ability to intentionally crash in various places for recovery testing * bugfix (dirty pages list found in checkpoint record was ignored) * smaller checkpoint record * misc small cleanups and comments mysql-test/include/maria_empty_logs.inc: maria-purge.test creates ~11 logs, remove them all mysql-test/r/maria-recovery-bitmap.result: result is good; without the _ma_bitmap_get_log_address() call, we got check error Bitmap at 0 has pages reserved outside of data file length mysql-test/r/maria-recovery.result: result update mysql-test/t/maria-recovery-bitmap.test: enable test of "bitmap-flush should flush whole log otherwise corrupted data file (bitmap ahead of data pages)". mysql-test/t/maria-recovery.test: test of checkpoint sql/sql_table.cc: comment storage/maria/ha_maria.cc: _ma_reenable_logging_for_table() now includes file->trn=0. At the end of repair() we don't need to re-enable logging, it is done already by caller (like copy_data_between_tables()); it sounds strange that this function could decide to re-enable, it should be up to caller who knows what other operations it plans. Removing this line led to assertion failure in maria_lock_database(F_UNLCK), fixed by removing the assertion: maria_lock_database() is here called in a context where F_UNLCK does not make the table visible to others so assertion is excessive, and external_lock() is already designed to honour the asserted condition. Ability to crash at the end of bulk insert when indices have been enabled. storage/maria/ma_bitmap.c: Better use pagecache_file_init() than set pagecache callbacks directly; and a new function to set those callbacks for bitmap so that we can reuse it. _ma_bitmap_get_log_address() is a pagecache get_log_address callback which causes the whole log to be flushed when a bitmap page is flushed by the page cache. This was required by WAL. storage/maria/ma_blockrec.c: get_log_address pagecache callback for data (non bitmap) pages: just reads the LSN from the page's content, like was hard-coded before in ma_pagecache.c. storage/maria/ma_blockrec.h: functions which need to be exported storage/maria/ma_check.c: create_new_data_handle() can be static. Ability to crash after rebuilding the index in OPTIMIZE, in REPAIR. my_lock() implemented already. storage/maria/ma_checkpoint.c: As MARIA_SHARE* is now accessible to pagecache_collect_changed_blocks_LSN(), we don't need to store kfile/dfile descriptors in checkpoint record, 2-byte-id of the table plus one byte to say if this is data or index file is enough. So we go from 4+4 bytes per table down to 2+1. storage/maria/ma_commit.c: removing duplicate functions (see _ma_tmp_disable_logging_for_table()) storage/maria/ma_extra.c: Monty fixed storage/maria/ma_key_recover.c: comment storage/maria/ma_locking.c: Sometimes other code does funny things with maria_lock_database(), like ha_maria::repair() calling it at start and end without going through ha_maria::external_lock(). So it happens that maria_lock_database() is called with now_transactional!=born_transactional. storage/maria/ma_loghandler.c: update to new prototype storage/maria/ma_open.c: set_data|index_pagecache_callbacks() need to be exported as they are now called when disabling/enabling transactionality. storage/maria/ma_pagecache.c: Removing PAGE_LSN_OFFSET, as much of the code relies on it being 0 anyway (let's not give impression we can just change this constant). When flushing a page to disk, call the get_log_address callback to know up to which LSN the log should be flushed. As we now can access MARIA_SHARE* we can know share->id and store it into the checkpoint record; we thus go from 4 bytes per dirty page to 2+1. storage/maria/ma_pagecache.h: get_log_address callback storage/maria/ma_panic.c: No reason to reset pagecache callbacks in HA_PANIC_READ: all we do is reopen files if they were closed; callbacks should be in place already as 'info' exists; we just want to modify the file descriptors, not the full PAGECACHE_FILE structure. If we open data file and it was closed, share->bitmap.file needs to be set. Note that the modified code is disabled anyway. storage/maria/ma_recovery.c: Checkpoint record does not contain kfile/dfile descriptors anymore so code can be simplified. Hash key in all_dirty_pages is not made from file_descriptor & pageno anymore, but index_or_data & table-short-id & pageno. If a table's create_rename_lsn is higher than record's LSN, we skip the table and don't fail if it's corrupted (because the LSNs say that we don't have to look at this table). If a table is skipped (for example due to create_rename_lsn), its UNDOs still cause undo_lsn to advance; this is so that if later we notice the transaction has to rollback we fail (as table should not be skipped in this case). Fixing a bug: the dirty_pages list was never used, because the LSN below which it was used was the minimum rec_lsn of dirty pages! It is now the min(checkpoint_start_log_horizon, min(trn's rec_lsn)). When we disable/reenable transactionality, we modify pagecache callbacks (needed for example for get_log_address: changing share->page_type is not enough anymore). storage/maria/ma_write.c: 'records' and 'checksum' are protected: they are updated under log's mutex in write-hooks when UNDO is written. storage/maria/maria_chk.c: remove use of duplicate functions. storage/maria/maria_def.h: set_data|index_pagecache_callbacks() need to be exported; _ma_reenable_logging_for_table() changes to a real function. storage/maria/unittest/ma_pagecache_consist.c: new prototype storage/maria/unittest/ma_pagecache_single.c: new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: new prototype |
||
unknown
|
098916f571 |
Parameter added to the log initializer to suppress unwanted
error messages during unit tests. |
||
unknown
|
bf9c8b8a10 |
Write failure callback added.
storage/maria/ma_key_recover.c: Initialization fixed (by Monty) |
||
unknown
|
18d408a913 |
Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria
into desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-callback storage/maria/Makefile.am: Auto merged storage/maria/ma_blockrec.c: Auto merged storage/maria/ma_check.c: Auto merged storage/maria/ma_create.c: Auto merged storage/maria/ma_loghandler.h: Auto merged storage/maria/ma_open.c: Auto merged storage/maria/ma_page.c: Auto merged storage/maria/ma_pagecache.c: Auto merged storage/maria/maria_chk.c: Auto merged storage/maria/maria_def.h: Auto merged storage/maria/unittest/ma_test_loghandler-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multithread-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_noflush-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_purge-t.c: Auto merged storage/maria/ma_bitmap.c: Merge. storage/maria/ma_loghandler.c: Merge. storage/maria/unittest/Makefile.am: Merge. |
||
unknown
|
7b19ba34fc |
Pagecache callbacks support added.
Page CRC check based on pagecache support added. Loghandler pagecache callbacks support added. Loghandler filecache rewritten. Support of deletting all logs added. storage/maria/Makefile.am: New file with functions for CRC calculation. storage/maria/ma_bitmap.c: Page CRC support. storage/maria/ma_blockrec.c: Removed code replaced by pagecache callbacks. storage/maria/ma_check.c: Page CRC support. storage/maria/ma_create.c: Page CRC support. storage/maria/ma_loghandler.c: Pagecache callbacks support. New file cache support. Removing log files support. storage/maria/ma_loghandler.h: CRC_LENGTH replaced with CRC_SIZE storage/maria/ma_open.c: Page CRC support. storage/maria/ma_page.c: Page CRC support. storage/maria/ma_pagecache.c: Pagecache callbacks support. storage/maria/ma_pagecache.h: Pagecache callbacks support. storage/maria/ma_panic.c: Page CRC support. storage/maria/maria_chk.c: Page CRC support. storage/maria/maria_def.h: Page CRC support. storage/maria/unittest/Makefile.am: New test of removing logs. storage/maria/unittest/ma_maria_log_cleanup.c: Memory leack fixed. storage/maria/unittest/ma_pagecache_consist.c: Pagecache callbacks support. storage/maria/unittest/ma_pagecache_single.c: Pagecache callbacks support. storage/maria/unittest/ma_test_loghandler-t.c: Fixed the test error processing. storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Fixed the test error processing. storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Fixed the test error processing. storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Fixed the test error processing. storage/maria/unittest/ma_test_loghandler_multithread-t.c: Fixed the test error processing. storage/maria/unittest/ma_test_loghandler_noflush-t.c: Fixed the test error processing. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Pagecache callbacks support. storage/maria/unittest/ma_test_loghandler_purge-t.c: Fixed the test error processing. storage/maria/unittest/test_file.c: Removed unneeded sync. |
||
unknown
|
fabdf6df6d |
Postmerge fix.
storage/maria/ma_loghandler.c: Postmerge fix. New state of the log added to proicess correctly shutdown. storage/maria/ma_loghandler.h: New state of the log added to proicess correctly shutdown. |
||
unknown
|
98a2c5e3cf |
Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria
into desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-readonly storage/maria/ha_maria.cc: Auto merged storage/maria/ma_create.c: Auto merged storage/maria/ma_open.c: Auto merged storage/maria/ma_test1.c: Auto merged storage/maria/ma_test2.c: Auto merged storage/maria/unittest/ma_test_loghandler-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multithread-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_noflush-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_purge-t.c: Auto merged storage/maria/ma_check.c: Merge. storage/maria/ma_loghandler.c: Merge. storage/maria/ma_loghandler.h: Merge. storage/maria/maria_read_log.c: Merge. storage/maria/unittest/Makefile.am: Merge. |
||
unknown
|
d225521c76 |
Transaction log behaviour in case of write
error fixed (switching to the read only mode). Added read only mode of transactions log handler. storage/maria/ha_maria.cc: Transaction log initialization parameters change. storage/maria/ma_check.c: New status variable of transactional log. storage/maria/ma_create.c: New status variable of transactional log. storage/maria/ma_loghandler.c: - New status variable added. - Checking the status variable in the loghandler interface functions added - All fails of loghandler functions revised. - UNRECOVERABLE_ERROR() removed. - Switching to read only mode added where it nead. - Checking of log state added before writes log content and changing status variables like sent_to_file, n_buffers_only, flushed. - Readonly loghandler initialization added. - Fixed problem with example table transactional log initialization. storage/maria/ma_loghandler.h: Readonly loghandler initialization added. Fixed problem with example table transactional log initialization. New status variable added. storage/maria/ma_open.c: New status variable of transactional log. storage/maria/ma_test1.c: Transaction log initialization parameters change. storage/maria/ma_test2.c: Transaction log initialization parameters change. storage/maria/maria_read_log.c: Transaction loghandler initialization in read only mode in case of only dysplay parameter. storage/maria/unittest/Makefile.am: Test of readonly mode added. storage/maria/unittest/ma_test_loghandler-t.c: Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. Fixed incorrect fprintf call parameters. storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Useing this test also as read only loghandler test. Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. storage/maria/unittest/ma_test_loghandler_multithread-t.c: Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. storage/maria/unittest/ma_test_loghandler_noflush-t.c: Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. storage/maria/unittest/ma_test_loghandler_purge-t.c: Transaction log initialization parameters change. Fixed problem with example table transactional log initialization. |
||
unknown
|
b9734f0d1a |
Merge bk-internal.mysql.com:/home/bk/mysql-maria
into mysql.com:/home/my/mysql-maria storage/maria/ha_maria.cc: Auto merged storage/maria/ma_loghandler.c: Auto merged storage/maria/ma_loghandler.h: Auto merged storage/maria/unittest/ma_test_loghandler-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multithread-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_noflush-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_purge-t.c: Auto merged storage/maria/unittest/test_file.c: Auto merged storage/maria/unittest/ma_pagecache_single.c: SCCS merged |
||
unknown
|
ebf7ab7bce |
Added error HA_ERR_FILE_TOO_SHORT to be used when files are shorter than expected (by my_read/my_pread)
Added debugger hook _my_dbug_put_break_here() that is called if we get a CRC that matches --debug-crc-break (my_crc_dbug_break) Fixed REDO_REPAIR to use all repair modes (repair, repair_by_sort, repair_paralell REDO_REPAIR now also logs used key map Fixed some bugs in REDO logging of key pages Better error messages from maria_read_log Added my_readwrite_flags to init_pagecache() to be able to get better error messages and simplify code. Don't allow pagecaches with less than 8 blocks (Causes strange crashes) Added EXTRA_DEBUG_KEY_CHANGES. When this is defined some REDO_INDEX entries contains page checksums (these are calculated and checked in DBUG mode, ignored otherwise) Fixed bug in ma_pagecache unit tests that caused program to sometimes fail Added some missing calls to MY_INIT() that caused some unit tests to fail Fixed that TRUNCATE works properly on temporary MyISAM files Updates some result files to new table checksums results (checksum when NULL fields are ignored) perl test-insert can be replayed with maria_read_log! sql/share/Makefile.am: Change mode to -rw-rw-r-- BitKeeper/etc/ignore: added storage/maria/unittest/page_cache_test_file_1 storage/maria/unittest/pagecache_debug.log include/maria.h: Added maria_tmpdir include/my_base.h: Added error HA_ERR_FILE_TOO_SHORT include/my_sys.h: Added variable my_crc_dbug_check Added function my_dbug_put_break_here() include/myisamchk.h: Added org_key_map (Needed for writing REDO record for REPAIR) mysql-test/r/innodb.result: Updated to new checksum algorithm (NULL ignored) mysql-test/r/mix2_myisam.result: Updated to new checksum algorithm (NULL ignored) mysql-test/r/myisam.result: Updated to new checksum algorithm (NULL ignored) mysql-test/t/myisam.test: Added used table mysys/checksum.c: Added DBUG for checksum results Added debugger hook so that _my_dbug_put_break_here() is called if we get matching CRC mysys/lf_alloc-pin.c: Fixed compiler warning mysys/my_handler.c: Added new error message mysys/my_init.c: If my_progname is not given, use 'unknown' form my_progname_short Added debugger function my_debug_put_break_here() mysys/my_pread.c: In case of too short file when MY_NABP or MY_FNABP is specified, give error HA_ERR_FILE_TO_SHORT mysys/my_read.c: In case of too short file when MY_NABP or MY_FNABP is specified, give error HA_ERR_FILE_TO_SHORT sql/mysqld.cc: Added debug option --debug-crc-break sql/sql_parse.cc: Trivial optimization storage/maria/ha_maria.cc: Renamed variable to be more logical Ensure that param.testflag is correct when calling repair Added extra argument to init_pagecache Set default value for maria_tempdir storage/maria/ma_blockrec.c: Test for HA_ERR_FILE_TOO_SHORT instead for -1 storage/maria/ma_cache.c: Test for HA_ERR_FILE_TOO_SHORT instead for -1 storage/maria/ma_check.c: Set param->testflag to match how repair is run (needed for REDO logging) Simple optimization Moved flag if page is node from pagelength to keypage-flag byte Log used key map in REDO log. storage/maria/ma_delete.c: Remember previous UNDO entry when writing undo (for future CLR records) Moved flag if page is node from pagelength to keypage-flag byte Fixed some bugs in redo logging Added CRC for some translog REDO_INDEX entries storage/maria/ma_dynrec.c: Test for HA_ERR_FILE_TOO_SHORT instead for -1 storage/maria/ma_ft_update.c: Fixed call to _ma_store_page_used() storage/maria/ma_key_recover.c: Added CRC for some translog REDO_INDEX entries Removed not needed pagecache_write() in _ma_apply_redo_index() storage/maria/ma_locking.c: Test for HA_ERR_FILE_TOO_SHORT instead for -1 storage/maria/ma_loghandler.c: Added used key map to REDO_REPAIR_TABLE storage/maria/ma_loghandler.h: Added operation for checksum of key pages storage/maria/ma_open.c: Allocate storage for undo lsn pointers storage/maria/ma_pagecache.c: Remove not needed include file Change logging to use fd: for file descritors as other code Added my_readwrite_flags to init_pagecache() to be able to get better error messages for maria_chk/maria_read_log Don't allow pagecaches with less than 8 blocks Remove wrong DBUG_ASSERT() storage/maria/ma_pagecache.h: Added readwrite_flags storage/maria/ma_recovery.c: Better error messages for maria_read_log: - Added eprint() for printing error messages - Print extra \n before error message if we are printing %0 %10 ... Added used key_map to REDO_REPAIR log entry More DBUG Call same repair method that was used by mysqld storage/maria/ma_rt_index.c: Moved flag if page is node from pagelength to keypage-flag byte storage/maria/ma_rt_key.c: Fixed call to _ma_store_page_used() storage/maria/ma_rt_split.c: Moved flag if page is node from pagelength to keypage-flag byte storage/maria/ma_static.c: Added maria_tmpdir storage/maria/ma_test1.c: Updated call to init_pagecache() storage/maria/ma_test2.c: Updated call to init_pagecache() storage/maria/ma_test3.c: Updated call to init_pagecache() storage/maria/ma_write.c: Removed #ifdef NOT_YET Moved flag if page is node from pagelength to keypage-flag byte Fixed bug in _ma_log_del_prefix() storage/maria/maria_chk.c: Fixed wrong min limit for page_buffer_size Updated call to init_pagecache() storage/maria/maria_def.h: Added EXTRA_DEBUG_KEY_CHANGES. When this is defined some REDO_INDEX entries contains page checksums Moved flag if page is node from pagelength to keypage-flag byte storage/maria/maria_ftdump.c: Updated call to init_pagecache() storage/maria/maria_pack.c: Updated call to init_pagecache() Reset share->state.create_rename_lsn & share->state.is_of_horizon storage/maria/maria_read_log.c: Better error messages Added --tmpdir option (needed to set temporary directory for REDO_REPAIR) Added --start-from-lsn Changed option for --display-only to 'd' (wanted to use -o for 'offset') storage/maria/unittest/lockman2-t.c: Added missing call to MY_INIT() storage/maria/unittest/ma_pagecache_consist.c: Updated call to init_pagecache() storage/maria/unittest/ma_pagecache_single.c: Fixed bug that caused program to sometimes fail Added some DBUG_ASSERTS() Changed some calls to malloc()/free() to my_malloc()/my_free() Create extra file to expose original hard-to-find bug storage/maria/unittest/ma_test_loghandler-t.c: Updated call to init_pagecache() storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Updated call to init_pagecache() storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Updated call to init_pagecache() storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Updated call to init_pagecache() storage/maria/unittest/ma_test_loghandler_multithread-t.c: Updated call to init_pagecache() storage/maria/unittest/ma_test_loghandler_noflush-t.c: Updated call to init_pagecache() storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Updated call to init_pagecache() storage/maria/unittest/ma_test_loghandler_purge-t.c: Updated call to init_pagecache() storage/maria/unittest/test_file.c: Changed malloc()/free() to my_malloc()/my_free() Fixed memory leak Changd logic a bit while trying to find bug in reset_file() storage/maria/unittest/trnman-t.c: Added missing call to MY_INIT() storage/myisam/mi_cache.c: Test for HA_ERR_FILE_TOO_SHORT instead for -1 storage/myisam/mi_create.c: Removed O_EXCL to get TRUNCATE to work for temporary files storage/myisam/mi_dynrec.c: Test for HA_ERR_FILE_TOO_SHORT instead for -1 storage/myisam/mi_locking.c: Test for HA_ERR_FILE_TOO_SHORT instead for -1 mysql-test/r/old-mode.result: New BitKeeper file ``mysql-test/r/old-mode.result'' mysql-test/t/old-mode-master.opt: New BitKeeper file ``mysql-test/t/old-mode-master.opt'' mysql-test/t/old-mode.test: New BitKeeper file ``mysql-test/t/old-mode.test'' |
||
unknown
|
3e90f0d86c |
Added variable to control log directory syncs.
Part of postreview fixes added. mysql-test/r/maria.result: Added variable to control log directory syncs. storage/maria/ha_maria.cc: Added variable to control log directory syncs. storage/maria/ma_loghandler.h: Added variable to control log directory syncs. storage/maria/ma_loghandler_lsn.h: postreview fix storage/maria/unittest/Makefile.am: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler_multigroup-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler_multithread-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler_noflush-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_test_loghandler_purge-t.c: New file with logrecord descriptors for tests. storage/maria/unittest/ma_loghandler_examples.c: New BitKeeper file ``storage/maria/unittest/ma_loghandler_examples.c'' |
||
unknown
|
f3e957b7c1 |
Support of rec_lsn added to pagecache_write call.
storage/maria/ma_pagecache.c: Support of rec_lsn added to pagecache_write call. A function for setting rec_lsn made. |
||
unknown
|
d0b9387b88 |
WL#3072 - Maria recovery.
* Recovery of the table's live checksum (CREATE TABLE ... CHECKSUM=1) is achieved in this patch. The table's live checksum (info->s->state.state.checksum) is updated in inwrite_rec_hook's under the log mutex when writing UNDO_ROW_INSERT|UPDATE|DELETE and REDO_DELETE_ALL. The checksum variation caused by the operation is stored in these UNDOs, so that the REDO phase, when it sees such UNDOs, can update the live checksum if it is older (state.is_of_lsn is lower) than the record. It is also used, as a nice add-on with no cost, to do less row checksum computation during the UNDO phase (as we have it in the record already). Doing this work, it became pressing to move in-write hooks (write_hook_for_redo() et al) to ma_blockrec.c. The 'parts' argument of inwrite_rec_hook is unpredictable (it comes mangled at this stage, for example by LSN compression) so it is replaced by a 'void* hook_arg', which is used to pass down information, currently only to write_hook_for_clr_end() (previous undo_lsn and type of undone record). * If from ha_maria, we print to stderr how many seconds (with one fractional digit) the REDO phase took, same for UNDO phase and for final table close. Just to give an indication for debugging and maybe also for Support. storage/maria/ha_maria.cc: question for Monty storage/maria/ma_blockrec.c: * log in-write hooks (write_hook_for_redo() etc) move from ma_loghandler.c to here; this is natural: the hooks are coupled to their callers (functions in ma_blockrec.c). * translog_write_record() now has a new argument "hook_arg"; using it to pass down to write_hook_for_clr_end() the transaction's previous_undo_lsn and the type of the being undone record, and also to pass down to all UNDOs the live checksum variation caused by the operation. * If table has live checksum, store in UNDO_ROW_INSERT|UPDATE|DELETE and in CLR_END the checksum variation ("delta") caused by the operation. For example if a DELETE caused the table's live checksum to change from 123 to 456, we store in the UNDO_ROW_DELETE, in 4 bytes, the value 333 (456-123). * Instead of hard-coded "1" as length of the place where we store the undone record's type in CLR_END, use a symbol CLR_TYPE_STORE_SIZE; use macros clr_type_store and clr_type_korr. * write_block_record() has a new parameter 'old_record_checksum' which is the pre-computed checksum of old_record; that value is used to update the table's live checksum when writing UNDO_ROW_UPDATE|CLR_END. * In allocate_write_block_record(), if we are executing UNDO_ROW_DELETE the row's checksum is already computed. * _ma_update_block_record2() now expect the new row's checksum into cur_row.checksum (was already true) and the old row's checksum into new_row.checksum (that's new). Its two callers, maria_update() and _ma_apply_undo_row_update(), honour this. * When executing an UNDO_ROW_INSERT|UPDATE|DELETE in UNDO phase, pick up the checksum delta from the log record. It is then used to update the table's live checksum when writing CLR_END, and saves us a computation of record. storage/maria/ma_blockrec.h: in-write hooks move from ma_loghandler.c storage/maria/ma_check.c: more straightforward size of buffer storage/maria/ma_checkpoint.c: <= is enough storage/maria/ma_commit.c: new prototype of translog_write_record() storage/maria/ma_create.c: new prototype of translog_write_record() storage/maria/ma_delete.c: The row's checksum must be computed before calling(*delete_record)(), not after, because it must be known inside _ma_delete_block_record() (to update the table's live checksum when writing UNDO_ROW_DELETE). If deleting from a transactional table, live checksum was already updated when writing UNDO_ROW_DELETE. storage/maria/ma_delete_all.c: @todo is now done (in ma_loghandler.c) storage/maria/ma_delete_table.c: new prototype of translog_write_record() storage/maria/ma_loghandler.c: * in-write hooks move to ma_blockrec.c. * translog_write_record() gets a new argument 'hook_arg' which is passed down to pre|inwrite_rec_hook. It is more useful that 'parts' for those hooks, because when those hooks are called, 'parts' has possibly been mangled (like with LSN compression) and is so unpredictable. * fix for compiler warning (unused buffer_start when compiling without debug support) * Because checksum delta is stored into UNDO_ROW_INSERT|UPDATE|DELETE and CLR_END, but only if the table has live checksum, these records are not PSEUDOFIXEDLENGTH anymore, they are now VARIABLE_LENGTH (their length is X if no live checksum and X+4 otherwise). * add an inwrite_rec_hook for UNDO_ROW_UPDATE, which updates the table's live checksum. Update it also in hooks of UNDO_ROW_INSERT| DELETE and REDO_DELETE_ALL and CLR_END. * Bugfix: when reading a record in translog_read_record(), it happened that "length" became negative, because the function assumed that the record extended beyond the page's end, whereas it may be shorter. storage/maria/ma_loghandler.h: * Instead of hard-coded "1" and "4", use symbols and macros to store/retrieve the type of record which the CLR_END corresponds to, and the checksum variation caused by the operation which logs the record * translog_write_record() gets a new argument 'hook_arg' which is passed down to pre|inwrite_rec_hook. It is more useful that 'parts' for those hooks, because when those hooks are called, 'parts' has possibly been mangled (like with LSN compression) and is so unpredictable. storage/maria/ma_open.c: fix for "empty body in if() statement" (when compiling without safemutex) storage/maria/ma_pagecache.c: <= is enough storage/maria/ma_recovery.c: * print the time that each recovery phase (REDO/UNDO/flush) took; this is enabled only when recovering from ha_maria. Is it printed n seconds with a fractional part of one digit (like 123.4 seconds). * In the REDO phase, update the table's live checksum by using the checksum delta stored in UNDO_ROW_INSERT|DELETE|UPDATE and CLR_END. Update it too when seeing REDO_DELETE_ALL. * In the UNDO phase, when executing UNDO_ROW_INSERT, if the table does not have live checksum then reading the record's header (as done by the master loop of run_undo_phase()) is enough; otherwise we do a translog_read_record() to have the checksum delta ready for _ma_apply_undo_row_insert(). * When at the end of the REDO phase we notice that there is an unfinished group of REDOs, don't assert in debug binaries, as I verified that it can happen in real life (with kill -9) * removing ' in #error as it confuses gcc3 storage/maria/ma_rename.c: new prototype of translog_write_record() storage/maria/ma_test_recovery.expected: Change in output of ma_test_recovery: now all live checksums of original tables equal those of tables recreated by the REDO phase and those of tables fixed by the UNDO phase. I.e. recovery of the live checksum looks like working (which was after all the only goal of this changeset). I checked by hand that it's not just all live checksums which are now 0 and that's why they match. They are the old values like 3757530372. maria.test has hard-coded checksum values in its result file so checks this too. storage/maria/ma_update.c: * It's useless to put up HA_STATE_CHANGED in 'key_changed', as we put up HA_STATE_CHANGED in info->update anyway. * We need to compute the old and new rows' checksum before calling (*update_record)(), as checksum delta must be known when logging UNDO_ROW_UPDATE which is done by _ma_update_block_record(). Note that some functions change the 'newrec' record (at least _ma_check_unique() does) so we cannot move the checksum computation too early in the function. storage/maria/ma_write.c: If inserting into a transactional table, live's checksum was already updated when writing UNDO_ROW_INSERT. The multiplication is a trick to save an if(). storage/maria/unittest/ma_test_loghandler-t.c: new prototype of translog_write_record() storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: new prototype of translog_write_record() storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: new prototype of translog_write_record() storage/maria/unittest/ma_test_loghandler_multigroup-t.c: new prototype of translog_write_record() storage/maria/unittest/ma_test_loghandler_multithread-t.c: new prototype of translog_write_record() storage/maria/unittest/ma_test_loghandler_noflush-t.c: new prototype of translog_write_record() storage/maria/unittest/ma_test_loghandler_pagecache-t.c: new prototype of translog_write_record() storage/maria/unittest/ma_test_loghandler_purge-t.c: new prototype of translog_write_record() storage/myisam/sort.c: fix for compiler warnings in pushbuild (write_merge_key* functions didn't have their declaration match MARIA_HA::write_key). |
||
unknown
|
73b073fff6 |
Check of transaction log descriptor table consistance added.
Small fixes made. storage/maria/ma_loghandler.c: Check of transaction log descriptor table consistance added.\ Incorrect record description fixed. Compiler warning fixed. storage/maria/ma_loghandler.h: fixed ident. storage/maria/unittest/ma_test_loghandler-t.c: Suppressing of automatic record writing storage/maria/unittest/ma_test_loghandler_first_lsn-t.c: Suppressing of automatic record writing storage/maria/unittest/ma_test_loghandler_max_lsn-t.c: Suppressing of automatic record writing storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Suppressing of automatic record writing storage/maria/unittest/ma_test_loghandler_multithread-t.c: Suppressing of automatic record writing storage/maria/unittest/ma_test_loghandler_noflush-t.c: Suppressing of automatic record writing storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Suppressing of automatic record writing storage/maria/unittest/ma_test_loghandler_purge-t.c: Suppressing of automatic record writing |
||
unknown
|
3d0f42a94c |
Log handler's unit test were broken by me on June 22, fixing this:
don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests, as they read their log and expect to find only the records they asked for, and so not LOGREC_LONG_TRANSACTION_ID. All log handler's unit tests pass. A way to run them faster: when their LONG_BUFFER_SIZE is too big (1GB) I divide it by ten and then I can run them on /dev/shm - they are then "instant". By the way, pushbuild was not displaying anymore "unit: failed" in the list of all pushes, which participates to the reasons why I didn't notice the breakage earlier. The other reason being that I was too lazy to run log handler unit tests on my machine as they took long (hadn't yet thought about the /dev/shm idea) and so I relied on pushbuild; Danny has now quickly fixed pushbuild - thanks :) storage/maria/unittest/ma_test_loghandler-t.c: don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests, as they read their log and expect to find only the records they asked for storage/maria/unittest/ma_test_loghandler_multigroup-t.c: don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests, as they read their log and expect to find only the records they asked for storage/maria/unittest/ma_test_loghandler_multithread-t.c: don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests, as they read their log and expect to find only the records they asked for storage/maria/unittest/ma_test_loghandler_pagecache-t.c: don't auto-log LOGREC_LONG_TRANSACTION_ID in log handler's unit tests, as they read their log and expect to find only the records they asked for |
||
unknown
|
8ba21c616e |
Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-maria
into a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-maria.prod BitKeeper/etc/ignore: auto-union include/maria.h: Auto merged include/my_base.h: Auto merged mysql-test/include/ps_conv.inc: Auto merged mysql-test/r/ps_2myisam.result: Auto merged mysql-test/r/ps_3innodb.result: Auto merged mysql-test/r/ps_4heap.result: Auto merged mysql-test/r/ps_5merge.result: Auto merged mysql-test/r/ps_7ndb.result: Auto merged mysys/Makefile.am: Auto merged mysys/mf_iocache.c: Auto merged mysys/mf_tempfile.c: Auto merged mysys/my_error.c: Auto merged mysys/my_init.c: Auto merged mysys/my_open.c: Auto merged sql/handler.cc: Auto merged sql/handler.h: Auto merged sql/lex.h: Auto merged sql/sql_show.cc: Auto merged sql/sql_yacc.yy: Auto merged sql/table.cc: Auto merged sql/table.h: Auto merged storage/maria/ha_maria.cc: Auto merged storage/maria/ha_maria.h: Auto merged storage/maria/ma_bitmap.c: Auto merged storage/maria/ma_blockrec.h: Auto merged storage/maria/ma_create.c: Auto merged storage/maria/ma_dynrec.c: Auto merged storage/maria/ma_loghandler.h: Auto merged storage/maria/ma_loghandler_lsn.h: Auto merged storage/maria/ma_open.c: Auto merged storage/maria/ma_pagecache.c: Auto merged storage/maria/ma_pagecache.h: Auto merged storage/maria/ma_test1.c: Auto merged storage/maria/ma_test2.c: Auto merged storage/maria/ma_update.c: Auto merged storage/maria/maria_chk.c: Auto merged storage/maria/unittest/ma_test_loghandler-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_multithread-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Auto merged storage/myisam/ft_stopwords.c: Auto merged storage/myisam/mi_close.c: Auto merged support-files/compiler_warnings.supp: Auto merged mysys/mf_keycache.c: Pulled from mysql-maria tree, manual merge. mysys/my_handler.c: Pulled from mysql-maria tree, manual merge. mysys/my_symlink2.c: Pulled from mysql-maria tree, manual merge. sql/mysqld.cc: Pulled from mysql-maria tree, manual merge. sql/sql_table.cc: Pulled from mysql-maria tree, manual merge. storage/maria/ma_blockrec.c: Pulled from mysql-maria tree, manual merge. storage/maria/ma_check.c: Pulled from mysql-maria tree, manual merge. storage/maria/ma_close.c: Pulled from mysql-maria tree, manual merge. storage/maria/ma_loghandler.c: Pulled from mysql-maria tree, manual merge. storage/maria/maria_def.h: Pulled from mysql-maria tree, manual merge. |
||
unknown
|
631ecaabea |
Merged with mysql-5.1 main tree.
BUILD/compile-pentium-debug-max: Added definition after macro was removed from main tree. This will be fixed back in main tree later. |
||
unknown
|
adac9798bf |
WL#3072 Maria Recovery
- new program maria_read_log to display and apply log records found in a Maria log (see file's revision comment) - minor, misc fixes storage/maria/Makefile.am: new program maria_read_log storage/maria/ha_maria.cc: create control file if missing storage/maria/ma_blockrec.c: 0 -> LSN_IMPOSSIBLE; comments storage/maria/ma_checkpoint.h: preparations for Checkpoint module storage/maria/ma_close.c: comment storage/maria/ma_control_file.c: renaming constants. Possibility to say "open control file but don't create it if it's missing" (used by maria_read_log which does not want to create anything) storage/maria/ma_control_file.h: renaming constants storage/maria/ma_create.c: I had duplicated "linkname" and "linkname_ptr", now I see it's not needed, reverting. Indeed those variables don't contain interesting information; fixing log record accordingly (the links are in ci->data/index_file_name). Storing keystart in log record is needed, to know at which size we must extend the file if we replay LOGREC_CREATE_TABLE. storage/maria/ma_loghandler.c: some structures need to be known to maria_read_log.c, taking them to ma_loghandler.h storage/maria/ma_loghandler.h: we have page_store, adding page_korr. translog_lock() made public, because Checkpoint will need it (to write to control file). Some structures moved from ma_loghandler.c because maria_read_log.c needs them (needs to know the execute-in-REDO-phase hooks of each record). storage/maria/ma_loghandler_lsn.h: constants defined in ma_control_file.h serve everywhere, and they relate to LSNs, so putting them in ma_loghandler_lsn.h. Stronger constraints in LSN_VALID(). storage/maria/ma_pagecache.c: renaming constants storage/maria/ma_recovery.h: copyright storage/maria/ma_test1.c: new prototype storage/maria/ma_test2.c: new prototype storage/maria/trnman_public.h: double-inclusion safe storage/maria/unittest/ma_control_file-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: constants renamed, new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: constants renamed, new prototype storage/myisam/mi_close.c: comment storage/maria/maria_read_log.c: program to read and print log records from a Maria transaction log, and optionally apply them to tables. Very basic, early version. Should serve as a base for Recovery's code. Designed to be idempotent. Create a log by running maria.test, then cd to var/master-data and run "maria_read_log --only-display" to see info about records; run "maria_read_log --display-and-apply" to also apply the records to tables (it's more interesting if you first wipe out the tables in var/master-data/test, to see how they get re-created). Only a few records are handled by now: LONG_TRANSACTION_ID, COMMIT, FILE_ID, REDO_CREATE_TABLE; place is ready for REDO_INSERT_ROW_HEAD where I could use Monty's help (search for "Monty" in the file). Note: changes to the index pages, index's header and bitmap pages are not properly logged yet, so don't expect the program to work with that. |
||
unknown
|
1a96259191 |
- WL#3239 "log CREATE TABLE in Maria"
- WL#3240 "log DROP TABLE in Maria" - similarly, log RENAME TABLE, REPAIR/OPTIMIZE TABLE, and DELETE no_WHERE_clause (== the DELETE which just truncates the files) - create_rename_lsn added to MARIA_SHARE's state - all these operations (except DROP TABLE) also update the table's create_rename_lsn, which is needed for the correctness of Recovery (see function comment of _ma_repair_write_log_record() in ma_check.c) - write a COMMIT record when transaction commits. - don't log REDOs/UNDOs if this is an internal temporary table like inside ALTER TABLE (I expect this to be a big win). There was already no logging for user-created "CREATE TEMPORARY" tables. - don't fsync files/directories if the table is not transactional - in translog_write_record(), autogenerate a 2-byte-id for the table and log the "id->name" pair (LOGREC_FILE_ID); log LOGREC_LONG_TRANSACTION_ID; automatically store the table's 2-byte-id in any log record. - preparations for Checkpoint: translog_get_horizon(); pausing Checkpoint when some dirty pages are unknown; capturing trn->rec_lsn, trn->first_undo_lsn for Checkpoint and log's low-water-mark computing. - assertions, comments. storage/maria/Makefile.am: more files to build storage/maria/ha_maria.cc: - logging a REPAIR log record if REPAIR/OPTIMIZE was successful. - ha_maria::data_file_type does not have to be set in every info() call, just do it once in open(). - if caller said that transactionality can be disabled (like if caller is ALTER TABLE) i.e. thd->transaction.on==FALSE, then we temporarily disable transactionality of the table in external_lock(); that will ensure that no REDOs/UNDOs are logged for this possibly massive write operation (they are not needed, as if any write fails, the table will be dropped). We re-enable in external_lock(F_UNLCK), which in ALTER TABLE happens before the tmp table replaces the original one (which is good, as thus the final table will have a REDO RENAME and a correct create_rename_lsn). - when we commit we also have to write a log record, so trnman_commit_trn() calls become ma_commit() calls - at end of engine's initialization, we are potentially entering a multi-threaded dangerous world (clients are going to be accepted) and so some assertions of mutex-owning become enforceable, for that we set maria_multi_threaded=TRUE (see ma_control_file.c) storage/maria/ha_maria.h: new member ha_maria::save_transactional (see also ha_maria.cc) storage/maria/ma_blockrec.c: - fixing comments according to discussion with Monty - if a table is transactional but temporarily non-transactional (like in ALTER TABLE), we need to give a sensible LSN to the pages (and, if we give 0, pagecache asserts). - translog_write_record() now takes care of storing the share's 2-byte-id in the log record storage/maria/ma_blockrec.h: fixing comment according to discussion with Monty storage/maria/ma_check.c: When REPAIR/OPTIMIZE modify the data/index file, if this is a transactional table, they must sync it; if they remove files or rename files, they must sync the directory, so that everything is durable. This is just applying to REPAIR/OPTIMIZE the logic already implemented in CREATE/DROP/RENAME a few months ago. Adding a function to write a LOGREC_REPAIR_TABLE at end of REPAIR/OPTIMIZE (called only by ha_maria, not by maria_chk), and to update the table's create_rename_lsn. storage/maria/ma_close.c: fix for a future bug storage/maria/ma_control_file.c: ensuring that if Maria is running in multi-threaded mode, anybody wanting to write to the control file and update last_checkpoint_lsn/last_logno owns the log's lock. storage/maria/ma_control_file.h: see ma_control_file.c storage/maria/ma_create.c: when creating a table: - sync it and its directory only if this is a transactional table and there is a log (no point in syncing in maria_chk) - decouple the two uses of linkname/linkname_ptr (for index file and for data file) into more variables, as we need to know all links until the moment we write the LOGREC_CREATE_TABLE. - set share.data_file_type early so that _ma_initialize_data_file() knows it (Monty's bugfix so that a table always has at least a bitmap page when it is created; so data-file is not 0 bytes anymore). - log a LOGREC_CREATE_TABLE; it contains the bytes which we have just written to the index file's header. Update table's create_rename_lsn. - syncing of kfile had been bugified in a previous merge, correcting - syncing of dfile is now needed as it's not empty anymore - in _ma_initialize_data_file(), use share's block_size and not the global one. This is a gratuitous change, both variables are equal, just that I find it more future-proof to use share-bound variable rather than global one. storage/maria/ma_delete_all.c: log a LOGREC_DELETE_ALL record when doing ma_delete_all_rows(); update create_rename_lsn then. storage/maria/ma_delete_table.c: - logging LOGREC_DROP_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - we need to sync directories only if the table is transactional storage/maria/ma_extra.c: questions storage/maria/ma_init.c: when maria_end() is called, engine is not multithreaded storage/maria/ma_loghandler.c: - translog_inited has to be visible to ma_create() (see how it is used in ma_create()) - checkpoint record will be a single record, not three - no REDO for TRUNCATE (TRUNCATE calls ma_create() internally so will log a REDO_CREATE) - adding REDO for DELETE no_WHERE_clause (fast DELETE of all rows by truncating the files), REPAIR. - MY_WAIT_IF_FULL to wait&retry if a log write hits a full disk - in translog_write_record(), if MARIA_SHARE does not yet have a 2-byte-id, generate one for it and log LOGREC_FILE_ID; automatically store this short id into log records. - in translog_write_record(), if transaction has not logged its long trid, log LOGREC_LONG_TRANSACTION_ID. - For Checkpoint, we need to know the current end-of-log: adding translog_get_horizon(). - For Control File, adding an assertion that the thread owns the log's lock (control file is protected by this lock) storage/maria/ma_loghandler.h: Changes in log records (see ma_loghandler.c). new prototypes, new functions. storage/maria/ma_loghandler_lsn.h: adding a type LSN_WITH_FLAGS especially for TRN::first_undo_lsn, where the most significant byte is used for flags. storage/maria/ma_open.c: storing the create_rename_lsn in the index file's header (in the state, precisely) and retrieving it from there. storage/maria/ma_pagecache.c: - my set_if_bigger was wrong, correcting it - if the first_in_switch list is not empty, it means that changed_blocks misses some dirty pages, so Checkpoint cannot run and needs to wait. A variable missing_blocks_in_changed_list is added to tell that (should it be named missing_blocks_in_changed_blocks?) - pagecache_collect_changed_blocks_with_lsn() now also tells the minimum rec_lsn (needed for low-water mark computation). storage/maria/ma_pagecache.h: see ma_pagecache.c storage/maria/ma_panic.c: comment storage/maria/ma_range.c: comment storage/maria/ma_rename.c: - logging LOGREC_RENAME_TABLE; knowing if this is needed, requires knowing if the table is transactional, which requires opening the table. - update create_rename_lsn - we need to sync directories only if the table is transactional storage/maria/ma_static.c: comment storage/maria/ma_test_all.sh: - tip for Valgrind-ing ma_test_all - do "export maria_path=somepath" before calling ma_test_all, if you want to run ma_test_all out of storage/maria (useful to have parallel runs, like one normal and one Valgrind, they must not use the same tables so need to run in different directories) storage/maria/maria_def.h: - state now contains, in memory and on disk, the create_rename_lsn - share now contains a 2-byte-id storage/maria/trnman.c: preparations for Checkpoint: capture trn->rec_lsn, trn->first_undo_lsn; minimum first_undo_lsn needed to know log's low-water-mark storage/maria/trnman.h: using most significant byte of first_undo_lsn to hold miscellaneous flags, for now TRANSACTION_LOGGED_LONG_ID. dummy_transaction_object is already declared in ma_static.c. storage/maria/trnman_public.h: dummy_transaction_object was declared in all files including trnman_public.h, while in fact it's a single object. new prototype storage/maria/unittest/ma_test_loghandler-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multigroup-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_multithread-t.c: update for new prototype storage/maria/unittest/ma_test_loghandler_pagecache-t.c: update for new prototype storage/maria/ma_commit.c: function which wraps: - writing a LOGREC_COMMIT record (==commit on disk) - calling trnman_commit_trn() (=commit in memory) storage/maria/ma_commit.h: new header file .tree-is-private: this file is now needed to keep our tree private (don't push it to public trees). When 5.1 is merged into mysql-maria, we can abandon our maria-specific post-commit trigger; .tree_is_private will take care of keeping commit mails private. Don't push this file to public trees. |
||
unknown
|
98e80921a8 |
Merge desktop.sanja.is.com.ua:/home/bell/mysql/bk/mysql-maria
into desktop.sanja.is.com.ua:/home/bell/mysql/bk/work-maria-lsn storage/maria/ma_loghandler.h: Auto merged storage/maria/unittest/ma_test_loghandler_multithread-t.c: Auto merged storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Auto merged storage/maria/ma_loghandler.c: merge storage/maria/unittest/ma_test_loghandler-t.c: merge storage/maria/unittest/ma_test_loghandler_multigroup-t.c: merge |
||
unknown
|
c548715c46 |
Fixed LSN codding to allow code all
relative LSN independed on "distance". Added support for test record descriptors to avoid interfere woth real record descriptors. Fixed descriptor of pseudofixed length record length, now it is length of record passed from/to client of the loghandler. BitKeeper/etc/ignore: Added storage/maria/unittest/ma_test_loghandler_long-t-big to the ignore list storage/maria/ma_init.c: Removed loghandler_init call because it is present in translog_init() storage/maria/ma_loghandler.c: Fixed LSN codding to allow code all relative LSN independed on "distance". Added support for test record descriptors to avoid interfere woth real record descriptors. Fixed length of LOGREC_REDO_INSERT_ROW_TAIL. Fixed descriptor of pseudofixed length record length, now it is length of record passed from/to client of the loghandler. storage/maria/ma_loghandler.h: Added support for test record descriptors to avoid interfere woth real record descriptors. storage/maria/unittest/Makefile.am: Made new test for log with reference over 63 files. Layout fixed. storage/maria/unittest/ma_test_loghandler-t.c: Added support for test record descriptors to avoid interfere woth real record descriptors. storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Added support for test record descriptors to avoid interfere woth real record descriptors. storage/maria/unittest/ma_test_loghandler_multithread-t.c: Added support for test record descriptors to avoid interfere woth real record descriptors. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Added support for test record descriptors to avoid interfere woth real record descriptors. |
||
unknown
|
704b39a158 |
- moving pagecache.h from include/ to storage/maria as it is Maria-
specific - adding TRN::first_undo_lsn, needed to know when a log can be deleted; this variable must be set under log's mutex and that leads to setting TRN::rec_lsn, TRN::undo_lsn and TRN::first_undo_lsn in a inwrite_rec_hook; adding implementation of one hook for REDOs and one for UNDOs. Thus translog_write_record() always uses TRN and so does not need a short_id argument, can find it from TRN. - Monty's patch for the last Valgrind error in the tree. - Log handler's unit tests fail but Sanja says it's known include/Makefile.am: pagecache.h moved and renamed include/maria.h: pagecache.h moved and renamed sql/handler.h: pagecache.h moved and renamed storage/maria/Makefile.am: pagecache.h moved and renamed storage/maria/ha_maria.cc: adding an assertion which sounds logical storage/maria/ma_blockrec.c: trn->rec_lsn and trn->undo_lsn are now set via hooks inside the log record's writing; this allows to also set trn->first_undo_lsn needed to compute the log's low-water mark. The PAGERANGE_STORE_SIZE -> PAGE_STORE_SIZE is Monty's fix to a Valgrind error. storage/maria/ma_loghandler.c: "tcb" renamed to "trn". Log handler now knows what is a transaction, and finds short_id from trn. trn's rec_lsn, undo_lsn, first_undo_lsn are now set by some inwrite_rec_hookS (one for REDOs, one for UNDOs). The HAVE_purify blocks are Monty's fix to a Valgrind error. storage/maria/ma_loghandler.h: Log handler functions use TRN, that needs a forward declaration storage/maria/ma_pagecache.c: pagecache.h was moved and renamed storage/maria/ma_pagecache.h: pagecache.h was moved and renamed storage/maria/ma_pagecaches.c: pagecache.h was moved and renamed storage/maria/trnman.c: initializing some members of TRN. storage/maria/trnman.h: TRN::first_undo_lsn needed for log's low-water mark calculation (which will serve to know which logs can be deleted) storage/maria/unittest/ma_test_loghandler-t.c: translog_write_record() now needs a valid TRN storage/maria/unittest/ma_test_loghandler_multigroup-t.c: translog_write_record() now needs a valid TRN storage/maria/unittest/ma_test_loghandler_multithread-t.c: translog_write_record() now needs a valid TRN storage/maria/unittest/ma_test_loghandler_pagecache-t.c: translog_write_record() now needs a valid TRN storage/maria/unittest/test_file.h: pagecache.h was moved and renamed |
||
unknown
|
fdfb51484c |
Fixed compiler warnings
Fixed bug in ma_dbug.c that gave valgrind warning (only relevant when using --debug) Fixed bug in blob logging (Fixes valgrind warning) maria_getint() -> maria_data_on_page() mysys/safemalloc.c: Added debug function to print out where a piece of memory was allocated sql/opt_range.cc: Remove DBUG_PRINT of unitailized memory storage/maria/ma_blockrec.c: Fixed bug in blob logging storage/maria/ma_check.c: Fixed compiler warning storage/maria/ma_dbug.c: Added missed end++; Caused usage of unitialized memory for nullable keys that was not NULL storage/maria/ma_delete.c: maria_getint() -> maria_data_on_page() storage/maria/ma_init.c: Added header file to get rid of warning storage/maria/ma_key.c: More debugging storage/maria/ma_loghandler.c: Removed some wrong ';' to get rid of compiler errors when compiling without debugging Indentation fixes Removed not needed 'break's Fixed some compiler warnings Added code to detect logging of unitialized memory storage/maria/ma_page.c: maria_getint() -> maria_data_on_page() Clear rest of index page before writing when used with valgrind (Fixes warning of writing pages with unitialized data) storage/maria/ma_range.c: maria_getint() -> maria_data_on_page() storage/maria/ma_rt_index.c: maria_getint() -> maria_data_on_page() storage/maria/ma_rt_index.h: maria_getint() -> maria_data_on_page() storage/maria/ma_rt_key.c: maria_getint() -> maria_data_on_page() storage/maria/ma_rt_split.c: maria_getint() -> maria_data_on_page() storage/maria/ma_search.c: maria_getint() -> maria_data_on_page() storage/maria/ma_test1.c: Fixed compiler warning storage/maria/ma_write.c: maria_getint() -> maria_data_on_page() storage/maria/maria_chk.c: maria_getint() -> maria_data_on_page() storage/maria/maria_def.h: maria_getint() -> maria_data_on_page() storage/maria/unittest/ma_pagecache_consist.c: Fixed compiler warning storage/maria/unittest/ma_pagecache_single.c: Fixed compiler warning storage/maria/unittest/ma_test_loghandler-t.c: Fixed compiler warning storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Fixed compiler warning storage/maria/unittest/ma_test_loghandler_multithread-t.c: Fixed compiler warning storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Fixed compiler warning storage/myisam/mi_dbug.c: Added missed end++; Caused usage of unitialized memory for nullable keys that was not NULL |
||
unknown
|
8f39541e7d |
This patch is a collection of patches from from Sanja, Sergei and Monty.
Added logging and pinning of pages to block format. Integration of transaction manager, log handler. Better page cache intergration Split trnman.h into two files, so that we don't have to include my_atomic.h into C++ programs. Renaming of structures, more comments, more debugging etc. Fixed problem with small head block + long varchar. Added extra argument to delete_record() and update_record() (needed for UNDO logging) Small changes to interface of pagecache and log handler. Change initialization of log_record_type_descriptors to not be depending on enum order. Use array of LEX_STRING's to send data to log handler Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. include/lf.h: Interface fixes Rename of structures (Patch from Sergei via Sanja) include/my_atomic.h: More comments include/my_global.h: Added MY_ERRPTR include/pagecache.h: Added undo LSN when unlocking pages mysql-test/r/maria.result: Updated results mysql-test/t/maria.test: Added autocommit around lock tables (Patch from Sanja) mysys/lf_alloc-pin.c: Post-review fixes, simple optimizations More comments Struct slot renames Check amount of memory on stack (Patch from Sergei) mysys/lf_dynarray.c: More comments mysys/lf_hash.c: More comments After review fixes (Patch from Sergei) storage/maria/ha_maria.cc: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) Move out all deferencing of the transaction structure. Transaction manager integrated (Patch from Sergei) storage/maria/ha_maria.h: Added prototype for start_stmt() storage/maria/lockman.c: Function call rename storage/maria/ma_bitmap.c: Mark deleted pages free from page cache storage/maria/ma_blockrec.c: Offset -> rownr More debugging Fixed problem with small head block + long varchar Added logging of changed pages Added logging of undo (Including only loggging of changed fields in case of update) Added pinning/unpinning of all changed pages More comments Added free_full_pages() as the same code was used in several places. fill_rows_parts() renamed as fill_insert_undo_parts() offset -> rownr Added some optimization of not transactional tables _ma_update_block_record() has new parameter, as we need original row to do efficent undo for update storage/maria/ma_blockrec.h: Added ROW_EXTENTS_ON_STACK Changed prototype for update and delete of row storage/maria/ma_check.c: Added original row to delete_record() call storage/maria/ma_control_file.h: Added ifdefs for C++ storage/maria/ma_delete.c: Added original row to delete_record() call (Needed for efficent undo logging) storage/maria/ma_dynrec.c: Added extra argument to delete_record() and update_record() Removed not used variable storage/maria/ma_init.c: Initialize log handler storage/maria/ma_loghandler.c: Removed not used variable Change initialization of log_record_type_descriptors to not be depending on enum order Use array of LEX_STRING's to send data to log handler storage/maria/ma_loghandler.h: New defines Use array of LEX_STRING's to send data to log handler storage/maria/ma_open.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Store in MARIA_SHARE->page_type if pages will have up to date LSN's storage/maria/ma_pagecache.c: Don't decrease number of readers when using pagecache_write()/pagecache_read() In pagecache_write() decrement request count if page was left pinned Added pagecache_delete_pages() Removed some casts Make trace output consistent with rest of code Simplify calling of DBUG_ASSERT(0) Only update LSN if the LSN is bigger than what's already on the page Added LSN parameter pagecache_unpin_page(), pagecache_unpin(), and pagecache_unlock() (Part of patch from Sanja) storage/maria/ma_static.c: Added 'dummy' transaction option to MARIA_INFO so that we can always assume 'trn' exists. Added default page cache storage/maria/ma_statrec.c: Added extra argument to delete_record() and update_record() storage/maria/ma_test1.c: Added option -T for transactions storage/maria/ma_test2.c: Added option -T for transactions storage/maria/ma_test_all.sh: Test with transactions storage/maria/ma_update.c: Changed prototype for update of row storage/maria/maria_def.h: Changed prototype for update & delete of row as block records need to access the old row Store in MARIA_SHARE->page_type if pages will have up to date LSN's Added MARIA_MAX_TREE_LEVELS to allow us to calculate the number of possible pinned pages we may need. Removed not used 'empty_bits_buffer' Added pointer to transaction object Added array for pinned pages Added log_row_parts array for logging of field data. Added MARIA_PINNED_PAGE to store pinned pages storage/maria/trnman.c: Added accessor functions to transaction object Added missing DBUG_RETURN() More debugging More comments Changed // comment of code to #ifdef NOT_USED Transaction manager integrated. Post review fixes Part of patch originally from Sergei storage/maria/trnman.h: Split trnman.h into two files, so that we don't have to include my_atomic.h into the .cc program. (Temporary fix to avoid bug in gcc) storage/maria/unittest/ma_pagecache_single.c: Added missing argument Added SKIP_BIG_TESTS (Patch from Sanja) storage/maria/unittest/ma_test_loghandler-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Test logging with new LEX_STRING parameter (Patch from Sanja) storage/maria/unittest/trnman-t.c: Stack overflow detection (Patch from Sergei) unittest/unit.pl: Command-line options --big and --verbose (Patch from Sergei) unittest/mytap/tap.c: Detect --big (Patch from Sergei) unittest/mytap/tap.h: Skip_big_tests and SKIP_BIG_TESTS (Patch from Sergei) storage/maria/trnman_public.h: New BitKeeper file ``storage/maria/trnman_public.h'' |
||
unknown
|
3411bfe05a |
merge from MyISAM into Maria (last step of merge of 5.1 into Maria).
Tests: "maria" and "ps_maria" fail like before merge (assertions), "ma_test_all" fails like before merge (ma_test2 segfaults, I'll try to find out why). mysys/mf_pagecache.c: using a more distinctive tag storage/maria/ha_maria.cc: merge from MyISAM into Maria storage/maria/ma_check.c: merge from MyISAM into Maria storage/maria/ma_close.c: TODO as a word storage/maria/ma_create.c: merge from MyISAM into Maria storage/maria/ma_delete_all.c: TODO as a word storage/maria/ma_delete_table.c: TODO as a word storage/maria/ma_dynrec.c: merge from MyISAM into Maria storage/maria/ma_extra.c: merge from MyISAM into Maria storage/maria/ma_ft_boolean_search.c: merge from MyISAM into Maria storage/maria/ma_locking.c: merge from MyISAM into Maria storage/maria/ma_loghandler.c: fix for compiler warning storage/maria/ma_open.c: merge from MyISAM into Maria. I will ask Monty to check the ASKMONTY-marked piece of code. storage/maria/ma_packrec.c: merge from MyISAM into Maria storage/maria/ma_range.c: merge from MyISAM into Maria storage/maria/ma_rename.c: TODO as a word storage/maria/ma_rt_index.c: merge from MyISAM into Maria storage/maria/ma_rt_split.c: merge from MyISAM into Maria storage/maria/ma_search.c: merge from MyISAM into Maria storage/maria/ma_sort.c: merge from MyISAM into Maria storage/maria/ma_update.c: merge from MyISAM into Maria storage/maria/ma_write.c: merge from MyISAM into Maria storage/maria/maria_chk.c: merge from MyISAM into Maria storage/maria/maria_def.h: merge from MyISAM into Maria storage/maria/maria_pack.c: merge from MyISAM into Maria storage/maria/unittest/ma_test_loghandler_pagecache-t.c: fix for compiler warning storage/myisam/ha_myisam.cc: merge from MyISAM into Maria storage/myisammrg/ha_myisammrg.cc: merge from MyISAM into Maria |
||
unknown
|
3b0e794aca |
unittest fixed
storage/maria/unittest/Makefile.am: Unittest cleunup added storage/maria/unittest/ma_control_file-t.c: Unittest fixed according new file format storage/maria/unittest/ma_test_loghandler-t.c: Unittest cleunup added Unittest progress report added storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Unittest cleunup added Unittest progress report added storage/maria/unittest/ma_test_loghandler_multithread-t.c: Unittest cleunup added Unittest progress report added storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Unittest cleunup added Unittest progress report added storage/maria/unittest/ma_maria_log_cleanup.c: New BitKeeper file ``storage/maria/unittest/ma_maria_log_cleanup.c'' |
||
unknown
|
3bc8f629dd |
Postreview changes.
Fixed befaviour when loghandler flags changed from one run to another one. Description of maria transaction log and control file added to the file commandÍs magic number file. mysys/mf_pagecache.c: postreview changes storage/maria/ma_control_file.c: Postreview changes. storage/maria/ma_control_file.h: Postreview changes. storage/maria/ma_loghandler.c: Postreview changes. Fixed befaviour when loghandler flags changed from one run to another one. storage/maria/ma_loghandler.h: Postreview changes. Functions comment left only near the function body. storage/maria/ma_loghandler_lsn.h: Postreview changes. storage/maria/unittest/ma_test_loghandler-t.c: Postreview changes. storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Postreview changes. storage/maria/unittest/ma_test_loghandler_multithread-t.c: Postreview changes. storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Postreview changes. support-files/magic: Description of maria transaction log and control file added to the file commandÕs magic number file. |
||
unknown
|
91a8199773 |
Postmerge fix (including changing type of LSN)
Some debug info and comments added include/pagecache.h: postmerge fix mysys/mf_pagecache.c: Postmerge fix (including changing type of LSN) Additional DBUG_ASSERTs added Comment about pinning mechanism added storage/maria/ma_control_file.c: Used the same LSN storing procedure everywhere Postmerge fix (including changing type of LSN) storage/maria/ma_control_file.h: Postmerge fix (including changing type of LSN) storage/maria/ma_loghandler.c: Postmerge fix (including changing type of LSN) storage/maria/ma_loghandler.h: Postmerge fix (including changing type of LSN) storage/maria/ma_loghandler_lsn.h: Postmerge fix (including changing type of LSN) storage/maria/unittest/Makefile.am: Postmerge fix storage/maria/unittest/ma_control_file-t.c: Postmerge fix (including changing type of LSN) storage/maria/unittest/ma_test_loghandler-t.c: Postmerge fix (including changing type of LSN) storage/maria/unittest/ma_test_loghandler_multigroup-t.c: Postmerge fix (including changing type of LSN) storage/maria/unittest/ma_test_loghandler_multithread-t.c: Postmerge fix (including changing type of LSN) storage/maria/unittest/ma_test_loghandler_pagecache-t.c: Postmerge fix (including changing type of LSN) storage/maria/unittest/mf_pagecache_consist.c: Postmerge fix (including changing type of LSN) storage/maria/unittest/mf_pagecache_single.c: Postmerge fix (including changing type of LSN) |