mirror of
https://github.com/MariaDB/server.git
synced 2025-01-20 05:52:27 +01:00
534417c285
Bulk insert: don't log REDO/UNDO for rows, log one UNDO which will truncate files; this is an optimization and a bugfix (table was left half-repaired by crash). Repair: mark table crashed-on-repair at start, bump skip_redo_lsn at start, this is easier for recovery (tells it to skip old REDOs or even UNDO phase) and user (tells it to repair) in case of crash, sync files in the end. Recovery skips missing or corrupted table and moves to next record (in REDO or UNDO phase) to be more robust; warns if happens in UNDO phase. Bugfix for UNDO_KEY_DELETE_WITH_ROOT (tested in ma_test_recovery) and maria_enable_indexes(). Create missing bitmaps when needed (there can be more than one to create, in rare cases), log a record for this. include/myisamchk.h: new flag: bulk insert repair mustn't bump create_rename_lsn mysql-test/lib/mtr_report.pl: skip normal warning in maria-recovery.test mysql-test/r/maria-recovery.result: result: crash before bulk insert is committed, causes proper rollback, and crash right after OPTIMIZE replaces index file with new index file leads to table marked corrupted and recovery not failing. mysql-test/t/maria-recovery.test: - can't check the table or it would commit the transaction, but check is made after recovery. - test of crash before bulk-insert-with-repair is committed (to see if it is rolled back), and of crash after OPTIMIZE has replaced index file but not finished all operations (to see if recovery fails - it used to assert when trying to execute an old REDO on the new index). storage/maria/CMakeLists.txt: new file storage/maria/Makefile.am: new file storage/maria/ha_maria.cc: - If bulk insert on a transactional table using an index repair: table is initially empty, so don't log REDO/UNDO for data rows (optimization), just log an UNDO_BULK_INSERT_WITH_REPAIR which will, if executed, empty the data and index file. Re-enable logging in end_bulk_insert(). - write log record for repair operation only after it's fully done, index sort including (maria_repair*() used to write the log record). - Adding back file->trn=NULL which was removed by mistake earlier. storage/maria/ha_maria.h: new member (see ha_maria.cc) storage/maria/ma_bitmap.c: Functions to create missing bitmaps: - one function which creates missing bitmaps in page cache, except the missing one with max offset which it does not put into page cache as it will be modified very soon. - one function which the one above calls, and creates bitmaps in page cache - one function to execute REDO_BITMAP_NEW_PAGE which uses the second one above. storage/maria/ma_blockrec.c: - when logging REDO_DELETE_ALL, not only 'records' and 'checksum' has to be reset under log's mutex. - execution of REDO_INSERT_ROW_BLOBS now checks the dirty pages' list - execution of UNDO_BULK_INSERT_WITH_REPAIR storage/maria/ma_blockrec.h: new functions storage/maria/ma_check.c: - table-flush-before-repair is moved to a separate function reused by maria_sort_index(); syncing is added - maria_repair() is allowed to re-enable logging only if it is the one which disabled it. - "_ma_flush_table_files_after_repair" was a bad name, it's not after repair now, and it should not sync as we do more changes to the files shortly after (sync is postponed to when writing the log record) - REDO_REPAIR record should be written only after all repair operations (in particular after sorting index in ha_mara::repair()) - close to the end of repair by sort, flushing of pages must happen also in the non-quick case, to prepare for the sync at end. - in parallel repair, some page flushes are not needed as done by initialize_variables_for_repair(). storage/maria/ma_create.c: Update skip_redo_lsn, create_rename_lsn optionally. storage/maria/ma_delete_all.c: Need to sync files at end of maria_delete_all_rows(), if transactional. storage/maria/ma_extra.c: During repair, we sometimes call _ma_flush_table_files() (via _ma_flush_table_files_before_swap()) while there is a WRITE_CACHE. storage/maria/ma_key_recover.c: - when we see CLR_END for UNDO_BULK_INSERT_WITH_REPAIR, re-enable indices. - fixing bug: _ma_apply_undo_key_delete() parsed UNDO_KEY_DELETE_WITH_ROOT wrongly, leading to recovery failure storage/maria/ma_key_recover.h: new prototype storage/maria/ma_locking.c: DBUG_VOID_RETURN missing storage/maria/ma_loghandler.c: UNDO for bulk insert with repair, and REDO for creating bitmaps. LOGREC_FIRST_FREE to not have to change the for() every time we add a new record type. storage/maria/ma_loghandler.h: new UNDO and REDO storage/maria/ma_open.c: Move share.kfile.file=kfile up a bit, so that _ma_update_state_lsns() can get its value, this fixes a bug where LSN_REPAIRED_BY_MARIA_CHK was not corrected on disk by maria_open(). Store skip_redo_lsn in index' header. maria_enable_indexes() had a bug for BLOCK_RECORD, where an empty file has one page, not 0 bytes. storage/maria/ma_recovery.c: - Skip a corrupted, missing, or repaired-with-maria_chk, table in recovery: don't fail, just go to next REDO or UNDO; but if an UNDO is skipped in UNDO phase we issue warnings. - Skip REDO|UNDO in REDO phase if <skip_redo_lsn. - If UNDO phase fails, delete transactions to not make trnman assert. - Update skip_redo_lsn when playing REDO_CREATE_TABLE - Don't record UNDOs for old transactions which we don't know (long_trid==0) - Bugfix for UNDO_KEY_DELETE_WITH_ROOT (see ma_key_recover.c) - Execution of UNDO_BULK_INSERT_WITH_REPAIR - Don't try to find a page number in REDO_DELETE_ALL - Pieces moved to ma_recovery_util.c storage/maria/ma_rename.c: name change storage/maria/ma_static.c: I modified layout of the index' header (inserted skip_redo_lsn in its middle) storage/maria/ma_test2.c: allow breaking the test towards the end, tests execution of UNDO_KEY_DELETE_WITH_ROOT storage/maria/ma_test_recovery.expected: 6 as testflag instead of 4 storage/maria/ma_test_recovery: Increase the amount of rollback work to do when testing recovery with ma_test2; this reproduces the UNDO_KEY_DELETE_WITH_ROOT bug. storage/maria/maria_chk.c: skip_redo_lsn should be updated too, for consistency. Write a REDO_REPAIR after all operations (including sort-records) have been done. No reason to flush blocks after maria_chk_data_link() and maria_sort_records(), there is maria_close() in the end. write_log_record() is a function, to not clutter maria_chk(). storage/maria/maria_def.h: New member skip_redo_lsn in the state, and comments storage/maria/maria_pack.c: skip_redo_lsn should be updated too, for consistency storage/maria/ma_recovery_util.c: _ma_redo_not_needed_for_page(), defined in ma_recovery.c, is needed by ma_blockrec.c; this causes link issues, resolved by putting _ma_redo_not_needed_for_page() into a new file (so that it is not in the same file as repair-related objects of ma_recovery.c). storage/maria/ma_recovery_util.h: new file |
||
|---|---|---|
| .. | ||
| 1st.test | ||
| alias.test | ||
| almost_full.test | ||
| alter_table-big.test | ||
| alter_table.test | ||
| analyse.test | ||
| analyze.test | ||
| ansi.test | ||
| archive-big.test | ||
| archive.test | ||
| archive_bitfield.test | ||
| archive_gis.test | ||
| auto_increment.test | ||
| backup-master.sh | ||
| backup.test | ||
| bdb_notembedded.test | ||
| bench_count_distinct.test | ||
| bigint.test | ||
| binary.test | ||
| binlog_unsafe.test | ||
| bool.test | ||
| bootstrap.test | ||
| bulk_replace.test | ||
| cache_innodb-master.opt | ||
| cache_innodb.test | ||
| case.test | ||
| cast.test | ||
| change_user.test | ||
| check.test | ||
| client_xml.test | ||
| comments.test | ||
| compare.test | ||
| compress.test | ||
| concurrent_innodb-master.opt | ||
| concurrent_innodb.test | ||
| connect.test | ||
| consistent_snapshot.test | ||
| constraints.test | ||
| contributors.test | ||
| count_distinct.test | ||
| count_distinct2-master.opt | ||
| count_distinct2.test | ||
| count_distinct3.test | ||
| crash_commit_before-master.opt | ||
| crash_commit_before.test | ||
| create-big.test | ||
| create.test | ||
| create_not_windows.test | ||
| create_select_tmp.test | ||
| csv.test | ||
| ctype_ascii.test | ||
| ctype_big5.test | ||
| ctype_collate.test | ||
| ctype_cp932_binlog_row.test | ||
| ctype_cp932_binlog_stm.test | ||
| ctype_cp1250_ch.test | ||
| ctype_cp1251.test | ||
| ctype_create.test | ||
| ctype_eucjpms.test | ||
| ctype_euckr.test | ||
| ctype_filename.test | ||
| ctype_gb2312.test | ||
| ctype_gbk.test | ||
| ctype_hebrew.test | ||
| ctype_latin1.test | ||
| ctype_latin1_de-master.opt | ||
| ctype_latin1_de.test | ||
| ctype_latin2.test | ||
| ctype_latin2_ch.test | ||
| ctype_ldml-master.opt | ||
| ctype_ldml.test | ||
| ctype_many.test | ||
| ctype_mb.test | ||
| ctype_recoding.test | ||
| ctype_sjis.test | ||
| ctype_tis620.test | ||
| ctype_uca.test | ||
| ctype_ucs.test | ||
| ctype_ucs2_def-master.opt | ||
| ctype_ucs2_def.test | ||
| ctype_ujis.test | ||
| ctype_ujis_ucs2.test | ||
| ctype_utf8.test | ||
| date_formats-master.opt | ||
| date_formats.test | ||
| ddl_i18n_koi8r.test | ||
| ddl_i18n_utf8.test | ||
| deadlock_innodb.test | ||
| default.test | ||
| delayed.test | ||
| delete.test | ||
| derived.test | ||
| dirty_close.test | ||
| disabled.def | ||
| distinct.test | ||
| drop.test | ||
| empty_table.test | ||
| endspace.test | ||
| error_simulation-master.opt | ||
| error_simulation.test | ||
| errors.test | ||
| events.test | ||
| events_bugs-master.opt | ||
| events_bugs.test | ||
| events_grant.test | ||
| events_logs_tests-master.opt | ||
| events_logs_tests.test | ||
| events_microsec.test | ||
| events_restart_phase1.test | ||
| events_restart_phase2-master.opt | ||
| events_restart_phase2.test | ||
| events_restart_phase3-master.opt | ||
| events_restart_phase3.test | ||
| events_scheduling.test | ||
| events_stress.test | ||
| events_time_zone.test | ||
| events_trans.test | ||
| events_trans_notembedded.test | ||
| exampledb.test | ||
| execution_constants.test | ||
| explain.test | ||
| federated.test | ||
| federated_archive.test | ||
| federated_bug_13118.test | ||
| federated_bug_25714.test | ||
| federated_innodb-slave.opt | ||
| federated_innodb.test | ||
| federated_server.test | ||
| federated_transactions-slave.opt | ||
| federated_transactions.test | ||
| fix_priv_tables.test | ||
| flush.test | ||
| flush2-master.opt | ||
| flush2.test | ||
| flush_block_commit.test | ||
| flush_block_commit_notembedded.test | ||
| flush_read_lock_kill-master.opt | ||
| flush_read_lock_kill.test | ||
| flush_table.test | ||
| foreign_key.test | ||
| fulltext.test | ||
| fulltext2.test | ||
| fulltext3.test | ||
| fulltext_cache.test | ||
| fulltext_distinct.test | ||
| fulltext_left_join.test | ||
| fulltext_multi.test | ||
| fulltext_order_by.test | ||
| fulltext_update.test | ||
| fulltext_var.test | ||
| func_compress.test | ||
| func_concat.test | ||
| func_crypt.test | ||
| func_date_add.test | ||
| func_default.test | ||
| func_des_encrypt.test | ||
| func_encrypt-master.opt | ||
| func_encrypt.test | ||
| func_encrypt_nossl.test | ||
| func_equal.test | ||
| func_gconcat.test | ||
| func_group.test | ||
| func_group_innodb.test | ||
| func_if.test | ||
| func_in.test | ||
| func_isnull.test | ||
| func_like.test | ||
| func_math.test | ||
| func_misc.test | ||
| func_op.test | ||
| func_regexp.test | ||
| func_sapdb.test | ||
| func_set.test | ||
| func_str.test | ||
| func_system.test | ||
| func_test.test | ||
| func_time.test | ||
| func_timestamp.test | ||
| gcc296.test | ||
| gis-rtree.test | ||
| gis.test | ||
| grant.test | ||
| grant2.test | ||
| grant3-master.opt | ||
| grant3.test | ||
| grant_cache_no_prot.test | ||
| grant_cache_ps_prot.test | ||
| greedy_optimizer.test | ||
| group_by.test | ||
| group_min_max.test | ||
| group_min_max_innodb.test | ||
| handler_innodb.test | ||
| handler_myisam.test | ||
| having.test | ||
| heap.test | ||
| heap_auto_increment.test | ||
| heap_btree.test | ||
| heap_hash.test | ||
| help.test | ||
| index_merge_innodb.test | ||
| index_merge_myisam.test | ||
| information_schema.test | ||
| information_schema_chmod.test | ||
| information_schema_db.test | ||
| information_schema_inno.test | ||
| information_schema_part.test | ||
| init_connect-master.opt | ||
| init_connect.test | ||
| init_file-master.opt | ||
| init_file.test | ||
| innodb-lock.test | ||
| innodb-master.opt | ||
| innodb-replace.test | ||
| innodb-semi-consistent-master.opt | ||
| innodb-semi-consistent.test | ||
| innodb-ucs2.test | ||
| innodb.test | ||
| innodb_autoinc_lock_mode_zero-master.opt | ||
| innodb_autoinc_lock_mode_zero.test | ||
| innodb_gis.test | ||
| innodb_mysql-master.opt | ||
| innodb_mysql.test | ||
| innodb_notembedded.test | ||
| innodb_timeout_rollback-master.opt | ||
| innodb_timeout_rollback.test | ||
| innodb_trx_weight.test | ||
| insert.test | ||
| insert_notembedded.test | ||
| insert_select.test | ||
| insert_update.test | ||
| join.test | ||
| join_crash.test | ||
| join_nested.test | ||
| join_outer.test | ||
| join_outer_innodb.test | ||
| key.test | ||
| key_cache-master.opt | ||
| key_cache.test | ||
| key_diff.test | ||
| key_primary.test | ||
| keywords.test | ||
| kill.test | ||
| limit.test | ||
| loaddata.test | ||
| loaddata_autocom_innodb.test | ||
| lock.test | ||
| lock_multi.test | ||
| lock_tables_lost_commit-master.opt | ||
| lock_tables_lost_commit.test | ||
| log_state-master.opt | ||
| log_state.test | ||
| log_tables-big-master.opt | ||
| log_tables-big.test | ||
| log_tables-master.opt | ||
| log_tables.test | ||
| long_tmpdir-master.opt | ||
| long_tmpdir-master.sh | ||
| long_tmpdir.test | ||
| lowercase_fs_off.test | ||
| lowercase_mixed_tmpdir-master.opt | ||
| lowercase_mixed_tmpdir-master.sh | ||
| lowercase_mixed_tmpdir.test | ||
| lowercase_table-master.opt | ||
| lowercase_table.test | ||
| lowercase_table2.test | ||
| lowercase_table3-master.opt | ||
| lowercase_table3.test | ||
| lowercase_table_grant-master.opt | ||
| lowercase_table_grant.test | ||
| lowercase_table_qcache-master.opt | ||
| lowercase_table_qcache.test | ||
| lowercase_view-master.opt | ||
| lowercase_view.test | ||
| maria-big.test | ||
| maria-connect.test | ||
| maria-purge.test | ||
| maria-recovery-bitmap-master.opt | ||
| maria-recovery-bitmap.test | ||
| maria-recovery-master.opt | ||
| maria-recovery.test | ||
| maria.test | ||
| merge-big.test | ||
| merge.test | ||
| merge_innodb.test | ||
| metadata.test | ||
| mix2_myisam.test | ||
| mix2_myisam_ucs2.test | ||
| multi_statement-master.opt | ||
| multi_statement.test | ||
| multi_update-master.opt | ||
| multi_update.test | ||
| myisam-blob-master.opt | ||
| myisam-blob.test | ||
| myisam-system.test | ||
| myisam.test | ||
| myisampack.test | ||
| mysql.test | ||
| mysql_client_test-master.opt | ||
| mysql_client_test.test | ||
| mysql_comments.sql | ||
| mysql_comments.test | ||
| mysql_cp932.test | ||
| mysql_delimiter.sql | ||
| mysql_delimiter_19799.sql | ||
| mysql_delimiter_source.sql | ||
| mysql_protocols.test | ||
| mysql_upgrade.test | ||
| mysqladmin.test | ||
| mysqlbinlog-cp932-master.opt | ||
| mysqlbinlog-cp932.test | ||
| mysqlbinlog-master.opt | ||
| mysqlbinlog.test | ||
| mysqlbinlog2.test | ||
| mysqlbinlog_base64.test | ||
| mysqlcheck.test | ||
| mysqldump-compat.opt | ||
| mysqldump-compat.test | ||
| mysqldump-max.test | ||
| mysqldump.test | ||
| mysqlshow.test | ||
| mysqlslap.test | ||
| mysqltest.test | ||
| named_pipe-master.opt | ||
| named_pipe.test | ||
| negation_elimination.test | ||
| no-threads-master.opt | ||
| no-threads.test | ||
| not_embedded_server-master.opt | ||
| not_embedded_server.test | ||
| not_partition.test | ||
| null.test | ||
| null_key.test | ||
| odbc.test | ||
| olap.test | ||
| old-mode-master.opt | ||
| old-mode.test | ||
| openssl_1.test | ||
| order_by.test | ||
| order_fill_sortbuf-master.opt | ||
| order_fill_sortbuf.test | ||
| outfile.test | ||
| outfile_loaddata.test | ||
| overflow.test | ||
| packet.test | ||
| parser.test | ||
| parser_bug21114_innodb.test | ||
| parser_precedence.test | ||
| partition-master.opt | ||
| partition.test | ||
| partition_archive.test | ||
| partition_blackhole.test | ||
| partition_bug18198.test | ||
| partition_charset.test | ||
| partition_csv.test | ||
| partition_datatype.test | ||
| partition_error.test | ||
| partition_federated.test | ||
| partition_grant.test | ||
| partition_hash.test | ||
| partition_innodb.test | ||
| partition_list.test | ||
| partition_mgm.test | ||
| partition_mgm_err.test | ||
| partition_mgm_err2.test | ||
| partition_not_windows-master.opt | ||
| partition_not_windows.test | ||
| partition_order.test | ||
| partition_pruning.test | ||
| partition_range.test | ||
| partition_symlink.test | ||
| partition_windows.test | ||
| perror.test | ||
| plugin-master.opt | ||
| plugin.test | ||
| preload.test | ||
| profiling.test | ||
| ps-master.opt | ||
| ps.test | ||
| ps_1general.test | ||
| ps_2myisam.test | ||
| ps_3innodb.test | ||
| ps_4heap.test | ||
| ps_5merge.test | ||
| ps_10nestset.test | ||
| ps_11bugs.test | ||
| ps_ddl.test | ||
| ps_grant.test | ||
| ps_maria.test | ||
| ps_not_windows.test | ||
| query_cache.test | ||
| query_cache_merge.test | ||
| query_cache_notembedded.test | ||
| query_cache_ps_no_prot.test | ||
| query_cache_ps_ps_prot.test | ||
| query_cache_with_views.test | ||
| range.test | ||
| read_many_rows_innodb.test | ||
| read_only.test | ||
| read_only_innodb.test | ||
| rename.test | ||
| renamedb.test | ||
| repair.test | ||
| replace.test | ||
| rollback.test | ||
| round.test | ||
| row.test | ||
| rowid_order_innodb.test | ||
| schema.test | ||
| select.test | ||
| select_found.test | ||
| select_safe.test | ||
| shm-master.opt | ||
| shm.test | ||
| show_check-master.opt | ||
| show_check.test | ||
| skip_grants-master.opt | ||
| skip_grants.test | ||
| skip_name_resolve-master.opt | ||
| skip_name_resolve.test | ||
| sp-big.test | ||
| sp-code.test | ||
| sp-destruct.test | ||
| sp-dynamic.test | ||
| sp-error.test | ||
| sp-prelocking.test | ||
| sp-security.test | ||
| sp-threads.test | ||
| sp-ucs2.test | ||
| sp-vars.test | ||
| sp.test | ||
| sp_gis.test | ||
| sp_notembedded.test | ||
| sp_stress_case.test | ||
| sp_trans.test | ||
| sp_trans_log.test | ||
| sql_mode.test | ||
| ssl-big.test | ||
| ssl.test | ||
| ssl_8k_key.test | ||
| ssl_compress.test | ||
| ssl_connect.test | ||
| status.test | ||
| strict.test | ||
| strict_autoinc_1myisam.test | ||
| strict_autoinc_2innodb.test | ||
| strict_autoinc_3heap.test | ||
| subselect.test | ||
| subselect2.test | ||
| subselect3.test | ||
| subselect_gis.test | ||
| subselect_innodb.test | ||
| subselect_notembedded.test | ||
| sum_distinct-big.test | ||
| sum_distinct.test | ||
| symlink.test | ||
| synchronization.test | ||
| sysdate_is_now-master.opt | ||
| sysdate_is_now.test | ||
| system_mysql_db.test | ||
| system_mysql_db_fix30020-master.opt | ||
| system_mysql_db_fix30020.test | ||
| system_mysql_db_fix40123-master.opt | ||
| system_mysql_db_fix40123.test | ||
| system_mysql_db_fix50030-master.opt | ||
| system_mysql_db_fix50030.test | ||
| system_mysql_db_fix50117-master.opt | ||
| system_mysql_db_fix50117.test | ||
| system_mysql_db_refs.test | ||
| tablelock.test | ||
| temp_table-master.opt | ||
| temp_table.test | ||
| timezone-master.opt | ||
| timezone.test | ||
| timezone2.test | ||
| timezone3-master.opt | ||
| timezone3.test | ||
| timezone4-master.opt | ||
| timezone4.test | ||
| timezone_grant.test | ||
| trigger-compat.test | ||
| trigger-trans.test | ||
| trigger.test | ||
| trigger_notembedded.test | ||
| truncate.test | ||
| type_binary.test | ||
| type_bit.test | ||
| type_bit_innodb.test | ||
| type_blob.test | ||
| type_date.test | ||
| type_datetime.test | ||
| type_decimal.test | ||
| type_enum.test | ||
| type_float.test | ||
| type_nchar.test | ||
| type_newdecimal-big.test | ||
| type_newdecimal.test | ||
| type_ranges.test | ||
| type_set.test | ||
| type_time.test | ||
| type_timestamp.test | ||
| type_uint.test | ||
| type_varchar.test | ||
| type_year.test | ||
| udf-master.opt | ||
| udf.test | ||
| union-master.opt | ||
| union.test | ||
| unsafe_binlog_innodb-master.opt | ||
| unsafe_binlog_innodb.test | ||
| update.test | ||
| upgrade.test | ||
| user_limits.test | ||
| user_var-binlog.test | ||
| user_var.test | ||
| varbinary.test | ||
| variables+c.test | ||
| variables-big.test | ||
| variables.test | ||
| view.test | ||
| view_grant.test | ||
| wait_timeout-master.opt | ||
| wait_timeout.test | ||
| warnings-master.opt | ||
| warnings.test | ||
| warnings_engine_disabled-master.opt | ||
| warnings_engine_disabled.test | ||
| windows.test | ||
| xa.test | ||
| xml.test | ||