mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-17 04:22:27 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/storage/innobase/include/dict0crea.h
Marko Mäkelä 0b47c126e3 MDEV-13542: Crashing on corrupted page is unhelpful 
The approach to handling corruption that was chosen by Oracle in
commit 177d8b0c12
is not really useful. Not only did it actually fail to prevent InnoDB
from crashing, but it is making things worse by blocking attempts to
rescue data from or rebuild a partially readable table.

We will try to prevent crashes in a different way: by propagating
errors up the call stack. We will never mark the clustered index
persistently corrupted, so that data recovery may be attempted by
reading from the table, or by rebuilding the table.

This should also fix MDEV-13680 (crash on btr_page_alloc() failure);
it was extensively tested with innodb_file_per_table=0 and a
non-autoextend system tablespace.

We should now avoid crashes in many cases, such as when a page
cannot be read or allocated, or an inconsistency is detected when
attempting to update multiple pages. We will not crash on double-free,
such as on the recovery of DDL in system tablespace in case something
was corrupted.

Crashes on corrupted data are still possible. The fault injection mechanism
that is introduced in the subsequent commit may help catch more of them.

buf_page_import_corrupt_failure: Remove the fault injection, and instead
corrupt some pages using Perl code in the tests.

btr_cur_pessimistic_insert(): Always reserve extents (except for the
change buffer), in order to prevent a subsequent allocation failure.

btr_pcur_open_at_rnd_pos(): Merged to the only caller ibuf_merge_pages().

btr_assert_not_corrupted(), btr_corruption_report(): Remove.
Similar checks are already part of btr_block_get().

FSEG_MAGIC_N_BYTES: Replaces FSEG_MAGIC_N_VALUE.

dict_hdr_get(), trx_rsegf_get_new(), trx_undo_page_get(),
trx_undo_page_get_s_latched(): Replaced with error-checking calls.

trx_rseg_t::get(mtr_t*): Replaces trx_rsegf_get().

trx_rseg_header_create(): Let the caller update the TRX_SYS page if needed.

trx_sys_create_sys_pages(): Merged with trx_sysf_create().

dict_check_tablespaces_and_store_max_id(): Do not access
DICT_HDR_MAX_SPACE_ID, because it was already recovered in dict_boot().
Merge dict_check_sys_tables() with this function.

dir_pathname(): Replaces os_file_make_new_pathname().

row_undo_ins_remove_sec(): Do not modify the undo page by adding
a terminating NUL byte to the record.

btr_decryption_failed(): Report decryption failures

dict_set_corrupted_by_space(), dict_set_encrypted_by_space(),
dict_set_corrupted_index_cache_only(): Remove.

dict_set_corrupted(): Remove the constant parameter dict_locked=false.
Never flag the clustered index corrupted in SYS_INDEXES, because
that would deny further access to the table. It might be possible to
repair the table by executing ALTER TABLE or OPTIMIZE TABLE, in case
no B-tree leaf page is corrupted.

dict_table_skip_corrupt_index(), dict_table_next_uncorrupted_index(),
row_purge_skip_uncommitted_virtual_index(): Remove, and refactor
the callers to read dict_index_t::type only once.

dict_table_is_corrupted(): Remove.

dict_index_t::is_btree(): Determine if the index is a valid B-tree.

BUF_GET_NO_LATCH, BUF_EVICT_IF_IN_POOL: Remove.

UNIV_BTR_DEBUG: Remove. Any inconsistency will no longer trigger
assertion failures, but error codes being returned.

buf_corrupt_page_release(): Replaced with a direct call to
buf_pool.corrupted_evict().

fil_invalid_page_access_msg(): Never crash on an invalid read;
let the caller of buf_page_get_gen() decide.

btr_pcur_t::restore_position(): Propagate failure status to the caller
by returning CORRUPTED.

opt_search_plan_for_table(): Simplify the code.

row_purge_del_mark(), row_purge_upd_exist_or_extern_func(),
row_undo_ins_remove_sec_rec(), row_undo_mod_upd_del_sec(),
row_undo_mod_del_mark_sec(): Avoid mem_heap_create()/mem_heap_free()
when no secondary indexes exist.

row_undo_mod_upd_exist_sec(): Simplify the code.

row_upd_clust_step(), dict_load_table_one(): Return DB_TABLE_CORRUPT
if the clustered index (and therefore the table) is corrupted, similar
to what we do in row_insert_for_mysql().

fut_get_ptr(): Replace with buf_page_get_gen() calls.

buf_page_get_gen(): Return nullptr and *err=DB_CORRUPTION
if the page is marked as freed. For other modes than
BUF_GET_POSSIBLY_FREED or BUF_PEEK_IF_IN_POOL this will
trigger a debug assertion failure. For BUF_GET_POSSIBLY_FREED,
we will return nullptr for freed pages, so that the callers
can be simplified. The purge of transaction history will be
a new user of BUF_GET_POSSIBLY_FREED, to avoid crashes on
corrupted data.

buf_page_get_low(): Never crash on a corrupted page, but simply
return nullptr.

fseg_page_is_allocated(): Replaces fseg_page_is_free().

fts_drop_common_tables(): Return an error if the transaction
was rolled back.

fil_space_t::set_corrupted(): Report a tablespace as corrupted if
it was not reported already.

fil_space_t::io(): Invoke fil_space_t::set_corrupted() to report
out-of-bounds page access or other errors.

Clean up mtr_t::page_lock()

buf_page_get_low(): Validate the page identifier (to check for
recently read corrupted pages) after acquiring the page latch.

buf_page_t::read_complete(): Flag uninitialized (all-zero) pages
with DB_FAIL. Return DB_PAGE_CORRUPTED on page number mismatch.

mtr_t::defer_drop_ahi(): Renamed from mtr_defer_drop_ahi().

recv_sys_t::free_corrupted_page(): Only set_corrupt_fs()
if any log records exist for the page. We do not mind if read-ahead
produces corrupted (or all-zero) pages that were not actually needed
during recovery.

recv_recover_page(): Return whether the operation succeeded.

recv_sys_t::recover_low(): Simplify the logic. Check for recovery error.

Thanks to Matthias Leich for testing this extensively and to the
authors of https://rr-project.org for making it easy to diagnose
and fix any failures that were found during the testing.
2022-06-06 14:03:22 +03:00

277 lines
9.9 KiB
C
Raw Blame History

/*****************************************************************************
Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2017, 2021, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA
*****************************************************************************/
/**************************************************//**
@file include/dict0crea.h
Database object creation
Created 1/8/1996 Heikki Tuuri
*******************************************************/
#ifndef dict0crea_h
#define dict0crea_h
#include "dict0dict.h"
#include "que0types.h"
#include "row0types.h"
#include "mtr0mtr.h"
#include "fil0crypt.h"
/*********************************************************************//**
Creates a table create graph.
@return own: table create node */
tab_node_t*
tab_create_graph_create(
/*====================*/
dict_table_t* table, /*!< in: table to create, built as
a memory data structure */
mem_heap_t* heap); /*!< in: heap where created */
/** Creates an index create graph.
@param[in] index index to create, built as a memory data structure
@param[in] table table name
@param[in,out] heap heap where created
@param[in] mode encryption mode (for creating a table)
@param[in] key_id encryption key identifier (for creating a table)
@param[in] add_v new virtual columns added in the same clause with
add index
@return own: index create node */
ind_node_t*
ind_create_graph_create(
dict_index_t* index,
const char* table,
mem_heap_t* heap,
fil_encryption_t mode,
uint32_t key_id,
const dict_add_v_col_t* add_v = NULL);
/***********************************************************//**
Creates a table. This is a high-level function used in SQL execution graphs.
@return query thread to run next or NULL */
que_thr_t*
dict_create_table_step(
/*===================*/
que_thr_t* thr); /*!< in: query thread */
/***********************************************************//**
Creates an index. This is a high-level function used in SQL execution
graphs.
@return query thread to run next or NULL */
que_thr_t*
dict_create_index_step(
/*===================*/
que_thr_t* thr); /*!< in: query thread */
/***************************************************************//**
Builds an index definition but doesn't update sys_table.
@return DB_SUCCESS or error code */
void
dict_build_index_def(
/*=================*/
const dict_table_t* table, /*!< in: table */
dict_index_t* index, /*!< in/out: index */
trx_t* trx); /*!< in/out: InnoDB transaction
handle */
/***************************************************************//**
Creates an index tree for the index if it is not a member of a cluster.
Don't update SYSTEM TABLES.
@return DB_SUCCESS or DB_OUT_OF_FILE_SPACE */
dberr_t
dict_create_index_tree(
/*===================*/
dict_index_t* index, /*!< in/out: index */
const trx_t* trx); /*!< in: InnoDB transaction handle */
/** Drop the index tree associated with a row in SYS_INDEXES table.
@param[in,out] pcur persistent cursor on rec
@param[in,out] trx dictionary transaction
@param[in,out] mtr mini-transaction
@return tablespace ID to drop (if this is the clustered index)
@retval 0 if no tablespace is to be dropped */
uint32_t dict_drop_index_tree(btr_pcur_t *pcur, trx_t *trx, mtr_t *mtr)
MY_ATTRIBUTE((nonnull(1,3), warn_unused_result));
/***************************************************************//**
Creates an index tree for the index if it is not a member of a cluster.
Don't update SYSTEM TABLES.
@return error code */
dberr_t
dict_create_index_tree_in_mem(
/*==========================*/
dict_index_t* index, /*!< in/out: index */
const trx_t* trx); /*!< in: InnoDB transaction handle */
/********************************************************************//**
Generate a foreign key constraint name when it was not named by the user.
A generated constraint has a name of the format dbname/tablename_ibfk_NUMBER,
where the numbers start from 1, and are given locally for this table, that is,
the number is not global, as it used to be before MySQL 4.0.18. */
UNIV_INLINE
dberr_t
dict_create_add_foreign_id(
/*=======================*/
ulint* id_nr, /*!< in/out: number to use in id
generation; incremented if used */
const char* name, /*!< in: table name */
dict_foreign_t* foreign); /*!< in/out: foreign key */
/** Adds the given set of foreign key objects to the dictionary tables
in the database. This function does not modify the dictionary cache. The
caller must ensure that all foreign key objects contain a valid constraint
name in foreign->id.
@param[in] local_fk_set set of foreign key objects, to be added to
the dictionary tables
@param[in] table table to which the foreign key objects in
local_fk_set belong to
@param[in,out] trx transaction
@return error code or DB_SUCCESS */
dberr_t
dict_create_add_foreigns_to_dictionary(
/*===================================*/
const dict_foreign_set& local_fk_set,
const dict_table_t* table,
trx_t* trx)
MY_ATTRIBUTE((nonnull, warn_unused_result));
/** Check if a foreign constraint is on columns server as base columns
of any stored column. This is to prevent creating SET NULL or CASCADE
constraint on such columns
@param[in] local_fk_set set of foreign key objects, to be added to
the dictionary tables
@param[in] table table to which the foreign key objects in
local_fk_set belong to
@return true if yes, otherwise, false */
bool
dict_foreigns_has_s_base_col(
const dict_foreign_set& local_fk_set,
const dict_table_t* table);
/********************************************************************//**
Add a foreign key definition to the data dictionary tables.
@return error code or DB_SUCCESS */
dberr_t
dict_create_add_foreign_to_dictionary(
/*==================================*/
const char* name, /*!< in: table name */
const dict_foreign_t* foreign,/*!< in: foreign key */
trx_t* trx) /*!< in/out: dictionary transaction */
MY_ATTRIBUTE((nonnull, warn_unused_result));
/* Table create node structure */
struct tab_node_t{
que_common_t common; /*!< node type: QUE_NODE_TABLE_CREATE */
dict_table_t* table; /*!< table to create, built as a
memory data structure with
dict_mem_... functions */
ins_node_t* tab_def; /*!< child node which does the insert of
the table definition; the row to be
inserted is built by the parent node */
ins_node_t* col_def; /*!< child node which does the inserts
of the column definitions; the row to
be inserted is built by the parent
node */
ins_node_t* v_col_def; /*!< child node which does the inserts
of the sys_virtual row definitions;
the row to be inserted is built by
the parent node */
/*----------------------*/
/* Local storage for this graph node */
ulint state; /*!< node execution state */
ulint col_no; /*!< next column definition to insert */
ulint base_col_no; /*!< next base column to insert */
mem_heap_t* heap; /*!< memory heap used as auxiliary
storage */
};
/* Table create node states */
#define TABLE_BUILD_TABLE_DEF 1
#define TABLE_BUILD_COL_DEF 2
#define TABLE_BUILD_V_COL_DEF 3
#define TABLE_ADD_TO_CACHE 4
#define TABLE_COMPLETED 5
/* Index create node struct */
struct ind_node_t{
que_common_t common; /*!< node type: QUE_NODE_INDEX_CREATE */
dict_index_t* index; /*!< index to create, built as a
memory data structure with
dict_mem_... functions */
const char* table_name; /*!< table name */
ins_node_t* ind_def; /*!< child node which does the insert of
the index definition; the row to be
inserted is built by the parent node */
ins_node_t* field_def; /*!< child node which does the inserts
of the field definitions; the row to
be inserted is built by the parent
node */
/*----------------------*/
/* Local storage for this graph node */
ulint state; /*!< node execution state */
uint32_t page_no; /* root page number of the index */
dtuple_t* ind_row; /* index definition row built */
ulint field_no; /* next field definition to insert */
mem_heap_t* heap; /*!< memory heap used as auxiliary
storage */
uint key_id; /*!< encryption key_id */
fil_encryption_t mode; /*!< encryption mode */
const dict_add_v_col_t*
add_v; /*!< new virtual columns that being
added along with an add index call */
};
/** Compose a column number for a virtual column, stored in the "POS" field
of Sys_columns. The column number includes both its virtual column sequence
(the "nth" virtual column) and its actual column position in original table
@param[in] v_pos virtual column sequence
@param[in] col_pos column position in original table definition
@return composed column position number */
UNIV_INLINE
ulint
dict_create_v_col_pos(
ulint v_pos,
ulint col_pos);
/** Get the column number for a virtual column (the column position in
original table), stored in the "POS" field of Sys_columns
@param[in] pos virtual column position
@return column position in original table */
UNIV_INLINE
ulint
dict_get_v_col_mysql_pos(
ulint pos);
/** Get a virtual column sequence (the "nth" virtual column) for a
virtual column, stord in the "POS" field of Sys_columns
@param[in] pos virtual column position
@return virtual column sequence */
UNIV_INLINE
ulint
dict_get_v_col_pos(
ulint pos);
/* Index create node states */
#define INDEX_BUILD_INDEX_DEF 1
#define INDEX_BUILD_FIELD_DEF 2
#define INDEX_CREATE_INDEX_TREE 3
#define INDEX_ADD_TO_CACHE 4
#include "dict0crea.inl"
#endif
Reference in a new issue View git blame Copy permalink