2012-08-01 17:27:34 +03:00
|
|
|
/*****************************************************************************
|
|
|
|
|
MDEV-14637: Fix hang due to persistent statistics
Similar to the tables SYS_FOREIGN and SYS_FOREIGN_COLS,
the tables mysql.innodb_table_stats and mysql.innodb_index_stats
are updated by the InnoDB internal SQL parser, which fails to
enforce the size limits of the data. Due to this, it is possible
for InnoDB to hang when there are persistent statistics defined on
partitioned tables where the total length of table name,
partition name and subpartition name exceeds the incorrectly
defined limit VARCHAR(64). That column should have been defined
as VARCHAR(199).
btr_node_ptr_max_size(): Interpret the VARCHAR(64) as VARCHAR(199),
to prevent a hang in the case that the upgrade script has not been
run.
dict_table_schema_check(): Ignore difference in the length of the
table_name column.
ha_innobase::max_supported_key_length(): For innodb_page_size=4k,
return a larger value so that the table mysql.innodb_index_stats
can be created. This could allow "impossible" tables to be created,
such that it is not possible to insert anything into a secondary
index when both the secondary key and the primary key are long,
but this is the easiest and most consistent way. The Oracle fix
would only ignore the maximum length violation for the two
statistics tables.
os_file_get_status_posix(), os_file_get_status_win32(): Handle
ENAMETOOLONG as well.
This patch is based on the following change in MySQL 5.7.23.
Not all changes were applied, and our variant allows persistent
statistics to work without hangs even if the table definitions
were not upgraded.
From fdbdce701ab8145ae234c9d401109dff4e4106cb Mon Sep 17 00:00:00 2001
From: Aditya A <aditya.a@oracle.com>
Date: Thu, 17 May 2018 16:11:43 +0530
Subject: [PATCH] Bug #26390736 THE FIELD TABLE_NAME (VARCHAR(64)) FROM
MYSQL.INNODB_TABLE_STATS CAN OVERFLOW.
In mysql.innodb_index_stats and mysql.innodb_table_stats
tables the table name column didn't take into consideration
partition names which can be more than varchar(64).
2018-08-02 17:35:24 +03:00
|
|
|
Copyright (c) 2009, 2018, Oracle and/or its affiliates. All Rights Reserved.
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
Copyright (c) 2017, 2018, MariaDB Corporation.
|
2012-08-01 17:27:34 +03:00
|
|
|
|
|
|
|
This program is free software; you can redistribute it and/or modify it under
|
|
|
|
the terms of the GNU General Public License as published by the Free Software
|
|
|
|
Foundation; version 2 of the License.
|
|
|
|
|
|
|
|
This program is distributed in the hope that it will be useful, but WITHOUT
|
|
|
|
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
|
|
|
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License along with
|
|
|
|
this program; if not, write to the Free Software Foundation, Inc.,
|
2019-05-11 19:25:02 +03:00
|
|
|
51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA
|
2012-08-01 17:27:34 +03:00
|
|
|
|
|
|
|
*****************************************************************************/
|
|
|
|
|
|
|
|
/**************************************************//**
|
|
|
|
@file include/dict0stats.h
|
|
|
|
Code used for calculating and manipulating table statistics.
|
|
|
|
|
|
|
|
Created Jan 06, 2010 Vasil Dimov
|
|
|
|
*******************************************************/
|
|
|
|
|
|
|
|
#ifndef dict0stats_h
|
|
|
|
#define dict0stats_h
|
|
|
|
|
|
|
|
#include "dict0types.h"
|
|
|
|
#include "trx0types.h"
|
|
|
|
|
MDEV-14637: Fix hang due to persistent statistics
Similar to the tables SYS_FOREIGN and SYS_FOREIGN_COLS,
the tables mysql.innodb_table_stats and mysql.innodb_index_stats
are updated by the InnoDB internal SQL parser, which fails to
enforce the size limits of the data. Due to this, it is possible
for InnoDB to hang when there are persistent statistics defined on
partitioned tables where the total length of table name,
partition name and subpartition name exceeds the incorrectly
defined limit VARCHAR(64). That column should have been defined
as VARCHAR(199).
btr_node_ptr_max_size(): Interpret the VARCHAR(64) as VARCHAR(199),
to prevent a hang in the case that the upgrade script has not been
run.
dict_table_schema_check(): Ignore difference in the length of the
table_name column.
ha_innobase::max_supported_key_length(): For innodb_page_size=4k,
return a larger value so that the table mysql.innodb_index_stats
can be created. This could allow "impossible" tables to be created,
such that it is not possible to insert anything into a secondary
index when both the secondary key and the primary key are long,
but this is the easiest and most consistent way. The Oracle fix
would only ignore the maximum length violation for the two
statistics tables.
os_file_get_status_posix(), os_file_get_status_win32(): Handle
ENAMETOOLONG as well.
This patch is based on the following change in MySQL 5.7.23.
Not all changes were applied, and our variant allows persistent
statistics to work without hangs even if the table definitions
were not upgraded.
From fdbdce701ab8145ae234c9d401109dff4e4106cb Mon Sep 17 00:00:00 2001
From: Aditya A <aditya.a@oracle.com>
Date: Thu, 17 May 2018 16:11:43 +0530
Subject: [PATCH] Bug #26390736 THE FIELD TABLE_NAME (VARCHAR(64)) FROM
MYSQL.INNODB_TABLE_STATS CAN OVERFLOW.
In mysql.innodb_index_stats and mysql.innodb_table_stats
tables the table name column didn't take into consideration
partition names which can be more than varchar(64).
2018-08-02 17:35:24 +03:00
|
|
|
#define TABLE_STATS_NAME "mysql/innodb_table_stats"
|
|
|
|
#define INDEX_STATS_NAME "mysql/innodb_index_stats"
|
|
|
|
|
2013-03-26 00:03:13 +02:00
|
|
|
enum dict_stats_upd_option_t {
|
2012-08-01 17:27:34 +03:00
|
|
|
DICT_STATS_RECALC_PERSISTENT,/* (re) calculate the
|
|
|
|
statistics using a precise and slow
|
|
|
|
algo and save them to the persistent
|
|
|
|
storage, if the persistent storage is
|
|
|
|
not present then emit a warning and
|
|
|
|
fall back to transient stats */
|
|
|
|
DICT_STATS_RECALC_TRANSIENT,/* (re) calculate the statistics
|
|
|
|
using an imprecise quick algo
|
|
|
|
without saving the results
|
|
|
|
persistently */
|
2013-03-26 00:03:13 +02:00
|
|
|
DICT_STATS_EMPTY_TABLE, /* Write all zeros (or 1 where it makes sense)
|
|
|
|
into a table and its indexes' statistics
|
|
|
|
members. The resulting stats correspond to an
|
|
|
|
empty table. If the table is using persistent
|
|
|
|
statistics, then they are saved on disk. */
|
|
|
|
DICT_STATS_FETCH_ONLY_IF_NOT_IN_MEMORY /* fetch the stats
|
2012-08-01 17:27:34 +03:00
|
|
|
from the persistent storage if the in-memory
|
|
|
|
structures have not been initialized yet,
|
|
|
|
otherwise do nothing */
|
|
|
|
};
|
|
|
|
|
2013-03-26 00:03:13 +02:00
|
|
|
/*********************************************************************//**
|
|
|
|
Set the persistent statistics flag for a given table. This is set only
|
|
|
|
in the in-memory table object and is not saved on disk. It will be read
|
|
|
|
from the .frm file upon first open from MySQL after a server restart. */
|
|
|
|
UNIV_INLINE
|
|
|
|
void
|
|
|
|
dict_stats_set_persistent(
|
|
|
|
/*======================*/
|
|
|
|
dict_table_t* table, /*!< in/out: table */
|
|
|
|
ibool ps_on, /*!< in: persistent stats explicitly enabled */
|
|
|
|
ibool ps_off) /*!< in: persistent stats explicitly disabled */
|
2016-06-21 14:21:03 +02:00
|
|
|
MY_ATTRIBUTE((nonnull));
|
2013-03-26 00:03:13 +02:00
|
|
|
|
2017-03-01 08:27:39 +02:00
|
|
|
/** @return whether persistent statistics is enabled for a given table */
|
2013-03-26 00:03:13 +02:00
|
|
|
UNIV_INLINE
|
2017-03-01 08:27:39 +02:00
|
|
|
bool
|
|
|
|
dict_stats_is_persistent_enabled(const dict_table_t* table)
|
2016-06-21 14:21:03 +02:00
|
|
|
MY_ATTRIBUTE((nonnull, warn_unused_result));
|
2013-03-26 00:03:13 +02:00
|
|
|
|
|
|
|
/*********************************************************************//**
|
|
|
|
Set the auto recalc flag for a given table (only honored for a persistent
|
|
|
|
stats enabled table). The flag is set only in the in-memory table object
|
|
|
|
and is not saved in InnoDB files. It will be read from the .frm file upon
|
|
|
|
first open from MySQL after a server restart. */
|
|
|
|
UNIV_INLINE
|
|
|
|
void
|
|
|
|
dict_stats_auto_recalc_set(
|
|
|
|
/*=======================*/
|
|
|
|
dict_table_t* table, /*!< in/out: table */
|
|
|
|
ibool auto_recalc_on, /*!< in: explicitly enabled */
|
|
|
|
ibool auto_recalc_off); /*!< in: explicitly disabled */
|
|
|
|
|
2017-03-01 08:27:39 +02:00
|
|
|
/** @return whether auto recalc is enabled for a given table*/
|
2013-03-26 00:03:13 +02:00
|
|
|
UNIV_INLINE
|
2017-03-01 08:27:39 +02:00
|
|
|
bool
|
|
|
|
dict_stats_auto_recalc_is_enabled(const dict_table_t* table)
|
|
|
|
MY_ATTRIBUTE((nonnull, warn_unused_result));
|
2013-03-26 00:03:13 +02:00
|
|
|
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*********************************************************************//**
|
|
|
|
Initialize table's stats for the first time when opening a table. */
|
2013-03-26 00:03:13 +02:00
|
|
|
UNIV_INLINE
|
|
|
|
void
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
dict_stats_init(
|
|
|
|
/*============*/
|
|
|
|
dict_table_t* table); /*!< in/out: table */
|
2013-03-26 00:03:13 +02:00
|
|
|
|
|
|
|
/*********************************************************************//**
|
|
|
|
Deinitialize table's stats after the last close of the table. This is
|
|
|
|
used to detect "FLUSH TABLE" and refresh the stats upon next open. */
|
|
|
|
UNIV_INLINE
|
|
|
|
void
|
|
|
|
dict_stats_deinit(
|
|
|
|
/*==============*/
|
|
|
|
dict_table_t* table) /*!< in/out: table */
|
2016-06-21 14:21:03 +02:00
|
|
|
MY_ATTRIBUTE((nonnull));
|
2012-08-01 17:27:34 +03:00
|
|
|
|
2018-08-02 08:19:57 +03:00
|
|
|
#ifdef WITH_WSREP
|
|
|
|
/** Update the table modification counter and if necessary,
|
|
|
|
schedule new estimates for table and index statistics to be calculated.
|
|
|
|
@param[in,out] table persistent or temporary table
|
|
|
|
@param[in] thd current session */
|
|
|
|
void dict_stats_update_if_needed(dict_table_t* table, THD* thd)
|
|
|
|
MY_ATTRIBUTE((nonnull(1)));
|
|
|
|
#else
|
MDEV-12698 innodb.innodb_stats_del_mark test failure
In my merge of the MySQL fix for Oracle Bug#23333990 / WL#9513
I overlooked some subsequent revisions to the test, and I also
failed to notice that the test is actually always failing.
Oracle introduced the parameter innodb_stats_include_delete_marked
but failed to consistently take it into account in FOREIGN KEY
constraints that involve CASCADE or SET NULL.
When innodb_stats_include_delete_marked=ON, obviously the purge of
delete-marked records should update the statistics as well.
One more omission was that statistics were never updated on ROLLBACK.
We are fixing that as well, properly taking into account the
parameter innodb_stats_include_delete_marked.
dict_stats_analyze_index_level(): Simplify an expression.
(Using the ternary operator with a constant operand is unnecessary
obfuscation.)
page_scan_method_t: Revert the change done by Oracle. Instead,
examine srv_stats_include_delete_marked directly where it is needed.
dict_stats_update_if_needed(): Renamed from
row_update_statistics_if_needed().
row_update_for_mysql_using_upd_graph(): Assert that the table statistics
are initialized, as guaranteed by ha_innobase::open(). Update the
statistics in a consistent way, both for FOREIGN KEY triggers and
for the main table. If FOREIGN KEY constraints exist, do not dereference
a freed pointer, but cache the proper value of node->is_delete so that
it matches prebuilt->table.
row_purge_record_func(): Update statistics if
innodb_stats_include_delete_marked=ON.
row_undo_ins(): Update statistics (on ROLLBACK of a fresh INSERT).
This is independent of the parameter; the record is not delete-marked.
row_undo_mod(): Update statistics on the ROLLBACK of updating key columns,
or (if innodb_stats_include_delete_marked=OFF) updating delete-marks.
innodb.innodb_stats_persistent: Renamed and extended from
innodb.innodb_stats_del_mark. Reduced the unnecessarily large dataset
from 262,144 to 32 rows. Test both values of the configuration
parameter innodb_stats_include_delete_marked.
Test that purge is updating the statistics.
innodb_fts.innodb_fts_multiple_index: Adjust the result. The test
is performing a ROLLBACK of an INSERT, which now affects the statistics.
include/wait_all_purged.inc: Moved from innodb.innodb_truncate_debug
to its own file.
2017-05-16 12:07:26 +03:00
|
|
|
/** Update the table modification counter and if necessary,
|
|
|
|
schedule new estimates for table and index statistics to be calculated.
|
|
|
|
@param[in,out] table persistent or temporary table */
|
2018-08-02 08:19:57 +03:00
|
|
|
void dict_stats_update_if_needed_func(dict_table_t* table)
|
MDEV-12698 innodb.innodb_stats_del_mark test failure
In my merge of the MySQL fix for Oracle Bug#23333990 / WL#9513
I overlooked some subsequent revisions to the test, and I also
failed to notice that the test is actually always failing.
Oracle introduced the parameter innodb_stats_include_delete_marked
but failed to consistently take it into account in FOREIGN KEY
constraints that involve CASCADE or SET NULL.
When innodb_stats_include_delete_marked=ON, obviously the purge of
delete-marked records should update the statistics as well.
One more omission was that statistics were never updated on ROLLBACK.
We are fixing that as well, properly taking into account the
parameter innodb_stats_include_delete_marked.
dict_stats_analyze_index_level(): Simplify an expression.
(Using the ternary operator with a constant operand is unnecessary
obfuscation.)
page_scan_method_t: Revert the change done by Oracle. Instead,
examine srv_stats_include_delete_marked directly where it is needed.
dict_stats_update_if_needed(): Renamed from
row_update_statistics_if_needed().
row_update_for_mysql_using_upd_graph(): Assert that the table statistics
are initialized, as guaranteed by ha_innobase::open(). Update the
statistics in a consistent way, both for FOREIGN KEY triggers and
for the main table. If FOREIGN KEY constraints exist, do not dereference
a freed pointer, but cache the proper value of node->is_delete so that
it matches prebuilt->table.
row_purge_record_func(): Update statistics if
innodb_stats_include_delete_marked=ON.
row_undo_ins(): Update statistics (on ROLLBACK of a fresh INSERT).
This is independent of the parameter; the record is not delete-marked.
row_undo_mod(): Update statistics on the ROLLBACK of updating key columns,
or (if innodb_stats_include_delete_marked=OFF) updating delete-marks.
innodb.innodb_stats_persistent: Renamed and extended from
innodb.innodb_stats_del_mark. Reduced the unnecessarily large dataset
from 262,144 to 32 rows. Test both values of the configuration
parameter innodb_stats_include_delete_marked.
Test that purge is updating the statistics.
innodb_fts.innodb_fts_multiple_index: Adjust the result. The test
is performing a ROLLBACK of an INSERT, which now affects the statistics.
include/wait_all_purged.inc: Moved from innodb.innodb_truncate_debug
to its own file.
2017-05-16 12:07:26 +03:00
|
|
|
MY_ATTRIBUTE((nonnull));
|
2018-08-02 08:19:57 +03:00
|
|
|
# define dict_stats_update_if_needed(t,thd) dict_stats_update_if_needed_func(t)
|
|
|
|
#endif
|
MDEV-12698 innodb.innodb_stats_del_mark test failure
In my merge of the MySQL fix for Oracle Bug#23333990 / WL#9513
I overlooked some subsequent revisions to the test, and I also
failed to notice that the test is actually always failing.
Oracle introduced the parameter innodb_stats_include_delete_marked
but failed to consistently take it into account in FOREIGN KEY
constraints that involve CASCADE or SET NULL.
When innodb_stats_include_delete_marked=ON, obviously the purge of
delete-marked records should update the statistics as well.
One more omission was that statistics were never updated on ROLLBACK.
We are fixing that as well, properly taking into account the
parameter innodb_stats_include_delete_marked.
dict_stats_analyze_index_level(): Simplify an expression.
(Using the ternary operator with a constant operand is unnecessary
obfuscation.)
page_scan_method_t: Revert the change done by Oracle. Instead,
examine srv_stats_include_delete_marked directly where it is needed.
dict_stats_update_if_needed(): Renamed from
row_update_statistics_if_needed().
row_update_for_mysql_using_upd_graph(): Assert that the table statistics
are initialized, as guaranteed by ha_innobase::open(). Update the
statistics in a consistent way, both for FOREIGN KEY triggers and
for the main table. If FOREIGN KEY constraints exist, do not dereference
a freed pointer, but cache the proper value of node->is_delete so that
it matches prebuilt->table.
row_purge_record_func(): Update statistics if
innodb_stats_include_delete_marked=ON.
row_undo_ins(): Update statistics (on ROLLBACK of a fresh INSERT).
This is independent of the parameter; the record is not delete-marked.
row_undo_mod(): Update statistics on the ROLLBACK of updating key columns,
or (if innodb_stats_include_delete_marked=OFF) updating delete-marks.
innodb.innodb_stats_persistent: Renamed and extended from
innodb.innodb_stats_del_mark. Reduced the unnecessarily large dataset
from 262,144 to 32 rows. Test both values of the configuration
parameter innodb_stats_include_delete_marked.
Test that purge is updating the statistics.
innodb_fts.innodb_fts_multiple_index: Adjust the result. The test
is performing a ROLLBACK of an INSERT, which now affects the statistics.
include/wait_all_purged.inc: Moved from innodb.innodb_truncate_debug
to its own file.
2017-05-16 12:07:26 +03:00
|
|
|
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*********************************************************************//**
|
|
|
|
Calculates new estimates for table and index statistics. The statistics
|
|
|
|
are used in query optimization.
|
2012-08-01 17:27:34 +03:00
|
|
|
@return DB_* error code or DB_SUCCESS */
|
2013-03-26 00:03:13 +02:00
|
|
|
dberr_t
|
2012-08-01 17:27:34 +03:00
|
|
|
dict_stats_update(
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*==============*/
|
|
|
|
dict_table_t* table, /*!< in/out: table */
|
|
|
|
dict_stats_upd_option_t stats_upd_option);
|
|
|
|
/*!< in: whether to (re) calc
|
|
|
|
the stats or to fetch them from
|
|
|
|
the persistent storage */
|
|
|
|
|
|
|
|
/*********************************************************************//**
|
|
|
|
Removes the information for a particular index's stats from the persistent
|
|
|
|
storage if it exists and if there is data stored for this index.
|
|
|
|
This function creates its own trx and commits it.
|
2012-08-01 17:27:34 +03:00
|
|
|
@return DB_SUCCESS or error code */
|
2013-03-26 00:03:13 +02:00
|
|
|
dberr_t
|
|
|
|
dict_stats_drop_index(
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*==================*/
|
|
|
|
const char* tname, /*!< in: table name */
|
|
|
|
const char* iname, /*!< in: index name */
|
|
|
|
char* errstr, /*!< out: error message if != DB_SUCCESS
|
|
|
|
is returned */
|
|
|
|
ulint errstr_sz);/*!< in: size of the errstr buffer */
|
2012-08-01 17:27:34 +03:00
|
|
|
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*********************************************************************//**
|
|
|
|
Removes the statistics for a table and all of its indexes from the
|
|
|
|
persistent storage if it exists and if there is data stored for the table.
|
|
|
|
This function creates its own transaction and commits it.
|
2012-08-01 17:27:34 +03:00
|
|
|
@return DB_SUCCESS or error code */
|
2013-03-26 00:03:13 +02:00
|
|
|
dberr_t
|
|
|
|
dict_stats_drop_table(
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*==================*/
|
|
|
|
const char* table_name, /*!< in: table name */
|
|
|
|
char* errstr, /*!< out: error message
|
|
|
|
if != DB_SUCCESS is returned */
|
|
|
|
ulint errstr_sz); /*!< in: size of errstr buffer */
|
2012-08-01 17:27:34 +03:00
|
|
|
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*********************************************************************//**
|
|
|
|
Fetches or calculates new estimates for index statistics. */
|
|
|
|
void
|
|
|
|
dict_stats_update_for_index(
|
|
|
|
/*========================*/
|
|
|
|
dict_index_t* index) /*!< in/out: index */
|
2016-06-21 14:21:03 +02:00
|
|
|
MY_ATTRIBUTE((nonnull));
|
2013-03-26 00:03:13 +02:00
|
|
|
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*********************************************************************//**
|
|
|
|
Renames a table in InnoDB persistent stats storage.
|
|
|
|
This function creates its own transaction and commits it.
|
2013-03-26 00:03:13 +02:00
|
|
|
@return DB_SUCCESS or error code */
|
|
|
|
dberr_t
|
|
|
|
dict_stats_rename_table(
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
/*====================*/
|
|
|
|
const char* old_name, /*!< in: old table name */
|
|
|
|
const char* new_name, /*!< in: new table name */
|
|
|
|
char* errstr, /*!< out: error string if != DB_SUCCESS
|
|
|
|
is returned */
|
|
|
|
size_t errstr_sz); /*!< in: errstr size */
|
2016-09-06 09:43:16 +03:00
|
|
|
|
|
|
|
/** Save an individual index's statistic into the persistent statistics
|
|
|
|
storage.
|
|
|
|
@param[in] index index to be updated
|
|
|
|
@param[in] last_update timestamp of the stat
|
|
|
|
@param[in] stat_name name of the stat
|
|
|
|
@param[in] stat_value value of the stat
|
|
|
|
@param[in] sample_size n pages sampled or NULL
|
|
|
|
@param[in] stat_description description of the stat
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
@param[in,out] trx in case of NULL the function will
|
|
|
|
allocate and free the trx object. If it is not NULL then it will be
|
|
|
|
rolled back only in the case of error, but not freed.
|
2016-09-06 09:43:16 +03:00
|
|
|
@return DB_SUCCESS or error code */
|
|
|
|
dberr_t
|
|
|
|
dict_stats_save_index_stat(
|
|
|
|
dict_index_t* index,
|
2017-12-08 13:10:41 +00:00
|
|
|
ib_time_t last_update,
|
2016-09-06 09:43:16 +03:00
|
|
|
const char* stat_name,
|
|
|
|
ib_uint64_t stat_value,
|
|
|
|
ib_uint64_t* sample_size,
|
|
|
|
const char* stat_description,
|
MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511
MDEV-14511 tried to avoid some consistency problems related to InnoDB
persistent statistics. The persistent statistics are being written by
an InnoDB internal SQL interpreter that requires the InnoDB data dictionary
cache to be locked.
Before MDEV-14511, the statistics were written during DDL in separate
transactions, which could unnecessarily reduce performance (each commit
would require a redo log flush) and break atomicity, because the statistics
would be updated separately from the dictionary transaction.
However, because it is unacceptable to hold the InnoDB data dictionary
cache locked while suspending the execution for waiting for a
transactional lock (in the mysql.innodb_index_stats or
mysql.innodb_table_stats tables) to be released, any lock conflict
was immediately be reported as "lock wait timeout".
To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring
transactional locks on the user tables in both the statistics and DDL
operations was made, but it would still not entirely prevent lock conflicts
on the mysql.innodb_index_stats and mysql.innodb_table_stats tables.
Fixing the remaining problems would require a change that is too intrusive
for a GA release series, such as MariaDB 10.2.
Thefefore, we revert the change MDEV-14511. To silence the
MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.
2018-01-21 18:23:28 +02:00
|
|
|
trx_t* trx);
|
2016-09-06 09:43:16 +03:00
|
|
|
|
2017-05-05 10:25:29 +03:00
|
|
|
/** Report an error if updating table statistics failed because
|
|
|
|
.ibd file is missing, table decryption failed or table is corrupted.
|
|
|
|
@param[in,out] table Table
|
|
|
|
@param[in] defragment true if statistics is for defragment
|
|
|
|
@retval DB_DECRYPTION_FAILED if decryption of the table failed
|
|
|
|
@retval DB_TABLESPACE_DELETED if .ibd file is missing
|
|
|
|
@retval DB_CORRUPTION if table is marked as corrupted */
|
|
|
|
dberr_t
|
|
|
|
dict_stats_report_error(dict_table_t* table, bool defragment = false)
|
|
|
|
MY_ATTRIBUTE((nonnull, warn_unused_result));
|
|
|
|
|
2013-03-26 00:03:13 +02:00
|
|
|
#include "dict0stats.ic"
|
|
|
|
|
2016-08-12 11:17:45 +03:00
|
|
|
#ifdef UNIV_ENABLE_UNIT_TEST_DICT_STATS
|
|
|
|
void test_dict_stats_all();
|
|
|
|
#endif /* UNIV_ENABLE_UNIT_TEST_DICT_STATS */
|
|
|
|
|
2012-08-01 17:27:34 +03:00
|
|
|
#endif /* dict0stats_h */
|