Adding tests demonstrating that columns:
- mysql.innodb_table_stats.last_update
- mysql.innodb_index_stats.last_update
contain sane values close to NOW() rathar than a garbage.
Tests cover these three underlying TIMESTAMP data formats:
- MariaDB Field_timestamp0 - UINT4 based
Like in a MariaDB native installation running with mysql56_temporal_format=0
- MariaDB Field_timestampf - BINARY(4) based, with UNSIGNED_FLAG
Like in a MariaDB native installation running with mysql56_temporal_format=1
- MySQL-alike Field_timestampf - BINARY(4) based, without UNSIGNED_FLAG
Like with a MariaDB server running over a MySQL-5.6 directory
(e.g. during a migragion).
Problem:
Field_timestampf implementations differ in MySQL and MariaDB:
- MariaDB sets the UNSIGNED_FLAG in Field::flags
- MySQL does not
The reference table structures
(defined in table_stats_schema and index_stats_schema)
expected the last_update column to have the DATA_UNSIGNED flag,
because MariaDB's Field_timestampf has the UNSIGNED_FLAG.
It worked fine on pure MariaDB installations.
However, if a MariaDB server starts over a MySQL-5.7 data directory during
a migration, the last_update column does not have DATA_UNSIGNED flag,
because MySQL's Field_timestampf does not have the UNSIGNED_FLAG.
This made InnoDB (after the migration from MySQL) complain into the server
error log about the unexpected data type.
The actual fix is done in storage/innobase/dict/dict0stats.cc:
It removes DATA_UNSIGNED from the prtype_mask member of the reference columns,
so now it does not require the underlying columns to have this flag.
The rest of the fix is needed for MTR tests.
The new data type plugin TYPE_MYSQL_TIMESTAMP implements a slightly modified
version of Field_timestampf, which removes the unsigned flag, so
it works like MySQL's Field_timestampf.
The MTR test ALTERs the data type of the columns
table_stats_schema.last_update and index_stats_schema.last_update
from TIMESTAMP to TYPE_MYSQL_TIMESTAMP, then makes InnoDB
verify the structure of the two statistics tables by creating
and populating an InnoDB table t1.
Without the fix made storage/innobase/dict/dict0stats.cc,
MTR complains about unexpected warnings in the server error log:
[ERROR] InnoDB: Column last_update in table mysql.innodb_table_stats is ...
[ERROR] InnoDB: Column last_update in table mysql.innodb_index_stats is ...
With the fix made storage/innobase/dict/dict0stats.cc these warnings
go away.
AWS KMS plugin saves all key files under the root folder of data
directory. Increasing of the key IDs and key rotations will generate a
lot of key files under the root folder, looks messy and hard to
maintain the folder permission etc.
Now introduce a new plugin parameter `aws_key_management_keyfile_dir` to
define the directory for saving the key files for better maintenance.
Detailed parameter information as following:
```
VARIABLE_NAME: AWS_KEY_MANAGEMENT_KEYFILE_DIR
SESSION_VALUE: NULL
GLOBAL_VALUE: <Directory path>
GLOBAL_VALUE_ORIGIN: COMMAND-LINE
DEFAULT_VALUE:
VARIABLE_SCOPE: GLOBAL
VARIABLE_TYPE: VARCHAR
VARIABLE_COMMENT: Define the directory in which to save key files
for the AWS key management plugin. If not set,
the root datadir will be used
READ_ONLY: YES
COMMAND_LINE_ARGUMENT: REQUIRED
GLOBAL_VALUE_PATH: NULL
```
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
While performing SAST scanning using Cppcheck against source code of
commit 81196469, several code vulnerabilities were found.
Fix following issues:
1. Parameters of `snprintf` function are incorrect.
Cppcheck error:
client/mysql_plugin.c:1228: error: snprintf format string requires 6 parameters but only 5 are given.
It is due to commit 630d7229 introduced option `--lc-messages-dir`
in the bootstrap command. However the parameter was not even given
in the `snprintf` after changing the format string.
Fix:
Restructure the code logic and correct the function parameters for
`snprintf`.
2. Null pointer is used in a `snprintf` which could cause a crash.
Cppcheck error:
extra/mariabackup/xbcloud.cc:2534: error: Null pointer dereference
The code intended to print the swift_project name, if the
opt_swift_project_id is NULL but opt_swift_project is not NULL.
However the parameter of `snprintf` was mistakenly using
`opt_swift_project_id`.
Fix:
Change to use the correct string from `opt_swift_project`.
3. Potential double release of a memory
Cppcheck error:
plugin/auth_pam/testing/pam_mariadb_mtr.c:69: error: Memory pointed to by 'resp' is freed twice.
A pointer `resp` is reused and allocated new memory after it has been
freed. However, `resp` was not set to NULL after freed.
Potential double release of the same pointer if the call back
function doesn't allocate new memory for `resp` pointer.
Fix:
Set the `resp` pointer to NULL after the first free() to make sure
the same address is not freed twice.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
The main difference in code path between EQ_REF and REF is that for
REF we have to do an extra read_next on the index to check that there
is no more matching rows.
Before this patch we added a preference of EQ_REF by ensuring that REF
would always estimate to find at least 2 rows.
This patch adds the cost of the extra key read_next to REF access and
removes the code that limited REF to at least 2 rows. For some queries
this can have a big effect as the total estimated rows will be halved
for each REF table with 1 rows.
multi_range cost calculations are also changed to take into account
the difference between EQ_REF and REF.
The effect of the patch to the test suite:
- About 80 test case changed
- Almost all changes where for EXPLAIN where estimated rows for REF
where changed from 2 to 1.
- A few test cases using explain extended had a change of 'filtered'.
This is because of the estimated rows are now closer to the
calculated selectivity.
- A very few test had a change of table order.
This is because the change of estimated rows from 2 to 1 or the small
cost change for REF
(main.subselect_sj_jcl6, main.group_by, main.dervied_cond_pushdown,
main.distinct, main.join_nested, main.order_by, main.join_cache)
- No key statistics and the estimated rows are now smaller which cased
estimated filtering to be lower.
(main.subselect_sj_mat)
- The number of total rows are halved.
(main.derived_cond_pushdown)
- Plans with 1 row changed to use RANGE instead of REF.
(main.group_min_max)
- ALL changed to REF
(main.key_diff)
- Key changed from ref + index_only to PRIMARY key for InnoDB, as
OPTIMIZER_ROW_LOOKUP_COST + OPTIMIZER_ROW_NEXT_FIND_COST is smaller than
OPTIMIZER_KEY_LOOKUP_COST + OPTIMIZER_KEY_NEXT_FIND_COST.
(main.join_outer_innodb)
- Cost changes printouts
(main.opt_trace*)
- Result order change
(innodb_gis.rtree)
This patch is the result of running
run-clang-tidy -fix -header-filter=.* -checks='-*,modernize-use-equals-default' .
Code style changes have been done on top. The result of this change
leads to the following improvements:
1. Binary size reduction.
* For a -DBUILD_CONFIG=mysql_release build, the binary size is reduced by
~400kb.
* A raw -DCMAKE_BUILD_TYPE=Release reduces the binary size by ~1.4kb.
2. Compiler can better understand the intent of the code, thus it leads
to more optimization possibilities. Additionally it enabled detecting
unused variables that had an empty default constructor but not marked
so explicitly.
Particular change required following this patch in sql/opt_range.cc
result_keys, an unused template class Bitmap now correctly issues
unused variable warnings.
Setting Bitmap template class constructor to default allows the compiler
to identify that there are no side-effects when instantiating the class.
Previously the compiler could not issue the warning as it assumed Bitmap
class (being a template) would not be performing a NO-OP for its default
constructor. This prevented the "unused variable warning".
Starting with commit baf276e6d4 (MDEV-19229)
the parameter innodb_undo_tablespaces can be increased from its
previous default value 0 while allowing an upgrade from old databases.
We will change the default setting to innodb_undo_tablespaces=3
so that the space occupied by possible bursts of undo log records
can be reclaimed after SET GLOBAL innodb_undo_log_truncate=ON.
We will not enable innodb_undo_log_truncate by default, because it
causes some observable performance degradation.
Special thanks to Thirunarayanan Balathandayuthapani for diagnosing
and fixing a number of bugs related to this new default setting.
Tested by: Matthias Leich, Axel Schwenke, Vladislav Vaintroub
(with both values of innodb_undo_log_truncate)
The MDEV-25004 test innodb_fts.versioning is omitted because ever since
commit 685d958e38 InnoDB would not allow
writes to a database where the redo log file ib_logfile0 is missing.
