Analysis:
========
Writes to 'rli->log_space_total' needs to be synchronized, otherwise both
SQL_THREAD and IO_THREAD can try to modify the variable simultaneously
resulting in incorrect rli->log_space_total. In the current test scenario
SQL_THREAD is trying to decrement 'rli->log_space_total' in 'purge_first_log'
and IO_THREAD is trying to increment the 'rli->log_space_total' in
'queue_event' simultaneously. Hence test occasionally fails with result
mismatch.
Fix:
===
Convert 'rli->log_space_total' variable to atomic type.
A follow-up fix, for original fix for MDEV-21577, which did not handle well
temporary tables.
OPTIMIZE and REPAIR TABLE statements can take a list of tables as argument,
and some of the tables may be temporary. Proper handling of temporary tables
is to skip them and continue working on the real tables. The bad version, skipped all tables,
if a single temporary table was in the argument list. And this resulted so that FK parent
tables were not scnanned for the remaining real tables. Some mtr tests, using OPTIMIZE or REPAIR
for temporary tables caused regressions bacause of this, e.g. galera.galera_optimize_analyze_multi
The fix in this PR opens temporary and real tables first, and in the table handling loop skips
temporary tables, FK parent scanning is done only for real tables.
The test has new scenario for OPTIMIZE and REPAIR issued for two tables of which one is temporary table.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
This PR fixes same issue as MDEV-21577 for TRUNCATE TABLE.
MDEV-21577 fixed TOI replication for OPTIMIZE, REPAIR and ALTER TABLE
operating on FK child table. It was later found out that also TRUNCATE
has similar problem and needs a fix.
The actual fix is to do FK parent table lookup before TRUNCATE TOI
isolation and append found FK parent table names in certification key
list for the write set.
PR contains also new test scenario in galera_ddl_fk_conflict test where
FK child has two FK parent tables and there are two DML transactions operating
on both parent tables.
For development convenience, new TO isolation macro was added:
WSREP_TO_ISOLATION_BEGIN_IF and WSREP_TO_ISOLATION_BEGIN_ALTER macro was changed
to skip the goto statement.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
As noted in commit 0b66d3f70d,
MariaDB does not support CREATE TABLESPACE for InnoDB.
Hence, some code that was added in
commit fec844aca8
and originally in
mysql/mysql-server@c71dd213bd
is unused in MariaDB and should be removed.
trx_undo_left(): Return 0 in case of an overflow, instead of
returning a negative number interpreted as a large positive number.
Also, add debug assertions to check that the pointer is within
the page area. This should allow us to catch bugs like
MDEV-24096 easier in the future.
The function ibuf_merge_or_delete_for_page() was always being
invoked with update_ibuf_bitmap=true ever since
commit cd623508df
fixed up something after MDEV-9566.
Furthermore, the parameter page_size is never being passed as a
null pointer, and therefore it should better be a reference to
a constant object.
- Patch 95bb3cb886 used `my_which` function in `10.2`
- Based on patch `355ee6877bec` from 10.3+ `command -v` is used instead of
`my_which` so we are changing in this patch also
- `ldconfig` is usually found in `/sbin` so make sure it is added in
`$PATH` variable
- Patch is solving generating report on warning
To repeat the error run single worker:
```
./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st --force --parallel 1
```
or `N` workers with `N+1` tests with failures and `force`
```
./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st grant5 --force --parallel 2
```
- Patch is doing cosmetic fix of `current_test` log file which holds the old log value of test `CURRENT TEST:..` in `mark_log()` in case of `unknown option` and as such
the logic which is using it's content doesn't output valid log content and doesn't generate valid `$test->{'comment'}` message.asdf
- Closing the socket/handler after the removing the handler from IO for
consistency
Reviewed by: serg@mariadb.com
MDEV-23672 (commit 7eda556196)
introduced a regression that can corrupt not only undo log pages,
but anything that resides in the InnoDB buffer pool.
trx_undo_left(): Add debug assertions for the assumptions.
If the pointer is out of bounds, we will return a positive
number, not a negative one. Thus, once a page overflow occurs,
further overflow to adjacent pages will be allowed.
This allows us to remove some more relaxed debug assertions
from some callers.
trx_undo_log_v_idx(): Correctly calculate the size limit.
This bug could manifest itself for a query with WHERE condition containing
top level OR formula such that each conjunct contained a single-range
condition supported by the same index. One of these range conditions must
be fully covered by another range condition that is used later in the OR
formula. Additionally at least one of these condition should be ANDed with
a sargable range condition supported by a different index.
There were several attempts to fix related problems for OR conditions after
the backport of range optimizer code from MySQL (commit
0e19f3e36f). Unfortunately the first of these
fixes contained typo remained unnoticed until recently. This typo bug led
to rejection of valid range accesses. This patch fixed this typo bug.
The fix revealed another two bugs: one in a constructor for SEL_ARG,
the other in the function tree_or(). Both are fixed in this patch.
Part#1: Revert the patch that caused it:
commit 291be49474
Author: Igor Babaev <igor@askmonty.org>
Date: Thu Sep 24 22:02:00 2020 -0700
MDEV-23811: With large number of indexes optimizer chooses an inefficient plan
The instrumentation that was added in
commit 90635c6fb5 (MDEV-7620)
was effectively reverted in MariaDB Server 10.2.2, in
commit 2e814d4702
(which stopped reporting the statistics) and
commit fec844aca8
(which stopped updating the statistics).
Let us remove the orphan data members to reduce the memory footprint.
- mysqlnd from PHP < 7.3
- mysql-connector-python any version
- mysql-connector-java any version
Relaxed check about garbage at the end of the packet in case of no parameters.
Added check for array binding.
Fixed test according to the new paradigm (allow junk at the end of the packet)
The parser of CREATE USER accepts ACCOUNT LOCK before PASSWORD
EXPIRE but not the other way around.
This just changes the SHOW CREATE USER to output a sql syntax that
is valid.
Thanks to Robert Bindar for analysis.
During graceful shutdowns, client connections are closed and
eventually and THD::awake() acquires LOCK_thd_data mutex which is
required later on in wsrep_thd_is_aborting(). Make sure LOCK_thd_data
is acquired, even if global wsrep_on is disabled.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Some DDL statements appear to acquire MDL locks for a table referenced by
foreign key constraint from the actual affected table of the DDL statement.
OPTIMIZE, REPAIR and ALTER TABLE belong to this class of DDL statements.
Earlier MariaDB version did not take this in consideration, and appended
only affected table in the certification key list in write set.
Because of missing certification information, it could happen that e.g.
OPTIMIZE table for FK child table could be allowed to apply in parallel
with DML operating on the foreign key parent table, and this could lead to
unhandled MDL lock conflicts between two high priority appliers (BF).
The fix in this patch, changes the TOI replication for OPTIMIZE, REPAIR and
ALTER TABLE statements so that before the execution of respective DDL
statement, there is foreign key parent search round. This FK parent search
contains following steps:
* open and lock the affected table (with permissive shared locks)
* iterate over foreign key contstraints and collect and array of Fk parent
table names
* close all tables open for the THD and release MDL locks
* do the actual TOI replication with the affected table and FK parent
table names as key values
The patch contains also new mtr test for verifying that the above mentioned
DDL statements replicate without problems when operating on FK child table.
The mtr test scenario #1, which can be used to check if some other DDL
(on top of OPTIMIZE, REPAIR and ALTER) could cause similar excessive FK
parent table locking.
Reviewed-by: Aleksey Midenkov <aleksey.midenkov@mariadb.com>
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Changed the test so that it does not rely on specific auto increment
ids. With Galera's default wsrep_auto_increment_control setting it is
not guaranteed that auto increments always start from 1. The test was
occasionally failing due to result content mismatch.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Prepared statements which were run over binary protocol crashed
a server if the statement did not have CF_PS_ARRAY_BINDING_OPTIMIZED
flag and the statement was executed in bulk mode and a BF abort occrurred.
This was because the bulk execution resulted in several statements without
calling wsrep_after_statement() between, which confused wsrep transaction
state tracking.
As a fix, call wsrep_after_statement() in bulk loop after each execution
if CF_PS_ARRAY_BINDING_OPTIMIZED is not set.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Bugs fixed:
- prepare_for_repair() didn't close all open files if table opened failed
because of out-of-memory
- If dd_recreate_table() failed, the data file was not properly restored
from it's temporary name
- Aria repair initializing code didn't properly clear all used structs
before calling error, which caused crashed in memory-free calls.
- maria_delete_table() didn't register if table open failed. This could
calls my_error() to be called without returning 1 to the caller, which
cased failures in my_ok()
Note when merging to 10.5:
- Remove the #if MYSQL_VERSION from sql_admin.cc
instant_alter_column_possible(): Relax a too strict debug assertion.
The existence of an index stub or a corrupted index on virtual columns
does not imply that virtual columns exist.
