In commit 54e2e70194
we relaxed a debug assertion in the POSIX version of
os_file_rename_func() only. Let us relax it also on Windows,
so that the test innodb.truncate_crash will pass.
In commit 91599701d0 (MDEV-25312)
some recovery code for TRUNCATE TABLE was broken
causing a regression in a case where undo log for a RENAME TABLE
operation had been durably written but the tablespace had not been
renamed yet.
row_rename_table_for_mysql(): Add a DEBUG_SYNC point for the
test case, and simplify the logic and trim the error messages.
fil_space_t::rename(): Simplify the operation. Merge the necessary
part of fil_rename_tablespace_check(). If there is no change to
the file name, do nothing.
dict_table_t::rename_tablespace(): Refactored from
dict_table_rename_in_cache().
row_undo_ins_parse_undo_rec(): On rolling back TRX_UNDO_RENAME_TABLE,
invoke dict_table_t::rename_tablespace() even if the table name matches.
os_file_rename_func(): Temporarily relax an assertion that would
fail during the recovery in the test innodb.truncate_crash.
after dfb41fddf6 tables that failed to drop are excluded from the
binlogged DROP TABLE statement. It means that the slave should not
expect any errors when executing DROP TABLE, and the binlog should
report that no error has happened, even if it was.
Do not write error code into the binlogged DROP TABLE,
and remove all code that was needed to compute it.
Cluster communication should be possible even when:
1. Node 2 is TCP
2. Node 1/3 is dynamic with SSL enabled
During test we shutdown Node 2 and enable SSL on it. It should connect
back to cluster successfully.
lock_discard_for_index(): New function, to discard locks for an
index whose index tree has been purged. By definition, such indexes
must be ones for which the MDL upgrade failed in inplace ALTER TABLE
and the ADD INDEX operation was never committed.
Note: Because we do not support online ADD SPATIAL INDEX, we only
have to traverse the lock_sys.rec_hash for B-trees and not the
hash tables for R-trees.
row_purge_remove_clust_if_poss_low(): Invoke lock_discard_for_index()
if necessary before dropping a B-tree for a SYS_INDEXES record.
btr_free_if_exists(): Always use the BUF_GET_POSSIBLY_FREED mode
when accessing pages, because due to MDEV-24589 the function
fil_space_t::set_stopping(true) can be called at any time during
the execution of this function.
mtr_t::m_freeing_tree: New data member for debugging purposes.
buf_page_get_low(): Assert that the BUF_GET mode is not being used
anywhere during the execution of btr_free_if_exists().
In all code related to freeing or allocating pages, we will add some
robustness, by making more use of BUF_GET_POSSIBLY_FREED and by
reporting an error instead of crashing in some cases of corruption.
fil_check_pending_ops(), fil_check_pending_io(): Remove.
These functions were actually duplicating each other ever since
commit 118e258aaa (MDEV-23855).
fil_space_t::check_pending_operations(): Replaces
fil_check_pending_operations() and incorporates the logic of
fil_check_pending_ops(). Avoid unnecessary lookups for the tablespace.
Just wait for the reference count to drop to zero.
fil_space_t::io(): Remove an unnecessary condition. We can (and
probably better should) refuse asynchronous reads of undo tablespaces
that are being truncated.
fil_truncate_prepare(): Remove.
trx_purge_truncate_history(): Implement the necessary steps that used
to be in fil_truncate_prepare().
innodb_adaptive_flushing_lwm is hit. (possible regression)
adaptive flushing should kick in if
a. dirty_pct (dirty pages in buffer pool) > innodb_max_dirty_pages_pct_lwm
OR
b. innodb_adaptive_flushing_lwm limit is reached (default to 10%)
both conditions are mutually exclusive and whichever is first to evaluate
true should kick-start the adaptive flushing.
After recent changes to simplify the flushing algorithm logic, (b) got ignored
that introduced the said regression.
row_merge_is_index_usable(): Allow access to any SEQUENCE, even if it was
created after the read view. SQL sequences are no-rollback tables with no
history at all.
Quoting MDEV reporter Daniel Lewart:
Starting MariaDB with default configuration causes the following problems:
"[Warning] Could not increase number of max_open_files to more than 16384 (request: 32186)"
silently reduces table_open_cache_instances from 8 (default) to 4
Default Server System Variables:
extra_max_connections = 1
max_connections = 151
table_open_cache = 2000
table_open_cache_instances = 8
thread_pool_size = 4
LimitNOFILE=16834 is in the following files:
support-files/mariadb.service.in
support-files/mariadb@.service.in
Looking at sql/mysqld.cc lines 3837-3917:
wanted_files= (extra_files + max_connections + extra_max_connections +
tc_size * 2 * tc_instances);
wanted_files+= threadpool_size;
Plugging in the default values:
wanted_files = (30 + 151 + 1 + 2000 * 2 * 8 + 4) = 32186
However, systemd configuration has LimitNOFILE = 16384, which is far smaller.
I suggest increasing LimitNOFILE to 32768.
This is a backport of
commit fd9ca2a742 (MDEV-23295) and
commit 9a156e1a23 (MDEV-23345) to 10.3.
An instant ADD/DROP/reorder column could create a dummy table
object with the wrong ROW_FORMAT when innodb_default_row_format
was changed between CREATE TABLE and ALTER TABLE.
prepare_inplace_alter_table_dict(): If we had promised that
ALGORITHM=INPLACE is supported, we must preserve the ROW_FORMAT.
The rest of the changes are related to adding
Alter_inplace_info::inplace_supported to cache the return value of
handler::check_if_supported_inplace_alter().
Back port upstream fix
commit 1800b015a1d487330f7b15f2020b887be348a66b
Author: Venkatesh Duggirala <venkatesh.duggirala@oracle.com>
Date: Fri Sep 8 20:29:22 2017 +0530
Bug#26027024 SLAVE_COMPRESSED_PROTOCOL DOESN'T WORK WITH
SEMI-SYNC REPLICATION IN MYSQL-5.7
Analysis: In mysql-5.6, dump thread (the thread that is created
on Master after Slave requested for a binlog dump) is also used
to receive acknowledgements from the Slave and act on them accordingly.
For performance reasons, a special thread called Ack Receiver thread
is added in mysql-5.7 Semi synchronous replication plugin.
This thread does not have special handling to receive acknowledgements
if Slave has enabled compression in the protocol. Hence Master is
unable to handle any slave if Slave_compressed_protocol is enabled
on it.
Fix: Enable compress flag on the communication channels if the Slave
has Slave_compressed_protocol ON.
(trivial backport to 10.2)
The optimizer removes redundant GROUP BY operations. If GROUP BY element
is a subselect, it is "eliminated".
However one must not eliminate the item if it is used both in the select
list and in the GROUP BY, like so:
select (select ... ) as SUBQ from ... group by SUBQ
Do not eliminate such items.
In the SUX_LOCK_GENERIC implementation, we can remember at most
one pending exclusive lock request. If multiple exclusive lock
requests are pending, the WRITER_WAITING flag will be cleared when
the first waiting writer acquires the exclusive lock.
ssux_lock_low::update_lock(): If WRITER_WAITING is set, wake up
the writer even if the UPDATER flag is set, because the waiting
writer may be in the process of upgrading its U lock to X.
rw_lock::read_unlock(): Also indicate that an X lock waiter must
be woken up if an U lock exists.
This fix may cause unnecessary wake-ups and system calls, but this
is the best that we can do. Ideally we would use the MDEV-25404
idea of a separate 'writer' mutex, but there is no portable way to
request that a non-recursive mutex be created, and InnoDB requires
the ability to transfer buf_block_t::lock ownership to an I/O thread.
To allow problems like this to be caught more reliably in the future,
we add a unit test for srw_mutex, srw_lock, ssux_lock, sux_lock.
The optimizer removes redundant GROUP BY operations. If GROUP BY element
is a subselect, it is "eliminated".
However one must not eliminate the item if it is used both in the select
list and in the GROUP BY, like so:
select (select ... ) as SUBQ from ... group by SUBQ
Do not eliminate such items.
It is possible that an object that was originally created by
open_purge_table() will remain cached and reused for SQL execution.
Our previous fix wrongly assumed that ha_innobase::open() would
always be called before SQL execution starts. Therefore, we must
invoke dict_stats_init() in ha_innobase::info_low() instead of
only doing it in ha_innobase::open().
Note: Concurrent execution of dict_stats_init() on the same table
is possible, but it also was possible between two calls to
ha_innobase::open(), with no ill effects observed.
This should fix the assertion failure on stat_initialized.
A possibly easy way to reproduce it would have been
to run the server with innodb_force_recovery=2 (disable the purge of
history), update a table so that an indexed virtual column will be
affected, and finally restart the server normally (purge enabled),
to observe a crash when the table is accessed from SQL.
The problem was first observed and this fix verified by
Elena Stepanova. Also Thirunarayanan Balathandayuthapani
repeated the problem.
row_sel_sec_rec_is_for_clust_rec(): If the field in the
clustered index record stored off page, always fetch it,
also when the secondary index field has been built on the
entire column. This was broken ever since the InnoDB Plugin
for MySQL Server 5.1 introduced ROW_FORMAT=DYNAMIC and
ROW_FORMAT=COMPRESSED for InnoDB tables. That code was first
introduced in this tree in
commit 3945d5e554.
For the original ROW_FORMAT=REDUNDANT and the MySQL 5.0.3
ROW_FORMAT=COMPRESSED, there was no problem, because for
those tables we always stored at least a 768-byte prefix of
each column in the clustered index record.
row_sel_sec_rec_is_for_blob(): Allow prefix_len==0 for matching
the full column.
Before FRM is written walk vcol expressions through
check_table_name_processor() and check if field items match (db,
table_name) qualifier.
We cannot do this in check_vcol_func_processor() as there is already
no table name qualifiers in expressions of written and loaded FRM.
Buffer overflow in ib_push_warning() fixed by using vsnprintf().
InnoDB parser was obsoleted by MDEV-16417.
Thanks to Nikita Malyavin for review and suggestion.
row_ins_clust_index_entry_low(): Do not enable bulk insert if
any record locks exist on the table. Bulk insert is assumed to
be covered only by an exclusive table lock, with no row-level
locking or undo logging.
Before this patch mergeable derived tables / view used in a multi-table
update / delete were merged before the preparation stage.
When the merge of a derived table / view is performed the on expression
attached to it is fixed and ANDed with the where condition of the select S
containing this derived table / view. It happens after the specification of
the derived table / view has been merged into S. If the ON expression refers
to a non existing field an error is reported and some other mergeable derived
tables / views remain unmerged. It's not a problem if the multi-table
update / delete statement is standalone. Yet if it is used in a stored
procedure the select with incompletely merged derived tables / views may
cause a problem for the second call of the procedure. This does not happen
for select queries using derived tables / views, because in this case their
specifications are merged after the preparation stage at which all ON
expressions are fixed.
This patch makes sure that merging of the derived tables / views used in a
multi-table update / delete statement is performed after the preparation
stage.
Approved by Oleksandr Byelkin <sanja@mariadb.com>
Before this patch mergeable derived tables / view used in a multi-table
update / delete were merged before the preparation stage.
When the merge of a derived table / view is performed the on expression
attached to it is fixed and ANDed with the where condition of the select S
containing this derived table / view. It happens after the specification of
the derived table / view has been merged into S. If the ON expression refers
to a non existing field an error is reported and some other mergeable derived
tables / views remain unmerged. It's not a problem if the multi-table
update / delete statement is standalone. Yet if it is used in a stored
procedure the select with incompletely merged derived tables / views may
cause a problem for the second call of the procedure. This does not happen
for select queries using derived tables / views, because in this case their
specifications are merged after the preparation stage at which all ON
expressions are fixed.
This patch makes sure that merging of the derived tables / views used in a
multi-table update / delete statement is performed after the preparation
stage.
Approved by Oleksandr Byelkin <sanja@mariadb.com>
While MDEV-24589 causes the regression
MDEV-25491 Race condition between DROP TABLE and purge of
SYS_INDEXES record
we must not revert it from 10.6, because MDEV-24589 is a prerequisite for
MDEV-25180 Atomic ALTER TABLE
that is targeting 10.6. The regression will be dealt with later.