... on semisync slave
To provide semisync master crash-recovery the same server-id transactions
were made to accept for execution on the semisync slave when the strict gtid
mode (see MDEV-27760).
That however caused out-of-order error on a master's transaction
server of the circular setup.
The error was fair in the sense of the gtid strict mode rule as indeed
under the condition of the circular setup the replicated transaction
already exists in the local binlog.
This is fixed by the commit to ignore on the gtid strict mode semisync
slave those gtids that exist in the slave's binlog that effectively restores
the default same-server-id ignore policy.
At the same time the fixes complies with MDEV-21117 semisync slave recovery
to accept the same server-id transactions that do not exist in local binlog.
SHOW ENGINE INNODB STATUS and SHOW GLOBAL VARIABLES were blocking on the locks used
to access the history length in MDEV-29141. While the reason for the blockage
was elsewhere, we should make these monitoring commands less blocking as there
is a trx_sys.history_size_approx function that can be used.
SHOW ENGINE INNODB STATUS and SHOW GLOBAL STATUS LIKE
'innodb_history_list_length' and Innodb Monitors can use
trx_sys.history_size_approx().
This commit is a fixup for MDEV-28762
Analysis: Some recursive json functions dont check for stack control
Fix: Add check_stack_overrun(). The last argument is NULL because it is not
used
Get rid of BTR_ESTIMATE and btr_cur_t::path_arr.
Before the fix btr_estimate_n_rows_in_range_low() used two
btr_cur_search_to_nth_level() calls to create two arrays of tree path,
the array per border. And then it tried to estimate the number of rows
diving level-by-level with the array elements. As the path pages are
unlatched during the arrays iterating, the tree could be modified, the
estimation function called itself until the number of attempts exceed.
After the fix the estimation happens during search process. Roughly, the
algorithm is the following. Dive in the left page, then if there are pages
between left and right ones, read a few pages to the right, if the right
page is reached, fetch it and count the exact number of rows, otherwise
count the estimated number of rows, and fetch the right page.
The latching order corresponds to WL#6326 rules, i.e.:
(2.1) [same as (1.1)]: Page latches must be acquired in descending order
of tree level.
(2.2) When acquiring a node pointer page latch at level L, we must hold
the left sibling page latch (at level L) or some ancestor latch
(at level>L).
When we dive to the level down, the parent page is unlatched only after
the the current level page is latched. When we estimate the number of rows
on some level, we latch the left border, then fetch the next page, and
then fetch the next page unlatching the previous page after the current
page is latched until the right border is reached. I.e. the left sibling
is always latched when we acquire page latch on the same level. When we
reach the right border, the current page is unlatched, and then the right
border is latched. Following to (2.2) rule, we can do this because the
right border's parent is latched.
In previous versions it was stated that MDEV-25968 was causing other
jobs in the pipeline to fail if not run with "-j 2" but this bug was not
affecting fedora-ninja. This is still true for the public gitlab runners.
However, running the fedora-ninja job on custom runners with more processors
without the "-j 2" flag will cause the compiler to crash.
When running the build with 2,4,8,16,32 threads, build times were
consistent indicating that the typical bottleneck is I/O and not CPU
cores. Therefore, "-j 2" is not a big drawback and it was chosen in
order to remain consistent with the other builds affected by MDEV-25968.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Step 1 in handling InnoDB AIO assertions better is to get
more detail of the cases of error.
This doesn't resolve MDEV-27593, but increases the level
of information in the assertion.
Fix the side effect of MDEV-4750 (reenabling innodb_stats_persistent),
so that sporadic MDL acquisition for this table does not interfere with
SELECT from information_schema.metadata_lock_info
Problem:
=======
ALTER TABLE in InnoDB fails to detect duplicate entries
for the unique index when the character set or collation of
an indexed column is changed in such a way that the character
encoding is compatible with the old table definition.
In this case, any secondary indexes on the changed columns
would be rebuilt (DROP INDEX, ADD INDEX).
Solution:
========
During ALTER TABLE, InnoDB keeps track of columns whose collation
changed, and will fill in the correct metadata when sorting the
index records, or applying changes from concurrent DML.
This metadata will be allocated in the dict_index_t::heap of
the being-created secondary indexes.
The fix was developed by Thirunarayanan Balathandayuthapani
and simplified by me.
Depending on OpenSSL version, and at least in 3.0.3, the client-side socket
timeout is reported as generic error (SSL_ERROR_SYSCALL), losing further
details (both errno and GetLastError() return 0). This results in client
reporting "Unknown OpenSSL error" 2026, instead of another generic
"Lost connection to server during query" 2013
Adjusted test case.
Part of MDEV-29000
OpenSSL 3.0.0+ does not support EVP_MD_CTX_FLAG_NON_FIPS_ALLOW any longer.
In OpenSSL 1.1.1 the non FIPS allowed flag is context specific, while
in 3.0.0+ it is a different EVP_MD provider.
Fixes#2010
part of MDEV-29000
Summary of changes
- MD_CTX_SIZE is increased
- EVP_CIPHER_CTX_buf_noconst(ctx) does not work anymore, points
to nobody knows where. The assumption made previously was that
(since the function does not seem to be documented)
was that it points to the last partial source block.
Add own partial block buffer for NOPAD encryption instead
- SECLEVEL in CipherString in openssl.cnf
had been downgraded to 0, from 1, to make TLSv1.0 and TLSv1.1 possible
(according to https://github.com/openssl/openssl/blob/openssl-3.0.0/NEWS.md
even though the manual for SSL_CTX_get_security_level claims that it
should not be necessary)
- Workaround Ssl_cipher_list issue, it now returns TLSv1.3 ciphers,
in addition to what was set in --ssl-cipher
- ctx_buf buffer now must be aligned to 16 bytes with openssl(
previously with WolfSSL only), ot crashes will happen
- updated aes-t , to be better debuggable
using function, rather than a huge multiline macro
added test that does "nopad" encryption piece-wise, to test
replacement of EVP_CIPHER_CTX_buf_noconst
part of MDEV-29000
ha_innobase::check_if_supported_inplace_alter(): Refuse to change the
collation of a column that would become or remain indexed as part of
the ALTER TABLE operation.
In MariaDB Server 10.6, we will allow this type of operation;
that fix depends on MDEV-15250.
Starting with 10.5, InnoDB crash recovery tests seem to time out
more easily under Valgrind, which emulates multiple threads by
interleaving them in a single operating system thread.
These tests will still be covered by
AddressSanitizer and MemorySanitizer.
- InnoDB mistakenly identifies the non-unique FTS_DOC_ID index as
FTS_DOC_ID_INDEX while loading the table. dict_load_indexes()
should check whether the index is unique before assigning
fts_doc_id_index
Before version 10, GCC would think that a right shift of an
unsigned char returns int. Let us explicitly cast that back,
to silence a bogus -Wconversion warning.
- In case of discarded tablespace, InnoDB can't read the root page to
assign the n_core_null_bytes. Consecutive instant DDL fails because
of non-matching n_core_null_bytes.
fil_space_t::acquire_low(): Introduce a parameter that specifies
which flags should be avoided. At all times, referenced() must not
be incremented if the STOPPING flag is set. When fil_system.mutex
is not being held by the current thread, the reference must not be
incremented if the CLOSING flag is set (unless NEEDS_FSYNC is set,
in fil_space_t::flush()).
fil_space_t::acquire(): Invoke acquire_low(STOPPING | CLOSING).
In this way, the reference count cannot be incremented after
fil_space_t::try_to_close() invoked fil_space_t::set_closing().
If the CLOSING flag was set, we must retry acquire_low() after
acquiring fil_system.mutex.
fil_space_t::prepare_acquired(): Replaces prepare(true).
fil_space_t::acquire_and_prepare(): Replaces prepare().
This basically retries fil_space_t::acquire() after
acquiring fil_system.mutex.
- Redundant InnoDB table fails to set the flags2 while loading
the table. It leads to "Upgrade index name failure" during alter
operation. InnoDB should set the flags2 to FTS_AUX_HEX_NAME when
fts is being loaded
recv_sys_t::recover_deferred(): Hold the exclusive page latch until
the tablespace has been set up. Otherwise, the write of the page
may be lost due to non-existent tablespace. This race only affects
the recovery of the first page in a newly created tablespace.
This race condition was introduced in MDEV-24626.
Log MDL state transitions. Trace-friendly message
format. DBUG_LOCK_FILE replaced by thread-local storage.
Logged states legend:
Seized lock was acquired without waiting
Waiting lock is waiting
Acquired lock was acquired after waiting
Released lock was released
Deadlock lock was aborted due to deadlock
Timeout lock was aborted due to timeout >0
Nowait lock was aborted due to zero timeout
Killed lock was aborted due to kill message
OOM can not acquire because out of memory
Usage:
mtr --mysqld=--debug=d,mdl,query:i:o,/tmp/mdl.log
Cleanup from garbage messages:
sed -i -re \
'/(mysql|performance_schema|sys|mtr)\// d; /MDL_BACKUP_/ d' \
/tmp/mdl.log
Ever since commit 9608773f75
the InnoDB persistent statistics are enabled on all InnoDB tables
by default. We must filter out any output that indicates that the
statistics tables are being internally accessed by InnoDB.
The issue was that flush_tables() didn't take a MDL lock on cached
TABLE_SHARE before calling open_table() to do a HA_EXTRA_FLUSH call.
Most engines seams to have no issue with it, but apparantly this conflicts
with InnoDB in 10.6 when using TRUNCATE
Fixed by taking a MDL lock before trying to open the table in
flush_tables().
There is no test case as it hard to repeat the scheduling that causes
the error. I did run the test case in MDEV-28897 to verify
that the bug is fixed.
The test was reported to fail sporadicaly with this diff:
--- mysql-test/main/information_schema_tables.result
+++ mysql-test/main/information_schema_tables.reject
@@ -21,6 +21,8 @@
disconnect con1;
connection default;
DROP VIEW IF EXISTS vv;
+Warnings:
+Note 4092 Unknown VIEW: 'test.vv'
in the "The originally reported non-deterministic test" part.
Disabling warnings around the DROP VIEW statement.
The Spider mixes the comma join with other join types, and thus
ERROR 1054 occurs. This is well-known issue caused by the higher
precedence of JOIN over the comma (,).
We can fix the problem simply by using JOINs instead of commas.
The bug is caused by a similar mechanism as MDEV-21027.
The function, check_insert_or_replace_autoincrement, failed to open
all the partitions on INSERT SELECT statements and it results in the
assertion error.
