Integration with status reporter in wsrep-lib.
Status reporter reports changes in wsrep state and logged errors/
warnings to a json file which then can be read and interpreted by
an external monitoring tool.
Rationale: until the server is fully initialized it is unaccessible
by client and the only source of information is an error log which
is not machine-friendly. Since wsrep node can spend a very long time
in initialization phase (state transfer), it may be a very long time
that automatic tools can't easily monitor its liveness and progression.
New variable: wsrep_status_file specifies the output file name.
If not set, no file is created and no reporting is done.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
By default, we should say "InnoDB: Buffered log writes" instead of
"InnoDB: File system buffers for log disabled" if we are able to
determine the physical block size on Linux.
The fix in commit 6e390a62ba (MDEV-26772)
was a step to the right direction, but implemented incorrectly.
When an InnoDB persistent statistics table cannot be locked immediately,
we must not let row_mysql_handle_errors() to roll back the transaction.
lock_table_for_trx(): Add the parameter no_wait (default false)
for an immediate return of DB_LOCK_WAIT in case of a conflict.
ha_innobase::delete_table(), ha_innobase::rename_table():
Pass no_wait=true to lock_table_for_trx() when needed,
instead of temporarily setting THDVAR(thd, lock_wait_timeout) to 0.
When thread is BF aborted by high priority service, ULL (user level
locks need to be removed and released). Calling directly release of lock for
MDL_EXPLICIT type doesn't clear also `thd->ull_hash`. Method
`mysql_ull_cleanup` will properly clear all information about ULL locks
for thread.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Using parallel slave applying can cause deadlock between between DDL and
other events. GTID with lower seqno can be blocked in galera when node
entered TOI mode, but DDL GTID which has higher node can be blocked
before previous GTIDs are applied locally.
Fix is to check prior commits before entering TOI.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
* Fix test galera.MW-44 to make it work with --ps-protocol
* Skip test galera.MW-328C under --ps-protocol This test
relies on wsrep_retry_autocommit, which has no effect
under ps-protocol.
* Return WSREP related errors on COM_STMT_PREPARE commands
Change wsrep_command_no_result() to allow sending back errors
when a statement is prepared. For example, to handle deadlock
error due to BF aborted transaction during prepare.
* Add sync waiting before statement prepare
When a statement is prepared, tables used in the statement may be
opened and checked for existence. Because of that, some tests (for
example galera_create_table_as_select) that CREATE a table in one node
and then SELECT from the same table in another node may result in errors
due to non existing table.
To make tests behave similarly under normal and PS protocol, we add a
call to sync wait before preparing statements that would sync wait
during normal execution.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Update wsrep-lib which contains a fixup introduced with MDEV-27553.
Also, adapt the corresponding test: after apply failure on ROLLBACK,
node will disconnect from cluster
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Handling BF abort for prepared statement execution so that EXECUTE processing will continue
until parameter setup is complete, before BF abort bails out the statement execution.
THD class has new boolean member: wsrep_delayed_BF_abort, which is set if BF abort is observed
in do_command() right after reading client's packet, and if the client has sent PS execute command.
In such case, the deadlock error is not returned immediately back to client, but the PS execution
will be started. However, the PS execution loop, will now check if wsrep_delayed_BF_abort is set, and
stop the PS execution after the type information has been assigned for the PS.
With this, the PS protocol type information, which is present in the first PS EXECUTE command, is not lost
even if the first PS EXECUTE command was marked to abort.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
This commit contains a test for reproducing the issue in MDEV-27649,
where a transaction, executing a prepared statment, is BF aborted.
The scenario, in MDEV-27649 has a transaction which has prepared a PS,
but not yet executed it, and this transaction is then BF aborted in this state.
When the BF aborted transaction tries to execute the PS, it will receive deadlock error.
But, when it tries to execute the PS second time, the node crashes.
Mtr test galera.galera_bf_abort_ps_bind, exercises this scenario.
However, mtr test platform does not have mechanism to control the execution of PS in required detail.
For this purpose, mysqltetst.cc was extended to contain 4 new commands:
PS_prepare - to prepare a prepared statement
PS_bind - to bind values for parameters for the PS
PS_execute - to execute the PS
PS_close - to close the PS
The support for controlling prepared statments in mtr scripts is quite minimal
in this commit. Limitations are:
* only one PS can be used by a connection, at a time
* only input parameters can be bound for the PS
* only varchar, integer or float type of parameters can be bound
added the result
fixes
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Since Debian Sid now has MariaDB 10.6, we can't do any upgrade tests in
Debian Sid for the 10.5 branch anymore. It would just fail with downgrade
errors.
Also, since MariaDB 10.5 is no longer in Sid, we can't even test 10.5.x
to 10.5.y upgrades in Sid.
Instead the 10.5 branch salsa-ci.yml should run all builds and tests based
on Debian Bullseye, which has MariaDB 10.5 (only).
To achieve this, essentially sync most the the salsa-ci.yml contents from
https://salsa.debian.org/mariadb-team/mariadb-10.5/-/tree/bullseye
Also add a couple Lintian overrides to make Salsa-CI pass.
NOTE TO MERGERS: This commit is intended for the 10.5 branch only, do not
merge anything from it on 10.6 or any other branch.
In commit 685d958e38 (MDEV-14425)
we ended up not enabling O_DIRECT writes on the redo log
by default, because back then, it was slightly slower on
some systems.
With commit a635c40648 (MDEV-27774)
the situation changed. A new test on a NVMe device shows 9%
improvement in throughput and over 15% reduction of latency
when O_DIRECT writes are enabled.
With this change, all the following settings will use O_DIRECT
on InnoDB data and log files:
innodb_flush_method=O_DIRECT
innodb_flush_method=O_DIRECT_NO_FSYNC
innodb_flush_method=O_DSYNC
Before MDEV-14425, log writes were always buffered on Linux.
Between MDEV-14425 and this change, unbuffered log writes
were only enabled for innodb_flush_method=O_DSYNC.
TYPELIBs for ENUM/SET columns could erroneously undergo redundant
hex-unescaping at the table open time.
Fix:
- Prevent multiple unescaping of the same TYPELIB
- Prevent sharing TYPELIBs between columns with different mbminlen
In commit ab38b7511b
an added "goto err" would seemingly cause a read of
an uninitialized variable old_info if errpos>=5.
However, because we would have errpos=0 at that point,
there was no real error.
- InnoDB purge waits at resume_FTS() while shutting down.
This happens after altering the FTS innodb partition table.
stop_FTS() has been called for each partition, but it calls
resume_FTS() only once and it leads to hang during shutdown.
This issue was introduced by
commit 1bd681c8b3c5213ce1f7976940a7dc38b48a0d39(MDEV-25506).
The data member log_sys.block_size is only defined in environments
where we can determine the physical block size of a file
(currently, Linux and Microsoft Windows). The accessor function
get_block_size() is available everywhere.
Thanks to Dmitry Shulga for reporting the build failure.
extra2_read_len resolved by keeping the implementation
in sql/table.cc by exposed it for use by ha_partition.cc
Remove identical implementation in unireg.h
(ref: bfed2c7d57)
Uses 500M+ of memory by repeating an 8 byte sequence 62.5M times.
Reduce the number of repeats on string reduced by 100 times.
Tested by applying against the reverted MDEV-24909 code. 1000 times
reduction was too much, but 100 still managed to trigger the bug.
Per Marko's comment in JIRA, sql_kill is passing the thread id
as long long. We change the format of the error messages to match,
and cast the thread id to long long in sql_kill_user.
The 10.5 test error main.grant_kill showed up a incorrect
thread id on a big endian architecture.
The cause of this is the sql_kill_user function assumed the
error was ER_OUT_OF_RESOURCES, when the the actual error was
ER_KILL_DENIED_ERROR. ER_KILL_DENIED_ERROR as an error message
requires a thread id to be passed as unsigned long, however a
user/host was passed.
ER_OUT_OF_RESOURCES doesn't even take a user/host, despite
the optimistic comment. We remove this being passed as an
argument to the function so that when MDEV-21978 is implemented
one less compiler format warning is generated (which would
have caught this error sooner).
Thanks Otto for reporting and Marko for analysis.
buf_flush_freed_pages(): Assert that neither buf_pool.mutex
nor buf_pool.flush_list_mutex are held. Simplify the loops.
Return the tablespace and the number of pages written or punched.
buf_flush_LRU_list_batch(), buf_do_flush_list_batch():
Release buf_pool.mutex before invoking buf_flush_space().
buf_flush_list_space(): Acquire the mutexes only after invoking
buf_flush_freed_pages().
Reviewed by: Thirunarayanan Balathandayuthapani
In commit a635c40648 (MDEV-27774)
a race condition was introduced between mtr_t::commit() and
a log checkpoint.
Between the time of assigning the log sequence number and adding
the changed pages to buf_pool.flush_list, the log_sys.latch must
be continuously held by the current thread, or otherwise a
log checkpoint could get the wrong result from
buf_pool.get_oldest_modification().
buf_pool_t::insert_into_flush_list(): Add a debug assertion for
increasing the probability of cathing this type of problem.
mtr_t::m_latch_ex: A flag that indicates whether the mini-transaction
is holding log_sys.latch in exclusive mode.
mtr_t::do_write(), mtr_t::finish_write(): Remove the parameter
"bool ex" and refer to m_latch_ex instead.
mtr_t::commit(): Release log_sys.latch according to m_latch_ex.
mtr_t::commit_shrink(), mtr_t::commit_files(): Set m_latch_ex.
mtr_t::do_write(): Do not release an exclusive log_sys.latch,
but instead set m_latch_ex if needed.
fil_space_t::try_to_close(): Tolerate a tablespace that has no
data files attached. The function fil_ibd_create() initially
creates and attaches a tablespace with no files, and invokes
fil_space_t::add() later.
fil_node_open_file(): After releasing and reacquiring fil_system.mutex,
check if the file was already opened by another thread. This avoids
an assertion failure !node->is_open() in fil_node_open_file_low().
These failures were reproduced with the test
innodb.table_definition_cache_debug and the fix of MDEV-27985.
