The test was passing some uninitialized data to libmariadb.
Mostly, the MemorySanitizer wrapper of send() detected that
some bytes were uninitialized.
The test_mdev19838() is for now disabled under MemorySanitizer,
to be fixed in MDEV-26761.
This is mostly working around a bad compiler optimization.
The Intel 80386 processor introduced some bit operations that would be
the perfect translation for atomic single-bit read-modify-and-write
operations. Alas, even the latest compilers as of today
(GCC 11, clang 13, Microsoft Visual C 19.29) would generate a loop around
LOCK CMPXCHG instead of emitting the instructions
LOCK BTS (fetch_or()), LOCK BTR (fetch_and()), LOCK BTC (fetch_xor()).
fil_space_t::clear_closing(): Clear the CLOSING flag.
fil_space_t::set_stopping_check(): Special variant of
fil_space_t::set_stopping() that will return the old value
of the STOPPING flag after atomically setting it.
fil_space_t::clear_stopping(): Use fetch_sub() to toggle
the STOPPING flag. The flag is guaranteed to be set upon
calling this function, hence we will toggle it to clear it.
On IA-32 and AMD64, this will translate into
the 80486 LOCK XADD instruction.
fil_space_t::check_pending_operations(): Replace a Boolean
variable with a goto label, to allow more compact code
generation for fil_space_t::set_stopping_check().
trx_rseg_t: Define private accessors ref_set() and ref_reset()
for setting and clearing the flags.
trx_lock_t::clear_deadlock_victim(), trx_lock_t::set_wsrep_victim():
Accessors for clearing and setting the flags.
Variable wsrep_forced_binlog_format has higher priority than
binlog_format. In situation where STATEMENT is used and DELAYED INSERT
is executing we should fall back to non-delay INSERT.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
Using `innodb_thread_concurrency` will call `wsrep_thd_is_aborting` to
check WSREP thread state. This call should be protected by taking
`LOCK_thd_data` before entering function.
Applier and TOI threads should no be affected with usage of
`innodb_thread_concurrency` variable so returning before any checks.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
If a table has no unique indexes, write set key information will be collected on all columns in the table.
The write set key information has space only for max 3500 bytes for individual column, and if a varchar colummn of such non-primary key table is longer than
this limit, currently a crash follows.
The fix in this commit, is to truncate key values extracted from such long varhar columns to max 3500 bytes.
This may potentially lead to false positive certification failures for transactions, which operate on separate cluster nodes, and update/insert/delete table rows, which differ only in the part of such long columns after 3500 bytes border.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
In commit 45ed9dd957 (MDEV-23855)
when removing fil_system.LRU we failed to rate-limit the output
for reporting violations of innodb_open_files or open_files_limit.
If the server is run with a small limit of open files that is
well below the number of .ibd files that are being accessed by
the workload, and if at the same time innodb_log_file_size is
very small so that log checkpoints will occur frequently,
the process of enforcing the open files limit may be run very often.
fil_space_t::try_to_close(): Display at most one message per call,
and only if at least 5 seconds have elapsed since the last time a
message was output.
fil_node_open_file(): Only output a summary message if
fil_space_t::try_to_close() displayed a message during this run.
(Note: multiple threads may execute fil_node_open_file() on
different files at the same time.)
fil_space_t::get(): Do not dereference a null pointer if n & STOPPING.
This was caught by the test case below.
Unfortunately, it is not possible to create a fully deterministic
test case (expecting exactly 1 message to be emitted). The following with
--innodb-open-files=10 --innodb-log-file-size=4m
would occasionally fail to find the message in the log:
--source include/have_innodb.inc
--source include/have_partition.inc
--source include/have_sequence.inc
call mtr.add_suppression("InnoDB: innodb_open_files=10 is exceeded");
CREATE TABLE t1 (pk INT AUTO_INCREMENT PRIMARY KEY) ENGINE=InnoDB
PARTITION BY key (pk) PARTITIONS 100;
INSERT INTO t1 SELECT * FROM seq_1_to_100;
--disable_query_log
let $n=400;
while ($n)
{
BEGIN; DELETE FROM t1; ROLLBACK;
dec $n;
}
--enable_query_log
let SEARCH_FILE= $MYSQLTEST_VARDIR/log/mysqld.1.err;
let SEARCH_PATTERN= \[Note\] InnoDB: Cannot close file;
-- source include/search_pattern_in_file.inc
DROP TABLE t1;
- Add new Ninja and Clang build jobs. This helps to ensure those
toolchains also work in addition to default CMake/gcc.
- Generate dependencies.dot/png to illustrate the CMake/Make/Ninja
build dependencies. Viewing this image and identifying bottle necks
in parallelism can help make the build run faster.
- Enable CUnit tests now as they are fixed on 10.6 (MDEV-25820).
- Limit parallel builds to 2 CPUs (full parallelism needs MDEV-25968) on
CMake/Make. Now only the Ninja builds run full parallel builds as only
Ninja is smart enough to prevent builds failing on resource
over-consumption.
- Enable Gitlab-CI cache for job 'centos8' for ccache so that it builds
faster. Don't use Gitlab-CI cache for other jobs, as it would too easily
use up all free tier storage on Gitlab.com and force users to get a paid
account just for MariaDB builds.
- On other jobs clean away ccache, as it only had a 5% hit rate on single
builds with no downloaded cache.
- Dump full database contents during the test install so that one can
use diff to compare the database contents at different stages and thus
track/debug potential bugs in mariadb-install-db and mariadb-upgrade
code.
Bugfixes:
- Zero out ccache stats before each run so that 'ccache -s' would actually
show the stats for the latest run.
Normally we disable caching of routines in "SHOW CREATE".
Introduce an exception, if debug_dbug="+d,cache_sp_in_show_create".
lock_sync.test needs a way to populate the cache without side effects,
or else it runs into debug_sync timeouts.
So, this possibility to cache will be remain only for very special tests.
Occasionally, after restart, additional transactions will have been
executed, possibly related to innodb_stats_auto_recalc.
We should only care that the transaction ID sequence does
not go backwards.
Let us mask the actual values of the defragmentation-related fields,
because they may vary. Also, remove the dependency on purge,
and instead delete records by a ROLLBACK of INSERT.
Microsoft does not define inline assembler syntax for AMD64,
but it defines the intrinsic function _interlockedbittestandset()
that we can use.
ssux_lock_impl<bool>::rd_wait(): Remove a call to yield,
because writer.wr_lock() will take care of context switches
between loop iterations.
This addresses suggestions by Vladislav Vaintroub.
Previously, neither our wrapper of Microsoft Windows SRWLOCK
nor the futex-less implementation SUX_LOCK_GENERIC supported spin loops.
This was suggested by Vladislav Vaintroub.
srw_mutex_impl<bool>::wait_and_lock(): In
commit a73eedbf3f we introduced
an std::atomic::fetch_or() in a loop. Alas, on the IA-32 and AMD64,
that was being translated into a loop around LOCK CMPXCHG.
To avoid a nested loop, it is better to explicitly invoke
std::atomic::compare_exchange_weak() in the loop, but only if
the attempt has a chance to succeed (the HOLDER flag is not set).
It is even more efficient to use LOCK BTS, but contemporary compilers
fail to translate std::atomic::fetch_or(x) & x into that when x is
a single-bit constant. On GCC-compatible compilers, we will use
inline assembler to achieve that.
On other ISA than IA-32 and AMD64, we will continue to use
std::atomic::fetch_or().
ssux_lock_impl<spinloop>::rd_wait(): Use rd_lock_try().
A loop around std::atomic::compare_exchange_weak() should be
cheaper than fetch_add(), fetch_sub() and a wakeup system call.
These deficiencies were pointed out and the use of LOCK BTS was
suggested by Thiago Macieira.
The reason for this behavior is that SP get cached, per connection.
The stored_program_cache is size of this cache, which amounts to 256
routines by default. A compiled stored procedure can easily be several
megabytes in size. Thus calling SHOW CREATE PROCEDURE for all stored
procedures, like mysqldump does, can require significant amount of memory.
Fixed by bypassing the cache for "SHOW CREATE". This should normally be
fine also perfomance-wise, as cache is meant to be used for repeated
execution, not repeated SHOW CREATEs.
Added a test to verify that CREATE PROCEDURE + SHOW CREATE PROCEURE do not
cache, i.e amount of allocated memory does not change.
Note, there is a change in existing behavior in an edge case :
If "SHOW CREATE PROCEDURE p1" called from p1, after p1 was altered, now
this will now return altered code. Previour behavior - relied on caching
and would return old code. The previous behavior might was not necessarily
correct.
Use in_sum_func (and so nest_level) only in LEX to which SELECT lex belong to
Reduce usage of current_select (because it does not always point on the correct
SELECT_LEX, for example with prepare.
Change context for all classes inherited from Item_ident (was only for Item_field) in case of pushing down it to HAVING.
Now name resolution context have to have SELECT_LEX reference if the context is present.
Fixed feedback plugin stack usage.
Problem:
========
InnoDB fails to fetch the page0 from dblwr if page0 is
corrupted.In that case, InnoDB defers the tablespace
and doesn't find the INIT_PAGE redo log record for page0
and it leads to failure.
Solution:
=========
InnoDB should recover page0 from dblwr if space_id can
be found for deferred tablespace.
