TOCTOU bug. The path is checked to be valid, symlinks are resolved.
Then the resolved path is opened. Between the check and the open,
there's a window when one can replace some path component with a
symlink, bypassing validity checks.
Fix: after we resolved all symlinks in the path, don't allow open()
to resolve symlinks, there should be none.
Compared to the old MyISAM/Aria code:
* fastpath. Opening of not-symlinked files is just one open(),
no fn_format() and lstat() anymore.
* opening of symlinked tables doesn't do fn_format() and lstat() either.
it also doesn't to realpath() (which was lstat-ing every path
component), instead if opens every path component with O_PATH.
* share->data_file_name stores realpath(path) not readlink(path). So,
SHOW CREATE TABLE needs to do lstat/readlink() now (see ::info()),
and certain error messages (cannot open file "XXX") show the real
file path with all symlinks resolved.
Don't let my_register_filename() fail because strdup() failed. Better to
have NULL for a filename, then to fail the already successful open().
Filenames are only used for error reporting and there was already code
to ignore OOMs (my_fdopen()) and to cope with missing filenames
(my_filename()).
While DEBUG_SYNC doesn't make sense without a valid THD, it might
be put in the code that's invoked both within and outside of a
THD context (e.g. in maria_open(), there is no THD during recovery).
Workaround for join_cache + index on vcols + keyread bug.
Initialize the record to avoid caching garbage in non-read fields.
A proper fix (do not cache non-read fields at all) is done in 10.2
in commits 5d7607f340f..8d99166c697
mysql_prune_stmt_list() was walking the list following
element->next pointers, but inside the loop it was invoking
list_add(element) that modified element->next. So, mysql_prune_stmt_list()
failed to visit and reset all elements, and some of them were left
with pointers to invalid MYSQL.
'Not exists' optimization can be used for nested outer joins
only if IS NULL predicate from the WHERE condition is activated.
So we have to check that all guards that wrap this predicate
are in the 'open' state.
This patch supports usage of 'Not exists' optimization for any
outer join, no matter how it's nested in other outer joins.
This patch is also considered as a proper fix for bugs
#49322/#58490 and LP #817360.
Before the MDEV-11520 fixes, fil_extend_space_to_desired_size()
in MariaDB Server 5.5 incorrectly passed the desired file size as the
third argument to posix_fallocate(), even though the length of the
extension should have been passed. This looks like a regression
that was introduced in the 5.5 version of MDEV-5746.
Remove the unused variable desired_size.
Also, correct the expression for the posix_fallocate() start_offset,
and actually test that it works with a multi-file system tablespace.
Before MDEV-11520, the expression was wrong in both innodb_plugin and
xtradb, in different ways.
The start_offset formula was tested with the following:
./mtr --big-test --mysqld=--innodb-use-fallocate \
--mysqld=--innodb-data-file-path='ibdata1:5M;ibdata2:5M:autoextend' \
--parallel=auto --force --retry=0 --suite=innodb &
ls -lsh mysql-test/var/*/mysqld.1/data/ibdata2
Problem:- When MariaDB is compiled with jemalloc support, And we run mtr valgrind
test, valgrind interferes with libjemalloc and returns false errors.
Solution:- Run valgrind with --soname-synonyms=somalloc=libjemalloc* or
--soname-synonyms=somalloc=NONE depending on whether we are dynamically
linking or statically linking.
Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
a large memory buffer on Windows
fil_extend_space_to_desired_size(), os_file_set_size(): Use calloc()
for memory allocation, and handle failures. Properly check the return
status of posix_fallocate().
On Windows, instead of extending the file by at most 1 megabyte at a time,
write a zero-filled page at the end of the file.
According to the Microsoft blog post
https://blogs.msdn.microsoft.com/oldnewthing/20110922-00/?p=9573
this will physically extend the file by writing zero bytes.
(InnoDB never uses DeviceIoControl() to set the file sparse.)
For innodb_plugin, port the XtraDB fix for MySQL Bug#56433
(introducing fil_system->file_extend_mutex). The bug was
fixed differently in MySQL 5.6 (and MariaDB Server 10.0).
parts.partition_float_myisam, parts.partition_int_myisam,
parts.partition_float_innodb are all known to fail with timeouts
on slow builders. The tests are composed of several independent parts
for corresponding subtypes (float == float + double,
int == tinyint + smallint + mediumint + int + bigint). The solution
is to split them into separate tests. No test logic has been changed.
This patch is actually a complement for the fix of bug mdev-6892.
The procedure create_tmp_table() now must take into account
Item_direct_refs that wrap up constant fields of derived tables/views
that are used as inner tables in outer join operations.
On a slow builder, a delay between binlog events on master could
occur, which would cause a heartbeat which is not expected by the
test. The solution is to monitor the timing of binlog events
on the master and only perform the heartbeat check if no critical
delays have happened.
Additionally, an unused variable was removed (this change is
unrelated to the bugfix).
my_thread_global_init() + my_thrad_global_end() repeatadily.
This caused THR_KEY_mysys to be allocated multiple times.
Deletion of THR_KEY_mysys was originally in my_thread_global_end() but was
moved to my_end() as DBUG uses THR_KEY_mysys and DBUG is released after
my_thread_global_end() is called.
Releasing DBUG before my_thread_global_end() and move THR_KEY_mysys back
into my_thread_global_end() could be a solution, but as safe_mutex and other
things called by my_thread_global_end is using DBUG it may not be completely
safe.
To solve this, I used the simple solution to add a marker that THR_KEY_mysys
is created and not re-create it in my_thread_global_init if it already
exists.
As the function Item_subselect::fix_fields does it the function
Item_subselect::update_used_tables must ignore UNCACHEABLE_EXPLAIN
when deciding whether the subquery item should be considered as a
constant item.
When building different range and index-merge trees the range optimizer
could build an index-merge tree with an index scan containing less ranges
then needed. This index-merge could be chosen as the best. Following this
index-merge the executioner missed some rows in the result set.
The invalid index scan was built due to an inconsistency in the code
back-ported from mysql into 5.3 that fixed mysql bug #11765831:
the code added to key_or() could change shared keys of the second
ored tree. Partially the problem was fixed in the patch for mariadb
bug #823301, but it turned out that only partially.
It was used for get_datetime_value() and for thd->is_error().
But in fact, get_datetime_value() never used thd argument, because the
cache ptr argument was NULL. And thd->is_error() check was not needed
at that place at all.
it used current_thd->alloc() and allocated on the thd's execution arena,
not on table->expr_arena.
Remove THD::arena_for_cached_items that is temporarily set in
update_virtual_fields(), and replaces THD arena in get_datetime_value().
Instead set THD arena to table->expr_arena for the whole duration
of update_virtual_fields()
Item_func_le included Arg_comparator. Arg_comparator remembered
the current_thd during fix_fields and used that value during
execution to allocate Item_cache in get_datetime_value().
But for vcols fix_fields and val_int can happen in different threads.
Same bug for Item_func_in using in_datetime or cmp_item_datetime,
both also remembered current_thd at fix_fields() to use it later
for get_datetime_value().
As a fix, these objects no longer remember the current_thd,
and get_datetime_value() uses current_thd at run time. This
should not increase the number of current_thd calls much, as
Item_cache is created only once anyway.
c3cf7f47f0 reverted the patch
for BUG#24487120. After merging the reverting patch from MySQL
to MariaDB the problems described in MDEV-11079 and MDEV-11631 disappeared.
Adding test cases only.
The fix for bug mdev-5104 did not take into account that
for any call of setup_order the size of ref_array must
be big enough. This patch fixes this problem.
When the test is run as a part of the suite with valgrind,
only allow it to be executed if --big-test is set.
If the test is run by specifying its name explicitly, it
will still be executed, even with valgrind without big-test,
MTR has special logic for that
The guilty part of the test checks for performance degradation on
a query with numerous joins on an empty table. The test expects
the query to take less than 1 second, and fails if it is not so
(which can happen on very slow builders).
The solution is to add more JOINs to the query. On a fixed server,
it should not have any noticeable impact on the query execution,
while on the unfixed version the query would take several times
longer (e.g. 6.5 sec vs 1.5 sec). Thus, we can increase the margin
for the error, and make the test fail when the query takes longer
than 5 seconds.
Backport the fix to 5.5, because it fails there too
The patch fixes two test failures:
- on slow builders, sometimes a connection attempt which should
fail due to the exceeded number of thread_pool_max_threads
actually succeeds;
- on even slow builders, MTR sometimes cannot establish the
initial connection, and check-testcase fails prior to the
test start
The problem with check-testcase was caused by connect-timeout=2
which was set for all clients in the test config file. On slow
builders it might be not enough.
There is no way to override it for the pre-test check, so it needed
to be substantially increased or removed.
The other problem was caused by a race condition between sleeps
that the test performs in existing connections and the connect
timeout for the connection attempt which was expected to fail.
If sleeps finished before the connect-timeout was exceeded, it
would allow the connection to succeed.
To solve each problem without making the other one worse,
connect-timeout should be configured dynamically during the test.
Due to the nature of the test (all connections must be busy
at the moment when we need to change the timeout, and cannot execute
SET GLOBAL ...), it needs to be done independently from the server.
The solution:
- recognize 'connect_timeout' as a connection option in mysqltest's
"connect" command;
- remove connect-timeout from the test configuration file;
- use the new connect_timeout option for those connections which
are expected to fail;
- re-arrange the test flow to allow running a huge SLEEP
without affecting the test execution time (because it would be
interrupted after the main test flow is finished).
The test is still subject to false negatives, e.g. if the connection
fails due to timeout rather than due to the exceeded number of
allowed threads, or if the connection on extra port succeeds due
to a race condition and not because the special logic for the extra
port. But those false negatives have always been possible there
on slow builders, they should not be critical because faster builders
should catch such failures if they appear.
Conflicts:
client/mysqltest.cc
mysql-test/r/pool_of_threads.result
mysql-test/t/pool_of_threads.test
