TOCTOU bug. The path is checked to be valid, symlinks are resolved.
Then the resolved path is opened. Between the check and the open,
there's a window when one can replace some path component with a
symlink, bypassing validity checks.
Fix: after we resolved all symlinks in the path, don't allow open()
to resolve symlinks, there should be none.
Compared to the old MyISAM/Aria code:
* fastpath. Opening of not-symlinked files is just one open(),
no fn_format() and lstat() anymore.
* opening of symlinked tables doesn't do fn_format() and lstat() either.
it also doesn't to realpath() (which was lstat-ing every path
component), instead if opens every path component with O_PATH.
* share->data_file_name stores realpath(path) not readlink(path). So,
SHOW CREATE TABLE needs to do lstat/readlink() now (see ::info()),
and certain error messages (cannot open file "XXX") show the real
file path with all symlinks resolved.
Don't let my_register_filename() fail because strdup() failed. Better to
have NULL for a filename, then to fail the already successful open().
Filenames are only used for error reporting and there was already code
to ignore OOMs (my_fdopen()) and to cope with missing filenames
(my_filename()).
While DEBUG_SYNC doesn't make sense without a valid THD, it might
be put in the code that's invoked both within and outside of a
THD context (e.g. in maria_open(), there is no THD during recovery).
Workaround for join_cache + index on vcols + keyread bug.
Initialize the record to avoid caching garbage in non-read fields.
A proper fix (do not cache non-read fields at all) is done in 10.2
in commits 5d7607f340f..8d99166c697
mysql_prune_stmt_list() was walking the list following
element->next pointers, but inside the loop it was invoking
list_add(element) that modified element->next. So, mysql_prune_stmt_list()
failed to visit and reset all elements, and some of them were left
with pointers to invalid MYSQL.
'Not exists' optimization can be used for nested outer joins
only if IS NULL predicate from the WHERE condition is activated.
So we have to check that all guards that wrap this predicate
are in the 'open' state.
This patch supports usage of 'Not exists' optimization for any
outer join, no matter how it's nested in other outer joins.
This patch is also considered as a proper fix for bugs
#49322/#58490 and LP #817360.
SPORADICALLY ON PB2-5.5 FOR LINUX-VALGRIND
Description: Sporadic failure of variables-bug21503595 test
on pb2-5.5 for linux-valgrind platform.
Fix: This is a issue related to libc and not related to
MySQL code. During dlclose few blocks of memory left
unfreed. This is a known issue in libc and needs to be
suppressed.
Fix: Added a valgrind suppression.
Problem: CREATE TABLE using a fully qualified name with INDEX DIR/DATA DIR
option reports an error when the current database is not SET.
check_access() was incorrectly called with NULL as the database
argument in a situation where the database name was not needed for
the particular privilege being checked. This will cause the current
database to be used, or an error to be reported if there is no current
database.
Fix: Call check_access() with any_db as the database argument in this situation.
(with blocks are still reachable).
There was a similar suppression already, but it had an extra line
comparing to failures which we are getting, so it wasn't applied.
Added another variant of the suppression.
fil_extend_space_to_desired_size(): Use a proper type cast when
computing start_offset for the posix_fallocate() call on 32-bit systems
(where sizeof(ulint) < sizeof(os_offset_t)). This could affect 32-bit
systems when extending files that are at least 4 MiB long.
This bug existed in MariaDB 10.0 before MDEV-11520. In MariaDB 10.1
it had been fixed in MDEV-11556.
a large memory buffer on Windows
fil_extend_space_to_desired_size(), os_file_set_size(): Use calloc()
for memory allocation, and handle failures. Properly check the return
status of posix_fallocate(), and pass the correct arguments to
posix_fallocate().
On Windows, instead of extending the file by at most 1 megabyte at a time,
write a zero-filled page at the end of the file.
According to the Microsoft blog post
https://blogs.msdn.microsoft.com/oldnewthing/20110922-00/?p=9573
this will physically extend the file by writing zero bytes.
(InnoDB never uses DeviceIoControl() to set the file sparse.)
I tested that the file extension works properly with a multi-file
system tablespace, both with --innodb-use-fallocate and
--skip-innodb-use-fallocate (the default):
./mtr \
--mysqld=--innodb-use-fallocate \
--mysqld=--innodb-autoextend-increment=1 \
--mysqld=--innodb-data-file-path='ibdata1:5M;ibdata2:5M:autoextend' \
--parallel=auto --force --retry=0 --suite=innodb &
ls -lsh mysql-test/var/*/mysqld.1/data/ibdata2
(several samples while running the test)
Before the MDEV-11520 fixes, fil_extend_space_to_desired_size()
in MariaDB Server 5.5 incorrectly passed the desired file size as the
third argument to posix_fallocate(), even though the length of the
extension should have been passed. This looks like a regression
that was introduced in the 5.5 version of MDEV-5746.
Remove the unused variable desired_size.
Also, correct the expression for the posix_fallocate() start_offset,
and actually test that it works with a multi-file system tablespace.
Before MDEV-11520, the expression was wrong in both innodb_plugin and
xtradb, in different ways.
The start_offset formula was tested with the following:
./mtr --big-test --mysqld=--innodb-use-fallocate \
--mysqld=--innodb-data-file-path='ibdata1:5M;ibdata2:5M:autoextend' \
--parallel=auto --force --retry=0 --suite=innodb &
ls -lsh mysql-test/var/*/mysqld.1/data/ibdata2
Problem:- When MariaDB is compiled with jemalloc support, And we run mtr valgrind
test, valgrind interferes with libjemalloc and returns false errors.
Solution:- Run valgrind with --soname-synonyms=somalloc=libjemalloc* or
--soname-synonyms=somalloc=NONE depending on whether we are dynamically
linking or statically linking.
Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>
a large memory buffer on Windows
fil_extend_space_to_desired_size(), os_file_set_size(): Use calloc()
for memory allocation, and handle failures. Properly check the return
status of posix_fallocate().
On Windows, instead of extending the file by at most 1 megabyte at a time,
write a zero-filled page at the end of the file.
According to the Microsoft blog post
https://blogs.msdn.microsoft.com/oldnewthing/20110922-00/?p=9573
this will physically extend the file by writing zero bytes.
(InnoDB never uses DeviceIoControl() to set the file sparse.)
For innodb_plugin, port the XtraDB fix for MySQL Bug#56433
(introducing fil_system->file_extend_mutex). The bug was
fixed differently in MySQL 5.6 (and MariaDB Server 10.0).
The function trx_purge_stop() was calling os_event_reset(purge_sys->event)
before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set()
call in srv_purge_coordinator_suspend() is protected by that X-latch.
It would seem a good idea to consistently protect both os_event_set()
and os_event_reset() calls with a common mutex or rw-lock in those
cases where os_event_set() and os_event_reset() are used
like condition variables, tied to changes of shared state.
For each os_event_t, we try to document the mutex or rw-lock that is
being used. For some events, frequent calls to os_event_set() seem to
try to avoid hangs. Some events are never waited for infinitely, only
timed waits, and os_event_set() is used for early termination of these
waits.
os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro
on other systems than Windows. TODO: remove this altogether and disable
innodb_use_native_aio on Windows.
os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.
The failure happens due to a race condition between processing
a row event (INSERT) and an automatically generated event
DROP TEMPORARY TABLE. Even though DROP has a higher GTID, it can
become visible in @@gtid_slave_pos before the row event with
a lower GTID has been committed. Since the test makes the slave
to synchronize with the master using GTID, the waiting stops
as soon as GTID of the DROP TEMPORARY TABLE becomes visible,
and if changes from the previous event haven't been applied yet,
the error occurs.
According to Kristian (see the comment to MDEV-10631), the real
problem is that DROP TEMPORARY TABLE is logged in the row mode
at all. For this particular test, since DROP does not do anything,
nothing prevents it from competing with the prior transaction.
The workaround for the test is to add a meaningful event
after DROP TEMPORARY TABLE, so that the slave would wait on its
GTID instead of the one from DROP.
Additionally (unrelated to this problem) removed FLUSH TABLES,
which, as the comment stated, should have been removed after
MDEV-6403 was fixed.
The standalone warning is not a sign of a problem, just of slowness,
so it should be added to global suppressions. If a real problem
happens, there will be other errors
* Revert "Make --mem a pure flag. If there is need to specifically set the location"
This reverts commit 716621db3f.
* MDEV-11619: mtr: when --mem is pure flag, conflicts with $MTR_MEM
Conflicts occurs when MTR_MEM=/xxx/yy ./mtr --mem is invoked. Here
the --mem option overrides opt_mem leaving the default path to be chosen.
This change makes when MTR_MEM set, opt_mem, the flag, is also
set. Both the environment and flag can no be set without conflicting.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
* MDEV-11619: if opt_mem is a path include it first
* MDEV-11619: MTR_MEM locations - don't follow symlinks
From Bjorn Munch it seems symlinks can confuse some
tests. Lets just avoid those.
(ref: https://github.com/mysql/mysql-server/pull/116#issuecomment-268479774)
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
parts.partition_float_myisam, parts.partition_int_myisam,
parts.partition_float_innodb are all known to fail with timeouts
on slow builders. The tests are composed of several independent parts
for corresponding subtypes (float == float + double,
int == tinyint + smallint + mediumint + int + bigint). The solution
is to split them into separate tests. No test logic has been changed.
This patch is actually a complement for the fix of bug mdev-6892.
The procedure create_tmp_table() now must take into account
Item_direct_refs that wrap up constant fields of derived tables/views
that are used as inner tables in outer join operations.
On a slow builder, a delay between binlog events on master could
occur, which would cause a heartbeat which is not expected by the
test. The solution is to monitor the timing of binlog events
on the master and only perform the heartbeat check if no critical
delays have happened.
Additionally, an unused variable was removed (this change is
unrelated to the bugfix).
Fix of Bug#25088048 caused paths to be relative, not absolute, this
proved to be problematic.
Fix is to still ignore current working directory, however switch to
using full path of basedir, which is set to parent directory of bin/
directory where mysqld_safe is located.
References to legacy tool mysql_print_defaults are removed, only
my_print_defaults is used these days.
This will also fix:
Bug#11745176 (11192) MYSQLD_SAFE ONLY EVALUATES --DEFAULTS-FILE OPTION WHEN IT IS THE FIRST OP
Bug#23013510 (80866) MYSQLD_SAFE SHOULD NOT SEARCH $MY_BASEDIR_VERSION/VAR AS DATADIR
Bug#25244898 (84173) MYSQLD_SAFE --NO-DEFAULTS & SILENTLY DOES NOT WORK ANY MORE
Bug#25261472 (84219) INITSCRIPT ERRORS WHEN LAUCHING MYSQLD_SAFE IN NON DEFAULT BASEDIR
Bug#25319392 (84263) MYSQL.SERVER (MYSQL SERVER STARTUP SCRIPT) CAN NOT WORK,AND EXPORT SOME ERROR.
Bug#25319457 MYSQLD_SAFE MIGHT FAIL IF $DATADIR HAS TRAILING /
Bug#25341981 MYSQLD_SAFE ASSUMES INCORRECT BASEDIR WHEN EXECUTED WITH ABSOLUTE PATH
Bug#25356221 (84427) MYSQLD_SAFE FAILS TO START WHEN USING A FIFO FOR LOG-ERROR (REGRESSION)
Bug#25365194 (84447) MYSQLD_SAFE DOESN'T CHECK EXISTENCE OF GIVEN BASEDIR PARAMETER
Bug#25377815 ERRORS WHILE STARTING MYSQLD_SAFE WITH SYM LINK ENABLED
