Commit graph

175028 commits

Author SHA1 Message Date
Marko Mäkelä
adc91387e3 Merge 10.0 into 10.1 2017-03-03 13:27:12 +02:00
Marko Mäkelä
29c776cfd1 MDEV-11520: Retry posix_fallocate() after EINTR.
The function posix_fallocate() as well as the Linux system call
fallocate() can return EINTR when the operation was interrupted
by a signal. In that case, keep retrying the operation, except
if InnoDB shutdown has been initiated.
2017-03-03 12:03:33 +02:00
Daniel Black
bc28b305e5 Remove warning: unused variable 'volatile_var' [-Wunused-variable]
This occured in gcc-6.2.1.

The variable wasn't used so was no need to be volatile either.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-03-03 13:00:41 +04:00
Daniel Black
d04d835f64 MDEV-11610: support-files/mysql-log-rotate.sh not binlog either
For the same reason as removing binlog rotation on Debian.
2017-03-02 15:16:21 +04:00
Daniel Black
156cf86def MDEV-11610: Alter Debian log rotate to not rotate binary/relay logs
Rotating binary/relay logs can cause interuption to the processing
on the server. Binary and relay logs have their own mechanism already
for not getting out of control (expire_logs_days).

By no longer rotating binary and relay logs log rotation is limited to
the following logs:
* error log
* general log
* slow query log

Writing these to the binary log would cause any logrotation on the
slave to occur twice, once due to this and another due to the log-
rotate script on the slave. Now --local is passed to mysqladmin to
prevent this duplication.
2017-03-02 15:16:06 +04:00
Daniel Black
0af8b565f2 MDEV-11610: mysqladmin flush-X-log options
Add the following options to mysqladmin

+  flush-binary-log        Flush binary log
+  flush-engine-log        Flush engine log(s)
+  flush-error-log         Flush error log
+  flush-general-log       Flush general log
+  flush-relay-log         Flush relay log
+  flush-user-resources    Flush user resources
2017-03-02 15:16:00 +04:00
Daniel Black
33c1f20d8e MDEV-11610: Add --local to mysqladmin 2017-03-02 15:09:24 +04:00
Daniel Black
659047b820 MDEV-11386: Advance Toochain library cache workaround (temporary)
Due to the way Advance Toolchain before 10.0-3 and 8.0-8 is
packaged, shared libraries installed after the library cache
advance-toolchain-atX.Y-runtime package aren't updated in
/opt/atX.Y/etc/ld.so.cache. This results in mysqld,
configured with RUNPATH set to /opt/atX.Y/lib64, resulting
in the Advance Toolchain loader being used and if libraries
such as jemalloc, libssl or any other library that mysqld uses
is installed after Advance Toolchain, these libraries aren't in
the cache and therefore won't be found in the RPM postinstall
when mysqld is executed by mysql_install_db.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-03-02 14:04:08 +04:00
Sergey Vojtovich
71f53bf72d MDEV-11221 - main.events_restart failed in bb
This is an addition to original fix. Buildbot revealed another sporadic failure
in perfschema.threads_mysql test. Tests relies on data stored in
performance_schema.threads, while performing waits on
information_schema.processlist. These tables are not updated synchronously.

Fixed by performing waits on performance_schema.threads instead.
2017-03-02 12:35:31 +04:00
Jan Lindström
88b5eedef2 Merge pull request from grooverdan/10.0-MDEV-10515-stat_tables_par-test-fix
MDEV-10515: Correct stat_tables_par test results
2017-03-02 08:29:52 +02:00
Vladislav Vaintroub
ad2e38153c AWS key management plugin: Fix search for system installed AWS SDK libs 2017-03-01 10:07:54 +00:00
Sergey Vojtovich
c1c5b7a8d2 Fixed missing DBUG_RETURN 2017-03-01 11:41:48 +04:00
Sergey Vojtovich
e9ad4bdb42 MDEV-11221 - main.events_restart failed in bb
Applied lost in a merge revision 7f38a07:

MDEV-10043 - main.events_restart fails sporadically in buildbot (crashes upon
             shutdown)

There was race condition between shutdown thread and event worker threads.

Shutdown thread waits for thread_count to become 0 in close_connections(). It
may happen so that event worker thread was started but didn't increment
thread_count by this time. In this case shutdown thread may miss wait for this
working thread and continue deinitialization. Worker thread in turn may continue
execution and crash on deinitialized data.

Fixed by incrementing thread_count before thread is actually created like it is
done for connection threads.

Also let event scheduler not to inc/dec running threads counter for symmetry
with other "service" threads.
2017-03-01 11:35:24 +04:00
Sergei Golubchik
cc413ce9a3 MDEV-11753 Link failure on missing -L${LIBLZ4_LIBRARY_DIR}
On FreeBSD liblz4 is installed in /usr/local/lib.
Groonga uses pkg_check_modules to check for liblz4 (that is, pkg-config),
and then it used to set for libgroonga.a

  link_directories({$LIBLZ4_LIBRARY_DIRS})
  target_link_libraries(... ${LIBLZ4_LIBRARIES})

Now groonga is a static library, linked into ha_mroonga.so. CMake won't
link dynamic liblz4.so into libgroonga.a, instead it'll pass it as a
dependency and will link it into ha_mroonga.so. Fine so far. But it will
not pass link_directories from the static library as a dependency,
so ha_mroonga.so won't find liblz4.so

As suggested on cmake mailing list (e.g.
here: http://public.kitware.com/pipermail/cmake/2011-November/047468.html)
we switch to use the full path to liblz4.so, instead of the -l/-L pair.
2017-02-28 16:19:44 +01:00
Sergei Golubchik
370cf70136 MDEV-11757 KEY_BLOCK_SIZE strangeness when UNCOMPRESSing COMPRESSed InnoDB tables
in ALTER TABLE ... DROP KEY, ADD KEY, don't forget to compare old
and new keys' block sizes. If they differ - the key definition has changed.
2017-02-28 16:19:44 +01:00
Monty
6a12c05347 Fixed wrong arguments to sql_print_error() 2017-02-28 16:10:47 +01:00
Monty
84ed5e1d5f Fixed hang doing FLUSH TABLES WITH READ LOCK and parallel replication
The problem was that waiting for pause_for_ftwrl was done before
event_group was completed. This caused rpl_pause_for_ftwrl() to wait
forever during FLUSH TABLES WITH READ LOCK.
Now we only wait for FLUSH TABLES WITH READ LOCK when we are changing
to a new event group.
2017-02-28 16:10:47 +01:00
Monty
f3c65ce951 Add protection to not access is_open() without LOCK_log mutex
Protection added to reopen_file() and new_file_impl().

Without this we could get an assert in fn_format() as name == 0,
because the file was closed and name reset, atthe same time
new_file_impl() was called.
2017-02-28 16:10:47 +01:00
Monty
b624b41abb Don't allow one to kill START SLAVE while the slaves IO_THREAD or SQL_THREAD
is starting.

This is needed as if we kill the START SLAVE thread too early during
shutdown then the IO_THREAD or SQL_THREAD will not have time to properly
initlize it's replication or THD structures and clean_up() will try to
delete master_info structures that are still in use.
2017-02-28 16:10:47 +01:00
Michael Widenius
d7a9aed43f Fixed test failing as myisam table was deleted before oqgraph table. 2017-02-28 16:10:47 +01:00
Monty
4bad74e139 Added error checking for all calls to flush_relay_log_info() and stmt_done() 2017-02-28 16:10:47 +01:00
Monty
a2de378c00 Add protection for reinitialization of mutex in parallel replaction
Added mutex_lock/mutex_unlock of mutex that is to be destroyed in
wait_for_commit::reinit() in a similar fashion that we do in
~wait_for_commit
2017-02-28 16:10:47 +01:00
Monty
c5e25c8b40 Added a separate lock for start/stop/reset slave.
This solves some possible dead locks when one calls stop slave while slave
is starting.
2017-02-28 16:10:46 +01:00
Monty
e65f667bb6 MDEV-9573 'Stop slave' hangs on replication slave
The reason for this is that stop slave takes LOCK_active_mi over the
whole operation while some slave operations will also need LOCK_active_mi
which causes deadlocks.

Fixed by introducing object counting for Master_info and not taking
LOCK_active_mi over stop slave or even stop_all_slaves()

Another benefit of this approach is that it allows:
- Multiple threads can run SHOW SLAVE STATUS at the same time
- START/STOP/RESET/SLAVE STATUS on a slave will not block other slaves
- Simpler interface for handling get_master_info()
- Added some missing unlock of 'log_lock' in error condtions
- Moved rpl_parallel_inactivate_pool(&global_rpl_thread_pool) to end
  of stop_slave() to not have to use LOCK_active_mi inside
  terminate_slave_threads()
- Changed argument for remove_master_info() to Master_info, as we always
  have this available
- Fixed core dump when doing FLUSH TABLES WITH READ LOCK and parallel
  replication. Problem was that waiting for pause_for_ftwrl was not done
  when deleting rpt->current_owner after a force_abort.
2017-02-28 16:10:46 +01:00
Monty
d5c54f3990 Fixed compiler warnings
- Removed not used variables
- Added __attribute__()
- Added static to some local functions
  (gcc 5.4 gives a warning for external functions without an external definition)
2017-02-28 16:10:46 +01:00
iangilfillan
ce903428a8 Update MariaDB Foundation sponsors 2017-02-28 17:15:36 +04:00
Vladislav Vaintroub
d4baeca441 Windows : Fix server compile errors when compile with /Zc:strictStrings option 2017-02-28 12:57:33 +00:00
Marko Mäkelä
fc673a2c12 MDEV-12127 InnoDB: Assertion failure loop_count < 5 in file log0log.cc
As suggested in MySQL Bug#58536, increase the limit in this
debug assertion in order to avoid false positives on heavily
loaded systems.
2017-02-28 09:54:12 +02:00
Daniel Black
b54566d73b MDEV-11619: mtr --mem {no argument of a directory} ()
This removes functionality of where ./mtr --mem /tmp/dir could be a directory.

Now MTR_MEM=/tmp/dir ./mtr is needed.

The case where MTR_MEM=/tmp/dir ./mtr --mem has the equivalent effect.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

--mem works better as a pure flag, because it can be followed by command-line arguments (test names). If the option is allowed to have a value, the test name which directly follows it will be treated as the option value instead. It is possible to implement workarounds to avoid this, but they would not be completely reliable, and there is no practical purpose of such extension of functionality to justify them.
2017-02-28 01:08:12 +02:00
Oleksandr Byelkin
e5b877ce27 MDEV-11935: Queries in stored procedures with and EXISTS(SELECT * FROM VIEW) crashes and closes hte conneciton.
Use correct start point even for taken out from subselect items in process of exists2in conversion.
2017-02-27 10:12:17 +01:00
Alexey Botchkov
fdeeab01c0 MDEV-6390 CONVERT TO CHARACTER SET utf8 doesn't change DEFAULT CHARSET.
ALTER_CONVERT flag removed and replaced for ALTER_OPTIONS
        for the CONVERT TO CHARACTER SET command.
2017-02-26 23:01:23 +04:00
Elena Stepanova
ae142c21a5 MDEV-12106 Valgrind tests fail all over in buildbot on 10.0
(with blocks are still reachable).
There was a similar suppression already, but it had an extra line
comparing to failures which we are getting, so it wasn't applied.
Added another variant of the suppression.
2017-02-23 14:32:43 +02:00
Marko Mäkelä
7f7673033e Remove an unused variable. 2017-02-22 22:32:29 +02:00
Marko Mäkelä
ec4cf111c0 MDEV-11520 after-merge fix for 10.1: Use sparse files.
If page_compression (introduced in MariaDB Server 10.1) is enabled,
the logical action is to not preallocate space to the data files,
but to only logically extend the files with zeroes.

fil_create_new_single_table_tablespace(): Create smaller files for
ROW_FORMAT=COMPRESSED tables, but adhere to the minimum file size of
4*innodb_page_size.

fil_space_extend_must_retry(), os_file_set_size(): On Windows,
use SetFileInformationByHandle() and FILE_END_OF_FILE_INFO,
which depends on bumping _WIN32_WINNT to 0x0600.
FIXME: The files are not yet set up as sparse, so
this will currently end up physically extending (preallocating)
the files, wasting storage for unused pages.

os_file_set_size(): Add the parameter "bool sparse=false" to declare
that the file is to be extended logically, instead of being preallocated.
The only caller with sparse=true is
fil_create_new_single_table_tablespace().
(The system tablespace cannot be created with page_compression.)

fil_space_extend_must_retry(), os_file_set_size(): Outside Windows,
use ftruncate() to extend files that are supposed to be sparse.
On systems where ftruncate() is limited to files less than 4GiB
(if there are any), fil_space_extend_must_retry() retains the
old logic of physically extending the file.
2017-02-22 22:29:56 +02:00
Marko Mäkelä
e1e920bf63 Merge 10.0 into 10.1 2017-02-22 15:53:05 +02:00
Marko Mäkelä
a0ce92ddc7 MDEV-11520 post-fix
fil_extend_space_to_desired_size(): Use a proper type cast when
computing start_offset for the posix_fallocate() call on 32-bit systems
(where sizeof(ulint) < sizeof(os_offset_t)). This could affect 32-bit
systems when extending files that are at least 4 MiB long.

This bug existed in MariaDB 10.0 before MDEV-11520. In MariaDB 10.1
it had been fixed in MDEV-11556.
2017-02-22 12:32:17 +02:00
Marko Mäkelä
81695ab8b5 MDEV-11520 Extending an InnoDB data file unnecessarily allocates
a large memory buffer on Windows

fil_extend_space_to_desired_size(), os_file_set_size(): Use calloc()
for memory allocation, and handle failures. Properly check the return
status of posix_fallocate(), and pass the correct arguments to
posix_fallocate().

On Windows, instead of extending the file by at most 1 megabyte at a time,
write a zero-filled page at the end of the file.
According to the Microsoft blog post
https://blogs.msdn.microsoft.com/oldnewthing/20110922-00/?p=9573
this will physically extend the file by writing zero bytes.
(InnoDB never uses DeviceIoControl() to set the file sparse.)

I tested that the file extension works properly with a multi-file
system tablespace, both with --innodb-use-fallocate and
--skip-innodb-use-fallocate (the default):

./mtr \
--mysqld=--innodb-use-fallocate \
--mysqld=--innodb-autoextend-increment=1 \
--mysqld=--innodb-data-file-path='ibdata1:5M;ibdata2:5M:autoextend' \
--parallel=auto --force --retry=0 --suite=innodb &

ls -lsh mysql-test/var/*/mysqld.1/data/ibdata2
(several samples while running the test)
2017-02-22 12:21:44 +02:00
Elena Stepanova
6dc00f97b7 MDEV-11774 tokudb.locks-select-update-3 failed in buildbot with lock wait timeout
Increase tokudb_lock_timeout for the test
2017-02-21 15:03:34 +02:00
Marko Mäkelä
3c47ed4849 Merge 10.0 into 10.1 2017-02-20 14:02:40 +02:00
Marko Mäkelä
13493078e9 MDEV-11802 innodb.innodb_bug14676111 fails
The function trx_purge_stop() was calling os_event_reset(purge_sys->event)
before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set()
call in srv_purge_coordinator_suspend() is protected by that X-latch.

It would seem a good idea to consistently protect both os_event_set()
and os_event_reset() calls with a common mutex or rw-lock in those
cases where os_event_set() and os_event_reset() are used
like condition variables, tied to changes of shared state.

For each os_event_t, we try to document the mutex or rw-lock that is
being used. For some events, frequent calls to os_event_set() seem to
try to avoid hangs. Some events are never waited for infinitely, only
timed waits, and os_event_set() is used for early termination of these
waits.

os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro
on other systems than Windows. TODO: remove this altogether and disable
innodb_use_native_aio on Windows.

os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.
2017-02-20 12:20:52 +02:00
Marko Mäkelä
72994d6442 Revert the MDEV-4396 tweak to innodb.innodb_bug14676111.
MDEV-11802 aims to fix the root cause instead.
2017-02-20 12:20:52 +02:00
Elena Stepanova
343ba58562 MDEV-10631 rpl.rpl_mdev6386 failed in buildbot
The failure happens due to a race condition between processing
a row event (INSERT) and an automatically generated event
DROP TEMPORARY TABLE. Even though DROP has a higher GTID, it can
become visible in @@gtid_slave_pos before the row event with
a lower GTID has been committed. Since the test makes the slave
to synchronize with the master using GTID, the waiting stops
as soon as GTID of the DROP TEMPORARY TABLE becomes visible,
and if changes from the previous event haven't been applied yet,
the error occurs.

According to Kristian (see the comment to MDEV-10631), the real
problem is that DROP TEMPORARY TABLE is logged in the row mode
at all. For this particular test, since DROP does not do anything,
nothing prevents it from competing with the prior transaction.

The workaround for the test is to add a meaningful event
after DROP TEMPORARY TABLE, so that the slave would wait on its
GTID instead of the one from DROP.

Additionally (unrelated to this problem) removed FLUSH TABLES,
which, as the comment stated, should have been removed after
MDEV-6403 was fixed.
2017-02-20 01:48:26 +02:00
Elena Stepanova
72a822f2ec MDEV-11766 Tests failed in buildbot with semaphore waiting warnings
The standalone warning is not a sign of a problem, just of slowness,
so it should be added to global suppressions. If a real problem
happens, there will be other errors
2017-02-20 01:48:11 +02:00
Daniel Black
5e42c958a5 MDEV-11619: mtr --mem and $MTR_MEM support in sane and consistent manner (10.0) ()
* Revert "Make --mem a pure flag. If there is need to specifically set the location"

This reverts commit 716621db3f.

* MDEV-11619: mtr: when --mem is pure flag, conflicts with $MTR_MEM

Conflicts occurs when MTR_MEM=/xxx/yy ./mtr --mem is invoked. Here
the --mem option overrides opt_mem leaving the default path to be chosen.

This change makes when MTR_MEM set, opt_mem, the flag, is also
set. Both the environment and flag can no be set without conflicting.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

* MDEV-11619: if opt_mem is a path include it first

* MDEV-11619: MTR_MEM locations - don't follow symlinks

From Bjorn Munch it seems symlinks can confuse some
tests. Lets just avoid those.

(ref: https://github.com/mysql/mysql-server/pull/116#issuecomment-268479774)

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-02-20 01:43:42 +02:00
Sergey Vojtovich
ba4d0a1b35 There's no systemd socket activation support yet 2017-02-17 12:47:09 +04:00
Marko Mäkelä
01d5d6db4c Fix GCC 6.3.0 warnings. 2017-02-16 11:16:27 +02:00
Marko Mäkelä
32170cafad MDEV-12075 innodb_use_fallocate does not work in MariaDB Server 10.1.21
fil_space_extend_must_retry(): When innodb_use_fallocate=ON,
initialize pages_added = size - space->size so that posix_fallocate()
will actually attempt to extend the file, instead of keeping the same size.

This is a regression from MDEV-11556 which refactored
the InnoDB data file extension.
2017-02-16 11:12:24 +02:00
Daniel Black
e2d6760d8a MDEV-10515: Correct stat_tables_par test results
Test results are distorted by a small rounding error
during an intermediate stage of calculating the result.
By using the SQL ROUND function we stablise tests.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-02-16 16:42:25 +11:00
Nirbhay Choubey
74a5638a1d MDEV-11530: wsrep_info.plugin fails sporadically in buildbot
Add the warning to the global_suppressions list.
2017-02-13 18:40:24 -05:00
Elena Stepanova
66822f164f Follow-up to MDEV-10731 - fix the broken test 2017-02-11 01:14:06 +02:00