Commit graph

178557 commits

Author SHA1 Message Date
Daniel Black
866e5c250e MDEV-15503: mtr fix --strace
$glob_mysql_test_dir was the wrong directory for strace output as
it was for in-tree builds only so failed for:
* out of tree builds
* --parallel; and
* --mem

strace output wasn't saved.

strace-option never replaced existing arguments (so ammended
documentation).

strace-client didn't accept an argument as described.

Replaced specification of client with this with 'stracer' to be
consistent with --debugger option.

For consistency with debugger options, --client-strace was added to
execute the strace on the mysqltest.

Example: Running one test

$ ./mtr --strace --client-strace funcs_1.is_table_constraints
Logging: ./mtr  --strace --client-strace funcs_1.is_table_constraints
vardir: /home/anel/mariadb/5.5/mysql-test/var
Checking leftover processes...
Removing old var directory...
 - WARNING: Using the 'mysql-test/var' symlink
Creating var directory '/home/anel/mariadb/5.5/mysql-test/var'...
Checking supported features...
MariaDB Version 5.5.67-MariaDB-debug
Installing system database...
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...

==============================================================================

TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
funcs_1.is_table_constraints             [ pass ]   1270
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 1.270 of 3 seconds executing testcases

Completed: All 1 tests were successful

$ find -L . -name \*strace -ls
      653     56 -rw-r--r--   1 anel     anel        57147 Nov 29 15:08 ./var/log/mysqltest.strace
      646   1768 -rw-r--r--   1 anel     anel      1809855 Nov 29 15:08 ./var/log/mysqld.1.strace

Example: Running test in parallel

$ mysql-test/mtr --strace --client-strace  --mem --parallel=3  main.select
Logging: /home/dan/software_projects/mariadb-server/mysql-test/mysql-test-run.pl  --strace --client-strace --mem --parallel=3 main.select
vardir: /home/dan/software_projects/build-mariadb-10.3/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/dan/software_projects/build-mariadb-10.3/mysql-test/var'...
 - symlinking 'var' to '/dev/shm/var_auto_0v2E'
Checking supported features...
MariaDB Version 5.5.67-MariaDB
 - SSL connections supported
Collecting tests...
Installing system database...

==============================================================================

TEST                                  WORKER RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
worker[3]  - 'localhost:16040' was not free
worker[2] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
worker[3] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
main.select                              w1 [ pass ]   7310
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 7.310 of 11 seconds executing testcases

Completed: All 1 tests were successful.

$ find mysql-test/var/ -name \*strace -ls
  5213766   1212 -rw-r--r--   1  dan      dan       1237817 May 20 16:47 mysql-test/var/1/log/mysqltest.strace
  5214733  13016 -rw-r--r--   1  dan      dan      13328335 May 20 16:47 mysql-test/var/1/log/mysqld.1.strace

$ mysql-test/mtr --strace --client-strace --strace-option='-e' --strace-option='trace=openat'  --mem --parallel=3  main.select
...
$ find mysql-test/var/ -name \*strace -ls
  5220790      8 -rw-r--r--   1  dan      dan          6291 May 20 17:02 mysql-test/var/3/log/mysqltest.strace
  5224140    308 -rw-r--r--   1  dan      dan        314356 May 20 17:02 mysql-test/var/3/log/mysqld.1.strace
$ more mysql-test/var/3/mysqltest.strace
1692  openat(AT_FDCWD, "/home/dan/software_projects/mariadb-server/libmysql/.libs/tls/x86_64/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) =
-1 ENOENT (No such file or directory)
1692  openat(AT_FDCWD, "/home/dan/software_projects/mariadb-server/libmysql/.libs/tls/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOE
NT (No such file or directory)

Closes 
2019-11-29 15:33:49 +01:00
Eugene Kosov
2855edf9ee try to fix data races in DBUG
init_settings.keywords and it's pointee are shared data. Protect them
with mutex too.
2019-11-28 11:59:27 +07:00
Vicențiu Ciorbaru
6218bf1b41 cracklib-runtime needs a comma after to parse properly 2019-11-26 21:48:41 +02:00
Vicențiu Ciorbaru
427eedd012 MDEV-13288: Proper fix for cracklib-runtime
The required dependencies should be added through the autobake script,
to also cover distributions that do not support libcrack2.
2019-11-26 19:52:37 +02:00
Vicențiu Ciorbaru
ed2379f983 MDEV-13288: Upstream debian patch 2019-11-26 14:04:21 +02:00
Seth Shelnutt
23664bc7a5 Fix incorrect DBUG_ENTER message for join_read_last 2019-11-26 12:27:07 +02:00
seppo
38839854b7 MDEV-19572 async slave node fails to apply MyISAM only writes ()
The problem happens when MariaDB master replicates writes for only non InnoDB
tables (e.g. writes to MyISAM table(s)). Async slave node, in Galera cluster,
can apply these writes successfully, but it will, in the end, write gtid position in
mysql.gtid_slave_pos table. mysql.gtid_slave_pos table is InnoDB engine, and
this write makes innodb handlerton part of the replicated "transaction".
Note that wsrep patch identifies that write to gtid_slave_pos should not be replicated
and skips appending wsrep keys for these writes. However, as InnoDB was present
in the transaction, and there are replication events (for MyISAM table) in transaction
cache, but there are no appended keys, wsrep raises an error, and this makes the söave
thread to stop.

The fix is simply to not treat it as an error if async slave tries to replicate a write
set with binlog events, but no keys. We just skip wsrep replication and return successfully.

This commit contains also a mtr test which forces mysql.gtid_slave_pos table isto be
of InnoDB engine, and executes MyISAM only write through asyn replication.

There is additional fix for declaring IO and background slave threads as non wsrep.
These threads should not write anything for wsrep replication, and this is just a safeguard
to make sure nothing leaks into cluster from these slave threads.
2019-11-26 08:49:50 +02:00
Hashir Sarwar
d8ace23d26 Fixed some typos in mysql.cc
Closes 
2019-11-22 15:19:04 +01:00
Eugene Kosov
a51f3b09bb cleanup DBUG
DbugParse(): removed mutex lock/unlock which should protect file writes only.
And no file writes happen in this function.

DbugFlush(): move mutex_unlock out of this method because fflush() doesn't
need any locking.

Slow stuff like mutex lock/unlock and accessing errno (TLS)
is moved to a more narrow scope.
2019-11-21 11:31:27 +07:00
Eugene Kosov
8fbfcce911 cleanup: remove always true condition to fix clang warning 2019-11-19 17:09:43 +07:00
Jan Lindström
c6b097ab37 Remove excessive sleep from test. 2019-11-18 15:22:01 +02:00
seppo
5c68343db7 MDEV-18497 CTAS async replication from mariadb master crashes galera nodes ()
This PR contains a mtr test for reproducing a failure with replicating create table as select statement (CTAS) through asynchronous mariadb replication to mariadb galera cluster.
The problem happens when CTAS replication contains both create table statement followed by row events for populating the table. In such situation, the galera node operating as mariadb replication slave, will first replicate only the create table part into the cluster, and then perform another replication containing both the create table and row events. This will lead all other nodes to fail for duplicate table create attempt, and crash due to this failure.

PR contains also a fix, which identifies the situation when CTAS has been replicated, and makes further scan in async replication stream to see if there are following row events. The slave node will replicate either single TOI in case the CTAS table is empty, or if CTAS table contains rows, then single bundled write set with create table and row events is replicated to galera cluster.

This fix should keep master server's GTID's for CTAS replication in sync with GTID's in galera cluster.
2019-11-18 15:18:00 +02:00
Varun Gupta
2909725636 MDEV-21044: Wrong result when using a smaller size for sort buffer
Make sure that the sort buffers can store atleast one sort key.
This is needed to make sure that all merge buffers are read else
with no sort keys some merge buffers are skipped because the code
makes a conclusion there is no data to be read.
2019-11-18 11:50:58 +05:30
Daniel Black
214023aa0e systemd: mariadb@bootstrap doesn't bootstrap, galera_new_cluster does
Closes 
2019-11-14 14:17:13 -08:00
Sujatha
7df07c7666 MDEV-20953: binlog_encryption.rpl_corruption failed in buildbot due to wrong error code
Problem:
========
CURRENT_TEST: binlog_encryption.rpl_corruption

mysqltest: In included file "./include/wait_for_slave_io_error.inc":
...
At line 72: Slave stopped with wrong error code
**** Slave stopped with wrong error code: 1743 (expected 1595,1913) ****

Analysis:
========
The test emulates the corruption at the various stages of replication for
example in binlog file, in network and in relay log etc. It verifies that all
corruption cases are handled through appropriate error messages.

The test cases which emulate network failure expect following errors.
--ER_SLAVE_RELAY_LOG_WRITE_FAILURE (1595)
--ER_NETWORK_READ_EVENT_CHECKSUM_FAILURE (1743)

Ideally test should expect error codes as 1595 and 1743.
But the test actually waits on incorrect error code 1595,1913

Fix:
===
Added appropriate error code for 'ER_NETWORK_READ_EVENT_CHECKSUM_FAILURE'.
Replaced 1913 with 1743.
2019-11-12 16:31:08 +05:30
Andrei Elkin
13db50fc03 MDEV-19376 Repl_semi_sync_master::commit_trx assertion failure: ... || !m_active_tranxs->is_tranx_end_pos(trx_wait_binlog_name, trx_wait_binlog_pos)
The assert indicates that the current transaction got caught uncleaned from
the semisync master's cache when it is signaled to proceed upon its
ack receive.

The reason of missed cleanup turns out to be a flaw in the gtid
connect mode.
A submitted by connecting slave value of its last received event's
binlog file *name* was adopted into
{{Repl_semi_sync_master::m_reply_file_name}} as a part of semisync
initialization.

Notice that the initialization still refines the position part of the
submitted last received event's binlog coordinates.
The master side binlog filename:pos refinement is
specific to the gtid connect mode for purpose of computing the latest
binlog file to resume slave feeding from.
Effectively in the gtid connect mode the computed resumption filename:pos
may appear smaller in which case a new post-connect time committing
transaction may be logged with its filename:pos also less than the
submitted coordinates and that triggers the assert.

Fixed with making the semisync initialization to use the refined filename:pos.
It is guaranteed to be less than any new generated transaction's binlog:pos.
2019-11-10 16:16:37 +02:00
Daniel Bartholomew
dfd2d3d861
bump the VERSION 2019-11-08 09:50:14 -05:00
Hartmut Holzgraefe
c4a844ca51 MDEV-20981 wsrep_sst_mariabackup fails silently when mariabackup is not installed ()
Make sure failure to find mariabackup binary does not terminate
the script silently, terminate with a clear error message instead
2019-11-08 09:03:49 +02:00
Varun Gupta
b1ab2ba583 MDEV-20519: Query plan regression with optimizer_use_condition_selectivity > 1
The issue here is the wrong estimate of the cardinality of a partial join,
the cardinality is too high because the function table_cond_selectivity()
returns an absurd number 100 while selectivity cannot be greater than 1.

When accessing table t by outer reference t1.a via index we do not perform any
range analysis for t. Yet we see TABLE::quick_key_parts[key] and
TABLE->quick_rows[key] contain a non-zero value though these should have been
remained untouched and equal to 0.

Thus real cause of the problem is that TABLE::init does not clean the arrays
TABLE::quick_key_parts[] and TABLE::>quick_rows[].
It should have done it because the TABLE structure created for any
instance of a table can be reused for many queries.
2019-11-07 15:24:21 +05:30
Marko Mäkelä
4e99e67c4e Merge 5.5 into 10.1 2019-11-06 08:17:03 +02:00
Thirunarayanan Balathandayuthapani
5c3bbbd845 MDEV-20987 InnoDB fails to start when fts table has FK relation
InnoDB: Assertion failure in file .../dict/dict0dict.cc line ...
InnoDB: Failing assertion: table->can_be_evicted

This fixes a regression that was caused by the fix of MDEV-20621
(commit a41d429765).
MySQL 5.6 (and MariaDB 10.0) introduced eviction of tables from
the InnoDB data dictionary cache. Tables that are connected to
FOREIGN KEY constraints or FULLTEXT INDEX are exempt of the eviction.
With the problematic change, a table that would already be exempt
from eviction due to FOREIGN KEY would cause the problem if there
also was a FULLTEXT INDEX defined on it.

dict_load_table(): Only prevent eviction if table->can_be_evicted holds.
2019-11-06 08:12:00 +02:00
Daniel Bartholomew
6e48f3a062
bump the VERSION 2019-11-05 09:55:18 -05:00
Daniel Bartholomew
e23cb3835e
bump the VERSION 2019-11-05 09:53:45 -05:00
Vladislav Vaintroub
dc771113a6 Fix ninja build
Do not rely on existence of CMakeFiles/${target}.dir directory existence
It is not there for custom targets in Ninja build.
2019-11-05 00:20:13 +01:00
Sergei Golubchik
cd156e2c3e MDEV-20971 ASAN heap-use-after-free in list_delete / heap_close
Don't save/restore HP_INFO as it could be changed by a concurrent thread.
different parts of HP_INFO are protected by different mutexes and
the mutex that protect most of the HP_INFO does not protect its open_list
data.

As a bonus, make heap_check_heap() to take const HP_INFO* and not
make any changes there whatsoever.
2019-11-04 23:14:27 +01:00
Marko Mäkelä
5164f8c206 Fix GCC 9.2.1 -Wstringop-truncation
dict_table_rename_in_cache(): Use strcpy() instead of strncpy(),
because they are known to be equivalent in this case (the length
of old_name was already validated).

mariabackup: Invoke strncpy() with one less than the buffer size,
and explicitly add NUL as the last byte of the buffer.
2019-11-04 15:52:54 +02:00
pkubaj
eb56339b66 Fix build on !glibc/powerpc*
Do the same that newer branches do and don't include glibc-related headers on non-glibc environment.
2019-11-02 18:25:12 +04:00
Varun Gupta
8afe4bba2e MDEV-20424: New default value for optimizer_use_condition-selectivity leads to bad plan
In the function prev_record_reads where one finds the different row combinations for a
subset of partial join, it did not take into account the selectivity of tables
involved in the subset of partial join.
2019-11-01 17:22:05 +05:30
Robert Bindar
6f86150ab3 MDEV-17896 Assertion `pfs->get_refcount() > 0' failed
Unfortunate DROP TEMPORARY..IF EXISTS on a regular table may allow
subsequent CREATE TABLE statements to steal away the PFS_table_share
instance from the dropped table.
2019-11-01 11:10:04 +02:00
Elena Stepanova
d925aec1c1 List of unstable tests for 10.1.42 release 2019-10-31 18:07:19 +02:00
Marko Mäkelä
1bb857089f MDEV-20927: Remove duplicated code
In commit d1e6b0bcff
some code was supposed to be modified, but instead
it got duplicated. Remove the duplicated copy.
2019-10-31 07:44:18 +02:00
Sergei Golubchik
313855766f cleanup
* use compile_time_assert instead of DBUG_ASSERT

* don't use thd->clear_error(), because
  * the error was already consumed by the error handler, so there is
     nothing to clear
  * it's dangerous to clear errors indiscriminately, if the error came
    from outside of read_statistics_for_tables() it must not be cleared
2019-10-30 23:14:44 +01:00
Sergei Golubchik
5392b4a32c MDEV-20354 All but last insert ignored in InnoDB tables when table locked
mysql_insert() first opens all affected tables (which implicitly
starts a transaction in InnoDB), then stat tables.
A failure to open a stat table caused open_tables() to abort
the current stmt transaction (trans_rollback_stmt()). So, from the
server point of view the following ha_write_row()-s happened outside
of a transactions, and the server didn't bother to commit them.

The server has a mechanism to prevent a transaction being
unexpectedly committed or rolled back in the middle of a statement -
if an operation takes place _in a sub-statement_ it cannot change
the transaction state. Operations on stat tables are exactly that -
they are not allowed to change a transaction state. Put them in
a sub-statement to make sure they don't.
2019-10-30 23:14:44 +01:00
Oleksandr Byelkin
c8ba98206f Merge branch '10.1' into bb-10.1-release 2019-10-30 20:01:51 +01:00
Sergei Golubchik
8fce180765 MDEV-19432 Systemd service does not get re-enabled after upgrade
following Fedora recommendations (see %systemd_post macro in FC29)
let's do `systemctl preset` on the first installation of the server
2019-10-30 16:46:08 +01:00
Oleksandr Byelkin
d671f506b0 Merge branch '5.5' into 10.1 2019-10-30 14:47:35 +01:00
Sergei Golubchik
9277b6ec1c Revert "MDEV-14448: Ctrl-C should not exit the client"
This reverts commit 396313d301.
2019-10-30 14:44:56 +01:00
Oleksandr Byelkin
708d46158b Merge remote-tracking branch 'connect/10.1' into 10.1 2019-10-30 13:55:35 +01:00
Oleksandr Byelkin
b58c38a3a9 Merge branch 'merge-tokudb-5.6' into 10.1 2019-10-30 13:49:25 +01:00
Marko Mäkelä
d03a59c6ff XtraDB 5.6.45-86.1 2019-10-30 13:21:36 +02:00
Marko Mäkelä
d1e6b0bcff MDEV-20927 Duplicate key with auto increment
Apply the changes to InnoDB and XtraDB that had been
inadvertently skipped in the merge
commit ae476868a5

That merge failure sabotaged part of MDEV-20127:
>Revert a problematic auto_increment_increment 'fix' from 2014.
>This involves replacing the MDEV-8827 fix and in 10.1,
>removing some WSREP instrumentation.

The code changes were re-merged manually by executing the following:

 # Get the parent of the problematic merge.
git checkout ae476868a5394041a00e75a29c7d45917e8dfae8^
 # Perform the merge again.
git merge ae476868a5394041a00e75a29c7d45917e8dfae8^2
 # Get the conflict resolution from that merge.
git checkout ae476868a5 .
 # Note: Any changes to these files were removed (empty diff)!
git diff HEAD storage/{innobase,xtradb}/handler/ha_innodb.cc
 # Apply the code changes:
git diff cf40393471b10ca68cc1d2804c22ab9203900978^2..MERGE_HEAD \
storage/{innobase,xtradb}/handler/ha_innodb.cc|
patch -p1
2019-10-30 13:21:36 +02:00
Eugene Kosov
b8eff8bce4 avoid accessing TLS 2019-10-30 13:23:15 +03:00
Oleksandr Byelkin
a3553a1387 5.6.45-86.1 2019-10-30 11:15:05 +01:00
Eugene Kosov
a7e3008ab3 MDEV-20926 UBSAN: load of value 165, which is not a valid value for type bool
Basicaly it's an uninitialized read. 165 is 0xa5 which comes from TRASH_ALLOC()

Fix by calling a class ctor which initializes problematic
TMP_TABLE_PARAM::force_copy_fields field
2019-10-30 12:53:25 +03:00
Sergey Vojtovich
afb4878703 MDEV-18783 - Server crash in hp_rb_make_key
In debug build, whenever MEMORY table instance gets closed it performs
consistency check without protection. It may cause server crash if
executed concurrently with DML.

Moved consistency check to ha_heap::external_lock(F_UNLCK), so that it
is protected by THR_LOCK.
2019-10-30 12:52:19 +04:00
Sergei Golubchik
20b72a3fad compilation fix for Windows 2019-10-30 00:47:50 +01:00
Eugene Kosov
bc8a9ab5f3 remove unused struct members 2019-10-29 23:02:03 +03:00
Eugene Kosov
9ed4d06706 Merge 5.5 into 10.1 2019-10-29 22:10:43 +03:00
Eugene Kosov
84088d9458 add .clang-format file for InnoDB 2019-10-29 21:56:24 +03:00
Marko Mäkelä
2cc360bdf2 Remove bogus advice
Ever since MariaDB 10.0 (and MySQL 5.6.8), the innodb_log_file_size
and innodb_log_files_in_group can be changed between server restarts,
and the redo log files will be resized on server startup if needed.
2019-10-28 16:03:00 +02:00