Commit graph

179758 commits

Author SHA1 Message Date
Sergei Petrunia
1cb2e0333d MDEV-12466 : Assertion `thd->transaction.stmt.is_empty() || thd->in_sub_stmt || ...
When "FLUSH TABLE ... FOR EXPORT" fails, the SQL layer should rollback
the statement. Otherwise we hit an assert when we try to close the
tables while having a non-empty list of statement transaction participants.
2018-04-07 14:05:28 +03:00
Sergey Vojtovich
5ccf3f96ac Fix misuse of MY_CHECK_CXX_COMPILER_FLAG
- compile_flags already include from top CMakeLists.txt
- MY_CHECK_CXX_COMPILER_FLAG() accepts only one parameter
- output variable of MY_CHECK_CXX_COMPILER_FLAG() is have_CXX__Wa__nH
- same check for mariabackup

Based on contribution by satanson (PR#466).
2018-04-05 17:45:36 +04:00
Marko Mäkelä
4fde1361a6 MDEV-15553 Assertion failed in dict_table_get_col_name
dict_foreign_qualify_index(): Avoid a redundant and harmful
computation of col_name of a virtual column. This fixes the
assertion failure.

dict_foreign_push_index_error(): Do not call dict_table_get_col_name()
on a virtual column. (It is unclear if this condition is actually
reachable.)
2018-04-05 15:01:17 +03:00
Marko Mäkelä
bc2501453c Remove an unused variable 2018-04-03 16:58:35 +03:00
Marko Mäkelä
3c21eccb8c MDEV-15764 InnoDB may write uninitialized garbage to redo log
log_write_up_to(): Erase the end of the current log block.
Simplify the computation of pad_size.

log_buffer_switch(): Evaluate a condition only once.
2018-04-03 15:58:13 +03:00
Thirunarayanan Balathandayuthapani
d9c5a46678 MDEV-15737 assertion in mariabackup.exe!recv_calc_lsn_on_data_add()
- recovered_lsn shouldn't be initialized during xtrabackup_copy_logfile().
If partial redo log read during the end of xtrabackup_copy_logfile() then
recovered_lsn will be different from scanned_lsn. Re-initialization of
recovered_lsn could lead to partial read again. It is a regression of
MDEV-14545
2018-04-03 16:43:36 +05:30
Galina Shalygina
6223f1dd98 MDEV-15579 Crash in Item_field::used_tables() called by
Item::derived_field_transformer_for_having

The crash occurred due to an inappropriate handling of multiple equalities
when pushing conditions into materialized views/derived tables. If equalities
extracted from a multiple equality can be pushed into a materialized
view/derived table they should be plainly conjuncted with other pushed
predicates rather than form a separate AND sub-formula.
2018-04-03 00:17:45 +02:00
Vladislav Vaintroub
27c24808f7 MDEV-15636 mariabackup --lock-ddl-per-table hangs if ALTER table is running
concurrently.

There is a deadlock between

C1 mariabackup's connection that holds MDL locks
C2 Online ALTER TABLE that wants to have MDL exclusively
   and tries to upgrade its mdl lock.
C3 another mariabackup's connection that does FLUSH TABLES (or FTWRL)

C3 waits waits for C2,  which waits for C1, which waits for C3,
thus the deadlock.


MDL locks cannot be released until FLUSH  succeeds, because
otherwise it would allow ALTER to sneak in, causing backup to abort and
breaking lock-ddl-per-table's promise.

The fix here workarounds the deadlock, by killing connections in
"Waiting for metadata lock" status (i.e ALTER). This killing continues
until FTWRL succeeds.

Killing connections is skipped in case --no-locks parameter
was  passed to backup, because there won't be a FLUSH.

For the reference,in Percona's xtrabackup --lock-ddl-per-connection
silently implies --no-lock ie FLUSH is always skipped there.

A rather large part of fix is introducing DBUG capability to start
a query  the new connection at the right moment of backup
compensating somewhat for mariabackup' lack of send_query or DBUG_SYNC.
2018-04-01 14:26:06 +00:00
Vladislav Vaintroub
a1d68faa38 CMake : Move INNODB_DISALLOW_WRITES from top-level CMakeLists.txt to innodb 2018-04-01 14:26:06 +00:00
Marko Mäkelä
55f4e4800b MDEV-15325 fixup: Disable test for --embedded 2018-03-29 22:00:07 +03:00
Daniel Black
402c7584a8 MDEV-13785: move defination HAVE_LARGE_PAGES -> HAVE_LINUX_LARGE_PAGES
HAVE_LARGE_PAGES was always Linux but now there is
HAVE_SOLARIS_LARGE_PAGES in the code base. Innodb was using HAVE_LINUX_LARGE_PAGES
so keep this consistent everywhere.

Test plan:

$ grep Hugepagesize: /proc/meminfo
Hugepagesize:       2048 kB

$ sudo sysctl vm.nr_hugepages=1024
vm.nr_hugepages = 1024
$ sudo sysctl  kernel.shmmax=$(( 2 * 1024 *1024 * 1024 ))
kernel.shmmax = 2147483648

No errors in ouput:
$ sql/mysqld --skip-networking --datadir=/tmp/datadir --log-bin=/tmp/datadir/mysqlbin --socket /tmp/s.sock --lc-messages-dir=${PWD}/sql/share --verbose --large-pages=1
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld (mysqld 10.2.14-MariaDB-log) starting as process 25406 ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Uses event mutexes
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Compressed tables use zlib 1.2.11
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using Linux native AIO
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Number of pools: 1
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Using SSE2 crc32 instructions
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Completed initialization of buffer pool
2018-03-23 12:51:18 139696883590912 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Highest supported file format is Barracuda.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 128 out of 128 rollback segments are active.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Creating shared tablespace for temporary tables
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-03-23 12:51:18 139697428129984 [Note] InnoDB: 5.7.21 started; log sequence number 1620099
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Loading buffer pool(s) from /tmp/datadir/ib_buffer_pool
2018-03-23 12:51:18 139696713733888 [Note] InnoDB: Buffer pool(s) load completed at 180323 12:51:18
2018-03-23 12:51:18 139697428129984 [Note] Plugin 'FEEDBACK' is disabled.
2018-03-23 12:51:18 139697428129984 [Note] Reading of all Master_info entries succeded
2018-03-23 12:51:18 139697428129984 [Note] Added new Master_info '' to hash table
2018-03-23 12:51:18 139697428129984 [Note] sql/mysqld: ready for connections.
Version: '10.2.14-MariaDB-log'  socket: '/tmp/s.sock'  port: 0  Source distribution

$ grep -i huge /proc/25406/smaps | grep -v ' 0 kB'
Private_Hugetlb:    8192 kB
Private_Hugetlb:    2048 kB
$ grep huge /proc/25406/numa_maps
7f0d74400000 default file=/SYSV00000000\040(deleted) huge
7f0dbd200000 default file=/SYSV00000000\040(deleted) huge dirty=4 N0=4 kernelpagesize_kB=2048
7f0dc5600000 default file=/SYSV00000000\040(deleted) huge
7f0dd1200000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      935
HugePages_Rsvd:      177
HugePages_Surp:        0
Hugepagesize:       2048 kB

Ran again with --memlock
(note needs ulimit -l > size)

$ grep Huge  /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:     940
HugePages_Free:      758
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

$ grep huge /proc/26020/numa_maps
7fe870400000 default file=/SYSV00000000\040(deleted) huge dirty=62 N0=62 kernelpagesize_kB=2048
7fe8b3a00000 default file=/SYSV00000000\040(deleted) huge dirty=66 N0=66 kernelpagesize_kB=2048
7fe8bd600000 default file=/SYSV00000000\040(deleted) huge dirty=53 N0=53 kernelpagesize_kB=2048
7fe8c8400000 default file=/SYSV00000000\040(deleted) huge dirty=1 N0=1 kernelpagesize_kB=2048

$ grep -i huge /proc/26020/smaps | grep -v ' 0 kB'
Private_Hugetlb:  126976 kB
Private_Hugetlb:  135168 kB
Private_Hugetlb:  108544 kB
Private_Hugetlb:    2048 kB
2018-03-29 18:55:18 +04:00
Marko Mäkelä
014dfe473a MDEV-15719: Adjust a test case 2018-03-29 16:58:59 +03:00
Marko Mäkelä
3eb73bf630 Remove unnecessary SysTablespace references 2018-03-29 16:54:05 +03:00
Marko Mäkelä
622d21e2b8 row_drop_table_for_mysql(): Use a constant string
It does not hurt to delete non-existing records from SYS_TABLESPACES
and SYS_DATAFILES. Because MariaDB does not support CREATE TABLESPACE,
only the system tablespace (space_id=0) can contain multiple tables.
But, there are no entries for the system tablespace in these tables
(which actually are stored inside the system tablespace).
2018-03-29 16:54:05 +03:00
Sergei Petrunia
b922741074 MDEV-15472: Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failure
MariaDB differs from the upstream for "DDL-like" command. For these,
it sets binlog_format=STATEMENT for the duration of the statement.
This doesn't play well with MyRocks, which tries to prevent DML
commands with binlog_format!=ROW.

Also, if Locked_tables_list::reopen_tables() returned an error, then
close_cached_tables should propagate the error condition and not silently
consume it (it's difficult to have test coverage for this because this
error condition is rare)
2018-03-29 14:43:08 +03:00
Marko Mäkelä
adaee46a90 MDEV-15682 mariabackup.unsupported_redo fails in buildbot with wrong result code
Skip the test mariabackup.unsupported_redo if a checkpoint occurred
before mariabackup --backup completed. Remove the slow shutdowns
and restarts which were attempting to prevent the checkpoints from
occurring.
2018-03-29 13:22:59 +03:00
Marko Mäkelä
6cccef21a6 MDEV-15720 ib_buffer_pool unnecessarily includes the temporary tablespace
The purpose of the InnoDB buffer pool dump is to allow InnoDB to be
restarted with the same persistent data pages in the buffer pool.

The InnoDB temporary tablespace that was introduced in MariaDB 10.2.2
is always reinitialized on restart. Therefore, it does not make sense
to attempt to dump or restore any pages of the temporary tablespace.
2018-03-29 13:22:16 +03:00
Marko Mäkelä
4d9969c216 MDEV-15719 ALTER TABLE…ALGORITHM=INPLACE is unnecessarily refused due to innodb_force_recovery
ha_innobase::check_if_supported_inplace_alter(): Only check for
high_level_read_only. Do not unnecessarily refuse
ALTER TABLE...ALGORITHM=INPLACE if innodb_force_recovery was
specified as 1, 2, or 3.

innobase_start_or_create_for_mysql(): Block all writes from SQL
if the system tablespace was initialized with 'newraw'.
2018-03-29 13:20:59 +03:00
Sergei Petrunia
0d2fffb612 MDEV-15686: Loading MyRocks plugin back after it has been unloaded causes a crash
Adjust the testcase to handle all possible outcomes.
2018-03-29 11:08:32 +03:00
Marko Mäkelä
d18a66147c recv_validate_tablespace(): Fix -Wmissing-fallthrough 2018-03-28 20:40:09 +03:00
Marko Mäkelä
5beddfa08c fil_node_open_file(): Add a missing space to message 2018-03-28 20:39:57 +03:00
Sergei Petrunia
011586c04d MDEV-15686: Loading MyRocks plugin back after it has been unloaded causes a crash
- Disallow loading of MyRocks (or any auxilary) plugins after it has been
  unloaded.
- Do it carefully - Plugin's system variables may be accesssed (e.g. default
  value is set) after the first rocksdb_done_func() call but before
  the secon rocksdb_init_func() call.
2018-03-28 14:30:37 +03:00
Sergei Petrunia
907aae2502 MDEV-15708: rocksdb.mariadb_plugin fails on winx64, Cannot enable tc-log at run-time
Ignore the warning.
On Windows, Galera is not available so the server starts in non-XA mode
when both MyRocks and InnoDB are disabled.
2018-03-28 14:28:58 +03:00
Jan Lindström
3b119d9d30 MDEV-11531: encryption.innodb_lotoftables failed in buildbot
Test changes only.
2018-03-28 13:14:24 +03:00
Daniel Bartholomew
aafb9d44d6 bump the VERSION 2018-03-27 13:31:07 -04:00
Thirunarayanan Balathandayuthapani
73af8af094 MDEV-15325 Incomplete validation of missing tablespace during recovery
Problem:
=======
	During validation of missing tablespace, missing tablespace id is
being compared with hash table of redo logs (recv_sys->addr_hash). But if the
hash table ran out of memory then there is a possibility that it will not contain
the redo logs of all tablespace. In that case, Server will load the InnoDB
even though there is a missing tablespace.

Solution:
========
	If the recv_sys->addr_hash hash table ran out of memory then InnoDB needs
to scan the remaining redo log again to validate the missing tablespace.
2018-03-27 13:47:56 +05:30
Sergei Petrunia
60438451c3 MDEV-14843: Assertion `s_tx_list.size() == 0' failed in myrocks::Rdb_transaction::term_mutex
When the plugin is unloaded, walk the s_trx_list and delete the left over
Rdb_transaction objects.
It is responsibility of the SQL layer to make sure that the storage engine
has no open tables when the plugin is being unloaded.
2018-03-26 21:25:40 +03:00
Elena Stepanova
b3cdafcb93 Updated list of unstable tests for 10.2.14 2018-03-26 19:36:39 +03:00
Thirunarayanan Balathandayuthapani
dcb59373d5 - Fixing innodb.purge_secondary test case failure 2018-03-26 17:36:41 +05:30
Marko Mäkelä
c813d9485a Fix result after commit e27535093d 2018-03-26 13:38:14 +03:00
Thirunarayanan Balathandayuthapani
e27535093d - Follow-up fix to MDEV-15229 2018-03-26 15:48:27 +05:30
Sergei Golubchik
c764bc0a78 Merge branch '10.1' into 10.2 2018-03-25 13:02:52 +02:00
Alexey Botchkov
d702e46390 MDEV-15561 json_extract returns NULL with numbers in scientific notation.
Scientific notation handling fixed.
2018-03-25 00:15:11 +04:00
Sergei Golubchik
15795b9f9a save/restore auto_inc settings in galera_sst_rsync test
and remove redundant have_innodb.inc, it's included in
galera_cluster.inc anyway.
2018-03-24 14:24:20 +01:00
Sergei Golubchik
7454d5f952 save/restore auto_inc settings in galera_sst_mysqldump test
and remove redundant have_innodb.inc, it's included in
galera_cluster.inc anyway.
2018-03-24 14:17:31 +01:00
Sergei Golubchik
5fdbc3f66b compiler warning
extra/mariabackup/ds_buffer.c:145:9: warning: pointer targets in assignment differ in signedness [-Wpointer-sign]
2018-03-24 14:17:31 +01:00
Alexey Botchkov
0b74a1fa64 MDEV-14533 Provide information_schema tables using which hardware
information can be obtained.

plugin only enabled for Linux, as it fails building on BSD/MacOSX.
disks.test fixed.
2018-03-24 00:37:38 +04:00
Alexey Botchkov
3b644ac1f7 MDEV-14533 Provide information_schema tables using which hardware
information can be obtained.

disks.test moved to plugin's directory.
2018-03-24 00:30:28 +04:00
Sergei Golubchik
febe1e8503 Merge branch '10.0' into 10.1 2018-03-23 17:40:53 +01:00
Sergei Golubchik
d3681c18f9 followup for 89b0d5cb6e, backport 8c422bf48d 2018-03-23 17:40:09 +01:00
Vladislav Vaintroub
af86422f08 MDEV-13023 mariabackup does not preserve holes for page compressed tables.
Changed "local" datasink logic to detect page compressed Innodb tables.

Whenever such table is detected, holes in the copied files are created by
skipping over binary zeros at the end of each compressed page.
2018-03-23 15:30:01 +00:00
Sergey Vojtovich
ca291015bc MDEV-10269 - Killed queries from I_S stay in 'Killed' state for long
time and don't let server shut down

Queries from I_S in "Filling schema table" state didn't check killed
flag. For large tables this phase may take a while to complete.

Fixed by adding thd->killed flag check for each processed row.
2018-03-23 15:33:53 +04:00
Sergei Golubchik
a2e47f8c41 Merge branch '5.5' into 10.0 2018-03-23 11:44:29 +01:00
Thirunarayanan Balathandayuthapani
7003067a09 - Fixing innodb.purge_secondary test case. 2018-03-23 13:27:33 +05:30
Sergei Golubchik
f249d8467a MDEV-15570 Assertion `Item_cache_temporal::field_type() != MYSQL_TYPE_TIME' failed in Item_cache_temporal::val_datetime_packed
remove an assert. TIME value can be used (and cached) in a datetime context
2018-03-23 01:11:14 +01:00
Sergei Golubchik
4092f90655 MDEV-15409 make sure every sst script is tested in buildbot
galera SST tests have a debug part, but we don't want to limit them
to fulltest2 builder. So, add support for test files that
have a debug part:

* add maybe_debug.inc and maybe_debug.combinations
* 'debug' combination is run when debug is available
* 'release' combination is run otherwise
* test wraps debug parts in if($with_debug) { ... }
* and creates ,debug.rdiff for debug results
2018-03-23 00:55:20 +01:00
Sergei Golubchik
5ff7ed96d5 MDEV-15409 make sure every sst script is tested in buildbot
create galera.galera_sst_mariabackup
2018-03-23 00:55:20 +01:00
Sergei Golubchik
60d4abc1e5 MDEV-15409 make sure every sst script is tested in buildbot
* make galera.galera_sst_xtrabackup* not big
* auto-select between socat and nc, whatever available
* auto-skip xtrabackup tests if no xtrabackup or neither socat nor nc
2018-03-23 00:55:20 +01:00
Sergei Golubchik
4b1cbff7a8 MDEV-15409 make sure every sst script is tested in buildbot
make galera.galera_sst_rsync not big
2018-03-23 00:55:20 +01:00
Sergei Golubchik
8f1014e9a0 MDEV-15409 make sure every sst script is tested in buildbot
fix galera.galera_sst_mysqldump test to work:
* must connect to 127.0.0.1, where mysqld is listening
* disable wsrep_sync_wait in wsrep_sst_mysqldump, otherwise
  sst can deadlock
* allow 127.0.0.1 for bind_address and wsrep_sst_receive_address.
  (it's useful in tests, or when two nodes are on the same box,
  or when nodes are on different boxes, but the connection is
  tunelled, or whatever. Don't judge user's setup). MDEV-14070
* don't wait for client connections to die when doing
  mysqldump sst. they'll die in a due time, and if needed mysql
  will wait on locks until they do. MDEV-14069

Also don't mark it big, to make sure it's sufficiently tested
2018-03-23 00:55:20 +01:00