Commit graph

179698 commits

Author SHA1 Message Date
Sergey Vojtovich
e147a4a067 Fixed build failure 2018-03-23 00:32:16 +04:00
Vicențiu Ciorbaru
1123f87b5c Fix unused variable thd warning in embedded
The warning came up post merge and is visible when compiling without
WSREP.
2018-03-22 21:01:20 +02:00
Marko Mäkelä
fc05777eac Enable --suite=innodb_undo 2018-03-22 18:02:58 +02:00
Marko Mäkelä
e80a842000 Merge 10.1 into 10.2 2018-03-22 18:02:40 +02:00
Marko Mäkelä
2fb31821de MDEV-11984 Avoid accessing SYS_TABLESPACES unnecessarily
The following INFORMATION_SCHEMA views were unnecessarily retrieving
the data from the SYS_TABLESPACES table instead of directly fetching
it from the fil_system cache:

information_schema.innodb_tablespaces_encryption
information_schema.innodb_tablespaces_scrubbing

InnoDB always loads all tablespace metadata into memory at startup
and never evicts it while the tablespace exists.

With this fix, accessing these views will be much faster and use less
memory, and include data about all tablespaces, including undo
tablespaces.

The view information_schema.innodb_sys_tablespaces will still reflect
the contents of the SYS_TABLESPACES table.
2018-03-22 18:01:29 +02:00
Marko Mäkelä
b98d80eb84 Correct a wait condition in a disabled test
The test will still time out. That should be addressed in MDEV-8139.
2018-03-22 18:01:29 +02:00
Alexander Barkov
0cba2c1ccb MDEV-15633 Memory leak after MDEV-15005 2018-03-22 16:23:37 +04:00
Eugene Kosov
8d32959b09 fix data races
srv_last_monitor_time: make all accesses relaxed atomical

WARNING: ThreadSanitizer: data race (pid=12041)
  Write of size 8 at 0x000003949278 by thread T26 (mutexes: write M226445748578513120):
    #0 thd_destructor_proxy storage/innobase/handler/ha_innodb.cc:314:14 (mysqld+0x19b5505)

  Previous read of size 8 at 0x000003949278 by main thread:
    #0 innobase_init(void*) storage/innobase/handler/ha_innodb.cc:4180:11 (mysqld+0x1a03404)
    #1 ha_initialize_handlerton(st_plugin_int*) sql/handler.cc:522:31 (mysqld+0xc5ec73)
    #2 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) sql/sql_plugin.cc:1447:9 (mysqld+0x134908d)
    #3 plugin_init(int*, char**, int) sql/sql_plugin.cc:1729:15 (mysqld+0x13484f0)
    #4 init_server_components() sql/mysqld.cc:5345:7 (mysqld+0xbf720f)
    #5 mysqld_main(int, char**) sql/mysqld.cc:5940:7 (mysqld+0xbf107d)
    #6 main sql/main.cc:25:10 (mysqld+0xbe971b)

  Location is global 'srv_running' of size 8 at 0x000003949278 (mysqld+0x000003949278)

WARNING: ThreadSanitizer: data race (pid=27869)
  Atomic write of size 4 at 0x7b4800000c00 by thread T8:
    #0 __tsan_atomic32_exchange llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interface_atomic.cc:589 (mysqld+0xbd4eac)
    #1 TTASEventMutex<GenericPolicy>::exit() storage/innobase/include/ib0mutex.h:467:7 (mysqld+0x1a8d4cb)
    #2 PolicyMutex<TTASEventMutex<GenericPolicy> >::exit() storage/innobase/include/ib0mutex.h:609:10 (mysqld+0x1a7839e)
    #3 fil_validate() storage/innobase/fil/fil0fil.cc:5535:2 (mysqld+0x1abd913)
    #4 fil_validate_skip() storage/innobase/fil/fil0fil.cc:204:9 (mysqld+0x1aba601)
    #5 fil_aio_wait(unsigned long) storage/innobase/fil/fil0fil.cc:5296:2 (mysqld+0x1abbae6)
    #6 io_handler_thread storage/innobase/srv/srv0start.cc:340:3 (mysqld+0x21abe1e)

  Previous read of size 4 at 0x7b4800000c00 by main thread (mutexes: write M1273, write M1271):
    #0 TTASEventMutex<GenericPolicy>::state() const storage/innobase/include/ib0mutex.h:530:10 (mysqld+0x21c66e2)
    #1 sync_array_detect_deadlock(sync_array_t*, sync_cell_t*, sync_cell_t*, unsigned long) storage/innobase/sync/sync0arr.cc:746:14 (mysqld+0x21c1c7a)
    #2 sync_array_wait_event(sync_array_t*, sync_cell_t*&) storage/innobase/sync/sync0arr.cc:465:6 (mysqld+0x21c1708)
    #3 TTASEventMutex<GenericPolicy>::enter(unsigned int, unsigned int, char const*, unsigned int) storage/innobase/include/ib0mutex.h:516:6 (mysqld+0x1a8c206)
    #4 PolicyMutex<TTASEventMutex<GenericPolicy> >::enter(unsigned int, unsigned int, char const*, unsigned int) storage/innobase/include/ib0mutex.h:635:10 (mysqld+0x1a782c3)
    #5 fil_mutex_enter_and_prepare_for_io(unsigned long) storage/innobase/fil/fil0fil.cc:1131:3 (mysqld+0x1a9a92e)
    #6 fil_io(IORequest const&, bool, page_id_t const&, page_size_t const&, unsigned long, unsigned long, void*, void*, bool) storage/innobase/fil/fil0fil.cc:5082:2 (mysqld+0x1ab8de2)
    #7 buf_flush_write_block_low(buf_page_t*, buf_flush_t, bool) storage/innobase/buf/buf0flu.cc:1112:3 (mysqld+0x1cb970a)
    #8 buf_flush_page(buf_pool_t*, buf_page_t*, buf_flush_t, bool) storage/innobase/buf/buf0flu.cc:1270:3 (mysqld+0x1cb7d70)
    #9 buf_flush_try_neighbors(page_id_t const&, buf_flush_t, unsigned long, unsigned long) storage/innobase/buf/buf0flu.cc:1493:9 (mysqld+0x1cc9674)
    #10 buf_flush_page_and_try_neighbors(buf_page_t*, buf_flush_t, unsigned long, unsigned long*) storage/innobase/buf/buf0flu.cc:1565:13 (mysqld+0x1cbadf3)
    #11 buf_do_flush_list_batch(buf_pool_t*, unsigned long, unsigned long) storage/innobase/buf/buf0flu.cc:1825:3 (mysqld+0x1cbbcb8)
    #12 buf_flush_batch(buf_pool_t*, buf_flush_t, unsigned long, unsigned long, flush_counters_t*) storage/innobase/buf/buf0flu.cc:1895:16 (mysqld+0x1cbb459)
    #13 buf_flush_do_batch(buf_pool_t*, buf_flush_t, unsigned long, unsigned long, flush_counters_t*) storage/innobase/buf/buf0flu.cc:2065:2 (mysqld+0x1cbcfe1)
    #14 buf_flush_lists(unsigned long, unsigned long, unsigned long*) storage/innobase/buf/buf0flu.cc:2167:8 (mysqld+0x1cbd5a3)
    #15 log_preflush_pool_modified_pages(unsigned long) storage/innobase/log/log0log.cc:1400:13 (mysqld+0x1eefc3b)
    #16 log_make_checkpoint_at(unsigned long, bool) storage/innobase/log/log0log.cc:1751:10 (mysqld+0x1eefb16)
    #17 buf_dblwr_create() storage/innobase/buf/buf0dblwr.cc:335:2 (mysqld+0x1cd2141)
    #18 innobase_start_or_create_for_mysql() storage/innobase/srv/srv0start.cc:2539:10 (mysqld+0x21b4d8e)
    #19 innobase_init(void*) storage/innobase/handler/ha_innodb.cc:4193:8 (mysqld+0x1a5e3d7)
    #20 ha_initialize_handlerton(st_plugin_int*) sql/handler.cc:522:31 (mysqld+0xc74d33)
    #21 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) sql/sql_plugin.cc:1447:9 (mysqld+0x1376d5d)
    #22 plugin_init(int*, char**, int) sql/sql_plugin.cc:1729:15 (mysqld+0x13761c0)
    #23 init_server_components() sql/mysqld.cc:5348:7 (mysqld+0xc0d0ff)
    #24 mysqld_main(int, char**) sql/mysqld.cc:5943:7 (mysqld+0xc06f9d)
    #25 main sql/main.cc:25:10 (mysqld+0xbff71b)

WARNING: ThreadSanitizer: data race (pid=29031)
  Write of size 8 at 0x0000039e48e0 by thread T15:
    #0 srv_monitor_thread storage/innobase/srv/srv0srv.cc:1699:24 (mysqld+0x21a254e)

  Previous write of size 8 at 0x0000039e48e0 by thread T14:
    #0 srv_refresh_innodb_monitor_stats() storage/innobase/srv/srv0srv.cc:1165:24 (mysqld+0x21a3124)
    #1 srv_error_monitor_thread storage/innobase/srv/srv0srv.cc:1836:3 (mysqld+0x21a2d40)

  Location is global 'srv_last_monitor_time' of size 8 at 0x0000039e48e0 (mysqld+0x0000039e48e0)
2018-03-22 14:42:15 +04:00
Thirunarayanan Balathandayuthapani
b6d68c6aa3 MDEV-13561 Mariabackup is incompatible with retroactively created innodb_undo_tablespaces
- Mariabackup supports starting undo tablespace id which is greater
than 1.
2018-03-22 14:19:16 +05:30
Marko Mäkelä
03a80e20f7 pfs_os_file_read_no_error_handling_int_fd_func(): Remove a variable 2018-03-21 22:58:56 +02:00
Marko Mäkelä
3d7915f000 Merge 10.1 into 10.2 2018-03-21 22:58:52 +02:00
Jan Lindström
4629db0dd6 Fix test failure on galera_var_reject_queries. 2018-03-21 14:56:42 +02:00
Marko Mäkelä
ca9d9029e6 Partially revert commit 2a729b5f4b
Define CF_SKIP_WSREP_CHECK only once.
2018-03-21 12:32:38 +02:00
Marko Mäkelä
c704523195 Remove orphan wsrep_node_is_ready() 2018-03-21 11:58:17 +02:00
Vicențiu Ciorbaru
82aeb6b596 Merge branch '10.1' into 10.2 2018-03-21 10:36:49 +02:00
Alexey Botchkov
9652038453 MDEV-14533 Provide information_schema tables using which hardware
information can be obtained.

        DISKS plugin implementation added to the tree.
2018-03-21 12:33:38 +04:00
Marko Mäkelä
15051ab14a Disable a failing test 2018-03-21 08:13:43 +02:00
Marko Mäkelä
613be24b7a Merge 10.0 into 10.1 2018-03-20 19:25:08 +02:00
Marko Mäkelä
0492100059 Merge 5.5 into 10.0 2018-03-20 18:36:03 +02:00
Marko Mäkelä
69bc3c1976 PR #666: MDEV-15030 Add ASAN instrumentation
mem_heap_create_block(): Poison the payload area until
mem_heap_alloc() unpoisons it.
2018-03-20 18:18:57 +02:00
Marko Mäkelä
e0a0fe7d81 MDEV-12396 IMPORT TABLESPACE: Do not retry partial reads
fil_iterate(), fil_tablespace_iterate(): Replace os_file_read()
with os_file_read_no_error_handling().

os_file_read_func(), os_file_read_no_error_handling_func():
Do not retry partial reads. There used to be an infinite amount
of retries. Because InnoDB extends both data and log files upfront,
partial reads should be impossible during normal operation.
2018-03-20 15:31:39 +02:00
Marko Mäkelä
a80af35a85 MDEV-12396 IMPORT cleanup: ROW_FORMAT=COMPRESSED
Initialize block.page.zip only once.

PageConverter::update(): Initialize m_page_zip_ptr
as late as possible.
(We should really remove it at some point.)

PageConverter::operator(): Refer to block->page.zip instead of
m_page_zip_ptr.

AbstractCallback::get_frame(): Define static. Refer
to block->page.zip.data directly.

fil_iterate(): Refer to block->page.zip.data directly.

fil_tablespace_iterate(): Initialize block.page.zip.data as soon
as possible.
2018-03-20 15:31:39 +02:00
Marko Mäkelä
eaa7bfb59f MDEV-12396 IMPORT TABLESPACE: Simplify validation
fil_iterate(): Validate the pages directly.

import_page_status_t, PageConverter::validate(): Remove.

AbstractCallback::filename(): New accessor.

AbstractCallback::is_interrupted(): Replaces periodic_check().

PageConverter::trigger_corruption(): Remove.
2018-03-20 15:31:39 +02:00
Marko Mäkelä
6247c64c2a MDEV-12396 IMPORT TABLESPACE cleanup
Reduce unnecessary inter-module calls for IMPORT TABLESPACE.
Move some IMPORT-related code from fil0fil.cc to row0import.cc.

PageCallback: Remove. Make AbstractCallback the base class.

PageConverter: Define some member functions inline.
2018-03-20 15:31:39 +02:00
Thirunarayanan Balathandayuthapani
eee73ddfbb MDEV-12255 innodb_prefix_index_cluster_optimization hits debug build
assert on UTF-8 columns

Problem:
=======
(1) Multi-byte character cases are not considered during prefix index
cluster optimization check. It leads to fetch of improper results during
read operation.
(2) Strict assert in row_sel_field_store_in_mysql_format_func and it asserts
for prefix index record to mysql conversion.

Solution:
========
(1) Consider the case of multi-byte character during prefix index
cluster optimization check.
(2) Relax the assert in row_sel_field_store_in_mysql_format_func to allow
prefix index record to mysql format conversion.

The patch is taken from
1eee538087
2018-03-20 17:53:33 +05:30
Jan Lindström
bc2e7d7889 Fix test case MW-329. 2018-03-20 12:10:41 +02:00
Philip Stoev
33028f7c4b Refs: MW-245 - Adjust tests to account for the new behavior. 2018-03-20 12:10:41 +02:00
sjaakola
9a89614857 Refs: MW-245 - changed logic so that in non primary node it is possible to do SET + SHOW + SELECT from information and pfs schema, when dirty reads are not enabled - however, non table selects are not allowed (e.g. SELECT 1) 2018-03-20 12:10:41 +02:00
sjaakola
c5dd2abf4c Refs MW-245 - logic was wrong in detecting if queries are allowed in non primary node. it allowed select with no table list to execute even if dirty reads was not specified 2018-03-20 12:10:41 +02:00
Philip Stoev
84d4ab5be1 refs MW-245: Galera MTR Tests: additional tests for wsrep_reject_queries, wsrep_dirty_reads 2018-03-20 12:10:41 +02:00
sjaakola
8f717ed360 refs MW-245 - allowing USE <database> with dirty reads configuration - fix for logic of setting wsrep ready status 2018-03-20 12:10:41 +02:00
sjaakola
2a729b5f4b refs MW-245 - merged wsrep_dirty_reads and wsrep_reject_queries from PXC 2018-03-20 12:10:41 +02:00
Eugene Kosov
5a8f8f89d6 honor alignment rules and xtradb too 2018-03-20 10:46:57 +03:00
Varun Gupta
e390f7b675 MDEV-12737: tokudb_mariadb.mdev6657 fails in buildbot with different plan, and outside with valgrind warnings
Fixing the test by adding replace column for the rows column in the explain.
2018-03-20 11:05:28 +05:30
Jacob Mathew
7cf2428db3 MDEV-10991: Server crashes in spider_udf_direct_sql_create_conn - tests in spider/oracle* suites crash the server
The crash occurs due to code that is #ifdef'd out with HAVE_ORACLE_OCI that
pertains to the use of Spider with an Oracle data tier. Enabling this code
eliminates the crash.

The reason that MariaDB needs to support Oracle storage at the data tier is
to help customers migrate from Oracle. It is necessary to build Spider with
the additional build flag -DHAVE_ORACLE_OCI, and install and start Oracle
before running the Oracle test suite or any tests within it. Nevertheless,
if Spider is built normally and Oracle has not been started, these tests
should not cause the MariaDB server to crash. The bug fix replaces the
crash with the following error:
  ERROR 12501 (HY000) at line 4: The connect info 'ORACLE' is invalid

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.
2018-03-19 12:49:22 -07:00
Vicențiu Ciorbaru
24b353162f Merge branch '10.0-galera' into 10.1 2018-03-19 15:21:01 +02:00
Eugene Kosov
75c76dbb06 MDEV-15030 Add ASAN instrumentation
Learn both valgrind and asan to catch this bug:

  mem_heap_t* heap = mem_heap_create(1024);
  byte* p = reinterpret_cast<byte*>(heap) + sizeof(mem_heap_t);
  *p = 123;

Overflows of the last allocation in a block will be catched too.

mem_heap_create_block(): poison newly allocated memory
2018-03-19 16:18:53 +03:00
Alexander Barkov
f538a64817 MDEV-15005 ASAN: stack-buffer-overflow in my_strnncollsp_simple
cmp_item_sort_string::store_value() did not cache the string returned
from item->val_str(), whose result can point to various private members
such as Item_char_typecast::tmp_value.

- cmp_item_sort_string::store_value() remembered the pointer returned
  from item->val_str() poiting to tmp_value into cmp_item_string::value_res.
- Later, cmp_item_real::store_value() was called, which called
  Item_str_func::val_real(), which called Item_char_typecast::val_str(&tmp)
  using a local stack variable "String tmp". Item_char_typecast::tmp_value
  was overwritten and become a link to "tmp":
  tmp_value.Ptr freed its own buffer and set to point to the buffer
  owned by "tmp".
- On return from Item_str_func::val_real(), "String tmp" was destructed,
  but "tmp_value" still pointed to the buffer owned by "tmp",
  So tmp_value.Ptr became invalid.
- Then cmp_item_sort_string() passed cmp_item_string::value_res to sortcmp().
  At this point, value_res still pointed to an invalid value of
  Item_char_typecast::tmp_value.

Fix:
changing cmp_item_sort_string::store_value() to force copying
to cmp_item_string::value if item->val_str(&value) returned
a different pointer (instead of &value).
2018-03-19 13:07:41 +04:00
Jan Lindström
31e2ab513d MDEV-13549: Galera test failures
Fix test failure on galera_flush_local.
2018-03-19 10:01:40 +02:00
Jan Lindström
f46155a31b MDEV-13549: Galera test failures
Fix test failure on galera_concurrent_ctas
2018-03-19 09:09:02 +02:00
Jan Lindström
e5e83249c1 MDEV-14875: galera_new_cluster crashes mysqld when existing server contains databases
Fortify wsrep_hton so that wsrep calls are not done to NULL-pointers.
2018-03-19 08:48:14 +02:00
Vicențiu Ciorbaru
34b03da211 Update Connector/C 2018-03-18 15:11:48 +02:00
Vicențiu Ciorbaru
a0f9cbc931 Merge remote-tracking branch 'connect/10.2' into 10.2 2018-03-18 15:05:27 +02:00
Varun Gupta
e3dd9a95e5 MDEV-6736: Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ in WHERE and
HAVING, ORDER BY, materialization+semijoin

During cleanup a pointer to the materialised table that was freed was not set to NULL
2018-03-17 10:51:19 +05:30
Thirunarayanan Balathandayuthapani
6d1d5c3aeb MDEV-14545 Backup fails due to MLOG_INDEX_LOAD record
- Fixed the asan failure of the unsupported_redo test case
2018-03-16 20:55:55 +05:30
Marko Mäkelä
27c54b77c1 Make some locking primitives inline
lock_rec_trx_wait(): Merge to the only caller lock_prdt_rec_move().

lock_rec_reset_nth_bit(), lock_set_lock_and_trx_wait(),
lock_reset_lock_and_trx_wait(): Define in lock0priv.h.
2018-03-16 15:50:05 +02:00
Marko Mäkelä
d2a15092c1 lock_table_create(), lock_rec_create(): Clean up the WSREP code
By definition, c_lock->trx->lock.wait_lock==c_lock cannot hold.
That is, the owner transaction of a lock cannot be waiting for that
particular lock. It must have been waiting for some other lock.
Remove the dead code related to that. Also, test c_lock for NULLness
only once.
2018-03-16 15:50:05 +02:00
Marko Mäkelä
61e192fa40 lock_reset_lock_and_trx_wait(): Remove diagnostics 2018-03-16 15:50:05 +02:00
Marko Mäkelä
27d4333cb9 MDEV-13935 INSERT stuck at state Unlocking tables
Refactor lock_grant(). With innodb_lock_schedule_algorithm=VATS
some callers were passing an incorrect parameter owns_trx_mutex
to lock_grant().

lock_grant_after_reset(): Refactored from lock_grant(), without
the call to lock_reset_lock_and_trx_wait().

lock_grant_have_trx_mutex(): A variant of lock_grant() where the
caller already holds the lock->trx->mutex. The normal lock_grant()
will acquire and release lock->trx->mutex.

lock_grant(): Define as a wrapper that will acquire lock->trx->mutex.
2018-03-16 15:50:04 +02:00
Marko Mäkelä
f93a219c72 MDEV-13935 INSERT stuck at state Unlocking tables
lock_rec_queue_validate(): Restore some assertions.

DeadlockChecker::select_victim(): Reduce the WSREP-related diff.
2018-03-16 15:50:04 +02:00