Commit graph

189958 commits

Author SHA1 Message Date
Marko Mäkelä
9159b8976f MDEV-22871: Clean up hash_table_t
HASH_TABLE_SYNC_MUTEX was kind-of used for the adaptive hash index,
even though that hash table is already protected by btr_search_latches[].

HASH_TABLE_SYNC_RWLOCK was only being used for buf_pool.page_hash.
It is cleaner to decouple that synchronization from hash_table_t,
and move it to the actual user.

buf_pool_t::page_hash_latches[]: Synchronization for buf_pool.page_hash.

LATCH_ID_HASH_TABLE_MUTEX: Remove.

hash_table_t::sync_obj, hash_table_t::n_sync_obj: Remove.

hash_table_t::type, hash_table_sync_t: Remove.

HASH_ASSERT_OWN(), hash_get_mutex(), hash_get_nth_mutex(): Remove.

ib_recreate(): Merge to the only caller, buf_pool_resize_hash().

ib_create(): Merge to the callers.

ha_clear(): Merge to the only caller buf_pool_t::close().

buf_pool_t::create(): Merge the ib_create() and
hash_create_sync_obj() invocations.

ha_insert_for_fold_func(): Clarify an assertion.

buf_pool_t::page_hash_lock(): Simplify the logic.

hash_assert_can_search(), hash_assert_can_modify(): Remove.
These predicates were only being invoked for the adaptive hash index,
while they only are effective for buf_pool.page_hash.

HASH_DELETE_AND_COMPACT(): Merge to ha_delete_hash_node().

hash_get_sync_obj_index(): Remove.

hash_table_t::heaps[], hash_get_nth_heap(): Remove. It was actually unused!

hash_get_heap(): Remove. It was only used in ha_delete_hash_node(),
where we always use hash_table_t::heap.

hash_table_t::calc_hash(): Replaces hash_calc_hash().
2020-06-18 14:16:01 +03:00
Daniel Black
08f6513cb2 libutils: merge_archives_unix
MRI scripts cannot handle + in paths, and ubuntu CI makes use of
these.

So we remove the top level build dir from the script and
transform it into a relative path script.
2020-06-18 14:15:36 +03:00
Daniel Black
38774f8dcb libutils: merge static libraries only once
Because of common dependencies between the
static libraries list can contain duplicates.

We reduce these down to the single last one in
the list.

This reduces the relative time of a rebuild from:

$ (cd builddir/; time make -j)
...
real	0m30.789s
user	1m33.477s
sys	0m19.678s

and the LIB entries
$ grep ADDLIB builddir/libmysqld/mysqlserver-\$\<CONFIG\>.mri.tpl  | wc -l
179

$ du -h builddir/libmysqld/libmariadbd.a
4.1G	builddir/libmysqld/libmariadbd.a

To:

$ (cd builddir/; time make -j)
...
real	0m20.139s
user	1m32.423s
sys	0m12.208s

$ grep ADDLIB builddir/libmysqld/mysqlserver-\$\<CONFIG\>.mri.tpl  | wc -l
25

$ du -h builddir/libmysqld/libmariadbd.a
688M	builddir/libmysqld/libmariadbd.a
2020-06-18 14:10:57 +03:00
Marko Mäkelä
c515b1d092 Merge 10.4 into 10.5 2020-06-18 13:58:54 +03:00
Vlad Lesin
205b0ce6ad MDEV-22894: Mariabackup should not read [mariadb-client] option group
from configuration files
2020-06-18 12:19:48 +03:00
Vlad Lesin
0121a9e0bb MDEV-18215: mariabackup does not report unknown command line options
Post-push fix: add mysqd options in backup string in mariabackup sst
script for the case of logging not in syslog.
2020-06-18 12:19:48 +03:00
Marko Mäkelä
01ed614027 Fix the test mariabackup.mdev-14447
The test mariabackup.mdev-14447 started failing due to the option
--apply-log-only that became invalid since
commit 9bdf35e90f
and had been removed in
commit 8c71c6aa8b.
2020-06-18 12:13:31 +03:00
Sergei Golubchik
baff3ba6e3 S3 compilation error on x86 2020-06-18 09:44:05 +02:00
Sergei Golubchik
7c0cf20444 update libmarias3 2020-06-18 09:44:05 +02:00
Sergei Golubchik
7bb32cda05 more "removed" mysqld command-line options
and put them all together in mysqld.cc
2020-06-18 09:44:05 +02:00
Vladislav Vaintroub
083a344760 Fix error in cmake, when trying gcc on Windows. 2020-06-18 00:57:23 +02:00
Vladislav Vaintroub
e54723fa34 When compiling with RelWithDebInfo, always use -fno-omit-frame-pointer
(if this option is present). perf, and other tools that do stack walking
works much better with it.
2020-06-18 00:55:11 +02:00
Thirunarayanan Balathandayuthapani
d33c9ca1b0 MDEV-22902 Assertion `!page_has_siblings(block->frame)' failed in btr_pcur_store_position
- There is a possiblity that metadata record is the only record in
the leftmost leaf page. So change the assertion to check only
previous page.
2020-06-17 22:09:16 +05:30
Sergei Petrunia
b7324e133f Remove redundant code in opt_range.cc: print_key_value() 2020-06-17 19:30:19 +03:00
Thirunarayanan Balathandayuthapani
815fc98732 MDEV-22904 Compressed row format table tries to access freed blob
- This is a regression of MDEV-21174(56f6dab1d0).
InnoDB resets the BTR_EXTERN_LEN value at wrong offset.
2020-06-17 21:04:18 +05:30
Alexey Yurchenko
471d7a9762 Fix include statements in galera_ipv6_mariabackup_section and
galera_ipv6_mariabackup MTR tests
2020-06-17 13:31:46 +03:00
MikkoJaakola
beb1918354 MDEV-21759 galera.galera_parallel_autoinc_manytrx sporadic failures.
The galera.galera_parallel_autoinc_manytrx mtr test opens and runs test
scenario through 3 connections to node 1 and one connection to node 2.
In the test initialization phase, the test creates two tables 't1' and 'ten'
and then creates a stored procedure 'p1' to operate on these tables.
These 3 create DDL statements are issued through same connection to node 1.

In the next test phase, the mtr script uses send command to launch the call
for the p1 stored procedure through all 3 connections to node 1 and through
one connection to node 2. As the mtr send command is asynchronous,
this test phase is non blocking and fast operation.
Now, if the replication between nodes is slow, it may happen that the
initialization phase DDL statements have not been received or have not been
fully applied in node 2. Therefore there is no guarantee that the test tables
and the stored procedure have been created in node 2. Yet, the test is trying
to call p1 in node 2.

In the failure case error logs, there is error message
"MTR failed: query 'reap' failed: 1305: PROCEDURE test.p1 does not exist"

The reap command through connection to node 2, is the first place where test
execution may observe that test tables and/or stored procedure are not yet
created in node 2.

The fix in this commit adds a wait condition in connection to node 2, to wait
until the stored procedure is created before calling the stored procedure.
The wait is implemented by looking in information_schema.routines for the p1
stored procedure.
2020-06-17 13:31:37 +03:00
Jan Lindström
8655d4a202 Add global ignore for Sending JOIN failed warning. 2020-06-17 13:04:15 +03:00
Jan Lindström
bf039b9127 MDEV-22125 : galera.galera_drop_multi MTR failed: InnoDB: MySQL is trying to drop database fts.`` though there are still open handles
MDEV-22140 galera.galera_drop_database MTR failed: InnoDB: MySQL is trying to drop database `fts`.`` though there are still open handles

Add wait conditions to wait that all operations are done in both
nodes.
2020-06-17 12:58:33 +03:00
Vladislav Vaintroub
9c577c2b90 MDEV-22917 wolfssl might crash at startup when both SSL and encryption plugin are enabled
Make sure to initialize SSL early enough, when encryption plugins is loaded
2020-06-17 08:35:03 +02:00
Krunal Bauskar
a0d598a4d2 MDEV-22794: Avoid potential rollback segment contention with
increased scalability through even distribution

Rollback segments are allocated to transactions in round-robin fashion.
This is controlled by incrementing a static-scope counter named rseg_slot.

Said logic is not protected by any mutex or use of atomic for the counter.
This potentially can cause the same rollback segment to get allocated to
N different transactions (requesting allocation at the same time).
While this is not an issue as a rollback segment can host multiple
transactions from contention (performance) perspective it is better to
allocate these rollback segments in round-robin fashion.

Fix for the said issue ports use of atomic for the said counter that would
ensure the original design semantic (even distribution through round-robin)
is retained.
2020-06-17 09:12:56 +03:00
Sachin
592a10d079 MDEV-22370 safe_mutex: Trying to lock uninitialized mutex at /data/src/10.4-bug/sql/rpl_parallel.cc, line 470 upon shutdown during FTWRL
Problem:- When we issue FTWRL with shutdown in parallel, there is race between
FTWRL and shutdown. Shutdown might destroy the mutex (pool->LOCK_rpl_thread_pool)
before FTWRL can lock it. So we can get crash on FTWRL thread

Solution:- mysql_mutex_destroy(pool->LOCK_rpl_thread_pool) should wait for
FTWRL thread to complete its work , and then destroy.
So slave_prepare_for_shutdown will just deactivate the pool, and mutex is destroyed
later in end_slave()
2020-06-17 02:22:46 +05:30
MikkoJaakola
0128e13e62 MDEV-21759 galera.galera_parallel_autoinc_manytrx sporadic failures.
The galera.galera_parallel_autoinc_manytrx mtr test opens and runs test
scenario through 3 connections to node 1 and one connection to node 2.
In the test initialization phase, the test creates two tables 't1' and 'ten'
and then creates a stored procedure 'p1' to operate on these tables.
These 3 create DDL statements are issued through same connection to node 1.

In the next test phase, the mtr script uses send command to launch the call
for the p1 stored procedure through all 3 connections to node 1 and through
one connection to node 2. As the mtr send command is asynchronous,
this test phase is non blocking and fast operation.
Now, if the replication between nodes is slow, it may happen that the
initialization phase DDL statements have not been received or have not been
fully applied in node 2. Therefore there is no guarantee that the test tables
and the stored procedure have been created in node 2. Yet, the test is trying
to call p1 in node 2.

In the failure case error logs, there is error message
"MTR failed: query 'reap' failed: 1305: PROCEDURE test.p1 does not exist"

The reap command through connection to node 2, is the first place where test
execution may observe that test tables and/or stored procedure are not yet
created in node 2.

The fix in this commit adds a wait condition in connection to node 2, to wait
until the stored procedure is created before calling the stored procedure.
The wait is implemented by looking in information_schema.routines for the p1
stored procedure.
2020-06-16 11:43:31 +03:00
Alexey Yurchenko
49ac606a75 Fix include statements in galera_ipv6_mariabackup_section and
galera_ipv6_mariabackup MTR tests
2020-06-15 12:53:16 +03:00
Vlad Lesin
9bdf35e90f MDEV-18215: mariabackup does not report unknown command line options
MDEV-21298: mariabackup doesn't read from the [mariadbd] and [mariadbd-X.Y]
server option groups from configuration files
MDEV-21301: mariabackup doesn't read [mariadb-backup] option group in
configuration file

All three issues require to change the same code, that is why their
fixes are joined in one commit.

The fix is in invoking load_defaults_or_exit() and handle_options() for
backup-specific groups separately from client-server groups to let the last
handle_options() call fail on unknown backup-specific options.

The order of options procesing is the following:
1) Load server groups and process server options, ignore unknown
options
2) Load client groups and process client options, ignore unknown
options
3) Load backup groups and process client-server options, exit on
unknown option
4) Process --mysqld-args command line options, ignore unknown options

New global flag my_handle_options_init_variables was added to have
ability to invoke handle_options() for the same allowed options set
several times without re-initialising previously set option values.

--password value destroying is moved from option processing callback to
mariabackup's handle_options() function to have ability to invoke server's
handle_options() several times for the same possible allowed options
set.

Galera invokes wsrep_sst_mariabackup.sh with mysqld command line
options to configure mariabackup as close to the server as possible.
It is not known what server options are supported by mariabackup when the
script is invoked. That is why new mariabackup option "--mysqld-args" is added,
all unknown options that follow this option will be silently ignored.

wsrep_sst_mariabackup.sh was also changed to:
- use "--mysqld-args" mariabackup option to pass mysqld options,
- remove deprecated innobackupex mode,
- remove unsupported mariabackup options:
    --encrypt
    --encrypt-key
    --rebuild-indexes
    --rebuild-threads
2020-06-14 13:23:07 +03:00
Kentoku SHIBA
bd62a636a4 fix a compiler warning on GCC 9.3.0 2020-06-14 18:40:07 +09:00
Thirunarayanan Balathandayuthapani
d0c69ccab5 MDEV-22911: Fix the valgrind & MSAN instrumentation of MDEV-8139
MEM_GET_VBITS(): Save information about uninitialized data.

MEM_SET_VBITS(): Restore information about uninitialized data.
2020-06-16 20:03:35 +05:30
Vladislav Vaintroub
72fc4f3fef MDEV-22841 ut_new_get_key_by_file is unnecessarily expensive, followup
Make ut_new_get_key_by_file event less expensive
remove binary search, compute auto_event_keys offset at compile time.
2020-06-16 12:12:37 +02:00
Otto Kekäläinen
7803601dcb MDEV-22569: Run bin/mariadbd instead of bin/mysqld
Replace all references to /usr/sbin/mysqld (and bin and libexec) with
mariadbd, so that the binary server will always be 'mariadbd'.

Also update all places that reference the server binary in other ways,
such as AppArmor profiles and scripts that previously expected to find
a 'mysqld' in process lists.
2020-06-16 12:59:30 +03:00
Alexey Botchkov
e290e5a75d MDEV-22837 JSON_ARRAYAGG and JSON_OBJECTAGG treat JSON arguments as text.
Item_field::is_json_value() implemented.
2020-06-15 23:21:29 +04:00
Alexey Botchkov
6c573a9146 MDEV-22844 JSON_ARRAYAGG is limited by group_concat_max_len.
Warning message and function result fixed
2020-06-15 22:51:21 +04:00
Aleksey Midenkov
30d41c8102 MDEV-22881 Unexpected errors, corrupt output, Valgrind / ASAN errors in Item_ident::print or append_identifier
After this code

end_inplace:

  if (thd->locked_tables_list.reopen_tables(thd, false))
    goto err_with_mdl_after_alter;

table is not reopened (need_reopen is false) but
some_table_marked_for_reopen is reset to false.

Item_field is allocated on table lock and assigned new name on first
ALTER which is then freed at the end of the command. Second ALTER
accessess this Item_field and gets garbage value.
2020-06-15 19:10:39 +03:00
Vicențiu Ciorbaru
359d5f56c3 Fix drop_combinations for embedded
Make sure to replace the datadir absolute path.
2020-06-15 17:30:26 +03:00
Marko Mäkelä
4be4d57d36 MDEV-22887 unresolved symbol crc32c_vpmsum on Ubuntu Xenial
This is one more follow-up fix to MDEV-22641.

Explicitly specify the dependency of the innobase library on mysys.

Also, remove stale references to CRC32_LIBRARY, which should have
been removed in commit dec3f8ca69.
2020-06-15 15:26:16 +03:00
Oleksandr Byelkin
4080e3acef fixup of "Make error messages from DROP TABLE and DROP TABLE IF EXISTS consistent" results 2020-06-15 12:28:34 +02:00
Sergei Petrunia
517e9334f2 MDEV-22891: Optimizer trace: const tables are not clearly visible
Make mark_join_nest_as_const() print its action into the trace.
2020-06-15 13:00:43 +03:00
Monty
c9f5cb97af Added checks for uninitalized memory when writing to IO_CACHE
This was done to be able to track some cases of unallocated memory
in replication tests reported by MSAN.
2020-06-14 19:39:43 +03:00
Monty
e843033d02 Created a workaround for a bug in MSAN for va_arg(,double)
MDEV-22691 MSAN use-of-uninitialized-value in test maria.maria-recovery2

This caused all my_vsnprintf() using doubles to fail.
Thanks to the workaround, I was able to remove the disabling of
MSAN in dtoa().
2020-06-14 19:39:43 +03:00
Oleksandr Byelkin
d7a9cdc627 Fixed hang in Aria page cache with concurrent SELECT
MDEV-20302 Server hangs upon concurrent SELECT from partitioned S3
2020-06-14 19:39:43 +03:00
Monty
b3179b7e32 Cleaned up compile failure if DEBUG_SYNC is disabled in DEBUG builds 2020-06-14 19:39:43 +03:00
Monty
7f941c4ef6 Fixed bug in trans_check() where on error we gave wrong return value 2020-06-14 19:39:43 +03:00
Monty
56045ef94a Fix for crash in Aria LOCK TABLES + CREATE TRIGGER
MDEV-22829 SIGSEGV in _ma_reset_history on LOCK
2020-06-14 19:39:43 +03:00
Monty
ab7eedc185 Fixed typos in aria_read_log 2020-06-14 19:39:43 +03:00
Monty
654b593149 BINLOG with LOCK TABLES and SAVEPOINT could cause a crash in debug bin
MDEV-22048 Assertion `binlog_table_maps == 0 ||
           locked_tables_mode == LTM_LOCK_TABLES' failed in
           THD::reset_for_next_command
2020-06-14 19:39:43 +03:00
Monty
6a3b581b90 MDEV-19745 BACKUP STAGE BLOCK_DDL hangs on flush sequence table
Problem was that FLUSH TABLES where trying to read latest sequence state
which conflicted with a running ALTER SEQUENCE. Removed the reading
of the state, when opening a table for FLUSH, as it's not needed in this
case.

Other thing:
- Fixed a potential issue with concurrently running ALTER SEQUENCE where
  the later ALTER could potentially read old data
2020-06-14 19:39:43 +03:00
Monty
08d475c73b Fixed core dump in "echo shutdown | mysqld --bootstrap" 2020-06-14 19:39:43 +03:00
Monty
1cca83784f Updated code comments 2020-06-14 19:39:43 +03:00
Monty
d35616aab3 Fixed crash in failing instant alter table with partitioned table
MDEV-22649 SIGSEGV in ha_partition::create_partitioning_metadata on ALTER
MDEV-22804 SIGSEGV in ha_partition::create_partitioning_metadata
2020-06-14 19:39:42 +03:00
Monty
10b88deb74 Changes needed for ColumnStore and insert cache
MCOL-3875 Columnstore write cache

The main change is to change thr_lock function get_status to
return a value that indicates we have to abort the lock.

Other thing:
- Made start_bulk_insert() and end_bulk_insert() protected so that the
  insert cache can use these
2020-06-14 19:39:42 +03:00
Monty
74df3c8024 Changed some DBUG_PRINT that used error:
The reson for the change was to make it easier to find true errors
when searching in trace logs.
"error:" should mainly be used when we have a real error
2020-06-14 19:39:42 +03:00