Commit graph

190155 commits

Author SHA1 Message Date
Otto Kekäläinen
55c7536a02 Add man page for new command aria_s3_copy
Based on output of 'aria_s3_copy --help'.

Also fix a typo in command help.
2020-07-08 12:24:55 +03:00
Eugene Kosov
0e86254b05 MDEV-22930 Unnecessary contention on rw_lock_list_mutex in ibuf_dummy_index_create()
1. Do not initialize dict_table_t::stats_latch in ibuf
2. Remove overengineering in GenericPolicy to speed up things

dict_mem_table_create(): add new argument init_stats_latch

ibuf_dummy_index_create(): do not initialize dict_table_t::stats_latch

GenericPolicy: add new members m_filename and m_line

sync_file_create_register()
sync_file_created_deregister()
sync_file_created_get()
CreateTracker: remove

rw_lock_t::created: a new debug member
2020-07-08 04:08:30 +03:00
Eugene Kosov
1ae008d272 replace UT_LIST with ilist in rw_lock_list_t 2020-07-07 09:47:30 +03:00
Eugene Kosov
ee5841376a fix clang compilation 2020-07-06 22:03:58 +03:00
Oleksandr Byelkin
846174c5ba MDEV-23071 remove potentially dangerouws casting to Item_in_subselect
Remove types casting with a help of virtual functions.
2020-07-06 09:13:15 +02:00
Marko Mäkelä
79c166c56d Fix cmake -DWITH_MSAN=ON
commit ab4069909d accidentally
misplaced #endif directives.
2020-07-05 20:38:50 +03:00
Marko Mäkelä
ab4069909d After-merge fix for ASAN and MSAN
The merge commit 0fd89a1a89
of commit b6ec1e8bbf
seems to cause occasional MemorySanitizer failures,
because it failed to replace some MEM_UNDEFINED() calls
with MEM_MAKE_ADDRESSABLE().

my_large_free(): Correctly invoke MEM_MAKE_ADDRESSABLE() after
freeing memory. Failure to do so could cause bogus
AddressSanitizer failures for memory allocated by my_large_malloc().
On MemorySanitizer, we will do nothing.

buf_pool_t::chunk_t::create(): Replace the MEM_MAKE_ADDRESSABLE()
that had been added in commit 484931325e
to work around the issue.
2020-07-05 16:25:29 +03:00
Marko Mäkelä
90d5d90640 Merge 10.4 into 10.5 2020-07-04 22:19:49 +03:00
Marko Mäkelä
b99fa1e767 Merge 10.3 into 10.4 2020-07-04 22:11:16 +03:00
Marko Mäkelä
453dc4b300 Fixup the parent commit for MSAN and Valgrind
commit 484931325e was a necessary
fix for the buffer pool resizing tests in 10.5 in
AddressSanitizer. However, that change would break the tests
innodb.innodb_buffer_pool_resize and
innodb.innodb_buffer_pool_resize_with_chunks
when run in MemorySanitizer, or presumably in Valgrind as well.
(Those tests run "forever" in Valgrind.)

buf_pool_resize(): Cancel the effect of MEM_NOACCESS() in Valgrind
and ASAN. In MSAN, MEM_NOACCESS() is a no-op, and hence we must do
nothing special here.

MEM_MAKE_ADDRESSABLE() would declare the memory contents undefined.
In this particular case, we must actually declare the contents
defined for Valgrind.
2020-07-04 22:02:30 +03:00
Marko Mäkelä
a85f81af03 MDEV-22535 fixup: Define a single-caller function inline
Let us avoid any overhead in release builds, for an empty function.
2020-07-04 14:28:11 +03:00
Marko Mäkelä
2d00e003b2 After-merge fixes for ASAN
The merge commit 0fd89a1a89
of commit b6ec1e8bbf
was slightly incomplete.

ReadView::mem_valid(): Use the correct primitive
MEM_MAKE_ADDRESSABLE(), because MEM_UNDEFINED() now has
no effect on ASAN.

recv_sys_t::alloc(), recv_sys_t::add(): Use MEM_MAKE_ADDRESSABLE()
instead of MEM_UNDEFINED(), to get the correct behaviour for ASAN.
For Valgrind and MSAN, there is no change in behaviour.

recv_sys_t::free(), recv_sys_t::clear(): Before freeing memory to
buf_pool.free_list, invoke MEM_MAKE_ADDRESSABLE() on the entire
buf_block_t::frame, to cancel the effect of MEM_NOACCESS() in
recv_sys_t::alloc().
2020-07-04 14:28:11 +03:00
Sergei Golubchik
478591608e optimize ha_delete_table_force
first try discovering engines, then the rest.
otherwise every DROP TABLE non_existent; will do
lots of i/o trying to remove .MYI/.MYD/.MAI/.MAD/.CSV/etc files

this matches the old behavior where DROP TABLE always tried to discover
the table before dropping.
2020-07-04 01:44:47 +02:00
Sergei Golubchik
4227dd2ac6 continue DROP TEMPORARY TABLE t1, t2, t3 after error.
normal DROP TABLE with many tables continues after an error,
trying to drop as many tables as possible. But DROP TEMPORARY TABLE
was aborting on the first error. Change it to behave as DROP TABLE does.
2020-07-04 01:44:47 +02:00
Sergei Golubchik
6c52931680 replace HTON_AUTOMATIC_DELETE_TABLE with return -1 from drop_table() 2020-07-04 01:44:47 +02:00
Sergei Golubchik
4876651e0f remove mysql_declare_plugin declaration from some plugins 2020-07-04 01:44:47 +02:00
Sergei Golubchik
7c2ba9e9d7 MDEV-11412 Ensure that table is truly dropped when using DROP TABLE
don't do table discovery on DROP. DROP falls back to "force"
approach when a table isn't found and will try to drop in all
engines anyway. That is, trying to discover in all engines before
the drop is redundant and may be expensive.
2020-07-04 01:44:47 +02:00
Sergei Golubchik
79a3f96166 rewrite bug#26704 test case
drop-no_root needs DROP DATABASE to fail.
But `chmod 000` is not reliable after drop table force anymore.

Make DROP DATABASE to fail by creating an extra file in the db dir
2020-07-04 01:44:47 +02:00
Sergei Golubchik
529b6dffe9 small cleanup 2020-07-04 01:44:46 +02:00
Sergei Golubchik
35f566db8d cleanup: make dd_frm_type to work as documented
remove redundant argument, return all possible enum values
2020-07-04 01:44:46 +02:00
Sergei Golubchik
2bb5981c20 MDEV-11412 Ensure that table is truly dropped when using DROP TABLE
minor post-review fixes
* remove duplicate tests
* first file indicates table existance even with discovery
* don't duplicate trigger dropping code
2020-07-04 01:44:46 +02:00
Sergei Golubchik
b014720b6c optimization: use hton->drop_table in few simple cases 2020-07-04 01:44:46 +02:00
Sergei Golubchik
c55c292832 introduce hton->drop_table() method
first step in moving drop table out of the handler.
todo: other methods that don't need an open table

for now hton->drop_table is optional, for backward compatibility
reasons
2020-07-04 01:44:46 +02:00
Sergei Golubchik
f17f7a43ba test dropping of a MEMORY table 2020-07-04 01:44:46 +02:00
Sergei Golubchik
d2b852b4ca cleanup, less #ifdef's 2020-07-04 01:44:46 +02:00
Monty
7d75e43261 Disable rpl_parallel2 temporarly until we have a proper fix for it in 10.5 2020-07-04 02:19:02 +03:00
Monty
0fd89a1a89 Merge remote-tracking branch 'origin/10.4' into 10.5 2020-07-03 23:31:12 +03:00
Qi Wu
70684afef2 Atomic write support for ScaleFlux NVMe SSD's 2020-07-03 22:42:10 +03:00
Monty
e9f06b19e0 Merge remote-tracking branch 'origin/10.3' into 10.4 2020-07-03 20:26:09 +03:00
Monty
484931325e Fix for MSAN from Marko related to buf_pool_resize 2020-07-03 18:37:33 +03:00
Julius Goryavsky
1bf863a91a Merge branch '10.4-MDEV-22222' of https://github.com/codership/mariadb-server into 10.4-MDEV-22222 2020-07-03 16:17:59 +02:00
Sergei Golubchik
7a4afad969 compilation fix
include/my_valgrind.h:88:112: error: ‘void* memset(void*, int, size_t)’ writing to an object of non-trivial type ‘key_map’ {aka ‘class Bitmap<64>’}; use assignment instead [-Werror=class-memaccess]

in this case it's safe, Bitmap<> is trivial enough
2020-07-03 15:01:21 +02:00
Monty
e6595a06d6 Don't give errors for default value copy in create_tmp_table 2020-07-03 01:18:51 +03:00
Monty
6e81ba0c12 Don't give errors for default value copy in create_tmp_table 2020-07-03 01:16:31 +03:00
Monty
5211af1c16 Merge remote-tracking branch 'origin/10.3' into 10.4 2020-07-03 00:35:28 +03:00
Monty
53ecc354e3 Fixed errors found by MSAN 2020-07-02 23:50:56 +03:00
Marko Mäkelä
b6ec1e8bbf MDEV-20377 post-fix: Introduce MEM_MAKE_ADDRESSABLE
In AddressSanitizer, we only want memory poisoning to happen
in connection with custom memory allocation or freeing.

The primary use of MEM_UNDEFINED is for declaring memory uninitialized
in Valgrind or MemorySanitizer. We do not want MEM_UNDEFINED to
have the unwanted side effect that AddressSanitizer would no longer
be able to complain about accessing unallocated memory.

MEM_UNDEFINED(): Define as no-op for AddressSanitizer.

MEM_MAKE_ADDRESSABLE(): Define as MEM_UNDEFINED() or
ASAN_UNPOISON_MEMORY_REGION().

MEM_CHECK_ADDRESSABLE(): Wrap also __asan_region_is_poisoned().
2020-07-02 17:59:28 +03:00
Monty
65f831d17c Fixed bugs found by valgrind
- Some of the bug fixes are backports from 10.5!
- The fix in innobase/fil/fil0fil.cc is just a backport to get less
  error messages in mysqld.1.err when running with valgrind.
- Renamed HAVE_valgrind_or_MSAN to HAVE_valgrind
2020-07-02 17:57:34 +03:00
Monty
6cee9b1953 MDEV-22535 TABLE::initialize_quick_structures() takes 0.5% in oltp_read_only
Fixed by:
- Make all quick_* variable allocated according to real number keys instead
  of MAX_KEY
- Store all the quick* items in separated allocated structure (OPT_RANGE)
- Ensure we don't access any quick* variable without first checking
  opt_range_keys.is_set().  Thanks to this, we don't need any
  pre-initialization of quick* variables anymore.

Some renames was done to use the new structure:
table->quick_keys                -> table->opt_range_keys
table->quick_rows[X]             -> table->opt_range[X].rows
table->quick_key_parts[X]        -> table->opt_range[X].key_parts
table->quick_costs[X]            -> table->opt_range[X].cost
table->quick_index_only_costs[X] -> table->opt_range[X].index_only_cost
table->quick_n_ranges[X]         -> table->opt_range[X].ranges
table->quick_condition_rows      -> table->opt_range_condition_rows

This patch should both decrease memory needed for TABLE objects
(3528 -> 984 + keyinfo) and increase performance, thanks to less
initializations per query, and more localized memory, thanks to the
opt_range structure.
2020-07-02 16:59:14 +03:00
Monty
5cbb18cb44 Fixed typo in InnoDB when compiling with VALGRIND 2020-07-02 16:59:04 +03:00
Monty
29f9e679ad Don't copy uninitialized bytes when copying varstrings
When using field_conv(), which is called in case of field1=field2 copy in
fill_records(), full varstring's was copied, including unitialized bytes.
This caused valgrind to compilain about usage of unitialized bytes when
using Aria static length records.
Fixed by not using memcpy when copying varstrings but instead just copy
the real bytes.
2020-07-02 14:25:41 +03:00
Monty
3f2044ae99 MDEV-22535 TABLE::initialize_quick_structures() takes 0.5% in oltp_read_only
- Removed not needed bzero in void TABLE::initialize_quick_structures().
- Replaced bzero with TRASH_ALLOC() to have this change verfied with
  memory checkers
- Added missing table->quick_keys.is_set in table_cond_selectivity()
2020-07-02 14:25:41 +03:00
Marko Mäkelä
1813d92d0c Merge 10.4 into 10.5 2020-07-02 09:41:44 +03:00
Marko Mäkelä
0fe97c20b3 MDEV-23017: Regression due to unwanted read-ahead
MDEV-15053 inadvertently inverted the condition for invoking
buf_read_ahead_linear().

buf_page_get_low(): Apply the correct condition.

buf_page_optimistic_get(): Remove the read-ahead altogether.
This function should never access a block for the first time.
2020-07-02 07:45:30 +03:00
Marko Mäkelä
f347b3e0e6 Merge 10.3 into 10.4 2020-07-02 07:39:33 +03:00
Marko Mäkelä
1df1a63924 Merge 10.2 into 10.3 2020-07-02 06:17:51 +03:00
Marko Mäkelä
dba7e1e8e1 Merge 10.1 into 10.2 2020-07-02 06:05:13 +03:00
Marko Mäkelä
c43a666662 Revert "Fix result of merge."
This reverts commit e0793d3865.

In idiomatic C++, accessor functions should not discard qualifiers.
2020-07-02 06:04:42 +03:00
Marko Mäkelä
90d1e58ed0 MDEV-22941: Fix the DBUG_ENTER name 2020-07-02 06:04:31 +03:00
Marko Mäkelä
838a1046b2 MDEV-20377: Fix cmake -DPLUGIN_PERFSCHEMA=NO 2020-07-02 06:03:59 +03:00