Commit graph

368 commits

Author SHA1 Message Date
Marko Mäkelä
394784095e Merge 10.3 into 10.4 2022-04-21 11:33:59 +03:00
Marko Mäkelä
4730314a70 MDEV-28369 ibuf_bitmap_mutex is an unnecessary contention point
The only purpose of ibuf_bitmap_mutex is to prevent a deadlock between
two concurrent invocations of ibuf_update_free_bits_for_two_pages_low()
on the same pair of bitmap pages, but in opposite order.
The mutex is unnecessarily serializing the execution of the function
even when it is being invoked on totally different tablespaces.
To avoid deadlocks, it suffices to ensure that the two page latches
are being acquired in a deterministic (sorted) order.
2022-04-21 09:15:18 +03:00
Marko Mäkelä
f5ff7d09c7 Merge 10.3 into 10.4 2022-02-25 13:00:48 +02:00
Marko Mäkelä
9ba385a50d Merge 10.2 into 10.3 2022-02-25 12:40:26 +02:00
Marko Mäkelä
ed691eca99 Remove deprecated (in C++11) std::binary_function 2022-02-25 12:34:06 +02:00
Marko Mäkelä
749ecedfec MDEV-24188: Merge 10.3 into 10.4 2020-11-13 20:45:28 +02:00
Marko Mäkelä
f9f2f37495 MDEV-24188: Merge 10.2 into 10.3 2020-11-13 20:41:48 +02:00
Marko Mäkelä
bb328a2a27 MDEV-24188 Hang in buf_page_create() after reusing a previously freed page
The fix of MDEV-23456 (commit b1009ae5c1)
introduced a livelock between page flushing and a thread that is
executing buf_page_create().

buf_page_create(): If the current mini-transaction is holding
an exclusive latch on the page, do not attempt to acquire another
one, and do not care about any I/O fix.

mtr_t::have_x_latch(): Replaces mtr_t::get_fix_count().

dyn_buf_t::for_each_block(const Functor&) const: A new variant.

rw_lock_own(): Add a const qualifier.

Reviewed by: Thirunarayanan Balathandayuthapani
2020-11-13 20:16:39 +02:00
Marko Mäkelä
7b2bb67113 Merge 10.3 into 10.4 2020-10-29 13:38:38 +02:00
Marko Mäkelä
2b6f804490 Merge 10.2 into 10.3 2020-10-28 10:44:40 +02:00
Eugene Kosov
afc9d00c66 MDEV-23991 dict_table_stats_lock() has unnecessarily long scope
Patch removes dict_index_t::stats_latch. Table/index statistics now
protected with dict_sys->mutex. That way statistics computation can
happen in parallel in several threads and dict_sys->mutex will be locked
only for a short period of time.

This patch is a joint work with Marko Mäkelä

dict_index_t:🔒 make mutable which allows to pass const pointer
when only lock is touched in an object

btr_height_get()
btr_get_size(): make index argument const for better type safety

btr_estimate_number_of_different_key_vals(): now returns computed values
instead of setting fields in dict_index_t directly

remove everything related to dict_index_t::stats_latch

dict_stats_index_set_n_diff(): now returns computed values instead
of setting fields in dict_index_t directly

dict_stats_analyze_index():  now returns computed values instead
of setting fields in dict_index_t directly

Reviewed by: Marko Mäkelä
2020-10-27 19:09:20 +03:00
Marko Mäkelä
46957a6a77 Merge 10.3 into 10.4 2020-10-22 13:27:18 +03:00
Marko Mäkelä
e3d692aa09 Merge 10.2 into 10.3 2020-10-22 08:26:28 +03:00
Marko Mäkelä
199bc67144 Cleanup: Remove unused SYNC_REC_LOCK
SYNC_REC_LOCK was never used in the public history of InnoDB,
starting with commit 132e667b0b.
2020-10-05 09:12:12 +03:00
Marko Mäkelä
46890349bf Cleanup: Remove fts_t::bg_threads_mutex, fts_t::bg_threads
The unused fts_t::bg_threads was added in
mysql/mysql-server@4b1049625c.

Any usage of fts_t::bg_threads_mutex was removed in
mysql/mysql-server@33c2404b39.
2020-10-02 08:36:50 +03:00
Marko Mäkelä
bd64c2e8cc Cleanup: Remove unnecessary trx_i_s_cache_t::last_read_mutex
We can simply use C++11 std::atomic for avoiding undefined behaviour
related to concurrent stores to a shared variable. On most if not all
ISAs, std::memory_order_relaxed loads and stores will not really
differ from non-atomic loads or stores.
2020-10-01 13:51:31 +03:00
Marko Mäkelä
9ae608d24d Merge 10.3 into 10.4 2020-10-01 13:41:36 +03:00
Marko Mäkelä
323500bfa9 Merge 10.2 into 10.3 2020-09-30 16:25:06 +03:00
Marko Mäkelä
cd5f4d2a59 Cleanup: Remove unused fts_cache_t::optimize_lock
This has been unused from the very beginning
(mysql/mysql-server@d5e512ae7e).
2020-09-30 13:26:46 +03:00
Marko Mäkelä
eae968f62d Merge 10.3 into 10.4 2020-08-10 21:08:46 +03:00
Marko Mäkelä
bafc5c1321 Merge 10.2 into 10.3 2020-08-10 18:40:57 +03:00
Marko Mäkelä
3b6dadb5eb Merge 10.1 into 10.2 2020-08-10 17:57:14 +03:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Oleksandr Byelkin
c32f71af7e Merge branch '10.2' into 10.3 2020-08-03 13:41:29 +02:00
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Marko Mäkelä
52ccedd6dd MDEV-23268 SIGSEGV on srv_monitor_event if InnoDB is read-only
The srv_monitor_event and the srv_monitor_thread would not be
created when InnoDB is in read-only mode. Yet, some code would
unconditionally invoke os_event_set(srv_monitor_event).
2020-07-23 09:59:16 +03:00
Marko Mäkelä
5f2628d1ee MDEV-22778 Slow InnoDB shutdown on large instance
Starting with MDEV-17441 we would no longer have os_once,
and we would always initialize zip_pad_info_t::mutex and
dict_table_t::autoinc_mutex, even for tables are not in
ROW_FORMAT=COMPRESSED nor include any AUTO_INCREMENT column.

mutex_free() on those unnecessary objects would make shutdown very slow
compared to older versions.

Let us use std::mutex for those two mutexes, to reduce the overhead.
The critical sections protected by these mutexes is very small, and
therefore contention or the need for any instrumentation should
be unlikely.
2020-07-23 08:28:17 +03:00
Monty
5211af1c16 Merge remote-tracking branch 'origin/10.3' into 10.4 2020-07-03 00:35:28 +03:00
Marko Mäkelä
b6ec1e8bbf MDEV-20377 post-fix: Introduce MEM_MAKE_ADDRESSABLE
In AddressSanitizer, we only want memory poisoning to happen
in connection with custom memory allocation or freeing.

The primary use of MEM_UNDEFINED is for declaring memory uninitialized
in Valgrind or MemorySanitizer. We do not want MEM_UNDEFINED to
have the unwanted side effect that AddressSanitizer would no longer
be able to complain about accessing unallocated memory.

MEM_UNDEFINED(): Define as no-op for AddressSanitizer.

MEM_MAKE_ADDRESSABLE(): Define as MEM_UNDEFINED() or
ASAN_UNPOISON_MEMORY_REGION().

MEM_CHECK_ADDRESSABLE(): Wrap also __asan_region_is_poisoned().
2020-07-02 17:59:28 +03:00
Marko Mäkelä
f347b3e0e6 Merge 10.3 into 10.4 2020-07-02 07:39:33 +03:00
Marko Mäkelä
1df1a63924 Merge 10.2 into 10.3 2020-07-02 06:17:51 +03:00
Marko Mäkelä
c36834c832 MDEV-20377: Make WITH_MSAN more usable
MemorySanitizer (clang -fsanitize=memory) requires that all code
be compiled with instrumentation enabled. The only exception is the
C runtime library. Failure to use instrumented libraries will cause
bogus messages about memory being uninitialized.

In WITH_MSAN builds, we must avoid calling getservbyname(),
because even though it is a standard library function, it is
not instrumented, not even in clang 10.

Note: Before MariaDB Server 10.5, ./mtr will typically fail
due to the old PCRE library, which was updated in MDEV-14024.

The following cmake options were tested on 10.5
in commit 94d0bb4dbe:

cmake \
-DCMAKE_C_FLAGS='-march=native -O2' \
-DCMAKE_CXX_FLAGS='-stdlib=libc++ -march=native -O2' \
-DWITH_EMBEDDED_SERVER=OFF -DWITH_UNIT_TESTS=OFF -DCMAKE_BUILD_TYPE=Debug \
-DWITH_INNODB_{BZIP2,LZ4,LZMA,LZO,SNAPPY}=OFF \
-DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,CONNECT,SPIDER}=NO \
-DWITH_SAFEMALLOC=OFF \
-DWITH_{ZLIB,SSL,PCRE}=bundled \
-DHAVE_LIBAIO_H=0 \
-DWITH_MSAN=ON

MEM_MAKE_DEFINED(): An alias for VALGRIND_MAKE_MEM_DEFINED()
and __msan_unpoison().

MEM_GET_VBITS(), MEM_SET_VBITS(): Aliases for
VALGRIND_GET_VBITS(), VALGRIND_SET_VBITS(), __msan_copy_shadow().

InnoDB: Replace the UNIV_MEM_ macros with corresponding MEM_ macros.

ut_crc32_8_hw(), ut_crc32_64_low_hw(): Use the compiler built-in
functions instead of inline assembler when building WITH_MSAN.
This will require at least -msse4.2 when building for IA-32 or AMD64.
The inline assembler would not be instrumented, and would thus cause
bogus failures.
2020-07-01 17:23:00 +03:00
Marko Mäkelä
386f168ab3 MDEV-22456 after-merge fix: introduce Atomic_relaxed
In the merge 9e6e43551f
we made Atomic_counter a more generic wrapper of std::atomic
so that dict_index_t would support the implicit assignment operator.

It is better to revert the changes to Atomic_counter and
instead introduce Atomic_relaxed as a generic wrapper to std::atomic.

Unlike Atomic_counter, we will not define operator++, operator+=
or similar, because we want to make the operations more explicit
in the users of Atomic_wrapper, because unlike loads and stores,
atomic read-modify-write operations always incur some overhead.
2020-05-18 15:02:55 +03:00
Marko Mäkelä
66f1e288a1 Merge 10.3 into 10.4 2020-05-16 07:54:09 +03:00
Marko Mäkelä
9e6e43551f Merge 10.3 into 10.4
We will expose some more std::atomic internals in Atomic_counter,
so that dict_index_t::lock will support the default assignment operator.
2020-05-16 07:39:15 +03:00
Marko Mäkelä
38d62189a8 Merge 10.2 into 10.3 2020-05-16 06:37:24 +03:00
Krunal Bauskar
dcb0bd59ce MDEV-22544: Inconsistent and Incorrect rw-lock stats
- There are multiple inconsistency and incorrect way in which rw-lock
  stats are calculated.

- shared rw-lock stats:
  "rounds" counter is incremented only once for N rounds done
  in spin-cycle.

- all rw-lock stats:
  If the spin-cycle is short-circuited then attempts are re-counted.
  [If spin-cycle is interrupted, before it completes
   srv_n_spin_wait_rounds (default 30) rounds, spin_count is incremented
   to consider this. If thread resumes spin-cycle (due to unavailability
   of the locks) and is again interrupted or completed, spin_count
   is again incremented with the total count, failing to adjust the
   previous attempt increment].

- s/x rw-lock stats:
  spin_loop counter is not incremented at-all instead it is projected
  as 0 (in show engine output) and division to calculate spin-round per
  spin-loop is adjusted.
  As per the original semantics spin_loop counter should be incremented
  once per spin_loop execution.

- sx rw-lock stats:
  sx locks increments spin_loop counter but instead of incrementing it
  once for a spin_loop invocation it does it multiple times based on how
  many time spin_loop flow is repeated for same instance post os-wait.
2020-05-14 15:27:15 +08:00
Marko Mäkelä
38f6c47f8a Merge 10.3 into 10.4 2020-05-13 12:52:57 +03:00
Marko Mäkelä
15fa70b840 Merge 10.2 into 10.3 2020-05-13 11:45:05 +03:00
Marko Mäkelä
ba3d58ad4c MDEV-22523 index->rtr_ssn.mutex is wasting memory
As part of the SPATIAL INDEX implementation in InnoDB,
dict_index_t was expanded by a rtr_ssn_t field. There are only
3 operations for this field, all protected by rtr_ssn_t::mutex:

* btr_cur_search_to_nth_level() stores the least significant 32 bits
of the 64-bit value that is stored in the index root page.
(This would better be done when the table is opened for the
very first time.)
* rtr_get_new_ssn_id() increments the value by 1.
* rtr_get_current_ssn_id() reads the current value.

All these operations can be implemented equally safely by using
atomic memory access operations.
2020-05-11 14:23:37 +03:00
Marko Mäkelä
8fa759a576 Merge 10.3 into 10.4
We disable the MDEV-21189 test galera.galera_partition
because it times out.
2019-12-13 17:30:37 +02:00
Marko Mäkelä
0a20e5ab77 Merge 10.2 into 10.3 2019-12-12 14:41:51 +02:00
Marko Mäkelä
41e6a154ec MDEV-14482 - Cache line contention on ut_rnd_interval()
InnoDB RNG maintains global state, causing otherwise unnecessary bus
traffic. Even worse, this is cross-mutex traffic. That is, different
mutexes suffer from contention.

Fixed delay of 4 was verified to give best throughput by OLTP update
index and read-write benchmarks on Intel Broadwell (2/20/40) and
ARM (1/46/46).

This is a backport of ce04790065 from
MariaDB Server 10.3.
2019-12-10 17:01:36 +02:00
Marko Mäkelä
d146e3dcfe MDEV-21256: Simplify ut_rnd_interval()
ut_rnd_interval(): Remove the first parameter, which was mostly
passed as 0. Implement as a simple wrapper around ut_rnd_gen().
Trivially return 0 if the size of the interval is smaller than 2.

ut_rnd_ulint_counter, ut_rnd_gen_next_ulint(), ut_rnd_gen_ulint(): Remove.
2019-12-10 16:58:28 +02:00
Marko Mäkelä
425748f1b5 Cleanup: Replace a redundant statement with an assertion 2019-12-09 14:54:27 +02:00
Marko Mäkelä
9b5cdeeb0f Merge 10.3 into 10.4 2019-09-27 16:26:53 +03:00
Marko Mäkelä
2911a9a693 Merge 10.2 into 10.3 2019-09-27 15:56:15 +03:00
Marko Mäkelä
718fcee0a3 Reduce rw_lock_debug_mutex contention
rw_lock_own(), rw_lock_own_flagged(): Traverse the rw_lock_t::debug_list
only after quickly checking if the thread is holding X-latch or SX-latch.
2019-09-27 14:22:59 +03:00
Marko Mäkelä
e9c1701e11 Merge 10.3 into 10.4 2019-07-25 18:42:06 +03:00
Marko Mäkelä
fdef9f9b89 Merge 10.2 into 10.3 2019-07-25 15:31:11 +03:00