Commit graph

9848 commits

Author SHA1 Message Date
Eugene Kosov
18d8f06f31 intrusive::list fixes
namespace intrusive: removed

split class into two: ilist<T> and sized_ilist<T> which has a size field.

ilist<T> no more NULLify pointers to bring a slignly better performance.
As a consequence, fil_space_t::is_in_unflushed_spaces and
fil_space_t::is_in_rotation_list boolean members are needed now.
2020-05-27 02:34:25 +03:00
Marko Mäkelä
ca38b6e427 Merge 10.3 into 10.4 2020-05-26 11:54:55 +03:00
Marko Mäkelä
ecc7f305dd Merge 10.2 into 10.3 2020-05-25 19:41:58 +03:00
Oleksandr Byelkin
cf52dd174e MDEV-22545: my_vsnprintf behaves not as in C standard
Added parameter %T for string which should be visibly truncated.
2020-05-24 21:27:08 +02:00
Monty
fa0397849a Move c++ code from my_atomic.h to my_atomic_wrapper.h
This is because it breaks code that is using extern "C" when including
my_atomic, which is the case with ha_s3.cc
2020-05-19 14:07:34 +03:00
Marko Mäkelä
386f168ab3 MDEV-22456 after-merge fix: introduce Atomic_relaxed
In the merge 9e6e43551f
we made Atomic_counter a more generic wrapper of std::atomic
so that dict_index_t would support the implicit assignment operator.

It is better to revert the changes to Atomic_counter and
instead introduce Atomic_relaxed as a generic wrapper to std::atomic.

Unlike Atomic_counter, we will not define operator++, operator+=
or similar, because we want to make the operations more explicit
in the users of Atomic_wrapper, because unlike loads and stores,
atomic read-modify-write operations always incur some overhead.
2020-05-18 15:02:55 +03:00
Marko Mäkelä
9e6e43551f Merge 10.3 into 10.4
We will expose some more std::atomic internals in Atomic_counter,
so that dict_index_t::lock will support the default assignment operator.
2020-05-16 07:39:15 +03:00
Marko Mäkelä
6a6bcc53b8 Merge 10.2 into 10.3 2020-05-15 17:55:01 +03:00
Marko Mäkelä
ff66d65a09 Amend af784385b4: Avoid vtable overhead
When neither MSAN nor Valgrind are enabled, declare
Field::mark_unused_memory_as_defined() as an empty inline function,
instead of declaring it as a virtual function.
2020-05-15 17:23:08 +03:00
Eugene Kosov
1cac6d4828 span cleanup 2020-05-15 16:25:32 +03:00
Monty
af784385b4 Fix for using uninitialized memory
MDEV-22073 MSAN use-of-uninitialized-value in
collect_statistics_for_table()

Other things:
innodb.analyze_table was changed to mainly test statistic
collection. Was discussed with Marko.
2020-05-15 15:10:58 +03:00
Sergei Golubchik
5b0df7433d WolfSSL fixes
remove Timeval workaround (not needed anymore).
add template workaround.
comments.
2020-05-08 12:49:53 +02:00
Vladislav Vaintroub
403dc759d0 Update WolfSSL
Fix WolfSSL build:

- Do not build with TLSv1.0,it stopped working,at least with SChannel client
- Disable a test that depends on TLSv1.0
- define FP_MAX_BITS always, to fix 32bit builds.
- Increase MAX_AES_CTX_SIZE, to fix build on Linux
2020-05-08 11:51:03 +02:00
Sergei Petrunia
8d85715d50 MDEV-21794: Optimizer flag rowid_filter leads to long query
Rowid Filter check is just like Index Condition Pushdown check: before
we check the filter, we must check if we have walked out of the range
we are scanning. (If we did, we should return, and not continue the scan).

Consequences of this:
- Rowid filtering doesn't work for keys that have partially-covered
  blob columns (just like Index Condition Pushdown)
- The rowid filter function has three return values: CHECK_POS (passed)
  CHECK_NEG (filtered out), CHECK_OUT_OF_RANGE.

All of the above is implemented in this patch
2020-05-07 12:27:17 +02:00
Marko Mäkelä
2c3c851d2c Merge 10.3 into 10.4 2020-05-05 20:33:10 +03:00
Oleksandr Byelkin
7fb73ed143 Merge branch '10.2' into 10.3 2020-05-04 16:47:11 +02:00
Oleksandr Byelkin
ca091e6372 Merge branch '10.1' into 10.2 2020-05-02 08:44:17 +02:00
Oleksandr Byelkin
23c6fb3e62 Merge branch '5.5' into 10.1 2020-04-30 17:36:41 +02:00
Sergei Golubchik
6a31aea5a1 BUG#30301356 - SOME EVENTS ARE DELAYED AFTER DROPPING EVENT
queues.c cleanup and refactoring.

Restore old version of _downhead() (from before cd483c5520)
that works well in an average case. Use it for queue_fix().

Move existing specialized version of _downhead() to queue_replace()
where it'll be handling the case it was specifically optimized for
(moving the element to the end of the queue).
And correct it to fix the heap not only down, but also up
(this fixes BUG#30301356).

Add unit tests.

Collateral cosmetic fixes.
2020-04-30 10:13:21 +02:00
Eugene Kosov
28325b0863 add WITH_DBUG_TRACE CMake variable
This is a way do disable DBUG_ENTER()/DBUG_EXIT() stuff which is
needed to dbug trace. Those who doesn't need it may avoid tests
slowdown with -DWITH_DBUG_TRACE=OFF

dbug/tests.c: add define which is neede always in this test

innodb.log_file_name_debug.test: do not depend on DBUG trace stuff
in test

Benchmark results: each test eats less CPU and you can have more
parallel jobs in MTR.

patched:
./mtr -mem -par=8 -suite=innodb  185.34s user 86.85s system 133% cpu 3:23.27 total
./mtr -mem -par=8 -suite=main    80.96s  user 36.01s system 182% cpu 1:04.07 total

main.select                              [ pass ]   1660
main.select                              [ pass ]   1513
main.select                              [ pass ]   1543
main.select                              [ pass ]   1660
main.select                              [ pass ]   1521
main.select                              [ pass ]   1511
main.select                              [ pass ]   1508
main.select                              [ pass ]   1520
main.select                              [ pass ]   1514
main.select                              [ pass ]   1522

vanilla:
./mtr -mem -par=8 -suite=innodb  203.61s user 92.16s system 140% cpu 3:30.16 total
./mtr -mem -par=8 -suite=main    94.11s  user 35.51s system 206% cpu 1:02.69 total

main.select                              [ pass ]   2032
main.select                              [ pass ]   2017
main.select                              [ pass ]   2040
main.select                              [ pass ]   2183
main.select                              [ pass ]   2253
main.select                              [ pass ]   2075
main.select                              [ pass ]   2109
main.select                              [ pass ]   2080
main.select                              [ pass ]   2098
main.select                              [ pass ]   2114
2020-04-29 20:13:14 +03:00
Sergei Petrunia
7bc6735736 MDEV-22401: Optimizer trace: multi-component range is not printed correctly
KEY_MULTI_RANGE::range_flag does not have correct flag bits for
per-endpoint flags (NEAR_MIN, NEAR_MAX, NO_MIN_RANGE, NO_MAX_RANGE).
It only has bits for flags that describe both endpoints.
So
- Document this.
- Switch optimizer trace to using {start|end}_key.flag values, instead.
  This fixes the bug.
- Switch records_in_column_ranges() to doing that too. (This used to
  work, because KEY_MULTI_RANGE::range_flag had correct flag value
  for the last key component, and EITS only uses one-component
  pseudo-indexes)
2020-04-29 16:31:16 +03:00
Daniel Black
c238e9b96a MDEV-20685: compile fixes for Solaris/OSX/AIX
sig_return: Solaris/OSX returns different function ptr
Move defination to my_alarm.h as its the only use.

prevents compile warnings (copied from 10.3 branch)

mysys/my_sync.c:136:19: error: 'cur_dir_name' defined but not used [-Werror=unused-const-variable=]
  136 | static const char cur_dir_name[]= {FN_CURLIB, 0};
      |                   ^~~~~~~~~~~~

fix compile error (DEPRECATED) leaked from ssl headers.

In file included from /export/home/dan/mariadb-server-10.4/sql/sys_vars.cc:37:
/export/home/dan/mariadb-server-10.4/sql/sys_vars.ic:69: error: "DEPRECATED" redefined [-Werror]
   69 | #define DEPRECATED(X) X
      |
In file included from /export/home/dan/mariadb-server-10.4/include/violite.h:150,
                 from /export/home/dan/mariadb-server-10.4/sql/sql_class.h:38,
                 from /export/home/dan/mariadb-server-10.4/sql/sys_vars.cc:36:
/usr/include/openssl/ssl.h:2356: note: this is the location of the previous definition
 2356 | # define DEPRECATED __attribute__((deprecated))
      |

Avoid Werror condition on non-Linux:

plugin/server_audit/server_audit.c:2267:7: error: variable 'db_len_off' set but not used [-Werror=unused-but-set-variable]
 2267 |   int db_len_off;
      |       ^~~~~~~~~~
plugin/server_audit/server_audit.c:2266:7: error: variable 'db_off' set but not used [-Werror=unused-but-set-variable]
 2266 |   int db_off;
      |       ^~~~~~

auth_gssapi fix include path for Solaris

Consistent with the upstream packaged patch:
https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/database/mariadb-103/patches/06-gssapi.h.patch

compile warnings on Solaris

[ 91%] Building C object plugin/server_audit/CMakeFiles/server_audit.dir/server_audit.c.o
/plugin/server_audit/server_audit.c: In function 'auditing_v8':
/plugin/server_audit/server_audit.c:2194:20: error: unused variable 'db_len_off' [-Werror=unused-variable]
 2194 |   static const int db_len_off= 128;
      |                    ^~~~~~~~~~
/plugin/server_audit/server_audit.c:2193:20: error: unused variable 'db_off' [-Werror=unused-variable]
 2193 |   static const int db_off= 120;
      |                    ^~~~~~
/plugin/server_audit/server_audit.c:2192:20: error: unused variable 'cmd_off' [-Werror=unused-variable]
 2192 |   static const int cmd_off= 4432;
      |                    ^~~~~~~
At top level:
/plugin/server_audit/server_audit.c:2192:20: error: 'cmd_off' defined but not used [-Werror=unused-const-variable=]
/plugin/server_audit/server_audit.c:2193:20: error: 'db_off' defined but not used [-Werror=unused-const-variable=]
 2193 |   static const int db_off= 120;
      |                    ^~~~~~
/plugin/server_audit/server_audit.c:2194:20: error: 'db_len_off' defined but not used [-Werror=unused-const-variable=]
 2194 |   static const int db_len_off= 128;
      |                    ^~~~~~~~~~
cc1: all warnings being treated as errors

tested on:
$ uname -a
SunOS openindiana 5.11 illumos-b97b1727bc i86pc i386 i86pc
2020-04-29 12:02:47 +03:00
Marko Mäkelä
b63446984c Merge 10.3 into 10.4 2020-04-27 17:38:17 +03:00
Marko Mäkelä
2e12d471ea Merge 10.2 into 10.3 2020-04-27 14:24:41 +03:00
Eugene Kosov
2c5067b689 cleanup THR_KEY_mysys
read TLS with my_thread_var
write TLS with set_mysys_var()

my_thread_var is no longer __attribute__ ((const)): this attribute
is simply incorrect here. Read gcc manual for more information.
sql/threadpool_generic.cc fails with that attribute.
2020-04-25 00:55:39 +03:00
Marko Mäkelä
edd38b50f6 MDEV-7962 wsrep_on() takes 0.14% in OLTP RO
The reason why we have wsrep_on() at all is that the macro WSREP(thd)
depends on the definition of THD, and that is intentionally an opaque
data type for InnoDB. So, we cannot avoid invoking wsrep_on(), but
we can evaluate the less expensive conditions thd && WSREP_ON before
calling the function.

Global_read_lock: Use WSREP_NNULL(thd) instead of wsrep_on(thd)
because we not only know the definition of THD but also that
the pointer is not null.

wsrep_open(): Use WSREP(thd) instead of wsrep_on(thd).

InnoDB: Replace thd && wsrep_on(thd) with wsrep_on(thd), now that
the condition has been merged to the definition of the macro
wsrep_on().
2020-04-24 16:01:10 +03:00
Marko Mäkelä
2c39f69d34 MDEV-22203: WSREP_ON is unnecessarily expensive WITH_WSREP=OFF
If the server is compiled WITH_WSREP=OFF, we should avoid evaluating
conditions on a global variable that is constant.

WSREP_ON_: Renamed from WSREP_ON. Defined only WITH_WSREP=ON.

WSREP_ON: Defined as unlikely(WSREP_ON_).

wsrep_on(): Defined as WSREP_ON && wsrep_service->wsrep_on_func().

The reason why we have wsrep_on() at all is that the macro WSREP(thd)
depends on the definition of THD, and that is intentionally an opaque
data type for InnoDB. So, we cannot avoid invoking wsrep_on(), but
we can evaluate the less expensive condition WSREP_ON before calling
the function.
2020-04-24 15:25:39 +03:00
Sergey Vojtovich
5679a2b6b3 Shrink my_atomic.h and my_cpu.h scope 2020-04-15 22:23:03 +04:00
Sergey Vojtovich
4bd9f82a8f slave_open_temp_tables to Atomic_counter 2020-04-15 21:05:21 +04:00
Sergey Vojtovich
5876ed9e5b Relay_log_info::executed_entries to Atomic_counter 2020-04-15 18:36:07 +04:00
Marko Mäkelä
e2f1f88fa6 Merge 10.3 into 10.4 2020-03-30 14:50:23 +03:00
Marko Mäkelä
1a9b6c4c7f Merge 10.2 into 10.3 2020-03-30 11:12:56 +03:00
Eugene Kosov
a7cbce06d4 unoptimized -fsanitize=undefined build on clang requires more stack space 2020-03-23 17:42:57 +03:00
Marko Mäkelä
bd3c8f47cd Merge 10.3 into 10.4 2020-03-20 22:06:55 +02:00
Eugene Kosov
1f53335d37 st_::span fixes
move span.h to a proper place to make it available for the whole server

Reformat it.

Constuctors from a contigous container are fixed
to use cont.data() instead of cont.begin()

span<>::index_type is replaced with span<>::size_type
2020-03-20 21:35:42 +03:00
Marko Mäkelä
44298e4dea Merge 10.2 into 10.3
Also, clean up the test innodb_gis.geometry a little further.
2020-03-20 18:12:17 +02:00
Marko Mäkelä
a66eebf57c MDEV-21981 Replace arithmetic + with bitwise OR when possible
Several macros such as sint2korr() and uint4korr() are using the
arithmetic + operator while a bitwise or operator would suffice.

GCC 5 and clang 5 and later can detect patterns consisting of
bitwise or and shifts by multiples of 8 bits, such as those used
in the InnoDB function mach_read_from_4(). They actually translate
that verbose low-level code into high-level machine language
(i486 bswap instruction or fused into the Haswell movbe instruction).

We should do the same for MariaDB Server code that is outside InnoDB.

Note: The Microsoft C compiler is lacking this optimization.
There, we might consider using _byteswap_ushort(), _byteswap_ulong(),
_byteswap_uint64(). But, those would lead to unaligned reads, which are
bad for reasons stated in MDEV-20277. Besides, outside InnoDB,
most data is already being stored in the native little-endian format
of that compiler.
2020-03-19 15:09:13 +02:00
Daniel Black
8eb0384556
mysys: remove windac my_security_attr_create (#1391)
No longer used.
2020-02-14 10:32:45 +01:00
Sachin
eed6d215f1 MDEV-20001 Potential dangerous regression: INSERT INTO >=100 rows fail for myisam table with HASH indexes
Problem:-

So the issue is when we do bulk insert with rows
> MI_MIN_ROWS_TO_DISABLE_INDEXES(100) , We try to disable the indexes to
speedup insert. But current logic also disables the long unique indexes.

Solution:- In ha_myisam::start_bulk_insert if we find long hash index
(HA_KEY_ALG_LONG_HASH) we will not disable the index.

This commit also refactors the mi_disable_indexes_for_rebuild function,
Since this is function is called at only one place, it is inlined into
start_bulk_insert

mi_clear_key_active is added into myisamdef.h because now it is also used
in ha_myisam.cc file.

(Same is done for Aria Storage engine)
2020-02-03 12:44:31 +05:30
Marko Mäkelä
4b291588bb MDEV-19845: Make my_cpu.h self-contained
Fix up commit f5c080c735
2020-02-01 14:56:05 +02:00
Oleksandr Byelkin
6918157e98 Merge branch '10.3' into 10.4 2020-01-21 23:15:02 +01:00
Oleksandr Byelkin
ade89fc898 Merge branch '10.2' into 10.3 2020-01-21 09:11:14 +01:00
Sergei Golubchik
1ecd0e0732 compilation fixes for new wolfssl 2020-01-20 22:35:54 +01:00
Oleksandr Byelkin
3a1716a7e7 Merge branch '10.1' into 10.2 2020-01-20 16:15:05 +01:00
Oleksandr Byelkin
10eacd5ff7 Merge branch 'merge-perfschema-5.6' into 10.1 2020-01-19 13:11:45 +01:00
Oleksandr Byelkin
3aff3f3679 5.6.47 2020-01-19 12:52:07 +01:00
Marko Mäkelä
d60dcabd0f Merge 10.3 into 10.4 2020-01-07 13:23:41 +02:00
Marko Mäkelä
eda719793a Merge 10.2 into 10.3 2020-01-07 12:14:35 +02:00
Eugene Kosov
fd899b3bbd Lets add another intrusive double linked list!
Features:
* STL-like interface
* Fast modification: no branches on insertion or deletion
* Fast iteration: one pointer dereference and one pointer comparison
* Your class can be a part of several lists

Modeled after std::list<T> but currently has fewer methods (not complete yet)

For even more performance it's possible to customize list with templates so
it won't have size counter variable or won't NULLify unlinked node.

How existing lists differ?

No existing lists support STL-like interface.

I_List:
* slower iteration (one more branch on iteration)
* element can't be a part of two lists simultaneously

I_P_List:
* slower modification (branches, except for the fastest push_back() case)
* slower iteration (one more branch on iteration)

UT_LIST_BASE_NODE_T:
* slower modification (branches)

Three UT_LISTs were replaced: two in fil_system_t and one in dyn_buf_t.
2020-01-04 13:39:14 +07:00
Marko Mäkelä
8fa759a576 Merge 10.3 into 10.4
We disable the MDEV-21189 test galera.galera_partition
because it times out.
2019-12-13 17:30:37 +02:00