Commit graph

9926 commits

Author SHA1 Message Date
Marko Mäkelä
eba2d10ac5 MDEV-22721 Remove bloat caused by InnoDB logger class
Introduce a new ATTRIBUTE_NOINLINE to
ib::logger member functions, and add UNIV_UNLIKELY hints to callers.

Also, remove some crash reporting output. If needed, the
information will be available using debugging tools.

Furthermore, remove some fts_enable_diag_print output that included
indexed words in raw form. The code seemed to assume that words are
NUL-terminated byte strings. It is not clear whether a NUL terminator
is always guaranteed to be present. Also, UCS2 or UTF-16 strings would
typically contain many NUL bytes.
2020-06-04 10:24:10 +03:00
Daniel Black
463a8fc5fd MDEV-22641: postfix - crc32{,c} fixups for ppc64 2020-06-03 18:59:20 +10:00
Vladislav Vaintroub
661ebd4699 Fix my_checksum declaration.
exporting data from the server needs MYSQL_PLUGIN_IMPORT.
2020-06-01 16:21:50 +02:00
mysqlonarm
dec3f8ca69
MDEV-22641: Provide SIMD optimized wrapper for zlib crc32() (#1558)
Existing implementation used my_checksum (from mysys)
for calculating table checksum and binlog checksum.

This implementation was optimized for powerpc only and lacked
SIMD implementation for x86 (using clmul) and ARM
(using ACLE) instead used zlib-crc32.

mariabackup had its own copy of the crc32 implementation
using hardware optimized implementation only for x86 and lagged
hardware based implementation for powerpc and ARM.

Patch helps unifies all such calls and help aggregate all of them
using an unified interface my_checksum().

Said unification also enables hardware optimized calls for all
architecture viz. x86, ARM, POWERPC.
Default always fallback to zlib crc32.

Thanks to Daniel Black for reviewing, fixing and testing
PowerPC changes. Thanks to Marko and Daniel for early code feedback.
2020-06-01 11:34:06 +03:00
Marko Mäkelä
4a0b56f604 Merge 10.4 into 10.5 2020-05-31 10:28:59 +03:00
Marko Mäkelä
6da14d7b4a Merge 10.3 into 10.4 2020-05-30 11:04:27 +03:00
Marko Mäkelä
dad7a8ee7d Merge 10.2 into 10.3 2020-05-27 17:10:39 +03:00
Eugene Kosov
5139cfabb3 fix compilation 2020-05-27 16:37:34 +03:00
Marko Mäkelä
1b3adaab25 Fix the build with GCC 4.1.2
The build was broken on CentOS 5 and CentOS 6 as a result of
commit 18d8f06f31 introducing
some constructs that old GCC versions could not cope with.
2020-05-27 10:58:28 +03:00
Marko Mäkelä
67496281ea Fix the RelWithDebInfo build
The build was broken in the parent
commit 18d8f06f31.
2020-05-27 09:51:46 +03:00
Eugene Kosov
18d8f06f31 intrusive::list fixes
namespace intrusive: removed

split class into two: ilist<T> and sized_ilist<T> which has a size field.

ilist<T> no more NULLify pointers to bring a slignly better performance.
As a consequence, fil_space_t::is_in_unflushed_spaces and
fil_space_t::is_in_rotation_list boolean members are needed now.
2020-05-27 02:34:25 +03:00
Marko Mäkelä
ca38b6e427 Merge 10.3 into 10.4 2020-05-26 11:54:55 +03:00
Marko Mäkelä
ecc7f305dd Merge 10.2 into 10.3 2020-05-25 19:41:58 +03:00
Oleksandr Byelkin
cf52dd174e MDEV-22545: my_vsnprintf behaves not as in C standard
Added parameter %T for string which should be visibly truncated.
2020-05-24 21:27:08 +02:00
Monty
c4bf4b7aef Fixed access to undefined memory found by valgrind and MSAN
When my_vsnprintf() is patched, the code protected disabled with
'WAITING_FOR_BUGFIX_TO_VSPRINTF' should be enabled again. Also all %b
formats in this patch should be revert to %s again
2020-05-23 12:29:10 +03:00
Marko Mäkelä
d2e96e492a Merge 10.4 into 10.5 2020-05-19 14:33:08 +03:00
Monty
fa0397849a Move c++ code from my_atomic.h to my_atomic_wrapper.h
This is because it breaks code that is using extern "C" when including
my_atomic, which is the case with ha_s3.cc
2020-05-19 14:07:34 +03:00
Vladislav Vaintroub
e2bc029211 MDEV-7021 Pass directory security descriptor from mysql_install_db.exe to bootstrap
This ensures that directory permissions are correct in all cases, even if
boostrap is passed non-standard locations for innodb.

Directory permissions are copied from the datadir.
2020-05-18 18:11:40 +02:00
Marko Mäkelä
23047d3ed4 Merge 10.4 into 10.5 2020-05-18 17:30:02 +03:00
Marko Mäkelä
386f168ab3 MDEV-22456 after-merge fix: introduce Atomic_relaxed
In the merge 9e6e43551f
we made Atomic_counter a more generic wrapper of std::atomic
so that dict_index_t would support the implicit assignment operator.

It is better to revert the changes to Atomic_counter and
instead introduce Atomic_relaxed as a generic wrapper to std::atomic.

Unlike Atomic_counter, we will not define operator++, operator+=
or similar, because we want to make the operations more explicit
in the users of Atomic_wrapper, because unlike loads and stores,
atomic read-modify-write operations always incur some overhead.
2020-05-18 15:02:55 +03:00
Marko Mäkelä
9e6e43551f Merge 10.3 into 10.4
We will expose some more std::atomic internals in Atomic_counter,
so that dict_index_t::lock will support the default assignment operator.
2020-05-16 07:39:15 +03:00
Marko Mäkelä
6a6bcc53b8 Merge 10.2 into 10.3 2020-05-15 17:55:01 +03:00
Marko Mäkelä
ff66d65a09 Amend af784385b4: Avoid vtable overhead
When neither MSAN nor Valgrind are enabled, declare
Field::mark_unused_memory_as_defined() as an empty inline function,
instead of declaring it as a virtual function.
2020-05-15 17:23:08 +03:00
Eugene Kosov
1cac6d4828 span cleanup 2020-05-15 16:25:32 +03:00
Monty
af784385b4 Fix for using uninitialized memory
MDEV-22073 MSAN use-of-uninitialized-value in
collect_statistics_for_table()

Other things:
innodb.analyze_table was changed to mainly test statistic
collection. Was discussed with Marko.
2020-05-15 15:10:58 +03:00
Marko Mäkelä
0186b0a077 Merge mariadb-10.5.3 into 10.5 2020-05-12 15:58:15 +03:00
Sergei Golubchik
13038e4705 Merge branch '10.4' into 10.5 2020-05-09 20:43:36 +02:00
Alexander Barkov
cfe5ee90c8 MDEV-22043 Special character leads to assertion in my_wc_to_printable_generic on 10.5.2 (debug)
The code did not take into account that:
- U+005C (backslash) can occupy more than mbminlen characters (e.g. in sjis)
- Some character sets do not have a code for U+005C (e.g. swe7)

Adding a new function my_wc_to_printable into MY_CHARSET_HANDLER to
cover all special cases easier.
2020-05-09 16:01:30 +04:00
Sergei Golubchik
5b0df7433d WolfSSL fixes
remove Timeval workaround (not needed anymore).
add template workaround.
comments.
2020-05-08 12:49:53 +02:00
Vladislav Vaintroub
403dc759d0 Update WolfSSL
Fix WolfSSL build:

- Do not build with TLSv1.0,it stopped working,at least with SChannel client
- Disable a test that depends on TLSv1.0
- define FP_MAX_BITS always, to fix 32bit builds.
- Increase MAX_AES_CTX_SIZE, to fix build on Linux
2020-05-08 11:51:03 +02:00
Sergei Petrunia
8d85715d50 MDEV-21794: Optimizer flag rowid_filter leads to long query
Rowid Filter check is just like Index Condition Pushdown check: before
we check the filter, we must check if we have walked out of the range
we are scanning. (If we did, we should return, and not continue the scan).

Consequences of this:
- Rowid filtering doesn't work for keys that have partially-covered
  blob columns (just like Index Condition Pushdown)
- The rowid filter function has three return values: CHECK_POS (passed)
  CHECK_NEG (filtered out), CHECK_OUT_OF_RANGE.

All of the above is implemented in this patch
2020-05-07 12:27:17 +02:00
Marko Mäkelä
7bcaa541aa Merge 10.4 into 10.5 2020-05-05 21:16:22 +03:00
Marko Mäkelä
2c3c851d2c Merge 10.3 into 10.4 2020-05-05 20:33:10 +03:00
Eugene Kosov
89ff4176c1 MDEV-22437 make THR_THD* variable thread_local
Now all access goes through _current_thd() and set_current_thd()
functions.

Some functions like THD::store_globals() can not fail now.
2020-05-05 18:13:31 +03:00
Marko Mäkelä
4337a3b5f9 Merge 10.4 into 10.5 2020-05-04 18:43:00 +03:00
Oleksandr Byelkin
7fb73ed143 Merge branch '10.2' into 10.3 2020-05-04 16:47:11 +02:00
Oleksandr Byelkin
ca091e6372 Merge branch '10.1' into 10.2 2020-05-02 08:44:17 +02:00
Oleksandr Byelkin
23c6fb3e62 Merge branch '5.5' into 10.1 2020-04-30 17:36:41 +02:00
Sergei Golubchik
6a31aea5a1 BUG#30301356 - SOME EVENTS ARE DELAYED AFTER DROPPING EVENT
queues.c cleanup and refactoring.

Restore old version of _downhead() (from before cd483c5520)
that works well in an average case. Use it for queue_fix().

Move existing specialized version of _downhead() to queue_replace()
where it'll be handling the case it was specifically optimized for
(moving the element to the end of the queue).
And correct it to fix the heap not only down, but also up
(this fixes BUG#30301356).

Add unit tests.

Collateral cosmetic fixes.
2020-04-30 10:13:21 +02:00
Eugene Kosov
28325b0863 add WITH_DBUG_TRACE CMake variable
This is a way do disable DBUG_ENTER()/DBUG_EXIT() stuff which is
needed to dbug trace. Those who doesn't need it may avoid tests
slowdown with -DWITH_DBUG_TRACE=OFF

dbug/tests.c: add define which is neede always in this test

innodb.log_file_name_debug.test: do not depend on DBUG trace stuff
in test

Benchmark results: each test eats less CPU and you can have more
parallel jobs in MTR.

patched:
./mtr -mem -par=8 -suite=innodb  185.34s user 86.85s system 133% cpu 3:23.27 total
./mtr -mem -par=8 -suite=main    80.96s  user 36.01s system 182% cpu 1:04.07 total

main.select                              [ pass ]   1660
main.select                              [ pass ]   1513
main.select                              [ pass ]   1543
main.select                              [ pass ]   1660
main.select                              [ pass ]   1521
main.select                              [ pass ]   1511
main.select                              [ pass ]   1508
main.select                              [ pass ]   1520
main.select                              [ pass ]   1514
main.select                              [ pass ]   1522

vanilla:
./mtr -mem -par=8 -suite=innodb  203.61s user 92.16s system 140% cpu 3:30.16 total
./mtr -mem -par=8 -suite=main    94.11s  user 35.51s system 206% cpu 1:02.69 total

main.select                              [ pass ]   2032
main.select                              [ pass ]   2017
main.select                              [ pass ]   2040
main.select                              [ pass ]   2183
main.select                              [ pass ]   2253
main.select                              [ pass ]   2075
main.select                              [ pass ]   2109
main.select                              [ pass ]   2080
main.select                              [ pass ]   2098
main.select                              [ pass ]   2114
2020-04-29 20:13:14 +03:00
Sergei Petrunia
7bc6735736 MDEV-22401: Optimizer trace: multi-component range is not printed correctly
KEY_MULTI_RANGE::range_flag does not have correct flag bits for
per-endpoint flags (NEAR_MIN, NEAR_MAX, NO_MIN_RANGE, NO_MAX_RANGE).
It only has bits for flags that describe both endpoints.
So
- Document this.
- Switch optimizer trace to using {start|end}_key.flag values, instead.
  This fixes the bug.
- Switch records_in_column_ranges() to doing that too. (This used to
  work, because KEY_MULTI_RANGE::range_flag had correct flag value
  for the last key component, and EITS only uses one-component
  pseudo-indexes)
2020-04-29 16:31:16 +03:00
Marko Mäkelä
496d0372ef Merge 10.4 into 10.5 2020-04-29 15:40:51 +03:00
Daniel Black
c238e9b96a MDEV-20685: compile fixes for Solaris/OSX/AIX
sig_return: Solaris/OSX returns different function ptr
Move defination to my_alarm.h as its the only use.

prevents compile warnings (copied from 10.3 branch)

mysys/my_sync.c:136:19: error: 'cur_dir_name' defined but not used [-Werror=unused-const-variable=]
  136 | static const char cur_dir_name[]= {FN_CURLIB, 0};
      |                   ^~~~~~~~~~~~

fix compile error (DEPRECATED) leaked from ssl headers.

In file included from /export/home/dan/mariadb-server-10.4/sql/sys_vars.cc:37:
/export/home/dan/mariadb-server-10.4/sql/sys_vars.ic:69: error: "DEPRECATED" redefined [-Werror]
   69 | #define DEPRECATED(X) X
      |
In file included from /export/home/dan/mariadb-server-10.4/include/violite.h:150,
                 from /export/home/dan/mariadb-server-10.4/sql/sql_class.h:38,
                 from /export/home/dan/mariadb-server-10.4/sql/sys_vars.cc:36:
/usr/include/openssl/ssl.h:2356: note: this is the location of the previous definition
 2356 | # define DEPRECATED __attribute__((deprecated))
      |

Avoid Werror condition on non-Linux:

plugin/server_audit/server_audit.c:2267:7: error: variable 'db_len_off' set but not used [-Werror=unused-but-set-variable]
 2267 |   int db_len_off;
      |       ^~~~~~~~~~
plugin/server_audit/server_audit.c:2266:7: error: variable 'db_off' set but not used [-Werror=unused-but-set-variable]
 2266 |   int db_off;
      |       ^~~~~~

auth_gssapi fix include path for Solaris

Consistent with the upstream packaged patch:
https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/database/mariadb-103/patches/06-gssapi.h.patch

compile warnings on Solaris

[ 91%] Building C object plugin/server_audit/CMakeFiles/server_audit.dir/server_audit.c.o
/plugin/server_audit/server_audit.c: In function 'auditing_v8':
/plugin/server_audit/server_audit.c:2194:20: error: unused variable 'db_len_off' [-Werror=unused-variable]
 2194 |   static const int db_len_off= 128;
      |                    ^~~~~~~~~~
/plugin/server_audit/server_audit.c:2193:20: error: unused variable 'db_off' [-Werror=unused-variable]
 2193 |   static const int db_off= 120;
      |                    ^~~~~~
/plugin/server_audit/server_audit.c:2192:20: error: unused variable 'cmd_off' [-Werror=unused-variable]
 2192 |   static const int cmd_off= 4432;
      |                    ^~~~~~~
At top level:
/plugin/server_audit/server_audit.c:2192:20: error: 'cmd_off' defined but not used [-Werror=unused-const-variable=]
/plugin/server_audit/server_audit.c:2193:20: error: 'db_off' defined but not used [-Werror=unused-const-variable=]
 2193 |   static const int db_off= 120;
      |                    ^~~~~~
/plugin/server_audit/server_audit.c:2194:20: error: 'db_len_off' defined but not used [-Werror=unused-const-variable=]
 2194 |   static const int db_len_off= 128;
      |                    ^~~~~~~~~~
cc1: all warnings being treated as errors

tested on:
$ uname -a
SunOS openindiana 5.11 illumos-b97b1727bc i86pc i386 i86pc
2020-04-29 12:02:47 +03:00
Marko Mäkelä
b63446984c Merge 10.3 into 10.4 2020-04-27 17:38:17 +03:00
Marko Mäkelä
2e12d471ea Merge 10.2 into 10.3 2020-04-27 14:24:41 +03:00
Marko Mäkelä
fbe2712705 Merge 10.4 into 10.5
The functional changes of commit 5836191c8f
(MDEV-21168) are omitted due to MDEV-742 having addressed the issue.
2020-04-25 21:57:52 +03:00
Eugene Kosov
2c5067b689 cleanup THR_KEY_mysys
read TLS with my_thread_var
write TLS with set_mysys_var()

my_thread_var is no longer __attribute__ ((const)): this attribute
is simply incorrect here. Read gcc manual for more information.
sql/threadpool_generic.cc fails with that attribute.
2020-04-25 00:55:39 +03:00
Marko Mäkelä
edd38b50f6 MDEV-7962 wsrep_on() takes 0.14% in OLTP RO
The reason why we have wsrep_on() at all is that the macro WSREP(thd)
depends on the definition of THD, and that is intentionally an opaque
data type for InnoDB. So, we cannot avoid invoking wsrep_on(), but
we can evaluate the less expensive conditions thd && WSREP_ON before
calling the function.

Global_read_lock: Use WSREP_NNULL(thd) instead of wsrep_on(thd)
because we not only know the definition of THD but also that
the pointer is not null.

wsrep_open(): Use WSREP(thd) instead of wsrep_on(thd).

InnoDB: Replace thd && wsrep_on(thd) with wsrep_on(thd), now that
the condition has been merged to the definition of the macro
wsrep_on().
2020-04-24 16:01:10 +03:00
Marko Mäkelä
2c39f69d34 MDEV-22203: WSREP_ON is unnecessarily expensive WITH_WSREP=OFF
If the server is compiled WITH_WSREP=OFF, we should avoid evaluating
conditions on a global variable that is constant.

WSREP_ON_: Renamed from WSREP_ON. Defined only WITH_WSREP=ON.

WSREP_ON: Defined as unlikely(WSREP_ON_).

wsrep_on(): Defined as WSREP_ON && wsrep_service->wsrep_on_func().

The reason why we have wsrep_on() at all is that the macro WSREP(thd)
depends on the definition of THD, and that is intentionally an opaque
data type for InnoDB. So, we cannot avoid invoking wsrep_on(), but
we can evaluate the less expensive condition WSREP_ON before calling
the function.
2020-04-24 15:25:39 +03:00
Sergey Vojtovich
5679a2b6b3 Shrink my_atomic.h and my_cpu.h scope 2020-04-15 22:23:03 +04:00