Commit graph

2631 commits

Author SHA1 Message Date
Marko Mäkelä
e685809a3b MDEV-23397 Remove deprecated InnoDB options in 10.6 2020-08-04 12:51:59 +03:00
Marko Mäkelä
9a7948e3f6 Merge 10.5 into 10.6 2020-08-04 07:55:16 +03:00
Marko Mäkelä
bbd70fcc43 MDEV-23379 Deprecate&ignore InnoDB concurrency throttling parameters
The parameters innodb_thread_concurrency and innodb_commit_concurrency
were useful years ago when both computing resources and the implementation
of some shared data structures were limited. MySQL 5.0 or 5.1 had trouble
scaling beyond 8 concurrent connections. Most of the scalability bottlenecks
have been removed since then, and the transactions per second delivered
by MariaDB Server 10.5 should not dramatically drop upon exceeding the
'optimal' number of connections.

Hence, enabling any concurrency throttling for InnoDB actually makes
things worse. We have seen many customers mistakenly setting this to a
small value like 16 or 64 and then complaining the server was slow.

Ignoring the parameters allows us to remove some normally unused code
and data structures, which could slightly improve performance.

innodb_thread_concurrency, innodb_commit_concurrency,
innodb_replication_delay, innodb_concurrency_tickets,
innodb_thread_sleep_delay, innodb_adaptive_max_sleep_delay:
Deprecate and ignore; hard-wire to 0.

The column INFORMATION_SCHEMA.INNODB_TRX.trx_concurrency_tickets
will always report 0.
2020-08-04 06:59:29 +03:00
Marko Mäkelä
50a11f396a Merge 10.4 into 10.5 2020-08-01 14:42:51 +03:00
Marko Mäkelä
9216114ce7 Merge 10.3 into 10.4 2020-07-31 18:09:08 +03:00
Marko Mäkelä
66ec3a770f Merge 10.2 into 10.3 2020-07-31 13:51:28 +03:00
Thirunarayanan Balathandayuthapani
fe39d02f51 MDEV-20638 Remove the deadcode from srv_master_thread() and srv_active_wake_master_thread_low()
- Due to commit fe95cb2e40 (MDEV-16125),
InnoDB master thread does not need to call srv_resume_thread()
and therefore there is no need to wake up the thread.
Due to the above patch, InnoDB should remove the following dead code.

srv_check_activity(): Makes the parameter as in,out and returns the
recent activity value

innobase_active_small(): Removed

srv_active_wake_master_thread(): Removed

srv_wake_master_thread(): Removed

srv_active_wake_master_thread_low(): Removed

Simplify srv_master_thread() and remove switch cases, added the assert.

Replace srv_wake_master_thread() with srv_inc_activity_count()

INNOBASE_WAKE_INTERVAL: Removed
2020-07-23 16:23:20 +05:30
Monty
46ffd47f42 Fixed wrong free in comp_err 2020-07-23 10:54:33 +03:00
Vladislav Vaintroub
9701759b3d MDEV-23043 Refactor Windows service handling
Removed the existing nt_service classes - they provide little
abstraction, and only obscure a relatively simple service handling.
This replaces by similar code inspired by MS docs samples.

Service handling is now moved into winmain.cc, which contains
the main() function for Windows.

winmain provides reporting callbacks, which should be used by external code
,to report transitions from starting to running to shutting down to stopped.

Removed a  do-nothing ServiceMain thread, and the
non-working service "pause/continue".  Removed a lot of #ifdef __WIN__
code from mysqld.cc
2020-07-04 18:24:40 +02:00
Vladislav Vaintroub
272828a171 Merge branch '10.5' into 10.6 2020-07-04 11:53:26 +02:00
Marko Mäkelä
1813d92d0c Merge 10.4 into 10.5 2020-07-02 09:41:44 +03:00
Marko Mäkelä
1df1a63924 Merge 10.2 into 10.3 2020-07-02 06:17:51 +03:00
Oleksandr Byelkin
b0f836053b MDEV-22983: Mariabackup's --help option disappeared
return --help option
2020-07-01 23:30:44 +03:00
Marko Mäkelä
ea2bc974dc Merge 10.1 into 10.2 2020-07-01 12:03:55 +03:00
Oleksandr Byelkin
5018b998a7 return --help option 2020-06-23 17:07:03 +02:00
Daniel Black
37c88445e3 mtr: use env for perl
On FreeBSD, perl isn't in /usr/bin, its in /usr/local/bin or
elsewhere in the path.

Like storage/{maria/unittest/,}ma_test_* , we use /usr/bin/env to
find perl and run it.
2020-06-23 03:24:46 +02:00
Marko Mäkelä
cfd3d70ccb MDEV-22871: Remove pointer indirection for InnoDB hash_table_t
hash_get_n_cells(): Remove. Access n_cells directly.

hash_get_nth_cell(): Remove. Access array directly.

hash_table_clear(): Replaced with hash_table_t::clear().

hash_table_create(), hash_table_free(): Remove.

hash0hash.cc: Remove.
2020-06-18 14:16:01 +03:00
Marko Mäkelä
c515b1d092 Merge 10.4 into 10.5 2020-06-18 13:58:54 +03:00
Vlad Lesin
205b0ce6ad MDEV-22894: Mariabackup should not read [mariadb-client] option group
from configuration files
2020-06-18 12:19:48 +03:00
Vlad Lesin
9bdf35e90f MDEV-18215: mariabackup does not report unknown command line options
MDEV-21298: mariabackup doesn't read from the [mariadbd] and [mariadbd-X.Y]
server option groups from configuration files
MDEV-21301: mariabackup doesn't read [mariadb-backup] option group in
configuration file

All three issues require to change the same code, that is why their
fixes are joined in one commit.

The fix is in invoking load_defaults_or_exit() and handle_options() for
backup-specific groups separately from client-server groups to let the last
handle_options() call fail on unknown backup-specific options.

The order of options procesing is the following:
1) Load server groups and process server options, ignore unknown
options
2) Load client groups and process client options, ignore unknown
options
3) Load backup groups and process client-server options, exit on
unknown option
4) Process --mysqld-args command line options, ignore unknown options

New global flag my_handle_options_init_variables was added to have
ability to invoke handle_options() for the same allowed options set
several times without re-initialising previously set option values.

--password value destroying is moved from option processing callback to
mariabackup's handle_options() function to have ability to invoke server's
handle_options() several times for the same possible allowed options
set.

Galera invokes wsrep_sst_mariabackup.sh with mysqld command line
options to configure mariabackup as close to the server as possible.
It is not known what server options are supported by mariabackup when the
script is invoked. That is why new mariabackup option "--mysqld-args" is added,
all unknown options that follow this option will be silently ignored.

wsrep_sst_mariabackup.sh was also changed to:
- use "--mysqld-args" mariabackup option to pass mysqld options,
- remove deprecated innobackupex mode,
- remove unsupported mariabackup options:
    --encrypt
    --encrypt-key
    --rebuild-indexes
    --rebuild-threads
2020-06-14 13:23:07 +03:00
Sergei Golubchik
4adc1269cc FreeBSD compilation fixes
* FreeBSD calls amd64 what Linux calls x86_64
* signal returns void (*)(int)
* struct pam_message has char*, not const char*
* krb5_free_unparsed_name exists, but is deprecated
2020-06-04 23:52:46 +02:00
mysqlonarm
dec3f8ca69
MDEV-22641: Provide SIMD optimized wrapper for zlib crc32() (#1558)
Existing implementation used my_checksum (from mysys)
for calculating table checksum and binlog checksum.

This implementation was optimized for powerpc only and lacked
SIMD implementation for x86 (using clmul) and ARM
(using ACLE) instead used zlib-crc32.

mariabackup had its own copy of the crc32 implementation
using hardware optimized implementation only for x86 and lagged
hardware based implementation for powerpc and ARM.

Patch helps unifies all such calls and help aggregate all of them
using an unified interface my_checksum().

Said unification also enables hardware optimized calls for all
architecture viz. x86, ARM, POWERPC.
Default always fallback to zlib crc32.

Thanks to Daniel Black for reviewing, fixing and testing
PowerPC changes. Thanks to Marko and Daniel for early code feedback.
2020-06-01 11:34:06 +03:00
Marko Mäkelä
4a0b56f604 Merge 10.4 into 10.5 2020-05-31 10:28:59 +03:00
Marko Mäkelä
ca38b6e427 Merge 10.3 into 10.4 2020-05-26 11:54:55 +03:00
Marko Mäkelä
ecc7f305dd Merge 10.2 into 10.3 2020-05-25 19:41:58 +03:00
Oleksandr Byelkin
cf52dd174e MDEV-22545: my_vsnprintf behaves not as in C standard
Added parameter %T for string which should be visibly truncated.
2020-05-24 21:27:08 +02:00
Monty
c4bf4b7aef Fixed access to undefined memory found by valgrind and MSAN
When my_vsnprintf() is patched, the code protected disabled with
'WAITING_FOR_BUGFIX_TO_VSPRINTF' should be enabled again. Also all %b
formats in this patch should be revert to %s again
2020-05-23 12:29:10 +03:00
Marko Mäkelä
5ece2155cb Merge 10.4 into 10.5 2020-05-20 17:46:05 +03:00
Marko Mäkelä
2bf93a8fd6 Merge 10.3 into 10.4 2020-05-19 21:18:15 +03:00
Marko Mäkelä
79ed33c184 Merge 10.2 into 10.3 2020-05-19 17:05:05 +03:00
Vlad Lesin
0f9bfcc323 MDEV-22554: "mariabackup --prepare" exits with code 0 even though innodb
error is logged

The fix is to set flag in ib::error::~error() and check it in
mariabackup.

ib::error::error() is replaced with ib::warn::warn() in
AIO::linux_create_io_ctx() because of two reasons:

1) if we leave it as is, then mariabackup MTR tests will fail with --mem
option, because Linux AIO can not be used on tmpfs,

2) when Linux AIO can not be initialized, InnoDB falls back to simulated
AIO, so such sutiation is not fatal error, it should be treated as warning.
2020-05-19 11:25:56 +03:00
Vladislav Vaintroub
e2bc029211 MDEV-7021 Pass directory security descriptor from mysql_install_db.exe to bootstrap
This ensures that directory permissions are correct in all cases, even if
boostrap is passed non-standard locations for innodb.

Directory permissions are copied from the datadir.
2020-05-18 18:11:40 +02:00
Vladislav Vaintroub
8cb3060c5c MDEV-22272 Windows installer - run service unter virtual service account
Change mysql_install_db.exe to run service under virtual account.
Set directory permissions so that service has full access to data files.

mariabackup --copy-back permission handling (MDEV-17008) needs to be
changed as well.
Now, whenever a directory is created in course of copy-back,
its permissions are copied from the datadir. This handling assumes,
that datadir already has the correct permissions for the Windows service.
2020-05-18 18:07:26 +02:00
Sergei Golubchik
13038e4705 Merge branch '10.4' into 10.5 2020-05-09 20:43:36 +02:00
Sergei Golubchik
5b0df7433d WolfSSL fixes
remove Timeval workaround (not needed anymore).
add template workaround.
comments.
2020-05-08 12:49:53 +02:00
Vladislav Vaintroub
403dc759d0 Update WolfSSL
Fix WolfSSL build:

- Do not build with TLSv1.0,it stopped working,at least with SChannel client
- Disable a test that depends on TLSv1.0
- define FP_MAX_BITS always, to fix 32bit builds.
- Increase MAX_AES_CTX_SIZE, to fix build on Linux
2020-05-08 11:51:03 +02:00
Marko Mäkelä
18a62eb76d MDEV-21133 follow-up: Use fil_page_get_type()
Let us use the common accessor function fil_page_get_type()
instead of accessing the page header field FIL_PAGE_TYPE directly.
2020-05-07 17:15:34 +03:00
Marko Mäkelä
ba573c4736 MDEV-21133 follow-up: More my_assume_aligned hints
fsp0pagecompress.h: Remove.
Invoke fil_page_get_type() and FSP_FLAGS_GET_PAGE_COMPRESSION_LEVEL
directly.

log_block_get_flush_bit(), log_block_set_flush_bit():
Access the byte directly.

dict_sys_read_row_id(): Remove (unused function).
2020-05-07 12:25:00 +03:00
Eugene Kosov
89ff4176c1 MDEV-22437 make THR_THD* variable thread_local
Now all access goes through _current_thd() and set_current_thd()
functions.

Some functions like THD::store_globals() can not fail now.
2020-05-05 18:13:31 +03:00
Marko Mäkelä
496d0372ef Merge 10.4 into 10.5 2020-04-29 15:40:51 +03:00
Marko Mäkelä
0632b8034b Merge 10.3 into 10.4 2020-04-29 09:05:15 +03:00
Marko Mäkelä
1fbdcada73 Merge 10.2 into 10.3 2020-04-28 22:29:13 +03:00
Vlad Lesin
d0150dc14e MDEV-20230: mariabackup --ftwrl-wait-timeout never times out on explicit
lock

--ftwrl-wait-timeout does not finish mariabackup execution when acquired
backup lock can't be grabbed for the certain amount of time, it just
waits for a long queries finishing before acquiring the lock to avoid
unnecessary locking.

This commit extends --ftwrl-wait-timeout so, that mariabackup execution
is finished if it waits for backup lock during certain amount of time.
2020-04-27 22:10:50 +03:00
Marko Mäkelä
fbe2712705 Merge 10.4 into 10.5
The functional changes of commit 5836191c8f
(MDEV-21168) are omitted due to MDEV-742 having addressed the issue.
2020-04-25 21:57:52 +03:00
Marko Mäkelä
88cf6f1c7f Merge 10.3 into 10.4 2020-04-22 18:18:51 +03:00
Marko Mäkelä
455cf6196c Merge 10.2 into 10.3 2020-04-22 14:45:55 +03:00
Vlad Lesin
0efe1971c6 MDEV-19347: Mariabackup does not honor ignore_db_dirs from server
config.

The solution is to read the system variable value on startup and to fill
databases_exclude_hash.

xb_load_list_string() became non-static and was reformatted. The system
variable value is read and processed in get_mysql_vars(), which was also
reformatted.
2020-04-21 10:34:37 +03:00
Monty
749b988794 Fixed memory leaks in resolve_stack_dump
- Remove memory leaks reported by safemalloc
- Changed that all 0x strings are converted. This is needed
  to easily be able to resolve safemalloc backtraces
2020-04-18 11:51:42 +03:00
Marko Mäkelä
af91266498 Merge 10.3 into 10.4
In main.index_merge_myisam we remove the test that was added in
commit a2d24def8c because
it duplicates the test case that was added in
commit 5af12e4635.
2020-04-16 12:12:26 +03:00
Marko Mäkelä
a215e2132d mbstream: Remove duplicate definition of datasink_buffer
For some reason, when mbstream is compiled with GCC 10,
ld will fail to link it due to a multiple definition of
datasink_buffer. The code appears to have been like that
since the very beginning.

Let us remove the duplicate dummy definition.
2020-04-15 14:50:17 +03:00