Commit graph

186055 commits

Author SHA1 Message Date
Marko Mäkelä
826f9d4f7e Merge 10.4 into 10.5 2019-05-23 10:32:21 +03:00
Marko Mäkelä
e5d71e0b3d MDEV-19551 Remove alias innodb_stats_sample_pages
The parameter innodb_stats_sample_pages became an alias for
innodb_stats_transient_sample_pages and was deprecated in
MariaDB Server 10.0. Let us finally remove that alias.
2019-05-23 10:25:12 +03:00
Marko Mäkelä
1a6f470464 MDEV-19544 Remove innodb_locks_unsafe_for_binlog
The transaction isolation levels READ COMMITTED and READ UNCOMMITTED
should behave similarly to the old deprecated setting
innodb_locks_unsafe_for_binlog=1, that is, avoid acquiring gap locks.

row_search_mvcc(): Reduce the scope of some variables, and clean up
the initialization and use of the variable set_also_gap_locks.
2019-05-23 10:25:12 +03:00
Marko Mäkelä
47cede646b MDEV-19543 Deprecate and ignore innodb_log_checksums
The parameter innodb_log_checksums that was introduced in MariaDB 10.2.2
via mysql/mysql-server@af0acedd88
does not make much sense. The original motivation of introducing this
parameter (initially called innodb_log_checksum_algorithm in
mysql/mysql-server@22ba38218e)
was that the InnoDB redo log used the slow and insecure innodb algorithm.
With hardware or SIMD accelerated CRC-32C, there should be no reason to
allow checksums to be disabled on the redo log.

The parameter innodb_encrypt_log already implies innodb_log_checksums=ON.

Let us deprecate the parameter innodb_log_checksums and always compute
redo log checksums, even if innodb_log_checksums=OFF is specified.

An upgrade from MariaDB 10.2.2 or later will only be possible after
using the default value innodb_log_checksums=ON. If the non-default
value innodb_log_checksums=OFF was in effect when the server was shut down,
a log block checksum mismatch will be reported and the upgraded server
will fail to start up.
2019-05-23 10:25:11 +03:00
Marko Mäkelä
3eef9f213f MDEV-17841: Fix -Wsometimes-uninitialized
maria_open(): Initialize share->mode=0 for S3
2019-05-23 10:18:55 +03:00
Marko Mäkelä
483536ec3a Adjust innodb.innodb-wl5522-debug result
The test result was affected by
commit 007f68c37f
but the adjustment was inadvertently omitted.
2019-05-23 10:17:00 +03:00
Alexander Barkov
55a2ca3e6a MDEV-19550 Move specific parts of log_event.cc to log_event_client.cc and log_event_server.cc 2019-05-23 05:23:42 +04:00
Monty
ebfe8c4e0e Make it trivial to get stack traces from external programs.
To get a stacktrace in a program like aria_chk, one only have to do:

#include <my_stacktrace.h>

call my_init_stacktrace(1) in main().
2019-05-23 02:28:23 +03:00
Monty
ab38b7511b MDEV-17841 S3 storage engine
A read-only storage engine that stores it's data in (aws) S3

To store data in S3 one could use ALTER TABLE:
ALTER TABLE table_name ENGINE=S3

libmarias3 integration done by Sergei Golubchik
libmarias3 created by Andrew Hutchings
2019-05-23 02:28:23 +03:00
Sergei Golubchik
2ca2dcac6a aria is mandatory now, so don't bother with CMAKE_DEPENDENT_OPTION 2019-05-23 01:20:18 +03:00
Monty
043a3a0176 Avoid not needed renames in ALTER TABLE
Removed not needed table renames when doing ALTER TABLE when engine
changes and both of the following is true:
- Either new or old engine does not store the table in files
- Neither old or new engine uses files from another engine

We also skip renames when ALTER TABLE does an explicit rename

This improves performance, especially for engines where rename is
a slow operation (like the upcoming S3 engine)
2019-05-23 01:20:18 +03:00
Monty
10e8ba13c6 ha_discover_table_names() now always remove duplicates
Discoverred to be needed by the upcoming S3 engine
2019-05-23 01:20:18 +03:00
Monty
007f68c37f Replace ha_notify_table_changed() with notify_tabledef_changed()
Reason for the change was that ha_notify_table_changed() was done
after table open when .frm had been replaced, which caused failure
in engines that checks on open if .frm matches the engines table
definition.

Other changes:
- Remove not needed open/close call at end of inline alter table.
  Some test that depended on the table beeing in the table cache after
  ALTER TABLE had to be updated.
2019-05-23 01:20:18 +03:00
Monty
96037a6f03 Updated error message for HA_ERR_INCOMPATIBLE_DEFINITION 2019-05-23 01:20:18 +03:00
Monty
e533ba9b84 Renamed tmp file using #sql_#_# to #sql-#-#
This was done to match the naming of the ALTER #sql-#-# tmp files
2019-05-23 01:20:18 +03:00
Monty
ebf372ddae Indentation cleanups 2019-05-23 01:20:17 +03:00
Monty
0b9f7f86f8 Removed not used function maria_clone() 2019-05-23 01:20:17 +03:00
Monty
2faa7dcd5c Updated debian packages to 10.5 2019-05-23 01:20:17 +03:00
Monty
31fe70290c Fixed sometimes wrong result in main.subselect_sj2_mat 2019-05-22 10:30:20 +03:00
Marko Mäkelä
437da7bc54 MDEV-19534 Make innodb_checksum_algorithm=full_crc32 by default, and remove innodb_checksums 2019-05-22 08:53:28 +03:00
Marko Mäkelä
b40c99a82c MDEV-17458: Clear more of the TRX_SYS page
trx_rseg_array_init(): Using the 10.4 specific MLOG_MEMSET record,
clear the entire TRX_SYS_WSREP_XID_INFO field.
2019-05-22 08:42:41 +03:00
Marko Mäkelä
cf77951fb6 Merge 10.3 into 10.4 2019-05-22 08:42:31 +03:00
Sergey Vojtovich
424dc49d41 Attempt fixing ERR_remove_state warning 2019-05-21 19:27:26 +04:00
Oleksandr Byelkin
1921df6697 MDEV-19540: 10.4 allow lock options with SELECT in brackets which previous version do not
Check locking options and brackets combinations.
2019-05-21 16:10:03 +02:00
Sergey Vojtovich
54b81cf6ca mysql_socket_accept() microoptimisations
No need to initialize socket_accept, it always gets assigned a value.

Pass addr_len directly to accept().

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
a61baa7a25 Maintain connection_count atomically
Removed LOCK_connection_count.

Removed duplicate denied_connections and connection_errors_max_connection
increment from create_new_thread(). Another increment done by
CONNECT::close_with_error().

Simplified away CONNECT::thread_count_incremented. Now close_with_error()
is always called with connection_count incremented.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
0bee021b78 Simplified away wake_thread
cached_thread_count can handle it alone.

Added "fast path": when thread is being added to the cache and there's
queued connection request, serve it immediately.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
34dfcbe3a6 Move thread re-initialisation out of cache_thread
It mostly duplicates code of do_handle_one_connection(). Move it there,
so that it is easier to simplify it in future.

Moved PSI_CALL_delete_current_thread()/_db_pop_() out of mutex. Call them
consistently in all cases, unlike before:
- called for reused threads
- called for flushed threads
- called for threads that go down after 5 minutes timeout
- not called for threads attempting to cache themselves during flush
- not called for threads attempting to cache themselves when cache is full

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ebc55c8577 Simplified away scheduler_functions::end_thread()
Code properly integrated into callers instead.

do_handle_one_connection(): no need to reset thd and thd->thread_stack
as they're not updated by cache_thread().

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
6900aaf417 Simplified away init_new_connection_thread()
It was meaningful only for one-thread-per-connection scheduler anyway,
so call init_new_connection_handler_thread() directly from
handle_one_connection().

It was somewhat harmful for no-threads scheduler, because it'd attempt to
detach main thread on AIX_3_2 and Siemens unix (and DEC OSF/1 3.2 too).
Also it does duplicate my_thread_init() call, which may produce a warning
under EXTRA_DEBUG_THREADS.

start_wsrep_THD() shouldn't have called it at all: it already detached
and called my_thread_init().

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
8d9d4aa6d6 Signal COND_thread_cache out of mutex
Gives ~10% throughput improvemet in sysbench connect benchmark.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5d183df77b Try accept a few times before falling back to poll
Gives ~3% throughput improvemet in sysbench connect benchmark.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
87775402cd Improved ha_close_connection() scalability
Rather than iterating global plugin collection, iterate THD local
collection. Removes two LOCK_plugin locks per connection.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5e139437a5 Adiue thd_ha_data(), you've broke many hearts
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ec926b0f40 Fixed RocksDB to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in rocksdb_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Removed transaction objects cleanup from rocksdb_done_func(). As we lock
plugin properly, there must be no transaction objects during RocksDB
shutdown.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
8c8d584f06 Fixed InnoDB to not use broken thd_ha_data()
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5e1b3cc8dc Fixed Aria to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ba59cc0f37 Fixed Sphinx to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in sphinx_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
00e533c78b Fixed Mroonga to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in mrn_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5c18ba6c88 Fixed Spider to follow THD ha_data protocol
Do not reset THD ha_data in spider_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
762d2b96fa Fixed FederatedX to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in ha_federatedx::disconnect(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ce30c99478 Moved vio allocation to connection thread
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
efb61c12a9 Simplified away CONNECT::real_id
It was wrong anyway, as it may not get updated by the time created thread
reads it. And it was 0 for cached threads.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
7192d7b700 Simplified away CONNECT::extra_port
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
c90c769807 Simplified away CONNECT::host
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
56b1cdde37 Removed duplicate thread cache check
It may be beneficial for slow-path, where new thread is to be created.
However it is harmful for fast-path, where thread cache can fulfil
request: it caused 2 extra loads of write-contended variables.

Also shrink variables scope.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
701e2a7edb Optimised fcntl() when accepting connections
Removed FD_CLOEXEC setting: already done by mysql_socket_accept().

Moved O_NONBLOCK setting out of accepting loop.

Removed blocking acceptance attempt, which was dead code. It was supposed
to be executed before the last iteration, however it was actually executed
during the last iteration. And it is not correct to block on one socket
leaving other sockets unattended anyway.

Gives ~5% throughput improvemet in sysbench connect benchmark.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
218a68bbc5 Less shared variables loads under the mutex
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
8268fa881f Moved set_timespec out of LOCK_thread_cache mutex
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ca847584eb Cleanup redundant abort_loop checks
abort_loop is intended to break accepting connections loop in main thread,
however it is being used for other purposes.

cache_thread() is governed by kill_cached_threads, no need to check
abort_loop here.

Check in create_new_thread() is redundant, abort_loop already checked
by caller.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00