Commit graph

186061 commits

Author SHA1 Message Date
Alexander Barkov
ac93d7d674 MDEV-19593 Split create_schema_table() into virtual methods in Type_handler 2019-05-25 11:41:18 +04:00
Yuqi
0928596a8b Armv8 CRC32 optimization (#772)
ARMv8 (AArch64) CPUs implement the CRC32 extension which is implemented by inline assembly ,
so they can also benefit from hardware acceleration in IO-intensive workloads.

The patch optimizes crc32c calculate with the armv8 crypto instruction(Intrinsics) when available
rather than original linear crc instructions.

Change-Id: I05d36a64c726d910c47befad93390108f4e6567f
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
2019-05-24 18:20:27 +04:00
Sergey Vojtovich
a74b01ea0e MDEV-16548 - Innodb fails to start on older kernels that don't support F_DUPFD_CLOEXEC
InnoDB duplicates file descriptor returned by create_temp_file() to
workaround further inconsistent use of this descriptor.

Use mysys file descriptors consistently for innobase_mysql_tmpfile(path).
Mostly close it by appropriate mysys wrappers.
2019-05-24 13:21:14 +04:00
Sergey Vojtovich
5f5a0b3bb6 MDEV-16548 - Innodb fails to start on older kernels that don't support F_DUPFD_CLOEXEC
InnoDB duplicates file descriptor returned by create_temp_file() to
workaround further inconsistent use of this descriptor.

Use mysys file descriptors consistently for innobase_mysql_tmpfile(NULL).
Mostly close it by appropriate mysys wrappers.
2019-05-24 13:21:09 +04:00
Marko Mäkelä
893472d005 MDEV-19570 Deprecate and ignore innodb_undo_logs, remove innodb_rollback_segments
The option innodb_rollback_segments was deprecated already in
MariaDB Server 10.0. Its misleadingly named replacement innodb_undo_logs
is of very limited use. It makes sense to always create and use the
maximum number of rollback segments.

Let us remove the deprecated parameter innodb_rollback_segments and
deprecate&ignore the parameter innodb_undo_logs (to be removed in a
later major release).

This work involves some cleanup of InnoDB startup. Similar to other
write operations, DROP TABLE will no longer be allowed if
innodb_force_recovery is set to a value larger than 3.
2019-05-23 17:34:47 +03:00
Alexander Barkov
c83018751c MDEV-19566 Remove Item::name related strlen() calls in constructors of some Item_string descendands 2019-05-23 14:58:03 +04:00
Marko Mäkelä
826f9d4f7e Merge 10.4 into 10.5 2019-05-23 10:32:21 +03:00
Marko Mäkelä
e5d71e0b3d MDEV-19551 Remove alias innodb_stats_sample_pages
The parameter innodb_stats_sample_pages became an alias for
innodb_stats_transient_sample_pages and was deprecated in
MariaDB Server 10.0. Let us finally remove that alias.
2019-05-23 10:25:12 +03:00
Marko Mäkelä
1a6f470464 MDEV-19544 Remove innodb_locks_unsafe_for_binlog
The transaction isolation levels READ COMMITTED and READ UNCOMMITTED
should behave similarly to the old deprecated setting
innodb_locks_unsafe_for_binlog=1, that is, avoid acquiring gap locks.

row_search_mvcc(): Reduce the scope of some variables, and clean up
the initialization and use of the variable set_also_gap_locks.
2019-05-23 10:25:12 +03:00
Marko Mäkelä
47cede646b MDEV-19543 Deprecate and ignore innodb_log_checksums
The parameter innodb_log_checksums that was introduced in MariaDB 10.2.2
via mysql/mysql-server@af0acedd88
does not make much sense. The original motivation of introducing this
parameter (initially called innodb_log_checksum_algorithm in
mysql/mysql-server@22ba38218e)
was that the InnoDB redo log used the slow and insecure innodb algorithm.
With hardware or SIMD accelerated CRC-32C, there should be no reason to
allow checksums to be disabled on the redo log.

The parameter innodb_encrypt_log already implies innodb_log_checksums=ON.

Let us deprecate the parameter innodb_log_checksums and always compute
redo log checksums, even if innodb_log_checksums=OFF is specified.

An upgrade from MariaDB 10.2.2 or later will only be possible after
using the default value innodb_log_checksums=ON. If the non-default
value innodb_log_checksums=OFF was in effect when the server was shut down,
a log block checksum mismatch will be reported and the upgraded server
will fail to start up.
2019-05-23 10:25:11 +03:00
Marko Mäkelä
3eef9f213f MDEV-17841: Fix -Wsometimes-uninitialized
maria_open(): Initialize share->mode=0 for S3
2019-05-23 10:18:55 +03:00
Marko Mäkelä
483536ec3a Adjust innodb.innodb-wl5522-debug result
The test result was affected by
commit 007f68c37f
but the adjustment was inadvertently omitted.
2019-05-23 10:17:00 +03:00
Alexander Barkov
55a2ca3e6a MDEV-19550 Move specific parts of log_event.cc to log_event_client.cc and log_event_server.cc 2019-05-23 05:23:42 +04:00
Monty
ebfe8c4e0e Make it trivial to get stack traces from external programs.
To get a stacktrace in a program like aria_chk, one only have to do:

#include <my_stacktrace.h>

call my_init_stacktrace(1) in main().
2019-05-23 02:28:23 +03:00
Monty
ab38b7511b MDEV-17841 S3 storage engine
A read-only storage engine that stores it's data in (aws) S3

To store data in S3 one could use ALTER TABLE:
ALTER TABLE table_name ENGINE=S3

libmarias3 integration done by Sergei Golubchik
libmarias3 created by Andrew Hutchings
2019-05-23 02:28:23 +03:00
Sergei Golubchik
2ca2dcac6a aria is mandatory now, so don't bother with CMAKE_DEPENDENT_OPTION 2019-05-23 01:20:18 +03:00
Monty
043a3a0176 Avoid not needed renames in ALTER TABLE
Removed not needed table renames when doing ALTER TABLE when engine
changes and both of the following is true:
- Either new or old engine does not store the table in files
- Neither old or new engine uses files from another engine

We also skip renames when ALTER TABLE does an explicit rename

This improves performance, especially for engines where rename is
a slow operation (like the upcoming S3 engine)
2019-05-23 01:20:18 +03:00
Monty
10e8ba13c6 ha_discover_table_names() now always remove duplicates
Discoverred to be needed by the upcoming S3 engine
2019-05-23 01:20:18 +03:00
Monty
007f68c37f Replace ha_notify_table_changed() with notify_tabledef_changed()
Reason for the change was that ha_notify_table_changed() was done
after table open when .frm had been replaced, which caused failure
in engines that checks on open if .frm matches the engines table
definition.

Other changes:
- Remove not needed open/close call at end of inline alter table.
  Some test that depended on the table beeing in the table cache after
  ALTER TABLE had to be updated.
2019-05-23 01:20:18 +03:00
Monty
96037a6f03 Updated error message for HA_ERR_INCOMPATIBLE_DEFINITION 2019-05-23 01:20:18 +03:00
Monty
e533ba9b84 Renamed tmp file using #sql_#_# to #sql-#-#
This was done to match the naming of the ALTER #sql-#-# tmp files
2019-05-23 01:20:18 +03:00
Monty
ebf372ddae Indentation cleanups 2019-05-23 01:20:17 +03:00
Monty
0b9f7f86f8 Removed not used function maria_clone() 2019-05-23 01:20:17 +03:00
Monty
2faa7dcd5c Updated debian packages to 10.5 2019-05-23 01:20:17 +03:00
Monty
31fe70290c Fixed sometimes wrong result in main.subselect_sj2_mat 2019-05-22 10:30:20 +03:00
Marko Mäkelä
437da7bc54 MDEV-19534 Make innodb_checksum_algorithm=full_crc32 by default, and remove innodb_checksums 2019-05-22 08:53:28 +03:00
Marko Mäkelä
b40c99a82c MDEV-17458: Clear more of the TRX_SYS page
trx_rseg_array_init(): Using the 10.4 specific MLOG_MEMSET record,
clear the entire TRX_SYS_WSREP_XID_INFO field.
2019-05-22 08:42:41 +03:00
Marko Mäkelä
cf77951fb6 Merge 10.3 into 10.4 2019-05-22 08:42:31 +03:00
Sergey Vojtovich
424dc49d41 Attempt fixing ERR_remove_state warning 2019-05-21 19:27:26 +04:00
Oleksandr Byelkin
1921df6697 MDEV-19540: 10.4 allow lock options with SELECT in brackets which previous version do not
Check locking options and brackets combinations.
2019-05-21 16:10:03 +02:00
Sergey Vojtovich
54b81cf6ca mysql_socket_accept() microoptimisations
No need to initialize socket_accept, it always gets assigned a value.

Pass addr_len directly to accept().

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
a61baa7a25 Maintain connection_count atomically
Removed LOCK_connection_count.

Removed duplicate denied_connections and connection_errors_max_connection
increment from create_new_thread(). Another increment done by
CONNECT::close_with_error().

Simplified away CONNECT::thread_count_incremented. Now close_with_error()
is always called with connection_count incremented.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
0bee021b78 Simplified away wake_thread
cached_thread_count can handle it alone.

Added "fast path": when thread is being added to the cache and there's
queued connection request, serve it immediately.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
34dfcbe3a6 Move thread re-initialisation out of cache_thread
It mostly duplicates code of do_handle_one_connection(). Move it there,
so that it is easier to simplify it in future.

Moved PSI_CALL_delete_current_thread()/_db_pop_() out of mutex. Call them
consistently in all cases, unlike before:
- called for reused threads
- called for flushed threads
- called for threads that go down after 5 minutes timeout
- not called for threads attempting to cache themselves during flush
- not called for threads attempting to cache themselves when cache is full

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ebc55c8577 Simplified away scheduler_functions::end_thread()
Code properly integrated into callers instead.

do_handle_one_connection(): no need to reset thd and thd->thread_stack
as they're not updated by cache_thread().

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
6900aaf417 Simplified away init_new_connection_thread()
It was meaningful only for one-thread-per-connection scheduler anyway,
so call init_new_connection_handler_thread() directly from
handle_one_connection().

It was somewhat harmful for no-threads scheduler, because it'd attempt to
detach main thread on AIX_3_2 and Siemens unix (and DEC OSF/1 3.2 too).
Also it does duplicate my_thread_init() call, which may produce a warning
under EXTRA_DEBUG_THREADS.

start_wsrep_THD() shouldn't have called it at all: it already detached
and called my_thread_init().

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
8d9d4aa6d6 Signal COND_thread_cache out of mutex
Gives ~10% throughput improvemet in sysbench connect benchmark.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5d183df77b Try accept a few times before falling back to poll
Gives ~3% throughput improvemet in sysbench connect benchmark.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
87775402cd Improved ha_close_connection() scalability
Rather than iterating global plugin collection, iterate THD local
collection. Removes two LOCK_plugin locks per connection.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5e139437a5 Adiue thd_ha_data(), you've broke many hearts
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ec926b0f40 Fixed RocksDB to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in rocksdb_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Removed transaction objects cleanup from rocksdb_done_func(). As we lock
plugin properly, there must be no transaction objects during RocksDB
shutdown.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
8c8d584f06 Fixed InnoDB to not use broken thd_ha_data()
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5e1b3cc8dc Fixed Aria to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ba59cc0f37 Fixed Sphinx to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in sphinx_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
00e533c78b Fixed Mroonga to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in mrn_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5c18ba6c88 Fixed Spider to follow THD ha_data protocol
Do not reset THD ha_data in spider_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
762d2b96fa Fixed FederatedX to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in ha_federatedx::disconnect(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ce30c99478 Moved vio allocation to connection thread
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
efb61c12a9 Simplified away CONNECT::real_id
It was wrong anyway, as it may not get updated by the time created thread
reads it. And it was 0 for cached threads.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
7192d7b700 Simplified away CONNECT::extra_port
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00