Commit graph

179931 commits

Author SHA1 Message Date
Sergey Vojtovich
159a6c2e60 MDEV-14505 - Threads_running becomes scalability bottleneck
Instead of updating global counter, calculate Threads_running on the fly.
All threads having command != COM_SLEEP are included.

Behaviour changes:

Previously SHOW STATUS and SHOW GLOBAL STATUS returned the same values
representing global status. Now SHOW STATUS always returns 1 indicating that
current session has 1 thread running.

Previously only threads that were executing dispatch_command() or running events
were accounted by Threads_running. Now it is rough equivalent of
SELECT COUNT(*) FROM INFORMATION_SCHEMA.PROCESSLIST WHERE state!='Sleep'
2017-12-13 12:46:33 +04:00
Jacob Mathew
751e0f6b1e Merge remote-tracking branch 'origin/bb-10.3-MDEV-14568' into 10.3 2017-12-12 10:58:39 -08:00
Aleksey Midenkov
364f42d091 Scripts: --tail-lines option for mtr (#493) 2017-12-12 19:03:42 +02:00
Monty
d4a4185451 MDEV Assertion `partition_id == m_extra_cache_part_id' failed in ha_partition::late_extra_no_cache
The problem was that multi_range_read_info_const() called
multi_range_key_create_key() which changed m_part_spec.start_part,
while there was an activ table scan ongoing.

Fixed by copying and restoring m_part_spec around
multi_range_key_create_calls()
2017-12-12 15:10:53 +02:00
Marko Mäkelä
0edd395e8d Merge bb-10.2-ext into 10.3 2017-12-12 13:32:49 +02:00
Marko Mäkelä
0628123617 Merge 10.2 into bb-10.2-ext 2017-12-12 13:27:35 +02:00
Marko Mäkelä
ccf22b8025 Post-merge fix: Adjust message codes in results 2017-12-12 13:27:16 +02:00
Sergei Petrunia
86c69263a4 MDEV-14389: MyRocks and NOPAD collations
Part #2: Take into accont the fact that three binary collations that
we handle are technically NOPAD collations
2017-12-12 13:31:41 +03:00
Marko Mäkelä
c2c4c1952d Remove an unused error code 2017-12-12 09:57:40 +02:00
Marko Mäkelä
34841d2305 Merge bb-10.2-ext into 10.3 2017-12-12 09:57:17 +02:00
Marko Mäkelä
a285e68018 Merge 10.2 into bb-10.2-ext 2017-12-12 09:15:17 +02:00
Varun Gupta
62eaf7b657 MDEV-14573: Upgrade from previous versions is broken for procedures 2017-12-12 12:09:52 +05:30
Sergei Petrunia
e12f77a7e3 MDEV-14389: MyRocks and NOPAD collations
Disallow use of NOPAD collations in indexed columns.
2017-12-12 01:33:03 +03:00
Vladislav Vaintroub
70d6c944cb MDEV-14113 Use abortive TCP close, in case server closes the connection
first, and we do not not care whether client has received all data.

This is a TCP optimization to avoid TIME_WAIT in TCP connection teardown.

This patch would abort connection on timeout, which usually happens when
client reads a large result set, at slower pace then the server can
write.

The patch also cleans up socket timeout handling, so that Windows
is consistent with another platforms (using nonblocking socket IO
 + waiting in poll/select on single socket, rather than setsockopt).
This makes identifying timeouts easier.


Also removed the superficial shutdown() before closesocket() in a few
places where it was used, because it  was never needed , and
reportedly breaks SO_LINGER on Windows.
2017-12-11 14:54:17 +00:00
Marko Mäkelä
8695c816e7 Wrap __atomic_compare_exchange_n() as valid C 2017-12-11 15:57:51 +02:00
Marko Mäkelä
e312a407b8 Merge 10.2 into bb-10.2-ext 2017-12-11 15:06:11 +02:00
Marko Mäkelä
13b9ec651a MDEV-14589 InnoDB should not lock a delete-marked record
When the transaction isolation level is SERIALIZABLE, or when
a locking read is performed in the REPEATABLE READ isolation level,
InnoDB must lock delete-marked records in order to prevent another
transaction from inserting something.

However, at READ UNCOMMITTED or READ COMMITTED isolation level or
when the parameter innodb_locks_unsafe_for_binlog is set, the
repeatability of the reads does not matter, and there is no need
to lock any records.

row_search_mvcc(): Skip locks on delete-marked committed records upfront,
instead of invoking row_unlock_for_mysql() afterwards. The unlocking
never worked for secondary index records.
2017-12-11 14:39:53 +02:00
Marko Mäkelä
434c9e6f0e MDEV-14614 InnoDB: Failing assertion in dict_stats_rename_table()
dict_stats_rename_table(): After DB_LOCK_WAIT_TIMEOUT
or DB_DUPLICATE_KEY, reset the trx->error_state before retrying.
Also, properly treat DB_DEADLOCK as a hard error.
2017-12-11 13:56:50 +02:00
Marko Mäkelä
1e6ac94451 Correct the comment of row_vers_impl_x_locked() 2017-12-11 13:56:36 +02:00
Sergei Petrunia
bdeb27a000 MDEV-14123: .rocksdb folder may break workflow which re-create data directory
Part #2: Adjust other parts of testsuite to take into account
that RocksDB files are in "#rocksdb" now.
2017-12-11 12:31:59 +03:00
Sergei Petrunia
ddc1d6904a MDEV-14123: .rocksdb folder may break workflow which re-create data directory
Part2: make MyRocks add its directory into @@ignore_db_dirs when starting.

This is necessary because apparently not everybody are using plugin's my.cnf
So load ha_rocksdb.{so,dll} manually and then hit MDEV-12451, MDEV-14461
etc.
2017-12-11 12:31:58 +03:00
Sergei Petrunia
b8a0373ed2 MDEV-14123: ".rocksdb folder may break workflow", and other MDEVs
Rename  Use '#rocksdb' instead of '.rocksdb' as subdirectory for RocksDB
datadir.
2017-12-11 12:31:58 +03:00
Marko Mäkelä
40eee1da17 MDEV-14614 InnoDB: Failing assertion: trx->error_state == DB_SUCCESS or lock wait timeout upon saving statistics
The assertion failure was caused by
MDEV-14511 Use fewer transactions for updating InnoDB persistent statistics

We are reusing a transaction object after commit, and sometimes,
even after a successful operation, the trx_t::error_state may be
something else than DB_SUCCESS. Reset the field when needed.
2017-12-11 10:12:14 +02:00
Marko Mäkelä
86beb08774 Remove an unnecessary dependency on persistent statistics 2017-12-11 09:31:28 +02:00
Varun Gupta
8f581e8bf1 Removing files sql/item_sum.cc.orig and sql/item_sum.h.orig added by
commit 6d63a03490
    MDEV-11297: Add support for LIMIT clause in GROUP_CONCAT()
2017-12-10 23:10:43 +05:30
Vesa Pentti
99bcec295d MDEV-12501 -- set --maturity-level by default
* Note: breaking change; since this commit, a plugin that has
    worked so far might get rejected due to plugin maturity
  * mariabackup is not affected (allows all plugins)
  * VERSION file defines SERVER_MATURITY, which defines the
    corresponding numeric value as SERVER_MATURITY_LEVEL in
    include/mysql_version.h
  * The default value for 'plugin_maturity' is SERVER_MATURITY_LEVEL - 1
  * Logs a warning if a plugin has maturity lower than
    SERVER_MATURITY_LEVEL
  * Tests suppress the plugin maturity warning
  * Tests use --plugin-maturity=unknown by default so as not to fail
    due to the stricter plugin maturity handling
2017-12-09 23:34:43 +00:00
Sergey Vojtovich
927dd9f355 Fixed LF_BACKOFF calls 2017-12-08 21:35:29 +04:00
Marko Mäkelä
0af52734a7 Remove the unused function row_is_magic_monitor_table()
Before MySQL 5.7 or MariaDB 10.2.2, there used to be some
magic InnoDB table names that would assign some InnoDB flags
on CREATE TABLE or DROP TABLE.
2017-12-08 16:36:19 +02:00
Marko Mäkelä
51bc407403 Remove dead code for "InnoDB table(space) monitor"
Starting with MySQL 5.7 (or MariaDB 10.2.2) InnoDB no longer contains
the "table monitor" or "tablespace monitor". The conditions on
srv_print_innodb_tablespace_monitor, srv_print_innodb_table_monitor
never hold. So, the code was dead.

Also, remove a bogus reference to dict_print(), which used to implement
the InnoDB table monitor.
2017-12-08 16:31:54 +02:00
Marko Mäkelä
3945049809 Remove space_name_list_t
fil_get_space_names(): Remove.

fts_drop_orphaned_tables(): Iterate fil_system->space_list directly.
2017-12-08 16:00:25 +02:00
Marko Mäkelä
094b0f869b Merge 10.2 into bb-10.2-ext 2017-12-08 16:00:10 +02:00
Sergey Vojtovich
c60095a818 Faster atomic loads and stores on windows 2017-12-08 17:55:41 +04:00
Sergey Vojtovich
b04f2a0f01 MDEV-14529 - InnoDB rw-locks: optimize memory barriers
Relax memory barrier for lock_word.

rw_lock_lock_word_decr() - used to acquire rw-lock, thus we only need to issue
ACQUIRE when we succeed locking.

rw_lock_x_lock_func_nowait() - same as above, but used to attempt to acquire
X-lock.

rw_lock_s_unlock_func() - used to release S-lock, RELEASE is what we need here.

rw_lock_x_unlock_func() - used to release X-lock. Ideally we'd need only RELEASE
here, but due to mess with waiters (they must be loaded after lock_word is
stored) we have to issue both ACQUIRE and RELEASE.

rw_lock_sx_unlock_func() - same as above, but used to release SX-lock.

rw_lock_s_lock_spin(), rw_lock_x_lock_func(), rw_lock_sx_lock_func() -
fetch-and-store to waiters has to issue only ACQUIRE memory barrier, so that
waiters are stored before lock_word is loaded.

Note that there is violation of RELEASE-ACQUIRE protocol here, because we do
on lock:

  my_atomic_fas32_explicit((int32*) &lock->waiters, 1, MY_MEMORY_ORDER_ACQUIRE);
  my_atomic_load32_explicit(&lock->lock_word, MY_MEMORY_ORDER_RELAXED);

on unlock

  my_atomic_add32_explicit(&lock->lock_word, X_LOCK_DECR, MY_MEMORY_ORDER_ACQ_REL);
  my_atomic_load32_explicit((int32*) &lock->waiters, MY_MEMORY_ORDER_RELAXED);

That is we kind of synchronize ACQUIRE on lock_word with ACQUIRE on waiters.
It was there before this patch. Simple fix may have negative performance impact.
Proper fix requires refactoring of lock_word.
2017-12-08 17:55:41 +04:00
Sergey Vojtovich
51bb18f989 MDEV-14529 - InnoDB rw-locks: optimize memory barriers
Relax memory barrier for waiters: these 2 stores must be completed before
os_event_set() finishes. This is guaranteed by RELEASE barrier issued by
mutex.exit() of os_event_set().
2017-12-08 17:55:41 +04:00
Sergey Vojtovich
5b624f00fc MDEV-14529 - InnoDB rw-locks: optimize memory barriers
Remove volatile modifier from waiters: it's not supposed for inter-thread
communication, use appropriate atomic operations instead.

Changed waiters to int32_t, my_atomic friendly type.
2017-12-08 17:55:41 +04:00
Sergey Vojtovich
57d20f1132 MDEV-14529 - InnoDB rw-locks: optimize memory barriers
Remove volatile modifier from lock_word: it's not supposed for inter-thread
communication, use appropriate atomic operations instead.
2017-12-08 17:55:41 +04:00
Sergey Vojtovich
c73e77da0f MDEV-14529 - InnoDB rw-locks: optimize memory barriers
Change lock_word from lint to int32_t: the latter is my_atomic_* friendly type.
2017-12-08 17:55:41 +04:00
Vladislav Vaintroub
bf96310657 Fix warnings 2017-12-08 13:10:41 +00:00
Monty
38908aaf81 Fix failing mtr tests
- Changed rocksdb.2pcgroup_commit.test to print information on error
- Updated myisam_views-big.result
2017-12-08 13:46:23 +02:00
Monty
c2118a08b1 Move all kill mutex protection to LOCK_thd_kill
LOCK_thd_data was used to protect both THD data and
ensure that the THD is not deleted while it was in use

This patch moves the THD delete protection to LOCK_thd_kill,
which already protects the THD for kill.

The benefits are:
- More well defined what LOCK_thd_data protects
- LOCK_thd_data usage is now much simpler and easier to verify
- Less chance of deadlocks in SHOW PROCESS LIST as there is less
  chance of interactions between mutexes
- Remove not needed LOCK_thread_count from
  thd_get_error_context_description()
- Fewer mutex taken for thd->awake()

Other things:
- Don't take mysys->var mutex in show processlist to check if thread
  is kill marked
- thd->awake() now automatically takes the LOCK_thd_kill mutex
  (Simplifies code)
- Apc uses LOCK_thd_kill instead of LOCK_thd_data
2017-12-08 13:46:23 +02:00
Monty
3574f9c7bc Updated MW-388.result file 2017-12-08 13:46:23 +02:00
Monty
7891a713da Don't wait too long in SHOW PROCESSLIST
This will allow show processlist to continue, without blocking
all new connections, if some threads gets stuck while holding
LOCK_thd_data or mysys_var->mutex

Connections that has mutex 'stuck' are marked as 'Busy' in 'Command'

Todo:
Make F_BACKOFF to do 'pause' instead of just (1)
2017-12-08 13:44:46 +02:00
Sergey Vojtovich
b3346c2f41 Restore LF_BACKOFF
Moved InnoDB UT_RELAX_CPU() to server. Restored cross-platform LF_BACKOFF
implementation basing on UT_RELAX_CPU().
2017-12-08 13:44:45 +02:00
Jan Lindström
2662228d18 Fix test failures. 2017-12-08 13:34:20 +02:00
Monty
db715ff392 Add extra mutex check to wsrep_aborting_thd_enqueue 2017-12-08 11:38:22 +02:00
Monty
68cd543539 Search for galera libraries also in /usr/lib64/galera-3
This is where Codership's offical rpm's puts them
2017-12-08 11:38:22 +02:00
Monty
c4581735d0 Cleanups
- Remove not used thd_rpl_is_parallel()
- Remove not used mysql_notify_thread_having_shared_lock()
- Remove not needed LOCK_thread_count from MYSQL_BIN_LOG::reset_logs()
  - LOCK_thread_count is not protecting against rollback, so this
    code and comment is not needed
- Remove mutex_locks in slave.cc that are not needed.
  Added THD::assert_not_linked() to ensure that it was safe to remove
- Fixed not repeatable test load_data_stmt_view
- Updated binlog_killed to test removal of mutex
  (thanks to Andrei Elkin for test)
- More code comments
2017-12-08 11:38:22 +02:00
Alexander Barkov
0e5eef886a MDEV-14350 Index use with collation utf8mb4_unicode_nopad_ci on LIKE pattern with wrong results 2017-12-08 13:19:19 +04:00
Marko Mäkelä
07e9ff1fe1 MDEV-14378 In ALGORITHM=INPLACE, use a common name for the intermediate tables or partitions
Allow DROP TABLE `#mysql50##sql-...._.` to drop tables that were
being rebuilt by ALGORITHM=INPLACE

NOTE: If the server is killed after the table-rebuilding ALGORITHM=INPLACE
commits inside InnoDB but before the .frm file has been replaced, then
the recovery will involve something else than DROP TABLE.

NOTE: If the server is killed in a true inplace ALTER TABLE commits
inside InnoDB but before the .frm file has been replaced, then we
are really out of luck. To properly handle that situation, we would
need a transactional mysql.ddl_fixup table that directs recovery to
rename or remove files.

prepare_inplace_alter_table_dict(): Use the altered_table->s->table_name
for generating the new_table_name.

table_name_t::part_suffix: The start of the partition name suffix.

table_name_t::dbend(): Return the end of the schema name.

table_name_t::dblen(): Return the length of the schema name, in bytes.

table_name_t::basename(): Return the name without the schema name.

table_name_t::part(): Return the partition name, or NULL if none.

row_drop_table_for_mysql(): Assert for #sql, not #sql-ib.
2017-12-08 10:07:51 +02:00
Marko Mäkelä
0a02c1a667 Merge 10.2 into bb-10.2-ext 2017-12-08 10:02:28 +02:00