Commit graph

4576 commits

Author SHA1 Message Date
Thirunarayanan Balathandayuthapani
80da232576 MDEV-21563 FTS thread aborts during shutdown
- Added the test case in innodb_fts suite
- Updated copyright year in row0mysql.cc
2020-02-07 15:22:23 +05:30
Thirunarayanan Balathandayuthapani
280bf17829 MDEV-21563 FTS thread aborts during shutdown
Problem:
=======
After discarding the table, fts_optimize_thread aborts during shutdown.
InnoDB fails to remove the table from fts_optimize_wq and it leads to
the fts_optimize_thread to lookup for the auxiliary table and fails.

Fix:
====
While discarding the fts table, remove the table from fts_optimize_wq.
2020-02-06 20:42:29 +05:30
Marko Mäkelä
0b36c27e0c MDEV-20307: Remove a useless debug check to save stack space
fil_space_encrypt(): Remove the debug check that decrypts the
just encrypted page. We are exercising the decryption of encrypted
pages enough via --suite=encryption,mariabackup. It is a waste of
computing resources to decrypt every page immediately after encrypting it.

The redundant check had been added in
commit 2bedc3978b (MDEV-9931).
2020-01-31 10:06:55 +02:00
Thirunarayanan Balathandayuthapani
afe9caa82d MDEV-21564 Assert in trx_purge_add_update_undo_to_history during shutdown
- dict_table_try_drop_aborted() can generate undo logs. So it should
be avoided after shutdown of background threads.
2020-01-30 17:54:49 +05:30
Marko Mäkelä
02af6278fb InnoDB 5.6.47 and XtraDB 5.6.46-86.2
The only change is a change of the version number.
In MySQL 5.6.46, the copyright comments in a number of files were changed
in mysql/mysql-server@f1a006ece7
but there was no functional change to InnoDB code.
This was also reflected by XtraDB. We are not changing the copyright
comments in MariaDB Server for now.

Between MySQL 5.6.46 and 5.6.47, InnoDB was not changed at all.

Actually, we had forgotten to update the InnoDB version number to
5.6.46. With this change, we are updating InnoDB
from 5.6.45 to 5.6.47 and XtraDB from 5.6.45-86.1 to 5.6.46-86.2.
2020-01-17 17:39:20 +02:00
Marko Mäkelä
bde7e0ba6e MDEV-21500 Server hang when using simulated AIO
The write-heavy test innodb_zip.wl6501_scale_1 timed out on
10.2 60d7011c5f for me.
Out of os_aio_n_segments=6, 5 are waiting for an event in
os_aio_simulated_handler(). One thread is waiting for a
write to complete in buf_dblwr_add_to_batch(), but that
would never happen, because nothing is waking up the simulated AIO
handler threads.

This hang appears to have been introduced in MySQL 5.6.12
in mysql/mysql-server@26cfde776c.
2020-01-16 11:40:21 +02:00
Eugene Kosov
a5a433e256 fix windows compilation 2019-12-18 23:35:33 +08:00
Eugene Kosov
3d3d8f1012 MDEV-21337 fix aligned_malloc()
do not fallback to malloc(), always return properly aligned buffer
2019-12-18 20:09:52 +08:00
Eugene Kosov
33cf4da183 cleanup: replace exit(1) with abort() 2019-11-30 18:19:20 +07:00
Eugene Kosov
6fe2aae3ce InnoDB: log unsuccessful calls to pthread_attr_init() and pthread_create() before crash 2019-11-30 12:14:00 +07:00
Thirunarayanan Balathandayuthapani
5c3bbbd845 MDEV-20987 InnoDB fails to start when fts table has FK relation
InnoDB: Assertion failure in file .../dict/dict0dict.cc line ...
InnoDB: Failing assertion: table->can_be_evicted

This fixes a regression that was caused by the fix of MDEV-20621
(commit a41d429765).
MySQL 5.6 (and MariaDB 10.0) introduced eviction of tables from
the InnoDB data dictionary cache. Tables that are connected to
FOREIGN KEY constraints or FULLTEXT INDEX are exempt of the eviction.
With the problematic change, a table that would already be exempt
from eviction due to FOREIGN KEY would cause the problem if there
also was a FULLTEXT INDEX defined on it.

dict_load_table(): Only prevent eviction if table->can_be_evicted holds.
2019-11-06 08:12:00 +02:00
Marko Mäkelä
5164f8c206 Fix GCC 9.2.1 -Wstringop-truncation
dict_table_rename_in_cache(): Use strcpy() instead of strncpy(),
because they are known to be equivalent in this case (the length
of old_name was already validated).

mariabackup: Invoke strncpy() with one less than the buffer size,
and explicitly add NUL as the last byte of the buffer.
2019-11-04 15:52:54 +02:00
Marko Mäkelä
1bb857089f MDEV-20927: Remove duplicated code
In commit d1e6b0bcff
some code was supposed to be modified, but instead
it got duplicated. Remove the duplicated copy.
2019-10-31 07:44:18 +02:00
Marko Mäkelä
d1e6b0bcff MDEV-20927 Duplicate key with auto increment
Apply the changes to InnoDB and XtraDB that had been
inadvertently skipped in the merge
commit ae476868a5

That merge failure sabotaged part of MDEV-20127:
>Revert a problematic auto_increment_increment 'fix' from 2014.
>This involves replacing the MDEV-8827 fix and in 10.1,
>removing some WSREP instrumentation.

The code changes were re-merged manually by executing the following:

 # Get the parent of the problematic merge.
git checkout ae476868a5394041a00e75a29c7d45917e8dfae8^
 # Perform the merge again.
git merge ae476868a5394041a00e75a29c7d45917e8dfae8^2
 # Get the conflict resolution from that merge.
git checkout ae476868a5 .
 # Note: Any changes to these files were removed (empty diff)!
git diff HEAD storage/{innobase,xtradb}/handler/ha_innodb.cc
 # Apply the code changes:
git diff cf40393471b10ca68cc1d2804c22ab9203900978^2..MERGE_HEAD \
storage/{innobase,xtradb}/handler/ha_innodb.cc|
patch -p1
2019-10-30 13:21:36 +02:00
Eugene Kosov
9ed4d06706 Merge 5.5 into 10.1 2019-10-29 22:10:43 +03:00
Eugene Kosov
84088d9458 add .clang-format file for InnoDB 2019-10-29 21:56:24 +03:00
Thirunarayanan Balathandayuthapani
a41d429765 MDEV-20621 FULLTEXT INDEX activity causes InnoDB hang
- fts_optimize_thread() uses dict_table_t object instead of table id.
So that it doesn't acquire dict_sys->mutex. It leads to remove the
hang of dict_sys->mutex between fts_optimize_thread() and other threads.

- in_queue to indicate whether the table is in fts_optimize_queue. It
is protected by fts_optimize_wq->mutex to avoid any race condition.

- fts_optimize_init() adds the fts table to the fts_optimize_wq
2019-10-25 16:27:41 +03:00
Thirunarayanan Balathandayuthapani
bd22650bbc MDEV-19073 FTS row mismatch after crash recovery
InnoDB stores synced_doc_id + 1 value in FTS_CONFIG table. But
while reading the synced doc id from FTS_CONFIG table after restart,
InnoDB should read synced_doc_id - 1 to get the actual synced
doc id value.
2019-10-25 16:17:59 +03:00
Marko Mäkelä
ae702d7643 MDEV-20813: Remove the buf_flush_init_for_writing() assertion
Old InnoDB/XtraDB versions only initialized FIL_PAGE_TYPE for
B-tree pages (to FIL_PAGE_INDEX), and left it uninitialized
(possibly containing FIL_PAGE_INDEX) for others. In MySQL
or MariaDB 5.5, the field is initialized on almost all pages,
but still not all of them.

In MariaDB 10.2 and later, buf_flush_init_for_writing() would
initialize the FIL_PAGE_TYPE on such old pages, but only after
passing the debug assertion that we are now removing from 10.1.
There, we will be able to modify fil_crypt_rotate_page() so
that it will skip the key rotation for pages that contain 0
in FIL_PAGE_TYPE.

In MariaDB 10.1, there is no logic that would initialize
FIL_PAGE_TYPE on data pages in old data files after an update.
So, encryption key rotation may routinely cause page flushes
on pages that contain 0 in FIL_PAGE_TYPE.
2019-10-14 16:38:28 +03:00
Marko Mäkelä
c0c003beb4 MDEV-20805 follow-up: Catch writes of bogus pages
buf_flush_init_for_writing(): Assert that FIL_PAGE_TYPE is set
except when creating a new data file with a dummy first page.

buf_dblwr_create(): Ensure that FIL_PAGE_TYPE on all pages
will be initialized. Reset buf_dblwr_being_created at the end.
2019-10-11 15:32:04 +03:00
Marko Mäkelä
cbfd6882f4 Merge 5.5 into 10.1 2019-10-11 15:19:55 +03:00
Marko Mäkelä
ea61b79694 MDEV-20805 ibuf_add_free_page() is not initializing FIL_PAGE_TYPE first
In the function recv_parse_or_apply_log_rec_body() there are debug checks
for validating the state of the page when redo log records are being
applied. Most notably, FIL_PAGE_TYPE should be set before anything else
is being written to the page.

ibuf_add_free_page(): Set FIL_PAGE_TYPE before performing any other changes.
2019-10-11 14:12:36 +03:00
Marko Mäkelä
d95f96ad1b Merge 5.5 into 10.1 2019-10-08 12:43:37 +03:00
Marko Mäkelä
db9a4d928d Remove orphan declaration buf_flush_wait_batch_end_wait_only()
The function was declared but not defined in
commit 9d6d1902e0
2019-10-07 17:18:10 +03:00
Thirunarayanan Balathandayuthapani
8a79fa0e4d MDEV-19529 InnoDB hang on DROP FULLTEXT INDEX
Problem:
=======
  During dropping of fts index, InnoDB waits for fts_optimize_remove_table()
and it holds dict_sys->mutex and dict_operaiton_lock even though the
table id is not present in the queue. But fts_optimize_thread does wait
for dict_sys->mutex to process the unrelated table id from the slot.

Solution:
========
  Whenever table is added to fts_optimize_wq, update the fts_status
of in-memory fts subsystem to TABLE_IN_QUEUE. Whenever drop index
wants to remove table from the queue, it can check the fts_status
to decide whether it should send the MSG_DELETE_TABLE to the queue.

Removed the following functions because these are all deadcode.
dict_table_wait_for_bg_threads_to_exit(),
fts_wait_for_background_thread_to_start(),fts_start_shutdown(), fts_shudown().
2019-09-18 13:22:08 +05:30
Thirunarayanan Balathandayuthapani
708f1e3419 MDEV-19647 Server hangs after dropping full text indexes and restart
- There is no need to add the table in fts_optimize_wq if there is
no fts indexes associated with it.
2019-09-17 20:47:58 +05:30
Marko Mäkelä
b01a95f6fc row_undo_mod_remove_clust_low(): Remove duplicated code
Some code was duplicated near the start of the function,
only for InnoDB, not XtraDB. This was noticed by
comparing the InnoDB between MariaDB and MySQL.
2019-08-22 17:37:13 +03:00
Marko Mäkelä
15c1ab52a9 Merge 5.5 into 10.1 2019-08-12 14:46:28 +03:00
Marko Mäkelä
1217e4a0c0 Fix -Wimplicit-fallthrough 2019-08-12 14:14:17 +03:00
Thirunarayanan Balathandayuthapani
47f8a18fec MDEV-20247 Replication hangs with "preparing" and never starts
- The commit ab6dd77408 wrongly sets the
condition inside innobase_srv_conc_enter_innodb().  Problem is that
InnoDB makes the thread to sleep indefinitely if it is a replication
slave thread.

Thanks to Sujatha Sivakumar for contributing the replication test case.
2019-08-07 12:35:04 +05:30
Marko Mäkelä
f6ea0389a4 Replace ut_timer() with my_interval_timer()
The function pointer ut_timer() was only used by the
InnoDB defragmenting thread. Let InnoDB use a single monotonic
high-precision timer, my_interval_timer() [in nanoseconds],
occasionally wrapped by microsecond_interval_timer().

srv_defragment_interval: Change from "timer" units to nanoseconds.

This concludes the InnoDB time function cleanup that was
motivated by MDEV-14154. Only ut_time_ms() will remain for now,
wrapping my_interval_timer().
2019-07-25 10:43:11 +03:00
Marko Mäkelä
0c7c61019d Remove the wrappers ut_time(), ut_difftime(), ib_time_t 2019-07-24 21:59:26 +03:00
Marko Mäkelä
c663a9414b MDEV-14154: Failing assertion: slot->last_run <= current_time in fts0opt.cc
The FTS optimizer thread made a false assumption that time(NULL)
is monotonic. The system clock can be adjusted to the past,
for example if the hardware clock was drifting to the future,
and it was adjusted by NTP.

fts_slot_t::interval_time: Replace with the constant
FTS_OPTIMIZE_INTERVAL_IN_SECS.

fts_slot_t::last_run, fts_slot_t::completed: Clarify the
documentation.

fts_optimize_get_time_limit(): Remove a type cast, and
add a FIXME comment about domain mismatch.

fts_optimize_compact(), fts_optimize_words(): Limit the time
also when the current time has been moved to the past.

fts_optimize_table_bk(): Check for wrap-around.

fts_optimize_how_many(): Check for wrap-around, and remove the
failing assertions.

fts_is_sync_needed(): Remove a redundant call to time(NULL).
2019-07-24 21:59:26 +03:00
Marko Mäkelä
9e5df96751 Reduce the amount of time(NULL) calls for lock processing
lock_t::requested_time: Document what the field is used for.

lock_t::wait_time: Document that the field is only used for
diagnostics and may be garbage if the system time is being adjusted.

srv_slot_t::suspend_time: Document that this is duplicating
trx_lock_t::wait_started.

lock_table_print(), lock_rec_print(): Declare in static scope.
Add a parameter for the current time.

lock_deadlock_check_and_resolve(), lock_deadlock_lock_print(),
lock_deadlock_joining_trx_print():
Add a parameter for the current time.
2019-07-24 21:59:26 +03:00
Marko Mäkelä
2b5bc761d3 MDEV-14154: Document some time_t fields better
srv_slot_t::suspend_time, os_aio_slot_t::reservation_time,
sync_cell_t::reservation_time: Explain what could happen
if the system time has is being adjusted.

fts_sync_t::start_time: Document that the field is mostly unused.
2019-07-24 21:59:26 +03:00
Marko Mäkelä
10727b6953 Always initialize trx_t::start_time_micro
This affects the function has_higher_priority() for internal or
recovered transactions.
2019-07-24 21:59:26 +03:00
Marko Mäkelä
10ee1b95b8 Remove ut_usectime(), ut_gettimeofday()
Replace ut_usectime() with my_interval_timer(),
which is equivalent, but monotonically counting nanoseconds
instead of counting the microseconds of real time.

os_event_wait_time_low(): Use my_hrtime() instead of ut_usectime().

FIXME: Set a clock attribute on the condition variable that allows
a monotonic clock to be chosen as the time base, so that the wait
is immune to adjustments of the system clock.
2019-07-24 21:59:26 +03:00
Marko Mäkelä
e764d5bc01 Correct the type of a parameter 2019-07-24 21:21:54 +03:00
Marko Mäkelä
ab6dd77408 MDEV-14154: Remove ut_time_us()
Use microsecond_interval_timer()
or my_interval_timer() [in nanoseconds] instead.
2019-07-24 21:21:54 +03:00
Marko Mäkelä
86767f4ac1 Remove unused ut_get_year_month_day() 2019-07-24 19:41:59 +03:00
Laurynas Biveinis
c385d80abd Fix PS-5388 (Enable hardware CRC32 under Valgrind)
Valgrind started supporting CRC32 instruction starting with version
3.6.1, released in 2011. Thus remove the fallback to software
implementation in case running under Valgrind.
2019-07-23 15:23:27 +03:00
Marko Mäkelä
d653db32f2 MDEV-14154: Make ut_time_ms(), ut_time_us() monotonic
This is motivated by PS-5221 in
percona/percona-server@2817c561fc

The coarser-precision ut_time() will still refer to the
system clock, meaning that bad things can happen if the
real time clock is adjusted backwards.
2019-07-23 15:23:27 +03:00
Marko Mäkelä
61b5e244d6 MDEV-20127 Merge new release of InnoDB 5.6.45 to 10.1
There is one directly applicable change to InnoDB:
commit 739f5239f1 in the
5.5 branch will be merged before the next MariaDB releases.

Another potentially applicable change will be tracked
separately as MDEV-20126.

Thus, here we only update the InnoDB version number and do
not change anything else.
2019-07-23 15:23:27 +03:00
Marko Mäkelä
9d93f9dac4 MDEV-8827: Fix the 32-bit build
Follow-up to 07ba5560da:
Use the correct 64-bit type name ulonglong instead of ulint,
like in mysql/mysql-server@4e0100d86b
2019-07-23 15:00:53 +03:00
Rahul Malik
739f5239f1 Bug#15851528 DUPLICATE KEY ERROR ON AUTO-INC PK WITH MIXED AUTO_INCREMENT_INCREMENT CLIENTS
Problem: Clients running different values for auto_increment_increment
and doing concurrent inserts leads to "Duplicate key error" in one of them.

Analysis:
When auto_increment_increment value is reduced in a session,
InnoDB uses last auto_increment_increment value
to recalculate the autoinc value.
In case, some other session has inserted a value
with different auto_increment_increment, InnoDB recalculate
autoinc values based on current session previous auto_increment_increment
instead of considering the auto_increment_increment used for last insert
across all session

Fix:
revert 7acdf29cb4
a.k.a. 7c12a9e5c3
as it causing the bug.

Reviewed By:
Bin <bin.x.su@oracle.com>
Kevin <kevin.lewis@oracle.com>
RB#21777

Note: In MariaDB Server, earlier changes in
ae5bc05988
for MDEV-533 require that the original test in
mysql/mysql-server@1ccd472d63
be adjusted for MariaDB.

Also, ef47b62551 (MDEV-8827)
had to be reverted after the upstream fix had been backported.
2019-07-23 11:26:11 +03:00
Marko Mäkelä
7153e15542 Revert "MDEV-8827 Duplicate key with auto increment"
This reverts commit ef47b62551.
The parent commit 07ba5560da
which is a backport of
mysql/mysql-server@1198267c33
fixes the issue differently.
2019-07-23 11:01:44 +03:00
Thirunarayanan Balathandayuthapani
07ba5560da Bug INNODB AUTO_INCREMENT PRODUCES SAME VALUE TWICE
Problem:
=======
Autoincrement value gives duplicate values because of the following reasons.

(1) In InnoDB handler function, current autoincrement value is not changed
based on newly set auto_increment_increment or auto_increment_offset variable.

(2) Handler function does the rounding logic and changes the current
autoincrement value and InnoDB doesn't aware of the change in current
autoincrement value.

Solution:
========
Fix the problem(1), InnoDB always respect the auto_increment_increment
and auto_increment_offset value in case of current autoincrement value.
By fixing the problem (2), handler layer won't change any current
autoincrement value.

Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 13748
2019-07-23 10:58:56 +03:00
Marko Mäkelä
a5e268a293 MDEV-20102 Phantom InnoDB table remains after interrupted CREATE...SELECT
This is a regression due to MDEV-16515 that affects some versions in
the MariaDB 10.1 server series starting with 10.1.35, and possibly
all versions starting with 10.2.17, 10.3.8, and 10.4.0.

The idea of MDEV-16515 is to allow DROP TABLE to be interrupted,
in case it was stuck due to some concurrent activity. We already
made some cases of internal DROP TABLE immune to kill in MDEV-18237,
MDEV-16647, MDEV-17470. We must include the cleanup of
CREATE TABLE...SELECT in the list of such internal DROP TABLE.

ha_innobase::delete_table(): Pass create_failed=true if the current
SQL statement is CREATE, so that the table will be dropped.

row_drop_table_for_mysql(): If create_failed=true, do not allow
the operation to be interrupted.
2019-07-22 14:55:46 +03:00
Eugene Kosov
5b205458d9 MDEV-20097 potential use-after-free
row_merge_read_clustered_index(): make buf always equals to merge_buf[i]
2019-07-18 22:28:11 +03:00
Marko Mäkelä
adbab0d4cd MDEV-13625: Add the test innodb.innodb-wl5980-debug 2019-07-18 17:01:59 +03:00