Commit graph

468 commits

Author SHA1 Message Date
Jan Lindström
a3acd72570 Merge InnoDB fixes from 5.5 revisions 4229, 4230, 4233, 4237 and 4238 i.e.
4229: MDEV-5670: Assertion failure in file buf0lru.c line 2355
      Add more status information if repeatable.

4230: MDEV-5673: Crash while parallel dropping multiple tables under heavy load
      Improve long semaphore wait output to include all semaphore waits
      and try to find out if there is a sequence of waiters.

4233: Fix compiler errors on product build.

4237: Fix too agressive long semaphore wait output and add guard against introducing
      compression failures on insert buffer.

4238: Fix test failure caused by simulated compression failure on
      IBUF_DUMMY table.
2014-07-25 10:30:16 +03:00
Sergey Vojtovich
6192f0bffa MDEV-6483 - Deadlock around rw_lock_debug_mutex on PPC64
This problem affects only debug builds on PPC64.

There are at least two race conditions around
rw_lock_debug_mutex_enter and rw_lock_debug_mutex_exit:

- rw_lock_debug_waiters was loaded/stored without setting
  appropriate locks/memory barriers.
- there is a gap between calls to os_event_reset() and
  os_event_wait() and in such case we're supposed to pass
  return value of the former to the latter.

Fixed by replacing self-cooked spinlocks with system mutexes.
These days system mutexes offer much better performance. OTOH
performance is not that critical for debug builds.
2014-07-24 18:12:32 +04:00
Jan Lindström
a1e41e3258 MDEV-6470: Restrict number of error messages about persistent statictic tables not found
If mysql.innodb_table_stats or mysql.innodb_index_stats is not found or has
unexpected structure output that error only once and no other error for
every table trying to use them. If they do exists, then print fetch or
recalculation errors only once / table or index.
2014-07-22 19:31:45 +03:00
Jan Lindström
ef67c3a239 MDEV-6443: Server crashed with assertaion failure in file ha_innodb.cc
line 8473

In case InnoDB index is not found, print the MySQL and InnoDB index
name we were trying to find and all MySQL and InnoDB index names there
is for this table.
2014-07-22 13:17:16 +03:00
Jan Lindström
fe3859cc9d MDEV-6443: Server crashed with assertaion failure in file
ha_innodb.cc line 8473

If index is not found from InnoDB make sure we print what we
were trying to find and all mysql and InnoDB index names there
is for this table.
2014-07-22 13:08:32 +03:00
Jan Lindström
0bb0230e3a MDEV-6426: Maria DB crashes randomly on creating indexes
Improve OS error messages on Windows.
2014-07-22 10:10:56 +03:00
Sergey Vojtovich
c0ebb3f388 MDEV-6450 - MariaDB crash on Power8 when built with advance tool
chain

InnoDB mutex_exit() function calls __sync_test_and_set() to release
the lock. According to manual this function is supposed to create
"acquire" memory barrier whereas in fact we need "release" memory
barrier at mutex_exit().

The problem isn't repeatable with gcc because it creates
"acquire-release" memory barrier for __sync_test_and_set().
ATC creates just "acquire" barrier.

Fixed by creating proper barrier at mutex_exit() by using
__sync_lock_release() instead of __sync_test_and_set().
2014-07-18 15:16:25 +04:00
Kristian Nielsen
501c56ef1e MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel replication causing replication to fail.
Merge the patches into MariaDB 10.0 main.

With this patch, parallel replication will now automatically retry a
transaction that fails due to deadlock or other temporary error, same as
single-threaded replication.

We catch deadlocks with InnoDB transactions due to enforced commit order. If
T1 must commit before T2 in parallel replication and T1 ends up waiting for T2
inside InnoDB, we kill T2 and retry it later to resolve the deadlock
automatically.
2014-07-11 12:06:47 +02:00
Kristian Nielsen
5b75891b7b Fix compile failure in non-debug build. 2014-07-10 14:24:53 +02:00
Kristian Nielsen
45f6262f54 MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel replication causing replication to fail.
After-review changes. Fix InnoDB coding style issues.
2014-07-09 13:02:52 +02:00
Kristian Nielsen
92577cc0eb MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel replication causing replication to fail.
Fix small (but nasty) typo.
2014-07-08 14:54:53 +02:00
Kristian Nielsen
98fc5b3af8 MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel replication causing replication to fail.
After-review changes.

For this patch in 10.0, we do not introduce a new public storage engine API,
we just fix the InnoDB/XtraDB issues. In 10.1, we will make a better public
API that can be used for all storage engines (MDEV-6429).

Eliminate the background thread that did deadlock kills asynchroneously.
Instead, we ensure that the InnoDB/XtraDB code can handle doing the kill from
inside the deadlock detection code (when thd_report_wait_for() needs to kill a
later thread to resolve a deadlock).

(We preserve the part of the original patch that introduces dedicated mutex
and condition for the slave init thread, to remove the abuse of
LOCK_thread_count for start/stop synchronisation of the slave init thread).
2014-07-08 12:54:47 +02:00
Jan Lindström
e005734058 MDEV-6424: Mariadb server crashes with assertion failure in file ha_innodb.cc
Analysis: For some reason table stats for a table pointed from a index 
is not initialized. Added additional warning output on this situation
and table stats initialization. This is better than asserting.
2014-07-08 21:05:18 +03:00
Jan Lindström
648fb98eb6 MDEV-6348: mariadb crash signal 11
Analysis: sync array output function, should make sure that all
used pointers are valid before using them.

Merge revision 4225 from lp:maria/5.5.
2014-07-08 18:51:34 +03:00
Jan Lindström
d2098b964d MDEV-6318: MariaDB with XtraDB uses times more of IO events
than with InnoDB plugin

Fix: os0file.h in XtraDB had OS_AIO_N_PENDING_IOS_PER_THREAD 256
when on InnoDB it is OS_AIO_N_PENDING_IOS_PER_THREAD 32. Changed
XtraDB also to use 32.
2014-07-04 08:09:27 +03:00
Jan Lindström
6bd2f900b2 MDEV-6288: Innodb causes server crash after disk full, then can't
ALTER TABLE any more.
2014-07-03 14:55:03 +03:00
unknown
bd4153a8c2 MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel
replication causing replication to fail.

Remove the temporary fix for MDEV-5914, which used READ COMMITTED for parallel
replication worker threads. Replace it with a better, more selective solution.

The issue is with certain edge cases of InnoDB gap locks, for example between
INSERT and ranged DELETE. It is possible for the gap lock set by the DELETE to
block the INSERT, if the DELETE runs first, while the record lock set by
INSERT does not block the DELETE, if the INSERT runs first. This can cause a
conflict between the two in parallel replication on the slave even though they
ran without conflicts on the master.

With this patch, InnoDB will ask the server layer about the two involved
transactions before blocking on a gap lock. If the server layer tells InnoDB
that the transactions are already fixed wrt. commit order, as they are in
parallel replication, InnoDB will ignore the gap lock and allow the two
transactions to proceed in parallel, avoiding the conflict.

Improve the fix for MDEV-6020. When InnoDB itself detects a deadlock, it now
asks the server layer for any preferences about which transaction to roll
back. In case of parallel replication with two transactions T1 and T2 fixed to
commit T1 before T2, the server layer will ask InnoDB to roll back T2 as the
deadlock victim, not T1. This helps in some cases to avoid excessive deadlock
rollback, as T2 will in any case need to wait for T1 to complete before it can
itself commit.

Also some misc. fixes found during development and testing:

 - Remove thd_rpl_is_parallel(), it is not used or needed.

 - Use KILL_CONNECTION instead of KILL_QUERY when a parallel replication
   worker thread is killed to resolve a deadlock with fixed commit
   ordering. There are some cases, eg. in sql/sql_parse.cc, where a KILL_QUERY
   can be ignored if the query otherwise completed successfully, and this
   could cause the deadlock kill to be lost, so that the deadlock was not
   correctly resolved.

 - Fix random test failure due to missing wait_for_binlog_checkpoint.inc.

 - Make sure that deadlock or other temporary errors during parallel
   replication are not printed to the the error log; there were some places
   around the replication code with extra error logging. These conditions can
   occur occasionally and are handled automatically without breaking
   replication, so they should not pollute the error log.

 - Fix handling of rgi->gtid_sub_id. We need to be able to access this also at
   the end of a transaction, to be able to detect and resolve deadlocks due to
   commit ordering. But this value was also used as a flag to mark whether
   record_gtid() had been called, by being set to zero, losing the value. Now,
   introduce a separate flag rgi->gtid_pending, so rgi->gtid_sub_id remains
   valid for the entire duration of the transaction.

 - Fix one place where the code to handle ignored errors called reset_killed()
   unconditionally, even if no error was caught that should be ignored. This
   could cause loss of a deadlock kill signal, breaking deadlock detection and
   resolution.

 - Fix a couple of missing mysql_reset_thd_for_next_command(). This could
   cause a prior error condition to remain for the next event executed,
   causing assertions about errors already being set and possibly giving
   incorrect error handling for following event executions.

 - Fix code that cleared thd->rgi_slave in the parallel replication worker
   threads after each event execution; this caused the deadlock detection and
   handling code to not be able to correctly process the associated
   transactions as belonging to replication worker threads.

 - Remove useless error code in slave_background_kill_request().

 - Fix bug where wfc->wakeup_error was not cleared at
   wait_for_commit::unregister_wait_for_prior_commit(). This could cause the
   error condition to wrongly propagate to a later wait_for_prior_commit(),
   causing spurious ER_PRIOR_COMMIT_FAILED errors.

 - Do not put the binlog background thread into the processlist. It causes
   too many result differences in mtr, but also it probably is not useful
   for users to pollute the process list with a system thread that does not
   really perform any user-visible tasks...
2014-06-10 10:13:15 +02:00
Sergey Vojtovich
23a5b2eb6d MDEV-6103 - Adding/removing non-materialized virtual column triggers
table recreation

Relaxed InnoDB/XtraDB checks to allow online add/drop of
non-materialized virtual columns.
2014-06-03 16:57:29 +04:00
unknown
629b822913 MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel
replication causing replication to fail.

In parallel replication, we run transactions from the master in parallel, but
force them to commit in the same order they did on the master. If we force T1
to commit before T2, but T2 holds eg. a row lock that is needed by T1, we get
a deadlock when T2 waits until T1 has committed.

Usually, we do not run T1 and T2 in parallel if there is a chance that they
can have conflicting locks like this, but there are certain edge cases where
it can occasionally happen (eg. MDEV-5914, MDEV-5941, MDEV-6020). The bug was
that this would cause replication to hang, eventually getting a lock timeout
and causing the slave to stop with error.

With this patch, InnoDB will report back to the upper layer whenever a
transactions T1 is about to do a lock wait on T2. If T1 and T2 are parallel
replication transactions, and T2 needs to commit later than T1, we can thus
detect the deadlock; we then kill T2, setting a flag that causes it to catch
the kill and convert it to a deadlock error; this error will then cause T2 to
roll back and release its locks (so that T1 can commit), and later T2 will be
re-tried and eventually also committed.

The kill happens asynchroneously in a slave background thread; this is
necessary, as the reporting from InnoDB about lock waits happen deep inside
the locking code, at a point where it is not possible to directly call
THD::awake() due to mutexes held.

Deadlock is assumed to be (very) rarely occuring, so this patch tries to
minimise the performance impact on the normal case where no deadlocks occur,
rather than optimise the handling of the occasional deadlock.

Also fix transaction retry due to deadlock when it happens after a transaction
already signalled to later transactions that it started to commit. In this
case we need to undo this signalling (and later redo it when we commit again
during retry), so following transactions will not start too early.

Also add a missing thd->send_kill_message() that got triggered during testing
(this corrects an incorrect fix for MySQL Bug#58933).
2014-06-03 10:31:11 +02:00
Jan Lindström
dabf471547 MDEV-4791: Assertion range_end >= range_start fails in log0online.c
on select from I_S.INNODB_CHANGED_PAGES

Analysis: limit_lsn_range_from_condition() incorrectly parses
start_lsn and/or end_lsn conditions.

Fix from SergeyP. Added some test cases.
2014-05-09 11:43:53 +03:00
Sergei Golubchik
99027efd14 post-fix for the merge of "Bug#16216513 INPLACE ALTER DISABLED FOR PARTITIONED TABLES"
make this innodb-only patch work for other engines as well
2014-05-08 10:25:09 +02:00
Sergei Golubchik
9927b36e87 merge of "Bug#16216513 INPLACE ALTER DISABLED FOR PARTITIONED TABLES"
revno: 4777
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: mysql-5.6
timestamp: Fri 2013-02-15 10:32:25 +0200
message:
  Bug#16216513 INPLACE ALTER DISABLED FOR PARTITIONED TABLES
2014-05-08 10:01:31 +02:00
Sergei Golubchik
a2807e41e8 xtradb 5.6.17-65.0 2014-05-07 17:33:33 +02:00
Sergei Golubchik
b968363aac MDEV-6184 10.0.11 merge
XtraDB 5.6.16-64.2
2014-05-06 10:21:34 +02:00
Sergei Golubchik
dc23a9501a Solaris compilation failure: xtradb is linked in statically, ha_innodb.so needs
the linker script.
2014-05-01 14:05:52 +02:00
Sergei Golubchik
948056c535 MDEV-5787 Server crashes in in row_mysql_convert_row_to_innobase on CREATE .. SELECT
XtraDB: don't accept MYSQL_TYPE_NULL as a column type
2014-03-19 09:57:57 +01:00
Jan Lindström
cdf6d3ec04 MDEV-5949: Performance of XtraDB slows down significantly on long benchmarks
when compressed tables are used.

Analysis: Number of flushed pages is incorrectly calculated at
buf_do_LRU_batch. This leads to problem when utility function
flushes dirty blocks from the end of the flush list of
all buffer pool instances in a loop until enough pages are flushed
or time limit is reached. As number of flushed pages is incorrectly
calculated, the loop mostly try to flush until time limit is
reached because the number of pages limit is not reached.

Fix: Fix the calculation of flushed pages (very short). This fix
was provided by Alexey Stroganov (Percona).
2014-03-26 15:17:12 +02:00
unknown
b352969118 MDEV-5914: Parallel replication deadlock due to InnoDB lock conflicts
Due to how gap locks work, two transactions could group commit together on the
master, but get lock conflicts and then deadlock due to different thread
scheduling order on slave.

For now, remove these deadlocks by running the parallel slave in READ
COMMITTED mode. And let InnoDB/XtraDB allow statement-based binlogging for the
parallel slave in READ COMMITTED.

We are also investigating a different solution long-term, which is based on
relaxing the gap locks only between the transactions running in parallel for
one slave, but not against possibly external transactions.
2014-03-21 13:30:55 +01:00
Jan Lindström
affe1731a1 MDEV-5830: Assertion failure mutex_get_waiters(mutex) == 0 at shutdown.
Analysis: XtraDB merge regression, at the end of mutex_spin_wait before goto mutex_loop
there is missing  

if (prio_mutex) { 
	os_atomic_decrement_ulint(&prio_mutex->high_priority_waiters, 1); 
}

Hence we get unbalanced waiter count.

Thanks to Laurynas Biveinis for finding this.
2014-03-21 08:39:04 +02:00
Jan Lindström
8250824a12 Remove assertions now that the actual bug has been repeated. 2014-03-20 09:32:37 +02:00
Jan Lindström
a60f227c04 Better to use ut_ad macro. 2014-03-19 19:35:42 +02:00
Jan Lindström
a092a40334 MDEV-5830: Assertion failure mutex_get_waiters(mutex) == 0 at shutdown. 2014-03-19 17:23:38 +02:00
Jan Lindström
f1ca1f37c9 MDEV-5878: Failing assertion: mutex_own(mutex) with innodb_use_fallocate=ON.
Analysis: This was merge error on file fil0fil.cc. fil_system mutex was taken twice because of this.

Fix: Remove unnecessary mutex_enter and fixed the issue with slow posix_fallocate usage.
2014-03-17 15:49:41 +02:00
Sergei Golubchik
68916bcab3 workaround for xtradb on gcc 4.1.2 RHEL5/x86, gcc atomic ops only work under -march=i686 2014-03-07 17:47:47 +01:00
Sergei Golubchik
a5fdd75980 XtraDB made the default 2014-03-07 15:21:07 +01:00
Sergei Golubchik
75124c5d2b xtradb, windows, aio: fix the bad merge 2014-03-04 22:25:34 +01:00
Sergei Golubchik
4b3cf4aa26 XtraDB compilation failures on Windows (again) 2014-02-28 21:04:58 +01:00
Sergei Golubchik
41c760b121 merge 2014-02-28 10:00:31 +01:00
Alexander Barkov
57cdc561fc Fixing AIX compilation failires 2014-02-27 19:44:00 +04:00
Jan Lindström
11826b1bcf Enable windows builds for XtraDB. 2014-02-27 16:41:49 +02:00
Sergei Golubchik
570c1a6fef MDEV-5672 MariaDB 10.0.8 doesn't compile without perfschema
apply the upstream patch
2014-02-27 12:25:51 +01:00
Sergei Golubchik
ac585e9ed5 Percona-Server-5.6.15-rel63.0.tar.gz merge 2014-02-26 19:21:23 +01:00
Sergei Golubchik
1b3c15f199 merge with 10.0-monty 2014-02-06 16:38:40 +01:00
Sergei Golubchik
37b8691cec fix the fix and update test results for MDEV-4439 2014-02-06 16:27:23 +01:00
Michael Widenius
313f18be5a Fixed errors and warnings found by buildbot
mysql-test/r/lowercase_table2.result:
  Updated result
  (The change happend because we don't try to open the table anymore as part of create table)
mysql-test/suite/rpl/r/create_or_replace_mix.result:
  Fixed result file
mysql-test/suite/rpl/r/create_or_replace_row.result:
  Fixed result file
mysql-test/suite/rpl/r/create_or_replace_statement.result:
  Fixed result file
mysql-test/suite/rpl/t/create_or_replace.inc:
  Drop open temporary table
mysys/my_delete.c:
  Added missing newline
plugin/metadata_lock_info/mysql-test/metadata_lock_info/r/user_lock.result:
  Fixed result
  (Lock names was before off by one. Was corrected by my previous patch)
sql/sql_select.cc:
  Fixed compiler warnings by adding missing casts
storage/connect/ha_connect.cc:
  Fixed compiler warnings
storage/innobase/os/os0file.cc:
  Fixed compiler warnings
storage/xtradb/btr/btr0btr.cc:
  Fixed compiler warnings
storage/xtradb/handler/ha_innodb.cc:
  removed not used function
strings/ctype-uca.c:
  Fixed compiler warnings
support-files/compiler_warnings.supp:
  Added suppression for warnings that are wrong or are not serious andthat we don't plan to fix.
2014-02-06 16:14:09 +02:00
Sergei Golubchik
5eb145858d more solaris fixes. xtradb and spider. 2014-02-05 17:27:41 +01:00
Sergei Golubchik
342b098cfb ha_xtradb.so fix for solaris, gcc 3.4.3 2014-02-04 19:29:58 +01:00
Sergei Golubchik
d929342b0f Merge the server part of MySQL WL#5522 - InnoDB transportable tablespaces.
Syntax. Server support. Test cases.
InnoDB bugfixes:
* don't mess around with system sprintf's, always use my_error() for errors.
* don't use InnoDB internal error codes where OS error codes are expected.
* don't say "file not found", when it was.
2014-02-02 10:00:36 +01:00
Sergei Golubchik
27d45e4696 MDEV-5574 Set AUTO_INCREMENT below max value of column.
Update InnoDB to 5.6.14
Apply MySQL-5.6 hack for MySQL Bug#16434374
Move Aria-only HA_RTREE_INDEX from my_base.h to maria_def.h (breaks an assert in InnoDB)
Fix InnoDB memory leak
2014-02-01 09:33:26 +01:00
Sergei Golubchik
348c962c49 fix xtradb I_S tables to load 2013-12-22 17:08:22 +01:00