Merge Facebook commit 25295d003cb0c17aa8fb756523923c77250b3294
authored by Steaphan Greene from https://github.com/facebook/mysql-5.6
This adds a pointer to the trx to each mtr.
This allows the trx to be accessed in parts of the code
where it was otherwise not available. This is needed later.
After-review changes.
For this patch in 10.0, we do not introduce a new public storage engine API,
we just fix the InnoDB/XtraDB issues. In 10.1, we will make a better public
API that can be used for all storage engines (MDEV-6429).
Eliminate the background thread that did deadlock kills asynchroneously.
Instead, we ensure that the InnoDB/XtraDB code can handle doing the kill from
inside the deadlock detection code (when thd_report_wait_for() needs to kill a
later thread to resolve a deadlock).
(We preserve the part of the original patch that introduces dedicated mutex
and condition for the slave init thread, to remove the abuse of
LOCK_thread_count for start/stop synchronisation of the slave init thread).
replication causing replication to fail.
Remove the temporary fix for MDEV-5914, which used READ COMMITTED for parallel
replication worker threads. Replace it with a better, more selective solution.
The issue is with certain edge cases of InnoDB gap locks, for example between
INSERT and ranged DELETE. It is possible for the gap lock set by the DELETE to
block the INSERT, if the DELETE runs first, while the record lock set by
INSERT does not block the DELETE, if the INSERT runs first. This can cause a
conflict between the two in parallel replication on the slave even though they
ran without conflicts on the master.
With this patch, InnoDB will ask the server layer about the two involved
transactions before blocking on a gap lock. If the server layer tells InnoDB
that the transactions are already fixed wrt. commit order, as they are in
parallel replication, InnoDB will ignore the gap lock and allow the two
transactions to proceed in parallel, avoiding the conflict.
Improve the fix for MDEV-6020. When InnoDB itself detects a deadlock, it now
asks the server layer for any preferences about which transaction to roll
back. In case of parallel replication with two transactions T1 and T2 fixed to
commit T1 before T2, the server layer will ask InnoDB to roll back T2 as the
deadlock victim, not T1. This helps in some cases to avoid excessive deadlock
rollback, as T2 will in any case need to wait for T1 to complete before it can
itself commit.
Also some misc. fixes found during development and testing:
- Remove thd_rpl_is_parallel(), it is not used or needed.
- Use KILL_CONNECTION instead of KILL_QUERY when a parallel replication
worker thread is killed to resolve a deadlock with fixed commit
ordering. There are some cases, eg. in sql/sql_parse.cc, where a KILL_QUERY
can be ignored if the query otherwise completed successfully, and this
could cause the deadlock kill to be lost, so that the deadlock was not
correctly resolved.
- Fix random test failure due to missing wait_for_binlog_checkpoint.inc.
- Make sure that deadlock or other temporary errors during parallel
replication are not printed to the the error log; there were some places
around the replication code with extra error logging. These conditions can
occur occasionally and are handled automatically without breaking
replication, so they should not pollute the error log.
- Fix handling of rgi->gtid_sub_id. We need to be able to access this also at
the end of a transaction, to be able to detect and resolve deadlocks due to
commit ordering. But this value was also used as a flag to mark whether
record_gtid() had been called, by being set to zero, losing the value. Now,
introduce a separate flag rgi->gtid_pending, so rgi->gtid_sub_id remains
valid for the entire duration of the transaction.
- Fix one place where the code to handle ignored errors called reset_killed()
unconditionally, even if no error was caught that should be ignored. This
could cause loss of a deadlock kill signal, breaking deadlock detection and
resolution.
- Fix a couple of missing mysql_reset_thd_for_next_command(). This could
cause a prior error condition to remain for the next event executed,
causing assertions about errors already being set and possibly giving
incorrect error handling for following event executions.
- Fix code that cleared thd->rgi_slave in the parallel replication worker
threads after each event execution; this caused the deadlock detection and
handling code to not be able to correctly process the associated
transactions as belonging to replication worker threads.
- Remove useless error code in slave_background_kill_request().
- Fix bug where wfc->wakeup_error was not cleared at
wait_for_commit::unregister_wait_for_prior_commit(). This could cause the
error condition to wrongly propagate to a later wait_for_prior_commit(),
causing spurious ER_PRIOR_COMMIT_FAILED errors.
- Do not put the binlog background thread into the processlist. It causes
too many result differences in mtr, but also it probably is not useful
for users to pollute the process list with a system thread that does not
really perform any user-visible tasks...
support ha_innodb.so as a dynamic plugin.
* remove obsolete *,innodb_plugin.rdiff files
* s/--plugin-load=/--plugin-load-add=/
* MYSQL_PLUGIN_IMPORT glob_hostname[]
* use my_error instead of push_warning_printf(ER_DEFAULT)
* don't use tdc_size and tc_size in a module
update test cases (XtraDB is 5.6.14, InnoDB is 5.6.10)
* copy new tests over
* disable some tests for (old) InnoDB
* delete XtraDB tests that no longer apply
small compatibility changes:
* s/HTON_EXTENDED_KEYS/HTON_SUPPORTS_EXTENDED_KEYS/
* revert unnecessary InnoDB changes to make it a bit closer to the upstream
fix XtraDB to compile on Windows (both as a static and a dynamic plugin)
disable XtraDB on Windows (deadlocks) and where no atomic ops are available (e.g. CentOS 5)
storage/innobase/handler/ha_innodb.cc:
revert few unnecessary changes to make it a bit closer to the original InnoDB
storage/innobase/include/univ.i:
correct the version to match what it was merged from
Introduce a new storage engine API method commit_checkpoint_request().
This is used to replace the fsync() at the end of every storage engine
commit with a single fsync() when a binlog is rotated.
Binlog rotation is now done during group commit instead of being
delayed until unlog(), removing some server stall and avoiding an
expensive lock/unlock of LOCK_log inside unlog().
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
hold the latch across the 2-phase commit, and it is already released at
start of prepare (if not before).
Rename trx->active_trans to active_flag; since we changed the semantics of
the field, renaming should help prevent silent merge errors.
The logic for how to check when to update the table cache for
INNODB_LOCKS with real data was flawed. This could result in both
not updating the cache often enough (when the table is queried
repeatedly with less than 100 milliseconds in-between) resulting
in stale data; as well as updating too often (when multiple
queries against the table start at around the same time).
This caused occasional test failures in innodb_information_schema.
Fix by updating the "last updated" timestamp in the right place,
when the cache is updated, not when it is read.
Fixed compiler warnings in xtradb
Added back resetting of null bitmap but now in row_search_for_mysql()
storage/xtradb/row/row0sel.c:
Added back resetting of null bitmap but now in row_search_for_mysql()
Fixed some wrong test cases
Fixed bug in null handling in XtraDB
extra/comp_err.c:
Fixed compiler warnings
extra/my_print_defaults.c:
Fixed compiler warnings
mysql-test/suite/binlog/t/binlog_killed.test:
Added support for timeouts
mysql-test/suite/funcs_1/r/is_columns_is.result:
Updated results (INNODB_SYS_TABLES had got new column)
scripts/mysql_install_db.sh:
Fixed typo
sql/mysql_priv.h:
Removed not needed argument for compare_record()
sql/sql_insert.cc:
Removed not needed argument for compare_record()
sql/sql_update.cc:
Removed not needed argument for compare_record()
The argument is not needed becasue we copy the full record[0] to record[1] and the comparison should work even if all columns are not read
sql/table.cc:
The comparison of rows is independent of HA_PARTIAL_COLUMN_READ
storage/maria/maria_chk.c:
Fixed compiler warnings
storage/maria/maria_read_log.c:
Fixed compiler warnings
storage/myisam/myisamchk.c:
Fixed compiler warnings
storage/myisam/myisampack.c:
Fixed compiler warnings
storage/xtradb/dict/dict0load.c:
Fixed compiler warnings
storage/xtradb/row/row0sel.c:
Fixed null handling in XtraDB. (See comment)
storage/xtradb/trx/trx0sys.c:
Fixed compiler warnings
support-files/compiler_warnings.supp:
Fixed compiler warnings
Don't use static link by default (in compile-pentium) as some new systems doesn't have all static libraries available
Change type for functions in plugin.h:str_mysql_ftparser_param() to const unsigned char and string lengths to size_t.
One effect of the above change is that one needs to include mysql_global.h or define size_t before including plugin.h
This fixes a case where mysql_client_test failed with newer gcc that enables strict-aliasing by default
BUILD/compile-pentium:
Don't use static link by default as some new systems doesn't have all static libraries available
client/mysql_upgrade.c:
Remove not used variable
cmd-line-utils/readline/config_readline.h:
Define some constants to get rid of compiler warnings on Linux
cmd-line-utils/readline/display.c:
Get rid of compiler warnings
cmd-line-utils/readline/history.c:
Got rid of compiler warnings:
- Defining some strings as const
- Added cast
cmd-line-utils/readline/rlmbutil.h:
Added cast to get rid of compiler warnings
cmd-line-utils/readline/text.c:
Remove not needed initialization to get rid of compiler warnings
cmd-line-utils/readline/xmalloc.c:
Changed types to 'const char* to get rid of compiler warnings
configure.in:
Ensure that we use MariaDB as suffix
include/mysql/plugin.h:
Changed types to 'const unsigned char* to get rid of compiler warnings (in other parts of the code)
Change length for not \0 terminated string to size_t
include/mysql/plugin.h.pp:
Update related to plugin.h
libmysql/libmysql.c:
Fixed bug that caused core dump with newer gcc when strict aliasing is not turned off
mysql-test/t/information_schema.test:
Test is depending on innodb
mysql-test/t/not_partition.test:
Fixed wrong directory name
(Not noticed before as we don't ususally run this test)
mysys/lf_hash.c:
Got rid of compiler warnings from -Wstrict-aliasing
mysys/my_redel.c:
Removed not used variable
regex/engine.c:
Changed types to 'const char* to get rid of compiler warnings
regex/engine.ih:
Changed types to 'const char* to get rid of compiler warnings
sql/sp_head.cc:
Got rid of compiler warning from -Wstrict-aliasing
sql/sql_base.cc:
Got rid of compiler warnings from -Wstrict-aliasing
(The original code was probably wrong as nj_col->table_field was
sql/sql_builtin.cc.in:
plugin.h needs to have size_t defined
sql/sql_parse.cc:
Remove used variable
sql/sql_select.cc:
Got rid of compiler warnings from -Wstrict-aliasing
sql/sql_show.cc:
Added #ifdef to get rid of compiler warning when not using partition engine
sql/table.cc:
Got rid of compiler warning from -Wstrict-aliasing
storage/maria/ha_maria.cc:
Got rid of compiler warnings from -Wstrict-aliasing:
- Use the thd_killed() API function
storage/maria/lockman.c:
Got rid of compiler warnings from -Wstrict-aliasing
storage/maria/ma_check.c:
Got rid of compiler warnings from -Wstrict-aliasing
Change to use new version of _ma_killed_ptr; Don't call it as often as before
storage/maria/ma_check_standalone.h:
Update to compatible _ma_killed_ptr() from ha_maria.cc
storage/maria/ma_ft_boolean_search.c:
Changed pointers from char -> const char* and length to size_t (to get rid of compiler warnings and casts)
storage/maria/ma_ft_nlq_search.c:
Got rid of compiler warnings from -Wstrict-aliasing
Ensure that 'subkeys' is 32 bit
storage/maria/ma_ft_parser.c:
Changed pointers from char -> const char* and length to size_t (to get rid of compiler warnings and casts)
storage/maria/ma_ftdefs.h:
Changed pointers from char -> const char* and length to size_t (to get rid of compiler warnings and casts)
storage/maria/ma_sort.c:
Change to use new version of _ma_killed_ptr; Don't call it as often as before
storage/maria/ma_state.c:
Got rid of compiler warnings from -Wstrict-aliasing
storage/maria/maria_def.h:
Redefine ma_killed_ptr()
storage/maria/maria_ftdump.c:
Got rid of compiler warnings from -Wstrict-aliasing
storage/maria/trnman.c:
Got rid of compiler warnings from -Wstrict-aliasing
storage/myisam/ft_boolean_search.c:
Changed pointers from char -> const char* and length to size_t (to get rid of compiler warnings and casts)
storage/myisam/ft_nlq_search.c:
Got rid of compiler warnings from -Wstrict-aliasing
storage/myisam/ft_parser.c:
Changed pointers from char -> const char* and length to size_t (to get rid of compiler warnings and casts)
storage/myisam/ft_stopwords.c:
Changed pointers from char -> const char* and length to size_t (to get rid of compiler warnings and casts)
storage/myisam/ftdefs.h:
Changed pointers from char -> const char* and length to size_t (to get rid of compiler warnings and casts)
storage/myisam/ha_myisam.cc:
Got rid of compiler warnings from -Wstrict-aliasing:
- Use the thd_killed() API function
storage/myisam/mi_check.c:
Use new killed_ptr() function
storage/myisam/myisam_ftdump.c:
Got rid of compiler warnings from -Wstrict-aliasing
storage/myisam/myisamchk.c:
Update to compatible killed_ptr() from ha_myisam.cc
storage/myisam/myisamdef.h:
Redefine killed_ptr()
storage/myisam/myisamlog.c:
Got rid of compiler warnings from -Wstrict-aliasing
storage/myisam/sort.c:
Change to use new version of killed_ptr; Don't call it as often as before
storage/xtradb/fil/fil0fil.c:
Fixedc ompiler warning
storage/xtradb/trx/trx0i_s.c:
Include mysql_plugin.h later to ensure that size_t is defined
