causes future shutdown hang
InnoDB would hang on shutdown if any XA transactions exist in the
system in the PREPARED state. This has been masked by the fact that
MySQL would roll back any PREPARED transaction on shutdown, in the
spirit of Bug #12161 Xa recovery and client disconnection.
[mysql-test-run] do_shutdown_server: Interpret --shutdown_server 0 as
a request to kill the server immediately without initiating a
shutdown procedure.
xid_cache_insert(): Initialize XID_STATE::rm_error in order to avoid a
bogus error message on XA ROLLBACK of a recovered PREPARED transaction.
innobase_commit_by_xid(), innobase_rollback_by_xid(): Free the InnoDB
transaction object after rolling back a PREPARED transaction.
trx_get_trx_by_xid(): Only consider transactions whose
trx->is_prepared flag is set. The MySQL layer seems to prevent
attempts to roll back connected transactions that are in the PREPARED
state from another connection, but it is better to play it safe. The
is_prepared flag was introduced in the InnoDB Plugin.
trx_n_prepared: A new counter, counting the number of InnoDB
transactions in the PREPARED state.
logs_empty_and_mark_files_at_shutdown(): On shutdown, allow
trx_n_prepared transactions to exist in the system.
trx_undo_free_prepared(), trx_free_prepared(): New functions, to free
the memory objects of PREPARED transactions on shutdown. This is not
needed in the built-in InnoDB, because it would collect all allocated
memory on shutdown. The InnoDB Plugin needs this because of
innodb_use_sys_malloc.
trx_sys_close(): Invoke trx_free_prepared() on all remaining
transactions.
Remove most references to thread id in InnoDB. Three references
remain: the current holder of a mutex, and the current x-lock holder
of a rw-lock, and some references in UNIV_SYNC_DEBUG checks. This
allows MySQL to change the thread associated to a client connection.
Tighten the UNIV_SYNC_DEBUG checks, trying to ensure that no InnoDB
mutex or x-lock is being held when returning control to MySQL. The
only semaphore that may be held is the btr_search_latch in shared mode.
sync_thread_levels_empty_except_dict(): A wrapper for
sync_thread_levels_empty_gen(TRUE).
sync_thread_levels_nonempty_trx(): Check that the current thread is
not holding any InnoDB semaphores, except btr_search_latch if
trx->has_search_latch.
sync_thread_levels_empty(): Unused function; remove.
trx_t: Remove mysql_thread_id and mysql_process_no.
srv_slot_t: Remove id and handle.
row_search_for_mysql(), srv_conc_enter_innodb(),
srv_conc_force_enter_innodb(), srv_conc_force_exit_innodb(),
srv_conc_exit_innodb(), srv_suspend_mysql_thread: Assert
!sync_thread_levels_nonempty_trx().
rb:634 approved by Sunny Bains
trx_get_trx_by_xid(): Invalidate trx->xid after a successful lookup,
so that subsequent callers will not find the same transaction.
The only callers of trx_get_trx_by_xid() will be invoking
innobase_commit_low() or innobase_rollback_trx(), and those code paths
should not depend on trx->xid.
rb://584 approved by Jimmy Yang
Remove trx_t::active_trans. Split into two separate fields with distinct
responsibilities. trx_t::is_registered and trx_t::owns_prepare_mutex.
There are wrapper functions for using this fields in ha_innodb.cc. The
wrapper functions check for invariants.
Fix some formatting to conform to InnoDB guidelines.
Remove innobase_register_stmt() and innobase_register_trx_and_stmt().
Add:
trx_is_started()
trx_deregister_from_2pc()
trx_register_for_2pc()
trx_is_registered_for_2pc()
trx_owns_prepare_commit_mutex_set()
trx_has_prepare_commit_mutex()
rb://479, Approved by Jimmy Yang.
Post-merge fixes: Remove the MYSQL_VERSION_ID checks, because they only
apply to the InnoDB Plugin. Fix potential race condition accessing
trx->op_info and trx->detailed_error.
------------------------------------------------------------
revno: 3466
revision-id: marko.makela@oracle.com-20100514130815-ym7j7cfu88ro6km4
parent: marko.makela@oracle.com-20100514130228-n3n42nw7ht78k0wn
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: mysql-5.1-innodb2
timestamp: Fri 2010-05-14 16:08:15 +0300
message:
Make the InnoDB FOREIGN KEY parser understand multi-statements. (Bug #48024)
Also make InnoDB thinks that /*/ only starts a comment. (Bug #53644).
This fixes the bugs in the InnoDB Plugin.
ha_innodb.h: Use trx_query_string() instead of trx_query() when
available (MySQL 5.1.42 or later).
innobase_get_stmt(): New function, to retrieve the currently running
SQL statement.
struct trx_struct: Remove mysql_query_str. Use innobase_get_stmt() instead.
dict_strip_comments(): Add and observe the parameter sql_length. Treat
/*/ as the start of a comment.
dict_create_foreign_constraints(), row_table_add_foreign_constraints():
Add the parameter sql_length.
Also make InnoDB thinks that /*/ only starts a comment. (Bug #53644).
struct trx_struct: Add mysql_query_len.
ha_innodb.cc: Use trx_query_string() instead of trx_query() and
initialize trx->mysql_query_len.
INNOBASE_COPY_STMT(thd, trx): New macro, to initialize
trx->mysql_query_str and trx->mysql_query_len.
dict_strip_comments(): Add and observe the parameter sql_length. Treat
/*/ as the start of a comment.
dict_create_foreign_constraints(), row_table_add_foreign_constraints():
Add the parameter sql_length.
lock monitor thread, it may have locks that are granted to waited to
waiting transactions. These waiting transactions will need to be woken
up but their trx->lock_wait_timeout flag will be FALSE causing the old
code to break. What we need is a flag that covers the entire lock
release process not individual transactions. The fix is to move the
flag out of trx_t and into srv_sys_t.
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin
bzr rm innobase # remove the builtin
Next step: build, test fixes.
problems
1) BUG#39320 - innodb crash in file btr/btr0pcur.c line 217 with
innodb_locks_unsafe_for_binlog
2) Fixes bug in multi-table semi consistent reads.
3) Fixes email address from dev@innodb.com to innodb_dev_ww@oracle.com
4) Fixes warning message generated by main.innodb test
Detailed revision comments:
r4399 | marko | 2009-03-12 09:38:05 +0200 (Thu, 12 Mar 2009) | 5 lines
branches/5.1: row_sel_get_clust_rec_for_mysql(): Store the cursor position
also for unlock_row(). (Bug #39320)
rb://96 approved by Heikki Tuuri.
r4400 | marko | 2009-03-12 10:06:44 +0200 (Thu, 12 Mar 2009) | 8 lines
branches/5.1: Fix a bug in multi-table semi-consistent reads.
Remember the acquired record locks per table handle (row_prebuilt_t)
rather than per transaction (trx_t), so that unlock_row should successfully
unlock all non-matching rows in multi-table operations.
This deficiency was found while investigating Bug #39320.
rb://94 approved by Heikki Tuuri.
r4481 | marko | 2009-03-19 15:01:48 +0200 (Thu, 19 Mar 2009) | 6 lines
branches/5.1: row_unlock_for_mysql(): Do not unlock records that were
modified by the current transaction. This bug was introduced or unmasked
in r4400.
rb://97 approved by Heikki Tuuri
r4573 | vasil | 2009-03-30 14:17:13 +0300 (Mon, 30 Mar 2009) | 4 lines
branches/5.1:
Fix email address from dev@innodb.com to innodb_dev_ww@oracle.com
r4574 | vasil | 2009-03-30 14:27:08 +0300 (Mon, 30 Mar 2009) | 38 lines
branches/5.1:
Restore the state of INNODB_THREAD_CONCURRENCY to silence this warning:
TEST RESULT TIME (ms)
------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 250, with reserved ports 12500..12509
main.innodb [ pass ] 8803
MTR's internal check of the test case 'main.innodb' failed.
This means that the test case does not preserve the state that existed
before the test case was executed. Most likely the test case did not
do a proper clean-up.
This is the diff of the states of the servers before and after the
test case was executed:
mysqltest: Logging to '/tmp/autotest.sh-20090330_033000-5.1.5Hg8CY/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/tmp/autotest.sh-20090330_033000-5.1.5Hg8CY/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result'.
mysqltest: Connecting to server localhost:12500 (socket /tmp/autotest.sh-20090330_033000-5.1.5Hg8CY/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: ... Connected.
mysqltest: Start processing test commands from './include/check-testcase.test' ...
mysqltest: ... Done processing test commands.
--- /tmp/autotest.sh-20090330_033000-5.1.5Hg8CY/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result 2009-03-30 14:12:31.000000000 +0300
+++ /tmp/autotest.sh-20090330_033000-5.1.5Hg8CY/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.reject 2009-03-30 14:12:41.000000000 +0300
@@ -99,7 +99,7 @@
INNODB_SUPPORT_XA ON
INNODB_SYNC_SPIN_LOOPS 20
INNODB_TABLE_LOCKS ON
-INNODB_THREAD_CONCURRENCY 8
+INNODB_THREAD_CONCURRENCY 16
INNODB_THREAD_SLEEP_DELAY 10000
INSERT_ID 0
INTERACTIVE_TIMEOUT 28800
mysqltest: Result content mismatch
not ok
r4576 | vasil | 2009-03-30 16:25:10 +0300 (Mon, 30 Mar 2009) | 4 lines
branches/5.1:
Revert a change to Makefile.am that I committed accidentally in c4574.
Bug #16979: AUTO_INC lock in InnoDB works a table level lock
Add a table level counter that tracks the number of AUTOINC locks that are
pending and/or granted on a table. We peek at this value to determine whether
a transaction doing a simple INSERT in innodb_autoinc_lock_mode = 1, needs to
acquire the AUTOINC lock or not. This change is related to Bug# 16979.
Bug #27950: Duplicate entry error in auto-inc after mysqld restart
We check whether the AUTOINC sub-system has been initialized (first) by
holding the AUTOINC mutex and if initialization is required then we
initialize using our normal procedure.
Bug #16979: AUTO_INC lock in InnoDB works a table level lock
- this is a major change in InnoDB auto-inc handling.
Bug #27950: Duplicate entry error in auto-inc after mysqld restart
- Init AUTOINC from delete_row().
Bug #28781: InnoDB increments auto-increment value incorrectly with ON DUPLICATE KEY UPDATE
- Use value specified by MySQL, in update_row().
After applying the snapshots, ensure that code conforms to the final version
of WL 3914.
It is signficant that, after these changes, InnoDB does not define MYSQL_SERVER,
and can be built as an independent storage engine plugin.
Fixes:
Bug#9709: InnoDB inconsistensy causes "Operating System Error 32/33"
Bug#18828: If InnoDB runs out of undo slots, it returns misleading 'table is full'
Bug#20090: InnoDB: Error: trying to declare trx to enter InnoDB
Bug#20352: Make ibuf_contract_for_n_pages tunable
Bug#21101: Wrong error on exceeding max row size for InnoDB table
Bug#21293: Deadlock detection prefers to kill long running FOR UPDATE queries
Bug#22819: SHOW INNODB STATUS crashes the server with an assertion failure under high load
Bug#25078: Make the replication thread to ignore innodb_thread_concurrency
Bug#25645: Assertion failure in file srv0srv.c
Bug#28138: indexing column prefixes produces corruption in InnoDB
Bug #27381: InnoDB exits when attempting to rename table to non-existant database
Fix Bug#27381 by calling os_file_handle_error_no_exit() instead of
os_file_handle_error().
"Server Variables for Plugins"
Implement support for plugins to declare server variables.
Demonstrate functionality by removing InnoDB specific code from sql/*
New feature for HASH - HASH_UNIQUE flag
New feature for DYNAMIC_ARRAY - initializer accepts preallocated ptr.
Completed support for plugin reference counting.
Bugs fixed:
- Bug #20791 valgrind errors in InnoDB
Remove Valgrind warning of Bug #20791 : in new database
creation, we read the doublewrite buffer magic number from
uninitialized memory; the code worked because it was extremely
unlikely that the memory would contain the magic number
- Bug #21784 DROP TABLE crashes 5.1.12-pre if concurrent
queries on the table
remove update_thd() in ::store_lock()
Also includes numerous coding style fixes, etc. See file-level
comments for details.
Changes in SQL parser:
* Change default mode of SELECT from "lock in share mode"
to "consistent read".
* Remove support from SELECT for specifying "consistent read".
* Add support in SELECT for specifying "lock in share mode".
* Change all uses of SQL parser to specify "lock in share mode".
* Modify syntax so that the only valid top-level statement is
a procedure definition, since it's the only one that actually
works.
* Add support for lock waits.
* Fix BUG#15650: "DELETE with LEFT JOIN crashes server with innodb_locks_unsafe_for binlog"
* Fix BUG#17134: "Partitions: uncommitted changes are visible"
* Fix BUG#17992: "Partitions: InnoDB, somehow rotten table after UPDATE"
row0ins.c: MySQL's partitioned table code does not set preduilt->sql_stat_start right
if it does an insert in the same statement after doing a search first in the same
partition table. We now write trx id always to the buffer, not just when flag
sql_stat_start is on. This will waste CPU time very sightly.
* Fix BUG#18077: "InnoDB uses full explicit table locks in stored FUNCTION"
* Fix BUG#18238: "When locks exhaust the buffer pool, InnoDB does not roll back the trx"
* Fix BUG#18252" "Disk space leak in updates of InnoDB BLOB rows in 5.0 and 5.1"
* Fix BUG#18283: "When InnoDB returns error 'lock table full', MySQL can write to binlog too much"
* Fix BUG#18350: "Use consistent read in CREATE ... SELECT ... if innodb_locks_unsafe_for_binlog"
* Fix BUG#18384: "InnoDB memory leak on duplicate key errors in 5.0 if row has many columns"
* Fix BUG#18934: "InnoDB crashes when table uses column names like DB_ROW_ID"
Refuse tables that use reserved column names.
* InnoDB's SQL parser:
- Add support for UNSIGNED types, EXIT keyword, quoted identifiers, user-function callbacks
for processing results of FETCH statements, bound literals, DATA_VARCHAR for bound literals.
- Allow bound literals of type non-INTEGER to be of length 0.
- Add make_flex.sh and update lexer/parser generation documentation.
- Add comment clarifying the difference between 'alias' and 'indirection' fields in sym_node_t.
- Remove never reached duplicate code in pars_set_dfield_type().
- Rewrite pars_info datatypes and APIs, add a few helper functions.
- Since the functions definitions in pars_info_t are accessed after pars_sql() returns
in the query graph execution stage, we can't free pars_info_t in pars_sql(). Instead,
make pars_sql() transfer ownership of pars_info_t to the created query graph, and
make que_graph_free() free it if needed.
- Allow access to system columns like DB_ROW_ID.
* Use bound literals in row_truncate_table_for_mysql, row_drop_table_for_mysql,
row_discard_tablespace_for_mysql, and row_rename_table_for_mysql.
* Setting an isolation level of the transaction to read committed weakens the locks for
this session similarly like the option innodb_locks_unsafe_for binlog. This patch removes
alnost all gap locking (used in next-key locking) and makes MySQL to release the row locks
on the rows which does not belong to result set. Additionally, nonlocking selects on
INSERT INTO SELECT, UPDATE ... (SELECT ...), and CREATE ... SELECT ... use a nonlocking
consistent read. If a binlog is used, then binlog format should be set to row based
binloging to make the execution of the complex SQL statements.
* Disable the statistic variables btr_search_n_hash_fail and n_hash_succ, n_hash_fail,
n_patt_succ, and n_searches of btr_search_t in builds without #ifdef UNIV_SEARCH_PERF_STAT.
* Make innodb.test faster. Group all consistent read test cases to a one test case and
wait their lock timeout after all have been send to the server. Decrease amount of rows
inserted in a certain test - this has no effect on the effectiveness of the test and
reduces the running time by ~10 sec. Remove temporary work-arounds from innodb.result
now that ALTER TABLE DROP FOREIGN KEY works once again.
* Make innodb_unsafe_binlog.test faster. Grout all consistent read test cases to a one
test case amd wait their lock timeout after all have been sent to the server. Remove
unnecessary option --loose_innodb_lock_wait_timeout.
* Print dictionary memory size in SHOW INNODB STATUS.
* Fix memory leaks in row_create_table_for_mysql() in rare corner cases.
* Remove code related to clustered tables. They were never implemented, and the
implementation would be challenging with ROW_FORMAT=COMPACT. Remove the table types
DICT_TABLE_CLUSTER_MEMBER and DICT_TABLE_CLUSTER and all related tests and functions.
dict_table_t: Remove mix_id, mix_len, mix_id_len, mix_id_buf, and cluster_name.
plan_t: Remove mixed_index.
dict_create_sys_tables_tuple(): Set MIX_ID=0, MIX_LEN=0, CLUSTER_NAME=NULL when
inserting into SYS_TABLES.
dict_tree_check_search_tuple(): Enclose in #ifdef UNIV_DEBUG.
* Move calling of thr_local_free() from trx_free_for_mysql() to
innobase_close_connection().
Fixed BUGS:
#3300: "UPDATE statement with no index column in where condition locks
all rows"
Implement semi-consistent read to reduce lock conflicts at the cost
of breaking serializability.
ha_innobase::unlock_row(): reset the "did semi consistent read" flag
ha_innobase::was_semi_consistent_read(),
ha_innobase::try_semi_consistent_read(): new methods
row_prebuilt_t, row_create_prebuilt(): add field row_read_type for
keeping track of semi-consistent reads
row_vers_build_for_semi_consistent_read(),
row_sel_build_committed_vers_for_mysql(): new functions
row_search_for_mysql(): implement semi-consistent reads
#9802: "Foreign key checks disallow alter table".
Added test cases.
#12456: "Cursor shows incorrect data - DML does not affect,
probably caching"
This patch implements a high-granularity read view to be used with
cursors. In this high-granularity consistent read view modifications
done by the creating transaction after the cursor is created or
future transactions are not visible. But those modifications that
transaction did before the cursor was created are visible.
#12701: "Support >4GB buffer pool and log files on 64-bit Windows"
Do not call os_file_create_tmpfile() at runtime. Instead, create all
tempfiles at startup and guard access to them with mutexes.
#13778: "If FOREIGN_KEY_CHECKS=0, one can create inconsistent FOREIGN KEYs".
When FOREIGN_KEY_CHECKS=0 we still need to check that datatypes between
foreign key references are compatible.
#14189: "VARBINARY and BINARY variables: trailing space ignored with InnoDB"
innobase_init(): Assert that
DATA_MYSQL_BINARY_CHARSET_COLL == my_charset_bin.number.
dtype_get_pad_char(): Do not pad VARBINARY or BINARY columns.
row_ins_cascade_calc_update_vec(): Refuse ON UPDATE CASCADE when trying
to change the length of a VARBINARY column that refers to or is referenced
by a BINARY column. BINARY columns are no longer padded on comparison,
and thus they cannot be padded on storage either.
#14747: "Race condition can cause btr_search_drop_page_hash_index() to crash"
Note that buf_block_t::index should be protected by btr_search_latch
or an s-latch or x-latch on the index page.
btr_search_drop_page_hash_index(): Read block->index while holding
btr_search_latch and use the cached value in the loop. Remove some
redundant assertions.
#15108: "mysqld crashes when innodb_log_file_size is set > 4G"
#15308: "Problem of Order with Enum Column in Primary Key"
#15550: "mysqld crashes in printing a FOREIGN KEY error in InnoDB"
row_ins_foreign_report_add_err(): When printing the parent record,
use the index in the parent table rather than the index in the child table.
#15653: "Slow inserts to InnoDB if many thousands of .ibd files"
Keep track on unflushed modifications to file spaces. When there are tens
of thousands of file spaces, flushing all files in fil_flush_file_spaces()
would be very slow.
fil_flush_file_spaces(): Only flush unflushed file spaces.
fil_space_t, fil_system_t: Add a list of unflushed spaces.
#15991: "innodb-file-per-table + symlink database + rename = cr"
os_file_handle_error(): Map the error codes EXDEV, ENOTDIR, and EISDIR
to the new code OS_FILE_PATH_ERROR. Treat this code as OS_FILE_PATH_ERROR.
This fixes the crash on RENAME TABLE when the .ibd file is a symbolic link
to a different file system.
#16157: "InnoDB crashes when main location settings are empty"
This patch is from Heikki.
#16298: "InnoDB segfaults in INSERTs in upgrade of 4.0 -> 5.0 tables
with VARCHAR BINARY"
dict_load_columns(): Set the charset-collation code
DATA_MYSQL_BINARY_CHARSET_COLL for those binary string columns
that lack a charset-collation code, i.e., the tables were created
with an older version of MySQL/InnoDB than 4.1.2.
#16229: "MySQL/InnoDB uses full explicit table locks in trigger processing"
Take a InnoDB table lock only if user has explicitly requested a table
lock. Added some additional comments to store_lock() and external_lock().
#16387: "InnoDB crash when dropping a foreign key <table>_ibfk_0"
Do not mistake TABLENAME_ibfk_0 for auto-generated id.
dict_table_get_highest_foreign_id(): Ignore foreign constraint
identifiers starting with the pattern TABLENAME_ibfk_0.
#16582: "InnoDB: Error in an adaptive hash index pointer to page"
Account for a race condition when dropping the adaptive hash index
for a B-tree page.
btr_search_drop_page_hash_index(): Retry the operation if a hash index
with different parameters was built meanwhile. Add diagnostics for the
case that hash node pointers to the page remain.
btr_search_info_update_hash(), btr_search_info_update_slow():
Document the parameter "info" as in/out.
#16814: "SHOW INNODB STATUS format error in LATEST FOREIGN KEY ERROR
section"
Add a missing newline to the LAST FOREIGN KEY ERROR section in SHOW
INNODB STATUS output.
dict_foreign_error_report(): Always print a newline after invoking
dict_print_info_on_foreign_key_in_create_format().
#16827: "Better InnoDB error message if ibdata files omitted from my.cnf"
#17126: "CHECK TABLE on InnoDB causes a short hang during check of adaptive
hash"
CHECK TABLE blocking other queries, by releasing the btr_search_latch
periodically during the adaptive hash table validation.
#17405: "Valgrind: conditional jump or move depends on unititialised values"
buf_block_init(): Reset magic_n, buf_fix_count and io_fix to avoid
testing uninitialized variables.
