mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-26 08:44:33 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
69697 commits 2779 branches 343 tags 1.6 GiB
Commit graph

1558 commits

Author SHA1 Message Date
Konstantin Osipov
265a6edd23 A pre-requisite patch for the fix for Bug#52044. 
Implement a few simple asserts in my_rwlock_t locks.

include/my_pthread.h:
  Declare two simple assert functions.
include/mysql/psi/mysql_thread.h:
  Add wrappers for new assert functions.
mysys/thr_rwlock.c:
  Add asserts.
sql/sql_base.cc:
  Silence a compiler warning for the case when
  SAFE_MUTEX is not ON.
 2010-08-11 01:12:01 +04:00
Konstantin Osipov
523066987d A fix for Bug#41158 "DROP TABLE holds LOCK_open during unlink()". 
Remove acquisition of LOCK_open around file system operations,
since such operations are now protected by metadata locks.
Rework table discovery algorithm to not require LOCK_open.

No new tests added since all MDL locking operations are covered
in lock.test and mdl_sync.test, and as long as these tests
pass despite the increased concurrency, consistency must be
unaffected.

mysql-test/t/disabled.def:
  Disable NDB tests due to Bug#55799.
sql/datadict.cc:
  No longer necessary to protect ha_create_table() with
  LOCK_open. Serial execution is now ensured by metadata
  locks.
sql/ha_ndbcluster.cc:
  Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.cc:
  Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.h:
  Update function signature.
sql/handler.cc:
  Implement ha_check_if_table_exists().
  @todo: some engines provide ha_table_exists_in_engine()
  handlerton call, for those we perhaps shouldn't
  call ha_discover(), to be more efficient.
  Since currently it's only NDB, postpone till
  integration with NDB.
sql/handler.h:
  Declare ha_check_if_table_exists() function.
sql/mdl.cc:
  Remove an obsolete comment.
sql/sql_base.cc:
  Update to a new signature of close_cached_tables():
  from now on we always call it without LOCK_open.
  Update comments.
  Remove get_table_share_with_create(), we should
  not attempt to create a table under LOCK_open.
  Introduce get_table_share_with_discover() instead,
  which would request a back off action if the table
  exists in engine.
  Remove acquisition of LOCK_open for 
  data dictionary operations, such as check_if_table_exists().
  Do not use get_table_share_with_create/discover for views,
  where it's not needed.
  Make tdc_remove_table() optionally acquire LOCK_open
  to simplify usage of this function.
  Use the right mutex in the partitioning code when
  manipulating with thd->open_tables.
sql/sql_base.h:
  Update signatures of changes functions.
sql/sql_insert.cc:
  Do not wrap quick_rm_table() with LOCK_open acquisition, 
  this is unnecessary.
sql/sql_parse.cc:
  Update to the new calling convention of tdc_remove_table().
  Update to the new signature of close_cached_tables().
  Update comments.
sql/sql_rename.cc:
  Update to the new calling convention of tdc_remove_table().
  Remove acquisition of LOCK_open around filesystem
  operations.
sql/sql_show.cc:
  Remove get_trigger_table_impl().
  Do not acquire LOCK_open for a dirty read of the trigger
  file.
sql/sql_table.cc:
  Do not acquire LOCK_open for filesystem operations.
sql/sql_trigger.cc:
  Do not require LOCK_open for trigger file I/O.
sql/sql_truncate.cc:
  Update to the new signature of tdc_remove_table().
sql/sql_view.cc:
  Do not require LOCK_open for view I/O.
  Use tdc_remove_table() to expel view share.
  Update comments.
sql/sys_vars.cc:
  Update to the new signature of close_cached_tables().
 2010-08-09 22:33:47 +04:00
Dmitry Lenev
8d0dc9b58b Part of fix for bug#52044 "FLUSH TABLES WITH READ LOCK and 
FLUSH TABLES <list> WITH READ LOCK are incompatible" to
be pushed as separate patch.

Replaced thread state name "Waiting for table", which was
used by threads waiting for a metadata lock or table flush, 
with a set of names which better reflect types of resources
being waited for.

Also replaced "Table lock" thread state name, which was used 
by threads waiting on thr_lock.c table level lock, with more
elaborate "Waiting for table level lock", to make it 
more consistent with other thread state names.

Updated test cases and their results according to these 
changes.

Fixed sys_vars.query_cache_wlock_invalidate_func test to not
to wait for timeout of wait_condition.inc script.

mysql-test/r/query_cache.result:
  Added test coverage for query_cache_wlock_invalidate
  behavior for implicitly locked tables.
mysql-test/suite/sys_vars/r/query_cache_wlock_invalidate_func.result:
  Fixed sys_vars.query_cache_wlock_invalidate_func test to not
  to wait for timeout of wait_condition.inc script. Reverted
  changes to test which introduced timeout and replaced waiting
  condition with a more appropriate one.
  Test coverage for query_cache_wlock_invalidate behavior for
  implicitly locked tables was added to query_cache.test.
mysql-test/suite/sys_vars/t/query_cache_wlock_invalidate_func.test:
  Fixed sys_vars.query_cache_wlock_invalidate_func test to not
  to wait for timeout of wait_condition.inc script. Reverted
  changes to test which introduced timeout and replaced waiting
  condition with a more appropriate one.
  Test coverage for query_cache_wlock_invalidate behavior for
  implicitly locked tables was added to query_cache.test.
mysql-test/t/query_cache.test:
  Added test coverage for query_cache_wlock_invalidate
  behavior for implicitly locked tables.
mysys/thr_lock.c:
  Replaced "Table lock" thread state name, which was used by 
  threads waiting on thr_lock.c table level lock, with more
  elaborate "Waiting for table level lock", to make it 
  consistent with thread state names which are used while
  waiting for metadata locks and table flush.
sql/mdl.cc:
  Replaced thread state name "Waiting for table", which was
  used by threads waiting for a metadata lock or table flush, 
  with a set of names which better reflect types of resources
  being waited for. 
  
  To implement this:
  - Adjusted MDL_wait::timed_wait() to take thread state name
    as parameter.
  - Introduced method of MDL_key class which allows to get
    thread state name to be used while waiting for resource
    corresponding to the key and changed code to use it.
    Added array translating namespaces to thread state names
    as part of this change.
sql/mdl.h:
  To implement this:
  - Adjusted MDL_wait::timed_wait() to take thread state name
    as parameter.
  - Introduced method of MDL_key class which allows to get
    thread state name to be used while waiting for resource
    corresponding to the key and changed code to use it.
    Added array translating namespaces to thread state names
    as part of this change.
sql/sql_base.cc:
  Replaced thread state name "Waiting for table", which was
  used by threads waiting for table flush, with a more elaborate
  "Waiting for table flush".
 2010-08-06 15:29:37 +04:00
Konstantin Osipov
a91a5ee3bf Cleanup: remove unused declarations from sql_base.h. 
sql/sql_base.cc:
  Update a comment to not refer to a non-existing
  function.
sql/sql_base.h:
  Remove unused declarations (bad merge with the header
  files split worklog).
sql/sql_test.cc:
  Cleanup: remove unused declarations.
 2010-08-04 20:29:13 +04:00
Alexander Nozdrin
32f3ab7933 Auto-merge from mysql-trunk-merge. 2010-07-29 16:32:11 +04:00
Konstantin Osipov
d85c628180 Merge trunk-bugfixing -> trunk-runtime. 2010-07-29 14:18:13 +04:00
Dmitry Lenev
00496b7acd Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH 
TABLES <list> WITH READ LOCK are incompatible".

The problem was that FLUSH TABLES <list> WITH READ LOCK
which was issued when other connection has acquired global
read lock using FLUSH TABLES WITH READ LOCK was blocked
and has to wait until global read lock is released.

This issue stemmed from the fact that FLUSH TABLES <list>
WITH READ LOCK implementation has acquired X metadata locks
on tables to be flushed. Since these locks required acquiring
of global IX lock this statement was incompatible with global
read lock.

This patch addresses problem by using SNW metadata type of
lock for tables to be flushed by FLUSH TABLES <list> WITH
READ LOCK. It is OK to acquire them without global IX lock
as long as we won't try to upgrade those locks. Since SNW
locks allow concurrent statements using same table FLUSH
TABLE <list> WITH READ LOCK now has to wait until old
versions of tables to be flushed go away after acquiring
metadata locks. Since such waiting can lead to deadlock
MDL deadlock detector was extended to take into account
waits for flush and resolve such deadlocks.

As a bonus code in open_tables() which was responsible for
waiting old versions of tables to go away was refactored.
Now when we encounter old version of table in open_table()
we don't back-off and wait for all old version to go away,
but instead wait for this particular table to be flushed.
Such approach supported by deadlock detection should reduce
number of scenarios in which FLUSH TABLES aborts concurrent
multi-statement transactions.

Note that active FLUSH TABLES <list> WITH READ LOCK still
blocks concurrent FLUSH TABLES WITH READ LOCK statement
as the former keeps tables open and thus prevents the
latter statement from doing flush.

mysql-test/include/handler.inc:
  Adjusted test case after changing status which is set
  when FLUSH TABLES waits for tables to be flushed from
  "Flushing tables" to "Waiting for table".
mysql-test/r/flush.result:
  Added test which checks that "flush tables <list> with
  read lock" is compatible with active "flush tables with
  read lock" but not vice-versa. This test also covers
  bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES
  <list> WITH READ LOCK are incompatible".
mysql-test/r/mdl_sync.result:
  Added scenarios in which wait for table to be flushed
  causes deadlocks to the coverage of MDL deadlock detector.
mysql-test/suite/perfschema/r/dml_setup_instruments.result:
  Adjusted test results after removal of COND_refresh
  condition variable.
mysql-test/suite/perfschema/r/server_init.result:
  Adjusted test and its results after removal of COND_refresh
  condition variable.
mysql-test/suite/perfschema/t/server_init.test:
  Adjusted test and its results after removal of COND_refresh
  condition variable.
mysql-test/t/flush.test:
  Added test which checks that "flush tables <list> with
  read lock" is compatible with active "flush tables with
  read lock" but not vice-versa. This test also covers
  bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES
  <list> WITH READ LOCK are incompatible".
mysql-test/t/kill.test:
  Adjusted test case after changing status which is set
  when FLUSH TABLES waits for tables to be flushed from
  "Flushing tables" to "Waiting for table".
mysql-test/t/lock_multi.test:
  Adjusted test case after changing status which is set
  when FLUSH TABLES waits for tables to be flushed from
  "Flushing tables" to "Waiting for table".
mysql-test/t/mdl_sync.test:
  Added scenarios in which wait for table to be flushed
  causes deadlocks to the coverage of MDL deadlock detector.
sql/ha_ndbcluster.cc:
  Adjusted code after adding one more parameter for
  close_cached_tables() call - timeout for waiting for
  table to be flushed.
sql/ha_ndbcluster_binlog.cc:
  Adjusted code after adding one more parameter for
  close_cached_tables() call - timeout for waiting for
  table to be flushed.
sql/lock.cc:
  Removed COND_refresh condition variable. See comment
  for sql_base.cc for details.
sql/mdl.cc:
  Now MDL deadlock detector takes into account information
  about waits for table flushes when searching for deadlock.
  To implement this change:
  - Declaration of enum_deadlock_weight and
    Deadlock_detection_visitor were moved to mdl.h header
    to make them available to the code in table.cc which
    implements deadlock detector traversal through edges
    of waiters graph representing waiting for flush.
  - Since now MDL_context may wait not only for metadata
    lock but also for table to be flushed an abstract
    Wait_for_edge class was introduced. Its descendants
    MDL_ticket and Flush_ticket incapsulate specifics
    of inspecting waiters graph when following through
    edge representing wait of particular type.
  
  We no longer require global IX metadata lock when acquiring
  SNW or SNRW locks. Such locks are needed only when metadata
  locks of these types are upgraded to X locks. This allows
  to use SNW locks in FLUSH TABLES <list> WITH READ LOCK
  implementation and keep the latter compatible with global
  read lock.
sql/mdl.h:
  Now MDL deadlock detector takes into account information
  about waits for table flushes when searching for deadlock.
  To implement this change:
  - Declaration of enum_deadlock_weight and
    Deadlock_detection_visitor were moved to mdl.h header
    to make them available to the code in table.cc which
    implements deadlock detector traversal through edges
    of waiters graph representing waiting for flush.
  - Since now MDL_context may wait not only for metadata
    lock but also for table to be flushed an abstract
    Wait_for_edge class was introduced. Its descendants
    MDL_ticket and Flush_ticket incapsulate specifics
    of inspecting waiters graph when following through
    edge representing wait of particular type.
  - Deadlock_detection_visitor now has m_table_shares_visited
    member which allows to support recursive locking for
    LOCK_open. This is required when deadlock detector
    inspects waiters graph which contains several edges
    representing waits for flushes or needs to come through
    the such edge more than once.
sql/mysqld.cc:
  Removed COND_refresh condition variable. See comment
  for sql_base.cc for details.
sql/mysqld.h:
  Removed COND_refresh condition variable. See comment
  for sql_base.cc for details.
sql/sql_base.cc:
  Changed approach to how threads are waiting for table
  to be flushed. Now thread that wants to wait for old
  table to go away subscribes for notification by adding
  Flush_ticket to table's share and waits using
  MDL_context::m_wait object. Once table gets flushed
  (i.e. all tables are closed and table share is ready
  to be destroyed) all such waiters are notified
  individually.
  Thanks to this change MDL deadlock detector can take
  such waits into account.
  
  To implement this/as result of this change:
  - tdc_wait_for_old_versions() was replaced with
    tdc_wait_for_old_version() which waits for individual
    old share to go away and which is called by open_table()
    after finding out that share is outdated. We don't
    need to perform back-off before such waiting thanks
    to the fact that deadlock detector now sees such waits.
  - As result Open_table_ctx::m_mdl_requests became
    unnecessary and was removed. We no longer allocate
    copies of MDL_request objects on MEM_ROOT when
    MYSQL_OPEN_FORCE_SHARED/SHARED_HIGH_PRIO flags are
    in effect.
  - close_cached_tables() and tdc_wait_for_old_version()
    share code which implements waiting for share to be
    flushed - the both use TABLE_SHARE::wait_until_flush()
    method. Thanks to this close_cached_tables() supports
    timeouts and has extra parameter for this.
  - Open_table_context::OT_MDL_CONFLICT enum element was
    renamed to OT_CONFLICT as it is now also used in cases
    when back-off is required to resolve deadlock caused
    by waiting for flush and not metadata lock.
  - In cases when we discover that current connection tries
    to open tables from different generation we now simply
    back-off and restart process of opening tables. To
    support this Open_table_context::OT_REOPEN_TABLES enum
    element was added.
  - COND_refresh condition variable became unnecessary and
    was removed.
  - mysql_notify_thread_having_shared_lock() no longer wakes
    up connections waiting for flush as all such connections
    can be waken up by deadlock detector if necessary.
sql/sql_base.h:
  - close_cached_tables() now has one more parameter -
    timeout for waiting for table to be flushed.
  - Open_table_context::OT_MDL_CONFLICT enum element was
    renamed to OT_CONFLICT as it is now also used in cases
    when back-off is required to resolve deadlock caused
    by waiting for flush and not metadata lock.
    Added new OT_REOPEN_TABLES enum element to be used in
    cases when we need to restart open tables process even
    in the middle of transaction.
  - Open_table_ctx::m_mdl_requests became unnecessary and
    was removed.
sql/sql_class.h:
  Added assert ensuring that we won't use LOCK_open mutex
  with THD::enter_cond(). Otherwise deadlocks can arise in
  MDL deadlock detector.
sql/sql_parse.cc:
  Changed FLUSH TABLES <list> WITH READ LOCK to take SNW
  metadata locks instead of X locks on tables to be flushed.
  Since we no longer require global IX lock to be taken
  when SNW locks are taken this makes this statement
  compatible with FLUSH TABLES WITH READ LOCK statement.
  Since SNW locks allow other connections to have table
  opened FLUSH TABLES <list> WITH READ LOCK now has to
  wait during open_tables() for old version to go away.
  Such waits can lead to deadlocks which will be detected
  by MDL deadlock detector which now takes waits for table
  to be flushed into account.
  
  Also adjusted code after adding one more parameter for
  close_cached_tables() call - timeout for waiting for
  table to be flushed.
sql/sql_yacc.yy:
  FLUSH TABLES <list> WITH READ LOCK now needs only SNW
  metadata locks on tables.
sql/sys_vars.cc:
  Adjusted code after adding one more parameter for
  close_cached_tables() call - timeout for waiting for
  table to be flushed.
sql/table.cc:
  Implemented new approach to how threads are waiting for
  table to be flushed. Now thread that wants to wait for
  old table to go away subscribes for notification by
  adding Flush_ticket to table's share and waits using
  MDL_context::m_wait object. Once table gets flushed
  (i.e. all tables are closed and table share is ready
  to be destroyed) all such waiters are notified
  individually. This change allows to make such waits
  visible inside of MDL deadlock detector.
  To do it:
  
  - Added list of waiters/Flush_tickets to TABLE_SHARE
    class.
  - Changed free_table_share() to postpone freeing of
    share memory until last waiter goes away and to
    wake up subscribed waiters.
  - Added TABLE_SHARE::wait_until_flushed() method which
    implements subscription to the list of waiters for
    table to be flushed and waiting for this event.
  
  Implemented interface which allows to expose waits for
  flushes to MDL deadlock detector:
  
  - Introduced Flush_ticket class a descendant of
    Wait_for_edge class.
  - Added TABLE_SHARE::find_deadlock() method which allows
    deadlock detector to find out what contexts are still
    using old version of table in question (i.e. to find
    out what contexts are waited for by owner of
    Flush_ticket).
sql/table.h:
  In order to support new strategy of waiting for table flush
  (see comment for table.cc for details) added list of
  waiters/Flush_tickets to TABLE_SHARE class.
  
  Implemented interface which allows to expose waits for
  flushes to MDL deadlock detector:
  - Introduced Flush_ticket class a descendant of
    Wait_for_edge class.
  - Added TABLE_SHARE::find_deadlock() method which allows
    deadlock detector to find out what contexts are still
    using old version of table in question (i.e. to find
    out what contexts are waited for by owner of
    Flush_ticket).
 2010-07-27 17:34:58 +04:00
Konstantin Osipov
36290c0923 A pre-requisite patch for the fix for Bug#52044. 
This patch also fixes Bug#55452 "SET PASSWORD is
replicated twice in RBR mode".

The goal of this patch is to remove the release of 
metadata locks from close_thread_tables().
This is necessary to not mistakenly release
the locks in the course of a multi-step
operation that involves multiple close_thread_tables()
or close_tables_for_reopen().

On the same token, move statement commit outside 
close_thread_tables().

Other cleanups:
Cleanup COM_FIELD_LIST.
Don't call close_thread_tables() in COM_SHUTDOWN -- there
are no open tables there that can be closed (we leave
the locked tables mode in THD destructor, and this
close_thread_tables() won't leave it anyway).

Make open_and_lock_tables() and open_and_lock_tables_derived()
call close_thread_tables() upon failure.
Remove the calls to close_thread_tables() that are now
unnecessary.

Simplify the back off condition in Open_table_context.

Streamline metadata lock handling in LOCK TABLES 
implementation.

Add asserts to ensure correct life cycle of 
statement transaction in a session.

Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.

mysql-test/r/variables.result:
  Update results: set @@autocommit and statement transaction/
  prelocked mode.
mysql-test/r/view.result:
  A harmless change in CHECK TABLE <view> status for a broken view.
  If previously a failure to prelock all functions used in a view 
  would leave the connection in LTM_PRELOCKED mode, now we call
  close_thread_tables() from open_and_lock_tables()
  and leave prelocked mode, thus some check in mysql_admin_table() that
  works only in prelocked/locked tables mode is no longer activated.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
  Fixed Bug#55452 "SET PASSWORD is replicated twice in
  RBR mode": extra binlog events are gone from the
  binary log.
mysql-test/t/variables.test:
  Add a test case: set autocommit and statement transaction/prelocked
  mode.
sql/event_data_objects.cc:
  Simplify code in Event_job_data::execute().
  Move sp_head memory management to lex_end().
sql/event_db_repository.cc:
  Move the release of metadata locks outside
  close_thread_tables().
  Make sure we call close_thread_tables() when
  open_and_lock_tables() fails and remove extra
  code from the events data dictionary.
  Use close_mysql_tables(), a new internal
  function to properly close mysql.* tables
  in the data dictionary.
  Contract Event_db_repository::drop_events_by_field,
  drop_schema_events into one function.
  When dropping all events in a schema,
  make sure we don't mistakenly release all
  locks acquired by DROP DATABASE. These
  include locks on the database name
  and the global intention exclusive
  metadata lock.
sql/event_db_repository.h:
  Function open_event_table() does not require an instance 
  of Event_db_repository.
sql/events.cc:
  Use close_mysql_tables() instead of close_thread_tables()
  to bootstrap events, since the latter no longer
  releases metadata locks.
sql/ha_ndbcluster.cc:
  - mysql_rm_table_part2 no longer releases
  acquired metadata locks. Do it in the caller.
sql/ha_ndbcluster_binlog.cc:
  Deploy the new protocol for closing thread
  tables in run_query() and ndb_binlog_index
  code.
sql/handler.cc:
  Assert that we never call ha_commit_trans/
  ha_rollback_trans in sub-statement, which
  is now the case.
sql/handler.h:
  Add an accessor to check whether THD_TRANS object
  is empty (has no transaction started).
sql/log.cc:
  Update a comment.
sql/log_event.cc:
  Since now we commit/rollback statement transaction in 
  mysql_execute_command(), we need a mechanism to communicate
  from Query_log_event::do_apply_event() to mysql_execute_command()
  that the statement transaction should be rolled back, not committed.
  Ideally it would be a virtual method of THD. I hesitate
  to make THD a virtual base class in this already large patch.
  Use a thd->variables.option_bits for now.
  
  Remove a call to close_thread_tables() from the slave IO
  thread. It doesn't open any tables, and the protocol
  for closing thread tables is more complicated now.
  
  Make sure we properly close thread tables, however, 
  in Load_data_log_event, which doesn't
  follow the standard server execution procedure
  with mysql_execute_command().
  @todo: this piece should use Server_runnable
  framework instead.
  Remove an unnecessary call to mysql_unlock_tables().
sql/rpl_rli.cc:
  Update Relay_log_info::slave_close_thread_tables()
  to follow the new close protocol.
sql/set_var.cc:
  Remove an unused header.
sql/slave.cc:
  Remove an unnecessary call to
  close_thread_tables().
sql/sp.cc:
  Remove unnecessary calls to close_thread_tables()
  from SP DDL implementation. The tables will
  be closed by the caller, in mysql_execute_command().
  When dropping all routines in a database, make sure
  to not mistakenly drop all metadata locks acquired
  so far, they include the scoped lock on the schema.
sql/sp_head.cc:
  Correct the protocol that closes thread tables
  in an SP instruction.
  Clear lex->sphead before cleaning up lex
  with lex_end to make sure that we don't
  delete the sphead twice. It's considered
  to be "cleaner" and more in line with
  future changes than calling delete lex->sphead
  in other places that cleanup the lex.
sql/sp_head.h:
  When destroying m_lex_keeper of an instruction,
  don't delete the sphead that all lex objects
  share. 
  @todo: don't store a reference to routine's sp_head
  instance in instruction's lex.
sql/sql_acl.cc:
  Don't call close_thread_tables() where the caller will
  do that for us.
  Fix Bug#55452 "SET PASSWORD is replicated twice in RBR 
  mode" by disabling RBR replication in change_password()
  function.
  Use close_mysql_tables() in bootstrap and ACL reload
  code to make sure we release all metadata locks.
sql/sql_base.cc:
  This is the main part of the patch:
  - remove manipulation with thd->transaction
  and thd->mdl_context from close_thread_tables().
  Now this function is only responsible for closing
  tables, nothing else.
  This is necessary to be able to easily use
  close_thread_tables() in procedures, that
  involve multiple open/close tables, which all
  need to be protected continuously by metadata
  locks.
  Add asserts ensuring that TABLE object
  is only used when is protected by a metadata lock.
  Simplify the back off condition of Open_table_context,
  we no longer need to look at the autocommit mode.
  Make open_and_lock_tables() and open_normal_and_derived_tables()
  close thread tables and release metadata locks acquired so-far 
  upon failure. This simplifies their usage.
  Implement close_mysql_tables().
sql/sql_base.h:
  Add declaration for close_mysql_tables().
sql/sql_class.cc:
  Remove a piece of dead code that has also become redundant
  after the fix for Bug 37521.
  The code became dead when my_eof() was made a non-protocol method,
  but a method that merely modifies the diagnostics area.
  The code became redundant with the fix for Bug#37521, when 
  we started to cal close_thread_tables() before
  Protocol::end_statement().
sql/sql_do.cc:
  Do nothing in DO if inside a substatement
  (the assert moved out of trans_rollback_stmt).
sql/sql_handler.cc:
  Add comments.
sql/sql_insert.cc:
  Remove dead code. 
  Release metadata locks explicitly at the
  end of the delayed insert thread.
sql/sql_lex.cc:
  Add destruction of lex->sphead to lex_end(),
  lex "reset" method called at the end of each statement.
sql/sql_parse.cc:
  Move close_thread_tables() and other related
  cleanups to mysql_execute_command()
  from dispatch_command(). This has become
  possible after the fix for Bug#37521.
  Mark federated SERVER statements as DDL.
  
  Next step: make sure that we don't store
  eof packet in the query cache, and move
  the query cache code outside mysql_parse.
  
  Brush up the code of COM_FIELD_LIST.
  Remove unnecessary calls to close_thread_tables().
  
  When killing a query, don't report "OK"
  if it was a suicide.
sql/sql_parse.h:
  Remove declaration of a function that is now static.
sql/sql_partition.cc:
  Remove an unnecessary call to close_thread_tables().
sql/sql_plugin.cc:
  open_and_lock_tables() will clean up
  after itself after a failure.
  Move close_thread_tables() above
  end: label, and replace with close_mysql_tables(),
  which will also release the metadata lock
  on mysql.plugin.
sql/sql_prepare.cc:
  Now that we no longer release locks in close_thread_tables()
  statement prepare code has become more straightforward.
  Remove the now redundant check for thd->killed() (used
  only by the backup project) from Execute_server_runnable.
  Reorder code to take into account that now mysql_execute_command()
  performs lex->unit.cleanup() and close_thread_tables().
sql/sql_priv.h:
  Add a new option to server options to interact
  between the slave SQL thread and execution
  framework (hack). @todo: use a virtual
  method of class THD instead.
sql/sql_servers.cc:
  Due to Bug 25705 replication of 
  DROP/CREATE/ALTER SERVER is broken.
  Make sure at least we do not attempt to 
  replicate these statements using RBR,
  as this violates the assert in close_mysql_tables().
sql/sql_table.cc:
  Do not release metadata locks in mysql_rm_table_part2,
  this is done by the caller.
  Do not call close_thread_tables() in mysql_create_table(),
  this is done by the caller. 
  Fix a bug in DROP TABLE under LOCK TABLES when,
  upon error in wait_while_table_is_used() we would mistakenly
  release the metadata lock on a non-dropped table.
  Explicitly release metadata locks when doing an implicit
  commit.
sql/sql_trigger.cc:
  Now that we delete lex->sphead in lex_end(),
  zero the trigger's sphead in lex after loading
  the trigger, to avoid double deletion.
sql/sql_udf.cc:
  Use close_mysql_tables() instead of close_thread_tables().
sql/sys_vars.cc:
  Remove code added in scope of WL#4284 which would
  break when we perform set @@session.autocommit along
  with setting other variables and using tables or functions.
  A test case added to variables.test.
sql/transaction.cc:
  Add asserts.
sql/tztime.cc:
  Use close_mysql_tables() rather than close_thread_tables().
 2010-07-27 14:25:53 +04:00
Luis Soares
9a714c935f BUG#55387: binlog.binlog_tmp_table crashes the server 
sporadically

There are two problems:

1. When closing temporary tables, during the THD clean up - and
   after the session connection was already closed, there is a
   chance we can push an error into the THD diagnostics area, if
   the writing of the implicit DROP event to the binary log fails
   for some reason. As a consequence an assertion can be
   triggered, because at that point the diagnostics area is
   already set.

2. Using push_warning with MYSQL_ERROR::WARN_LEVEL_ERROR is a 
   bug.

Given that close_temporary_tables is mostly called from
THD::cleanup - ie, with the session already closed, we fix
problem #1 by allowing the diagnostics area to be
overwritten. There is one other place in the code that calls
close_temporary_tables - while applying Start_log_event_v3. To
cover that case, we make close_temporary_tables to return the
error, thus, propagating upwards in the stack.

To fix problem #2, we replace push_warning with sql_print_error.

sql/log_event.cc:
  Added handling of error returned by close_temporary_tables to
  Start_log_event_v3::do_apply_event.
sql/sql_base.cc:
  Three changes to close_temporary_tables:
  1. it returns a boolean now (instead of void)
  2. it uses sql_print_error instead of push_warning when writing to
     binary log fails
  3. we set can_overwrite_status before writing to the binary log,
     thence not risking triggering an assertion by any other push
     into diagnostics area happening inside mysql_bin_log.write.
sql/sql_base.h:
  Changed the interface of close_temporary_tables so that it returns
  bool instead of void.
 2010-07-22 16:56:50 +01:00
Davi Arnaut
182599dd13 Merge of mysql-5.1-bugteam into mysql-trunk-merge. 2010-07-20 16:30:10 -03:00
Davi Arnaut
9a5fa17fd3 Bug#45288: pb2 returns a lot of compilation warnings on linux 
Fix warnings flagged by the new warning option -Wunused-but-set-variable
that was added to GCC 4.6 and that is enabled by -Wunused and -Wall. The
option causes a warning whenever a local variable is assigned to but is
later unused. It also warns about meaningless pointer dereferences.

client/mysql.cc:
  Meaningless pointer dereferences.
client/mysql_upgrade.c:
  Check whether reading from the file succeeded.
extra/comp_err.c:
  Unused.
extra/yassl/src/yassl_imp.cpp:
  Skip instead of reading data that is discarded.
include/my_pthread.h:
  Variable is only used in debug builds.
include/mysys_err.h:
  Add new error messages.
mysys/errors.c:
  Add new error message for permission related functions.
mysys/mf_iocache.c:
  Variable is only checked under THREAD.
mysys/my_copy.c:
  Raise a error if chmod or chown fails.
mysys/my_redel.c:
  Raise a error if chmod or chown fails.
regex/engine.c:
  Use a equivalent variable for the assert.
server-tools/instance-manager/instance_options.cc:
  Unused.
sql/field.cc:
  Unused.
sql/item.cc:
  Unused.
sql/log.cc:
  Do not ignore the return value of freopen: only set buffer if
  reopening succeeds.
  
  Adjust doxygen comment to the right function.
  
  Pass message lenght to log function.
sql/mysqld.cc:
  Do not ignore the return value of freopen: only set buffer if
  reopening succeeds.
sql/partition_info.cc:
  Unused.
sql/slave.cc:
  No need to set pointer to the address of '\0'.
sql/spatial.cc:
  Unused. Left for historical purposes.
sql/sql_acl.cc:
  Unused.
sql/sql_base.cc:
  Pointers are always set to the same variables.
sql/sql_parse.cc:
  End statement if reading fails.
  
  Store the buffer after it has actually been updated.
sql/sql_repl.cc:
  No need to set pointer to the address of '\0'.
sql/sql_show.cc:
  Put variable under the same ifdef block.
sql/udf_example.c:
  Set null pointer flag appropriately.
storage/csv/ha_tina.cc:
  Meaningless dereferences.
storage/example/ha_example.cc:
  Return the error since it's available.
storage/myisam/mi_locking.c:
  Remove unused and dead code.
 2010-07-20 15:07:36 -03:00
Jon Olav Hauglid
7467ee7bc6 merge from mysql-trunk-bugfixing 2010-07-19 10:27:53 +02:00
Davi Arnaut
711c318c07 Merge of mysql-trunk-bugfixing into mysql-trunk-merge. 2010-07-15 10:47:50 -03:00
Dmitry Lenev
a6a8e10c79 A pre-requisite for patch fixing bug #52044 "FLUSH TABLES 
WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are
incompatible", which adds information about waits caused by 
FLUSH TABLES statement to deadlock detector in MDL subsystem.

Remove API supporting caching of pointers to TABLE_SHARE 
object in MDL subsystem and all code related to it. 

The problem was that locking requirements of code 
implementing this API conflicted with locking requirements 
of code which adds information about waits caused by flushes 
to deadlock detector in MDL subsystem (the former needed to
lock LOCK_open or its future equivalent while having 
write-lock on MDL_lock's rwlock, and the latter needs to be 
able to read-lock MDL_lock rwlock while owning LOCK_open or 
its future equivalent).

Since caching of pointers to TABLE_SHARE objects in MDL 
subsystem didn't bring expected performance benefits we 
decided to remove caching API rather than try to come up 
with some complex solution for this problem.
 2010-07-13 22:01:54 +04:00
Jon Olav Hauglid
f45523188e merge from mysql-trunk-bugfixing 2010-07-13 10:39:24 +02:00
Davi Arnaut
f56dd32bf7 Bug#34043: Server loops excessively in _checkchunk() when safemalloc is enabled 
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.

Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.

Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost. 

The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.

Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.

client/mysqldump.c:
  Pass my_free directly as its signature is compatible with the
  callback type -- which wasn't the case for free_table_ent.
 2010-07-08 18:20:08 -03:00
Davi Arnaut
5641fb64b4 Merge of mysql-5.1-bugteam into mysql-trunk-merge. 2010-07-02 16:38:04 -03:00
Davi Arnaut
0eb26fdfa8 Bug#53445: Build with -Wall and fix warnings that it generates 
Apart strict-aliasing warnings, fix the remaining warnings
generated by GCC 4.4.4 -Wall and -Wextra flags.

One major source of warnings was the in-house function my_bcmp
which (unconventionally) took pointers to unsigned characters
as the byte sequences to be compared. Since my_bcmp and bcmp
are deprecated functions whose only difference with memcmp is
the return value, every use of the function is replaced with
memcmp as the special return value wasn't actually being used
by any caller.

There were also various other warnings, mostly due to type
mismatches, missing return values, missing prototypes, dead
code (unreachable) and ignored return values.

BUILD/SETUP.sh:
  Remove flags that are implied by -Wall and -Wextra.
  Do not warn about unused parameters in C++.
BUILD/check-cpu:
  Print only the compiler version instead of verbose banner.
  Although the option is gcc specific, the check was only
  being used for GCC specific checks anyway.
client/mysql.cc:
  bcmp is no longer defined.
client/mysqltest.cc:
  Pass a string to function expecting a format string.
  Replace use of bcmp with memcmp.
cmd-line-utils/readline/Makefile.am:
  Always define _GNU_SOURCE when compiling GNU readline.
  Required to make certain prototypes visible.
cmd-line-utils/readline/input.c:
  Condition for the code to be meaningful.
configure.in:
  Remove check for bcmp.
extra/comp_err.c:
  Use appropriate type.
extra/replace.c:
  Replace use of bcmp with memcmp.
extra/yassl/src/crypto_wrapper.cpp:
  Do not ignore the return value of fgets. Retrieve the file
  position if fgets succeed -- if it fails, the function will
  bail out and return a error.
extra/yassl/taocrypt/include/blowfish.hpp:
  Use a single array instead of accessing positions of the sbox_
  through a subscript to pbox_.
extra/yassl/taocrypt/include/runtime.hpp:
  One definition of such functions is enough.
extra/yassl/taocrypt/src/aes.cpp:
  Avoid potentially ambiguous conditions.
extra/yassl/taocrypt/src/algebra.cpp:
  Rename arguments to avoid shadowing related warnings.
extra/yassl/taocrypt/src/blowfish.cpp:
  Avoid potentially ambiguous conditions.
extra/yassl/taocrypt/src/integer.cpp:
  Do not define type within a anonymous union.
  Use a variable to return a value instead of
  leaving the result in a register -- compiler
  does not know the logic inside the asm.
extra/yassl/taocrypt/src/misc.cpp:
  Define handler for pure virtual functions.
  Remove unused code.
extra/yassl/taocrypt/src/twofish.cpp:
  Avoid potentially ambiguous conditions.
extra/yassl/testsuite/test.hpp:
  Function must have C language linkage.
include/m_string.h:
  Remove check which relied on bcmp being defined -- they weren't
  being used as bcmp is only visible when _BSD_SOURCE is defined.
include/my_bitmap.h:
  Remove bogus helpers which were used only in a few files and
  were causing warnings about dead code.
include/my_global.h:
  Due to G++ bug, always silence false-positive uninitialized
  variables warnings when compiling C++ code with G++.
  Remove bogus helper.
libmysql/Makefile.shared:
  Remove built-in implementation of bcmp.
mysql-test/lib/My/SafeProcess/safe_process.cc:
  Cast pid to largest possible type for a process identifier.
mysys/mf_loadpath.c:
  Leave space of the ending nul.
mysys/mf_pack.c:
  Replace bcmp with memcmp.
mysys/my_bitmap.c:
  Dead code removal.
mysys/my_gethwaddr.c:
  Remove unused variable.
mysys/my_getopt.c:
  Silence bogus uninitialized variable warning.
  Do not cast away the constant qualifier.
mysys/safemalloc.c:
  Cast to expected type.
mysys/thr_lock.c:
  Silence bogus uninitialized variable warning.
sql/field.cc:
  Replace bogus helper with a more appropriate logic which is
  used throughout the code.
sql/item.cc:
  Remove bogus logical condition which always evaluates to TRUE.
sql/item_create.cc:
  Simplify code to avoid signedness related warnings.
sql/log_event.cc:
  Replace use of bcmp with memcmp.
  No need to use helpers for simple bit operations.
sql/log_event_old.cc:
  Replace bmove_align with memcpy.
sql/mysqld.cc:
  Move use declaration of variable to the ifdef block where it
  is used. Remove now-unnecessary casts and arguments.
sql/set_var.cc:
  Replace bogus helpers with simple and classic bit operations.
sql/slave.cc:
  Cast to expected type and silence bogus warning.
sql/sql_class.h:
  Don't use enum values as bit flags, the supposed type safety is
  bogus as the combined bit flags are not a value in the enumeration.
sql/udf_example.c:
  Only declare variable when necessary.
sql/unireg.h:
  Replace use of bmove_align with memcpy.
storage/innobase/os/os0file.c:
  Silence bogus warning.
storage/myisam/mi_open.c:
  Remove bogus cast, DBUG_DUMP expects a pointer to unsigned
  char.
storage/myisam/mi_page.c:
  Remove bogus cast, DBUG_DUMP expects a pointer to unsigned
  char.
strings/bcmp.c:
  Remove built-in bcmp.
strings/ctype-ucs2.c:
  Silence bogus warning.
tests/mysql_client_test.c:
  Use a appropriate type as expected by simple_command().
 2010-07-02 15:30:47 -03:00
Dmitry Lenev
10582e2abe A follow-up for 5.5 version of fix for bug#54360 "Deadlock 
DROP/ALTER/CREATE DATABASE with open HANDLER".

Remove wait_for_condition() which became unused after 
database locks were replaced with MDL scoped locks.
If one needs functionality provided by this call one can 
always use THD::enter_cond()/exit_cond() methods.

Also removed an unused include from sql_db.cc and updated 
comment describing one of used includes to reflect current
situation.
 2010-07-01 18:58:47 +04:00
Jon Olav Hauglid
9ff272fbbd A 5.5 version of the fix for Bug #54360 "Deadlock DROP/ALTER/CREATE 
DATABASE with open HANDLER"

Remove LOCK_create_db, database name locks, and use metadata locks instead.
This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based
deadlock detector in MDL, and paves the way for a safe, deadlock-free
implementation of RENAME DATABASE.

Database DDL statements will now take exclusive metadata locks on
the database name, while table/view/routine DDL statements take
intention exclusive locks on the database name. This prevents race
conditions between database DDL and table/view/routine DDL.
(e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE)

By adding database name locks, this patch implements
WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and
WL#4985 "DDL locking: namespace/hierarchical locks".

The patch also changes code to use init_one_table() where appropriate.
The new lock_table_names() function requires TABLE_LIST::db_length to
be set correctly, and this is taken care of by init_one_table().

This patch also adds a simple template to help work with 
the mysys HASH data structure.

Most of the patch was written by Konstantin Osipov.
 2010-07-01 15:53:46 +02:00
Konstantin Osipov
c3b1a5751a Salvage comments added by Ingo while working on 
Bug#52114 and Bug#50788.
The bugs themselves are regressions that are introduced
by an incomplete fix for Bug#36171 and will not be pushed.
 2010-06-29 18:28:36 +04:00
Konstantin Osipov
52f361ecb8 A fix for Bug#54811 "Assert in mysql_lock_have_duplicate()". 
Remove mysql_lock_have_duplicate(), since now we always
have TABLE_LIST objects for MyISAMMRG children
in lex->query_tables and keep it till the end of the
statement (sub-statement).


mysql-test/r/merge.result:
  Update results (Bug#54811).
mysql-test/t/merge-big.test:
  Update to new wait state.
mysql-test/t/merge.test:
  Add a test case for Bug#54811.
sql/lock.cc:
  Remove a function that is now unused.
sql/lock.h:
  Remove a function that is now unused.
sql/sql_base.cc:
  Don't try to search for duplicate table among THR_LOCK objects, TABLE_LIST list contains all used tables.
 2010-06-29 18:01:33 +04:00
Luis Soares
e0ece40b64 BUG#54842: DROP TEMPORARY TABLE not binlogged after manual 
switching binlog format to ROW

BUG 52616 fixed the case which the user would switch from STMT to
ROW binlog format, but the server would silently ignore it. After
that fix thd->is_current_stmt_binlog_format_row() reports correct
value at logging time and events are logged in ROW (as expected)
instead of STMT as they were previously and wrongly logged.

However, the fix was only partially complete, because on
disconnect, at THD cleanup, the implicit logging of temporary
tables is conditionally performed. If the binlog_format==ROW and
thd->is_current_stmt_binlog_format_row() is true then DROPs are
not logged. Given that the user can switch from STMT to ROW, this
is wrong because the server cannot tell, just by relying on the
ROW binlog format, that the tables have been dropped before. This
is effectively similar to the MIXED scenario when a switch from
STMT to ROW is triggered.

We fix this by removing this condition from
close_temporary_tables.

mysql-test/extra/binlog_tests/drop_temp_table.test:
  Added binlog test case.
mysql-test/suite/binlog/r/binlog_row_drop_tmp_tbl.result:
  Result changes because:
  - there is a missing drop on three temporary tables
  - it now contains results for the test added
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Result now contains the implicit drop for the temporary table.
mysql-test/suite/binlog/r/binlog_stm_drop_tmp_tbl.result:
  Result file changed because it now contains results for added
  test case.
mysql-test/suite/rpl/r/rpl_drop_temp.result:
  Result file changed because it now contains results for added
  test case.
mysql-test/suite/rpl/t/rpl_drop_temp.test:
  Added replication test case.
sql/sql_base.cc:
  Removed the condition that would make the server to skip
  logging implicit drops when ROW binary log format mode was 
  in use.
  Additionally, deployed DBUG_ENTER/RETURN macros.
 2010-06-29 11:54:58 +01:00
Jon Olav Hauglid
9b5d1d98da Bug #53757 assert in mysql_truncate_by_delete 
The assert was triggered if a connection executing TRUNCATE
on a InnoDB table was killed during open_tables.

This bug was fixed in the scope of Bug #45643
"InnoDB does not support replication of TRUNCATE TABLE".

This patch adds test coverage to innodb_mysql_sync.test.
 2010-06-25 09:07:18 +02:00
Konstantin Osipov
429454f76e A new implementation for the TABLE_SHARE cache in MDL 
subsystem. Fix a number of caveates that the previous
implementation suffered from, including unprotected
access to shared data and lax resource accounting
(share->ref_count) that could lead to deadlocks.

The new implementation still suffers from a number
of potential deadlocks in some edge cases, and this is 
still not enabled by default. Especially since performance
testing has shown that it gives only marginable (not even 
exceeding measuring accuracy) improvements.

@todo: 
- Remove calls to close_cached_tables() with REFRESH_FAST,
and have_lock, because they break the MDL cache. 
- rework FLUSH TABLES <list> to not use close_cached_tables()
- make sure that whenever we set TABLE_SHARE::version to
0 we free MDL cache references to it.


sql/mdl.cc:
  We may cache references to TABLE_SHARE objects in 
  MDL_lock objects for tables. Create a separate
  MDL_lock class to represent a table.
sql/mdl.h:
  Adjust the MDL caching API to avoid races.
sql/sql_base.cc:
  Move all caching functionality close together.
  Implement a solution for deadlocks caused by 
  close_cached_tables() when MDL cache is enabled (incomplete).
sql/sql_yacc.yy:
  Adjust FLUSH rule to do the necessary initialization of
  TABLE_LIST elements used in for FLUSH TABLES <list>, and thus
  work OK with flush_mdl_cache() function.
 2010-06-18 20:14:10 +04:00
Konstantin Osipov
33c57e2f9e WL#5419 "LOCK_open scalability: make tdc_refresh_version 
an atomic counter"

Split the large LOCK_open section in open_table(). 
Do not call open_table_from_share() under LOCK_open.
Remove thd->version.

This fixes
Bug#50589 "Server hang on a query evaluated using a temporary 
table"
Bug#51557 "LOCK_open and kernel_mutex are not happy together"
Bug#49463 "LOCK_table and innodb are not nice when handler 
instances are created".

This patch has effect on storage engines that rely on
ha_open() PSEA method being called under LOCK_open.
In particular:

1) NDB is broken and left unfixed. NDB relies on LOCK_open
being kept as part of ha_open(), since it uses auto-discovery.
While previously the NDB open code was race-prone, now
it simply fails on asserts.

2) HEAP engine had a race in ha_heap::open() when
a share for the same table could be added twice
to the list of shares, or a dangling reference to a share
stored in HEAP handler. This patch aims to address this
problem by 'pinning' the newly created share in the 
internal HEAP engine share list until at least one
handler instance is created using that share.


include/heap.h:
  Add members to HP_CREATE_INFO.
  Declare heap_release_share().
sql/lock.cc:
  Remove thd->version, use thd->open_tables->s->version instead.
sql/repl_failsafe.cc:
  Remove thd->version.
sql/sql_base.cc:
  - close_thread_table(): move handler cleanup code outside the critical section protected by LOCK_open.
  - remove thd->version
  - split the large critical section in open_table() that
  opens a new table from share and is protected by LOCK_open
  into 2 critical sections, thus reducing the critical path.
  - make check_if_table_exists() acquire LOCK_open internally.
  - use thd->open_tables->s->version instead of thd->refresh_version to make sure that all tables in
  thd->open_tables are in the same refresh series.
sql/sql_base.h:
  Add declaration for check_if_table_exists().
sql/sql_class.cc:
  Remove init_open_tables_state(), it's now equal to
  reset_open_tables_state().
sql/sql_class.h:
  Remove thd->version, THD::init_open_tables_state().
sql/sql_plugin.cc:
  Use table->m_needs_reopen to mark the table as stale
  rather than manipulate with thd->version, which is no more.
sql/sql_udf.cc:
  Use table->m_needs_reopen to mark the table as stale
  rather than manipulate with thd->version, which is no more.
sql/table.h:
  Remove an unused variable.
sql/tztime.cc:
  Use table->m_needs_reopen to mark the table as stale
  rather than manipulate with thd->version, which is no more.
storage/heap/CMakeLists.txt:
  Add heap tests to cmake build files.
storage/heap/ha_heap.cc:
  Fix a race when ha_heap::ha_open() could insert two 
  HP_SHARE objects into the internal share list or store
  a dangling reference to a share in ha_heap instance,
  or wrongly set implicit_emptied.
storage/heap/hp_create.c:
  Optionally pin a newly created share in the list of shares
  by increasing its open_count. This is necessary to 
  make sure that a newly created share doesn't disappear while
  a HP_INFO object is being created to reference it.
storage/heap/hp_open.c:
  When adding a new HP_INFO object to the list of objects
  in the heap share, make sure the open_count is not increased
  twice.
storage/heap/hp_test1.c:
  Adjust the test to new function signatures.
storage/heap/hp_test2.c:
  Adjust the test to new function signatures.
 2010-06-11 19:28:18 +04:00
Konstantin Osipov
90a87cd52d A pre-requisite for WL#5419 "LOCK_open scalability: 
make tdc_refresh_version an 
atomic counter".
Backport the trivial changes from mysql-trunk-iplus.
 2010-06-10 15:43:32 +04:00
Konstantin Osipov
47b9f64c53 A pre-requisite patch for WL#5419 "LOCK_open scalability: 
make tdc_refresh_version an atomic counter".

To avoid orphaned TABLE_SHARE objects left in the
cache, make sure that wherever we set table->s->version
we take care of removing all unused table share objects
from the table cache. 

Always set table->s->version under LOCK_open, to make sure
that no other connection sees an old value of the
version and adds the table to unused_tables list.

Add an assert to table_def_unuse_table() that we never
'unuse' a talbe of a share that has an old version.

With this patch, only three places are left in the code
that manipulate with table->s->version:
- tdc_remove_table(). In most cases we have an X mdl lock
in tdc_remove_table(), the two remaining cases when we
don't are 'FLUSH TABLE' and mysql_admin_table().
- sql_view.cc - a crude hack that needs a separate fix
- initial assignment from refresh_version in table.cc.


sql/sql_base.cc:
  Add an assert.
  Don't manipulate with table->s->version in auto-repair, auto-discover.
  Use tdc_remove_table() in auto_repair_table() and drop_open_table().
sql/sql_table.cc:
  Remove dead code from mysql_admin_table().
  Manipulate with table->s->version through the TDC API.
 2010-06-10 15:31:19 +04:00
Konstantin Osipov
8d42534182 A review comment for WL#4441 " LOCK_open: Remove requirement of 
mutex protecting thd->open_tables".

We should not manipulate with table->s->version outside the 
table definition cache code, but use the TDC API
to achieve the desired result.

Fix one violation: close_all_tables_for_name().

sql/sql_base.cc:
  Use tdc_remove_table(TDC_RT_REMOVE_ALL) to expel the
  table share in close_all_tables_for_name, rather
  than manipulate with the table cache explicitly.
  
  Make sure that close_cached_tables() calls 
  close_all_tables_for_name() after closing all the involved
  handlers. The rest of the code was inspected to
  make sure that mysql_ha_rm_tables() is called.
sql/sql_handler.cc:
  Add a method to close all HANDLER cursors for a list
  of tables.
sql/sql_handler.h:
  Add declaration for mysql_ha_flush_tables().
 2010-06-09 12:39:09 +04:00
Konstantin Osipov
d4f1abaad2 WL#4441 "LOCK_open: Remove requirement of mutex protecting 
thd->open_tables"

thd->open_tables list is not normally accessed concurrently
except for one case: when the connection has open SQL
HANDLER tables, and we want to perform a DDL on the table,
we want to abort waits on MyISAM thr_lock of those connections
that prevent the DDL from proceeding, and iterate
over thd->open_tables list to find out the tables on which
the thread is waiting.

In 5.5 we mostly use deadlock detection and soft deadlock 
prevention, as opposed to "hard" deadlock prevention
of 5.1, which would abort any transaction that
may cause a deadlock. The only remaining case when
neither deadlock detection nor deadlock prevention
is implemented in 5.5 is HANDLER SQL, where we use
old good thr_lock_abort() technique form 5.1. 

Thus, replace use of LOCK_open to protect thd->open_tables
with thd->LOCK_ha_data (a lock protecting various session
private data).

This is a port of the work done for 5.5.4 for review
and inclusion into 5.5.5.

sql/sql_base.cc:
  Use thd->LOCK_ha_data (systematically) to set
  thd->open_tables.
sql/sql_class.h:
  Implement THD::set_open_tables().
sql/sql_cursor.cc:
  Use thd->LOCK_ha_data (systematically) to set
  thd->open_tables.
sql/sql_handler.cc:
  Use thd->LOCK_ha_data (systematically) to set
  thd->open_tables.
  Acquisition of LOCK_open is moved inside close_thread_table().
sql/sql_table.cc:
  Acquisition of LOCK_open is moved inside close_thread_tables().
 2010-06-08 12:08:46 +04:00
Konstantin Osipov
29f9fb7a0a Backport two small cleanups from trunk-iplus tree: 
1) No mutex and no function call if we're not using
plugins.
2) If we're above the table definition cache limit,
delete the oldest unused share, not the share on our hands.

sql/sql_base.cc:
  If we're above the table definition cache limit,
  delete the oldest unused share, not the share on our hands.
sql/sql_lex.cc:
  No mutex and no function call if we're not using
  plugins.
 2010-06-07 18:53:50 +04:00
Dmitry Lenev
9dbd9ce185 Patch that changes approach to how we acquire metadata 
locks for DML statements and changes the way MDL locks
are acquired/granted in contended case.

Instead of backing-off when a lock conflict is encountered
and waiting for it to go away before restarting open_tables()
process we now wait for lock to be released without releasing
any previously acquired locks. If conflicting lock goes away
we resume opening tables. If waiting leads to a deadlock we
try to resolve it by backing-off and restarting open_tables()
immediately.

As result both waiting for possibility to acquire and
acquiring of a metadata lock now always happen within the
same MDL API call. This has allowed to make release of a lock
and granting it to the most appropriate pending request an
atomic operation.
Thanks to this it became possible to wake up during release
of lock only those waiters which requests can be satisfied
at the moment as well as wake up only one waiter in case
when granting its request would prevent all other requests
from being satisfied. This solves thundering herd problem
which occured in cases when we were releasing some lock and
woke up many waiters for SNRW or X locks (this was the issue
in bug#52289 "performance regression for MyISAM in sysbench
OLTP_RW test".
This also allowed to implement more fair (FIFO) scheduling
among waiters with the same priority.
It also opens the door for introducing new types of requests
for metadata locks such as low-prio SNRW lock which is
necessary in order to support LOCK TABLES LOW_PRIORITY WRITE.

Notice that after this sometimes can report ER_LOCK_DEADLOCK
error in cases in which it has not happened before.
Particularly we will always report this error if waiting for
conflicting lock has happened in the middle of transaction
and resulted in a deadlock. Before this patch the error was
not reported if deadlock could have been resolved by backing
off all metadata locks acquired by the current statement.

mysql-test/r/mdl_sync.result:
  Added test coverage for some aspects of deadlock handling in
  metadata locking subsystem.
  Adjusted test case after removing back-off in general case
  when conflicting metadata lock is encountered during
  open_tables() (now this happens only if waiting for
  conflicting lock to go away leads to a deadlock).
mysql-test/r/sp_sync.result:
  Adjusted test case after removing back-off in general case
  when conflicting metadata lock is encountered during
  open_tables() (now this happens only if waiting for
  conflicting lock to go away leads to a deadlock).
mysql-test/suite/perfschema/r/dml_setup_instruments.result:
  Adjusted test results after renaming MDL_context::
  m_waiting_for_lock rwlock to m_LOCK_waiting_for.
mysql-test/suite/rpl/r/rpl_sp.result:
  Adjusted test case after implementing new approach to
  acquiring metadata locks in open_tables(). We no longer
  release all MDL locks acquired by statement before waiting
  for conflicting lock to go away. As result DROP FUNCTION
  statement has to wait for DML statement which managed to
  acquire metadata lock on function being dropped and now
  waits for other conflicting metadata lock to go away.
mysql-test/suite/rpl/t/rpl_sp.test:
  Adjusted test case after implementing new approach to
  acquiring metadata locks in open_tables(). We no longer
  release all MDL locks acquired by statement before waiting
  for conflicting lock to go away. As result DROP FUNCTION
  statement has to wait for DML statement which managed to
  acquire metadata lock on function being dropped and now
  waits for other conflicting metadata lock to go away.
mysql-test/t/mdl_sync.test:
  Added test coverage for some aspects of deadlock handling in
  metadata locking subsystem.
  Adjusted test case after removing back-off in general case
  when conflicting metadata lock is encountered during
  open_tables() (now this happens only if waiting for
  conflicting lock to go away leads to a deadlock).
mysql-test/t/sp_sync.test:
  Adjusted test case after removing back-off in general case
  when conflicting metadata lock is encountered during
  open_tables() (now this happens only if waiting for
  conflicting lock to go away leads to a deadlock).
sql/mdl.cc:
  Changed MDL subsystem to support new approach to acquring
  metadata locks in open tables and more fair and efficient
  scheduling of metadata locks. To implement this:
  - Made releasing of the lock and granting it to the most
    appropriate pending request atomic operation. As result it
    became possible to wake up only those waiters requests from
    which can be satisfied at the moment as well as wake-up
    only one waiter in case when granting its request would
    prevent all other requests from being satisfied.
    This solved thundering herd problem which occured in cases
    when we were releasing some lock and woke up many waiters
    for SNRW or X locks (this was the issue in Bug #52289
    "performance regression for MyISAM in sysbench OLTP_RW
    test".
    To emphasize above changes wake_up_waiters() was renamed
    to MDL_context::reschedule_waiters().
  - Changed code to add tickets for new requests to the back of
    waiters queue and to select tickets to be satisfied from
    the head of the queue if possible (this makes scheduling of
    requests with the same priority fair). To be able to do
    this efficiently we now use for waiting and granted queues
    version of I_P_List class which provides fast push_back()
    method.
  - Members and methods of MDL_context related to sending
    and waiting for signal were moved to separate MDL_wait
    class.
  - Since in order to avoid race conditions we must grant the
    lock only to the context which was not chosen as a victim
    of deadlock, killed or aborted due to timeout
    MDL_wait::set_status() (former awake()) was changed not to
    send signal if signal slot is already occupied and to
    indicate this fact through its return value. As another
    consequence MDL_wait::timed_wait() method was changed to
    handle timeout (optionally) and abort due to kill as
    signals which make signal slot occupied.
  - Renamed MDL_context::acquire_lock_impl() to acquire_lock().
    Changed it to be able correctly process requests for shared
    locks when there are open HANDLERs, made this method more
    optimized for acquisition of shared locks. As part of this
    change moved code common between try_acquire_lock() and
    acquire_lock() to new try_acquire_lock_impl() method.
    Also adjusted acquire_lock()'s code to take into account
    the fact that in cases when lock is granted as result of
    MDL_context::reschedule_waiters() call (i.e. when it is
    granted after waiting for lock conflict to go away)
    updating MDL_lock state is responsibility of the thread
    calling reschedule_waiters().
  - Changed MDL_context::find_deadlock() to send VICTIM
    signal even if victim is the context which has initiated
    deadlock detection. This is required in order to avoid
    races in cases when the same context simultaneously is
    chosen as a victim and its request for lock is satisfied.
    As result return value of this method became unnecessary
    and it was changed to return void.
    Adjusted MDL_lock::find_deadlock() method to take into
    account that now there can be a discrepancy between
    MDL_context::m_waiting_for value being set and real state
    of the ticket this member points to.
  - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
    and MDL_context::stop_waiting() to done_waiting_for().
  - Finally, removed MDL_context::wait_for_lock() method.
sql/mdl.h:
  Changed MDL subsystem to support new approach to acquring
  metadata locks in open tables and more fair and efficient
  scheduling of metadata locks. To implement this:
  - Members and methods of MDL_context related to sending
    and waiting for signal were moved to separate MDL_wait
    class.
  - Since now in order to avoid race conditions we must grant
    the lock only to the context which was not chosen as a
    victim of deadlock, killed or aborted due to timeout
    MDL_wait::set_status (former awake()) was changed not to
    send signal if signal slot is already occupied and to
    indicate this fact through its return value.
    Also NORMAL_WAKE_UP signal became GRANTED, and timeouts
    and aborts due to kill became full blown signals rather
    than simple return values.
  - MDL_wait::timed_wait() now takes extra parameter that
    indicates whether signal should be set if timeout is
    reached.
  - Enabled fast push_back() operation in MDL_context::m_tickets
    list to make move_ticket_after_trans_sentinel() method more
    efficient.
  - Removed MDL_context::wait_for_lock() method.
  - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
    and MDL_context::stop_waiting() to done_waiting_for().
  - MDL_context::acquire_lock_impl() became acquire_lock().
  - Introduced MDL_context::try_acquire_lock_impl() as a
    place for code shared by try_acquire_lock and
    acquire_lock().
  - Due to fact that now VICTIM signal is sent even if victim
    is the context which has initiated deadlock detection
    find_deadlock() no longer needs a return value.
sql/sql_base.cc:
  Implemented new approach to acquiring metadata locks in
  open_tables(). We no longer perform back-off when conflicting
  metadata lock is encountered. Instead we wait for this lock
  to go away while holding all locks which were acquired so
  far. Back-off is only used in situation when further waiting
  will cause a deadlock which could be avoided by performing
  back-off and restarting open_tables() process. Absence of
  waiting between back-off and restart of acquiring metadata
  locks can't lead to livelocks as MDL subsystem was changed
  to make release of lock and granting it to waiting lock
  an atomic action, so back-off will automatically give way
  to other participants of deadlock loop.
  Accordingly:
  - open_table_get_mdl_lock() and open_and_process_routine()
    were changed to wait for conflicting metadata lock to
    go away without back-off. Only if such wait leads to a
    deadlock back-off is requested. As part of this change
    new error handler class was introduced which converts,
    if possible, ER_LOCK_DEADLOCK error to a request for
    back-off and re-start of open_tables() process.
  - Open_table_context::recover_from_failed_open() was changed
    not to wait in case of metadata lock conflict. Instead we
    immediately proceed to re-acquiring locks.
  - Open_table_context::request_backoff_action() now always
    emits error if back-off is requested in the middle of
    transaction as we can't be sure that releasing lock
    which were acquired only by current statement will
    resolve a deadlock. Before this patch such situations were
    successfully detected thanks to the fact that we called
    MDL_context::wait_for_lock() method in
    recover_from_failed_open().
  - In order to avoid deadlocks open_tables() code was adjusted
    to flush open HANDLERs for which there are pending requests
    for X locks before restarting the process of acquiring
    metadata locks.
  - Changed close_tables_for_reopen() not to reset MDL_request
    for tables belonging to the tail of prelocking list. It is
    no longer necessary as these MDL_request objects won't be
    used for any waiting.
  - Adjusted comment in tdc_wait_for_old_version() to avoid
    mentioning removed MDL_context::wait_for_lock() method.
sql/sql_base.h:
  As we no longer wait for conflicting metadata lock away in
  Open_table_context::recover_from_failed_open() method,
  Open_table_context::OT_WAIT_MDL_LOCK action was renamed to
  OT_MDL_CONFLICT.
  Also Open_table_context::m_failed_mdl_request became
  unnecessary and was removed.
sql/sql_plist.h:
  Extended I_P_List template to support efficient push_back()
  operation if it is parameterized with an appropriate policy
  class.
sql/sql_show.cc:
  Adjusted code after removal of MDL_context::wait_for_lock()
  method. Now if one needs to acquire metadata lock with waiting
  one has to use a variant of MDL_context::acquire_lock() method.
 2010-06-07 11:06:55 +04:00
Jon Olav Hauglid
142a162c66 manual merge from mysql-trunk-bugfixing 
Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h
 2010-06-06 13:19:29 +02:00
Konstantin Osipov
5fa66315e9 A follow up patch for the fix for Bug#51263 "Deadlock between 
transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Make open flags part of Open_table_context.
This allows to simplify some code and (in future)
enforce the invariant that we don't, say, request a back 
off on the table when there is MYSQL_OPEN_IGNORE_FLUSH 
flag.



sql/sql_base.cc:
  open_table() flags are part of Open_table_context.
  Remove dead code that would check for OPEN_VIEW_NO_PARSE,
  which is not an open table flag.
sql/sql_base.h:
  Move flags to Open_table_context. Reorder Open_table_context
  members to compact the structure footprint.
sql/sql_insert.cc:
  Update with a new calling signature of open_table().
sql/sql_table.cc:
  Update with a new calling signature of open_table().
 2010-06-01 14:49:35 +04:00
Konstantin Osipov
a96d23a57a A follow up patch for the fix for Bug#51263 "Deadlock between 
transactional SELECT and ALTER TABLE ...  REBUILD PARTITION".

Move declarations of sql_base.cc classes to sql_base.h
(previously declared in sql_class.h).
Became possible after a header file split.





sql/sql_base.cc:
  Make sql_base.h the first include in sql_base.cc.
sql/sql_base.h:
  Add declarations of Prelocking_strategy and Open_table_context.
sql/sql_class.h:
  Remove declarations of Prelocking_strategy and Open_table_context
  (moved).
 2010-06-01 14:19:05 +04:00
Alfranio Correia
c54aabc93b auto-merge mysql-trunk-bugfixing (local) --> mysql-trunk-bugfixing.merge 2010-06-01 14:49:31 +01:00
Alexander Nozdrin
59a9912963 Auto-merge from mysql-trunk. 2010-05-28 09:47:58 +04:00
Dmitry Lenev
0a35e5bd18 A 5.1-only version of fix for bug #46947 "Embedded SELECT 
without FOR UPDATE is causing a lock".

SELECT statements with subqueries referencing InnoDB tables
were acquiring shared locks on rows in these tables when they
were executed in REPEATABLE-READ mode and with statement or
mixed mode binary logging turned on.

This was a regression which were introduced when fixing
bug 39843.

The problem was that for tables belonging to subqueries
parser set TL_READ_DEFAULT as a lock type. In cases when
statement/mixed binary logging at open_tables() time this
type of lock was converted to TL_READ_NO_INSERT lock at
open_tables() time and caused InnoDB engine to acquire
shared locks on reads from these tables. Although in some
cases such behavior was correct (e.g. for subqueries in
DELETE) in case of SELECT it has caused unnecessary locking.

This patch implements minimal version of the fix for the
specific problem described in the bug-report which supposed
to be not too risky for pushing into 5.1 tree.
The 5.5 tree already contains a more appropriate solution
which also addresses other related issues like bug 53921
"Wrong locks for SELECTs used stored functions may lead
to broken SBR".

This patch tries to solve the problem by ensuring that
TL_READ_DEFAULT lock which is set in the parser for
tables participating in subqueries at open_tables()
time is interpreted as TL_READ_NO_INSERT or TL_READ.
TL_READ is used only if we know that this is a SELECT
and that this particular table is not used by a stored
function.

Test coverage is added for both InnoDB and MyISAM.

This patch introduces an "incompatible" change in locking
scheme for subqueries used in SELECT ... FOR UPDATE and
SELECT .. IN SHARE MODE.

In 4.1 (as well as in 5.0 and 5.1 before fix for bug 39843)
the server would use a snapshot InnoDB read for subqueries
in SELECT FOR UPDATE and SELECT .. IN SHARE MODE statements,
regardless of whether the binary log is on or off.

If the user required a different type of read (i.e. locking
read), he/she could request so explicitly by providing FOR
UPDATE/IN SHARE MODE clause for each individual subquery.

The patch for bug 39843 broke this behaviour (which was not
documented or tested), and started to use locking reads for
all subqueries in SELECT ... FOR UPDATE/IN SHARE MODE.
This patch restores 4.1 behaviour.

This patch should be mostly null-merged into 5.5 tree.

mysql-test/include/check_concurrent_insert.inc:
  Added auxiliary script which allows to check if statement
  reading table allows concurrent inserts in it.
mysql-test/include/check_no_concurrent_insert.inc:
  Added auxiliary script which allows to check that statement
  reading table doesn't allow concurrent inserts in it.
mysql-test/include/check_no_row_lock.inc:
  Added auxiliary script which allows to check if statement
  reading table doesn't take locks on its rows.
mysql-test/include/check_shared_row_lock.inc:
  Added auxiliary script which allows to check if statement
  reading table takes shared locks on some of its rows.
mysql-test/r/bug39022.result:
  After bug #46947 'Embedded SELECT without FOR UPDATE is
  causing a lock' was fixed test case for bug 39022 has to
  be adjusted in order to trigger execution path on which
  original problem was encountered.
mysql-test/r/innodb_mysql_lock2.result:
  Added coverage for handling of locking in various cases when
  we read data from InnoDB tables (includes test case for
  bug #46947 'Embedded SELECT without FOR UPDATE is causing a
  lock').
mysql-test/r/lock_sync.result:
  Added coverage for handling of locking in various cases when
  we read data from MyISAM tables.
mysql-test/t/bug39022.test:
  After bug #46947 'Embedded SELECT without FOR UPDATE is
  causing a lock' was fixed test case for bug 39022 has to
  be adjusted in order to trigger execution path on which
  original problem was encountered.
mysql-test/t/innodb_mysql_lock2.test:
  Added coverage for handling of locking in various cases when
  we read data from InnoDB tables (includes test case for
  bug #46947 'Embedded SELECT without FOR UPDATE is causing a
  lock').
mysql-test/t/lock_sync.test:
  Added coverage for handling of locking in various cases when
  we read data from MyISAM tables.
sql/mysql_priv.h:
  Function read_lock_type_for_table() now takes pointers to
  LEX and TABLE_LIST elements as its arguments since to
  correctly determine lock type it needs to know what
  statement is being performed and whether table element for
  which lock type to be determined belongs to prelocking list.
sql/sql_base.cc:
  Changed read_lock_type_for_table() to return a weak TL_READ
  type of lock in cases when we are executing SELECT (and so
  won't update tables directly) and table doesn't belong to
  statement's prelocking list and thus can't be used by a
  stored function. It is OK to do so since in this case table
  won't be used by statement or function call which will be
  written to the binary log, so serializability requirements
  for it can be relaxed.
  One of results from this change is that SELECTs on InnoDB
  tables no longer takes shared row locks for tables which
  are used in subqueries (i.e. bug #46947 is fixed).
  Another result is that for similar SELECTs on MyISAM tables
  concurrent inserts are allowed.
  In order to implement this change signature of
  read_lock_type_for_table() function was changed to
  take pointers to LEX and TABLE_LIST objects.
sql/sql_update.cc:
  Function read_lock_type_for_table() now takes pointers to
  LEX and TABLE_LIST elements as its arguments since to
  correctly determine lock type it needs to know what
  statement is being performed and whether table element for
  which lock type to be determined belongs to prelocking list.
 2010-05-28 00:07:40 +04:00
Alfranio Correia
8357596035 BUG#53437 @@session.sql_bin_log support in substatements is incorrect 
The thd->variables.option_bits & OPTION_BIN_LOG is currently abused: 
it's both a system variable and an implementation switch. The current
approach to this option bit breaks the session variable encapsulation. 
      
Besides it is allowed to change @@session.sql_bin_log within a
transaction what may lead to not correctly logging a transaction.
      
To fix the problems,  we created a thd->variables variable to represent
the "sql_log_bin" and prohibited its update inside a transaction or
sub-statement.

mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
  Updated result file. The reason the warnings are removed is
  related to BUG#50312.
mysql-test/suite/binlog/r/binlog_switch_inside_trans.result:
  Checks when is possible to change the option @@session.sql_log_bin.
mysql-test/suite/binlog/t/binlog_switch_inside_trans.test:
  Checks when is possible to change the option @@session.sql_log_bin.
mysql-test/suite/rpl/r/rpl_non_direct_stm_mixing_engines.result:
  Updated the result file with warnings that were being printed
  due to the wrong use of the thd->variables.option_bits and
  sql_log_bin_top_level variables.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  Updated the result file with warnings that were being printed
  due to the wrong use of the thd->variables.option_bits and
  sql_log_bin_top_level variables.
sql/share/errmsg-utf8.txt:
  Introduces two error messages to notify that the @@session.sql_log_bin
  cannot be changed inside a sub-statement or transaction.
sql/sql_base.cc:
  Removes the variable sql_log_bin_toplevel and uses
  the session variable sql_log_bin.
sql/sql_class.cc:
  Replaces the variable sql_log_bin_toplevel by the
  (variables.option_bits & OPTION_BIN_LOG).
sql/sql_class.h:
  Removes the variable sql_log_bin_toplevel and creates 
  a session variable sql_log_bin.
sql/sys_vars.cc:
  Checks when the sql_log_bin can be correctly updated.
 2010-05-27 16:43:08 +01:00
Dmitry Lenev
ae1ae4bd63 Fix for bug #51263 "Deadlock between transactional 
SELECT and ALTER TABLE ...  REBUILD PARTITION".

ALTER TABLE on InnoDB table (including partitioned tables)
acquired exclusive locks on rows of table being altered.
In cases when there was concurrent transaction which did
locking reads from this table this sometimes led to a
deadlock which was not detected by MDL subsystem nor by
InnoDB engine (and was reported only after exceeding
innodb_lock_wait_timeout).

This problem stemmed from the fact that ALTER TABLE acquired
TL_WRITE_ALLOW_READ lock on table being altered. This lock
was interpreted as a write lock and thus for table being
altered handler::external_lock() method was called with
F_WRLCK as an argument. As result InnoDB engine treated
ALTER TABLE as an operation which is going to change data
and acquired LOCK_X locks on rows being read from old
version of table.

In case when there was a transaction which already acquired
SR metadata lock on table and some LOCK_S locks on its rows
(e.g. by using it in subquery of DML statement) concurrent
ALTER TABLE was blocked at the moment when it tried to
acquire LOCK_X lock before reading one of these rows.
The transaction's attempt to acquire SW metadata lock on
table being altered led to deadlock, since it had to wait
for ALTER TABLE to release SNW lock. This deadlock was not
detected and got resolved only after timeout expiring
because waiting were happening in two different subsystems.

Similar deadlocks could have occured in other situations.
This patch tries to solve the problem by changing ALTER TABLE
implementation to use TL_READ_NO_INSERT lock instead of
TL_WRITE_ALLOW_READ. After this step handler::external_lock()
is called with F_RDLCK as an argument and InnoDB engine
correctly interprets ALTER TABLE as operation which only
reads data from original version of table. Thanks to this
ALTER TABLE acquires only LOCK_S locks on rows it reads.
This, in its turn, causes inter-subsystem deadlocks to go
away, as all potential lock conflicts and thus deadlocks will
be limited to metadata locking subsystem:

- When ALTER TABLE reads rows from table being altered it
  can't encounter any locks which conflict with LOCK_S row
  locks. There should be no concurrent transactions holding
  LOCK_X row locks. Such a transaction should have been
  acquired SW metadata lock on table first which would have
  conflicted with ALTER's SNW lock.
- Vice versa, when DML which runs concurrently with ALTER
  TABLE tries to lock row it should be requesting only LOCK_S
  lock which is compatible with locks acquired by ALTER,
  as otherwise such DML must own an SW metadata lock on table
  which would be incompatible with ALTER's SNW lock.

mysql-test/r/innodb_mysql_lock2.result:
  Added test for bug #51263 "Deadlock between transactional
  SELECT and ALTER TABLE ... REBUILD PARTITION".
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
  Since CREATE TRIGGER no longer acquires write lock on table
  it is no longer interpreted as an operation which modifies
  table data and therefore no longer fails if invoked for
  SBR-only engine in ROW mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
  Since CREATE TRIGGER no longer acquires write lock on table
  it is no longer interpreted as an operation which modifies
  table data and therefore no longer fails if invoked for
  SBR-only engine in ROW mode.
mysql-test/t/innodb_mysql_lock2.test:
  Added test for bug #51263 "Deadlock between transactional
  SELECT and ALTER TABLE ... REBUILD PARTITION".
sql/ha_partition.cc:
  When ALTER TABLE creates a new partition to be filled from
  other partition lock it in F_WRLCK mode instead of using
  mode which was used for locking the whole table (it is
  F_RDLCK now).
sql/lock.cc:
  Replaced conditions which used TL_WRITE_ALLOW_READ
  lock type with equivalent conditions using
  TL_WRITE_ALLOW_WRITE. This should allow to get rid
  of TL_WRITE_ALLOW_READ lock type eventually.
sql/mdl.cc:
  Updated outdated comment to reflect current situation.
sql/sql_base.cc:
  Replaced conditions which used TL_WRITE_ALLOW_READ
  lock type with equivalent conditions using
  TL_WRITE_ALLOW_WRITE. This should allow to get rid
  of TL_WRITE_ALLOW_READ lock type eventually.
sql/sql_table.cc:
  mysql_admin_table():
    Use TL_WRITE_ALLOW_WRITE lock type instead of
    TL_WRITE_ALLOW_READ to determine that we need to acquire
    upgradable metadata lock. This should allow to completely
    get rid of TL_WRITE_ALLOW_READ in long term.
  mysql_recreate_table():
    ALTER TABLE now requires TL_READ_NO_INSERT thr_lock.c lock
    instead of TL_WRITE_ALLOW_READ.
sql/sql_trigger.cc:
  Changed CREATE/DROP TRIGGER implementation to use
  TL_READ_NO_INSERT lock instead of TL_WRITE_ALLOW_READ lock.
  The latter is no longer necessary since:
  a) We now can rely on metadata locks to achieve proper
     isolation between two DDL statements or DDL and DML
     statements.
  b) This statement does not change any data in table so there
     is no need to inform storage engine about it.
sql/sql_yacc.yy:
  Changed implementation of ALTER TABLE (and CREATE/DROP INDEX
  as a consequence) to use TL_READ_NO_INSERT lock instead of
  TL_WRITE_ALLOW_READ lock. This is possible since:
  a) We now can rely on metadata locks to achieve proper
     isolation between two DDL statements or DDL and DML
     statements.
  b) This statement only reads data in table being open.
     We write data only to the new version of table and
     then replace with it old version of table under
     X metadata lock.
  
  Thanks to this change InnoDB will no longer acquire LOCK_X
  locks on rows being read by ALTER TABLE (instead LOCK_S
  locks will be acquired) and thus cause of bug #51263
  "Deadlock between transactional SELECT and ALTER TABLE ...
  REBUILD PARTITION" is removed.
  
  Did the similar change for CREATE TRIGGER (see comments
  for sql_trigger.cc for details).
 2010-05-26 16:18:08 +04:00
Davi Arnaut
4f18083b08 Bug#42643: InnoDB does not support replication of TRUNCATE TABLE 
The problem was that TRUNCATE TABLE didn't take a exclusive
lock on a table if it resorted to truncating via delete of
all rows in the table. Specifically for InnoDB tables, this
could break proper isolation as InnoDB ends up aborting some
granted locks when truncating a table.

The solution is to take a exclusive metadata lock before
TRUNCATE TABLE can proceed. This guarantees that no other
transaction is using the table.

Incompatible change: Truncate via delete no longer fails
if sql_safe_updates is activated (this was a undocumented
side effect).

libmysqld/CMakeLists.txt:
  Add new files to the build list.
libmysqld/Makefile.am:
  Add new files to the build list.
mysql-test/extra/binlog_tests/binlog_truncate.test:
  Add test case for Bug#42643
mysql-test/include/mix1.inc:
  Update test case as TRUNCATE TABLE now grabs a exclusive lock.
  Ensure that TRUNCATE waits for granted locks on the table.
mysql-test/suite/binlog/t/binlog_truncate_innodb.test:
  As with other data modifying statements, TRUNCATE is still not
  possible in a transaction with isolation level READ COMMITTED
  or READ UNCOMMITED. It would be possible to implement so, but
  it is not worth the effort.
mysql-test/suite/binlog/t/binlog_truncate_myisam.test:
  Test under different binlog formats.
mysql-test/suite/binlog/t/disabled.def:
  Re-enable test case.
mysql-test/t/innodb_bug38231.test:
  Truncate no longer works with row-level locks.
mysql-test/t/mdl_sync.test:
  Ensure that a acquired lock is not given up due to a conflict.
mysql-test/t/partition_innodb_semi_consistent.test:
  End transaction as to release metadata locks.
mysql-test/t/truncate.test:
  A metadata lock is now taken before the object is verified.
sql/CMakeLists.txt:
  Add new files to the build list.
sql/Makefile.am:
  Add new files to the build list.
sql/datadict.cc:
  Introduce a new file specific for data dictionary operations.
sql/datadict.h:
  Add header file.
sql/sql_base.cc:
  Rename data dictionary function.
sql/sql_bitmap.h:
  Include dependency.
sql/sql_delete.cc:
  Move away from relying on mysql_delete() to delete all rows of
  a table. Thus, move any bits related to truncate to sql_truncate.cc
sql/sql_delete.h:
  Remove parameter.
sql/sql_parse.cc:
  Add protection against the global read lock -- a intention
  exclusive lock can be acquired in the truncate path.
sql/sql_show.cc:
  Add sync point for testing scenarios where a pending flush
  is ignored.
sql/sql_truncate.cc:
  Acquire a shared metadata lock before accessing table metadata.
  Upgrade the lock to a exclusive one if the table can be re-created.
  Rework binlog rules to better reflect the requirements.
sql/sql_yacc.yy:
  Set appropriate lock types for table to be truncated.
sql/table.h:
  Move to data dictionary header.
 2010-05-25 17:01:38 -03:00
Dmitry Lenev
bee0f214fd Pre-requisite patch for bug #51263 "Deadlock between 
transactional SELECT and ALTER TABLE ... REBUILD PARTITION".

The goal of this patch is to decouple type of metadata
lock acquired for table by open_tables() from type of
table-level lock to be acquired on it.

To achieve this we change approach to how we determine what
type of metadata lock should be acquired on table to be open.
Now instead of inferring it at open_tables() time from flags
and type of table-level lock we rely on that type of metadata
lock is properly set at parsing time and is not changed
further.

sql/ha_ndbcluster.cc:
  Now one needs to properly initialize table list element's
  MDL_request object before calling mysql_rm_table_part2().
sql/lock.cc:
  lock_table_names() no longer initializes table list elements'
  MDL_request objects. Now proper initialization of these
  requests is a responsibility of the caller.
sql/lock.h:
  Removed MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag which became
  unnecessary. Thanks to the fact that we don't reset type of
  requests for metadata locks between re-executions we now can
  figure out that upgradable locks are requested by simply
  looking at their type which were set in the parser. As result
  this flag became redundant.
sql/mdl.h:
  Added version of new operator which simplifies allocation of
  MDL_request objects on a MEM_ROOT.
sql/sp_head.cc:
  Added comment explaining why it is OK to infer type of
  metadata lock to request from type of table-level lock
  for prelocking.
  Added enum_mdl_type argument to sp_add_to_query_tables()
  to simplify its usage in trigger implementation.
sql/sp_head.h:
  Added enum_mdl_type argument to sp_add_to_query_tables()
  to simplify its usage in trigger implementation.
sql/sql_base.cc:
  - open_table_get_mdl_lock():
    Preserve type of MDL_request for table list element which
    was set in the parser by creating MDL_request objects on
    memory root if MYSQL_OPEN_FORCE_SHARED_MDL or
    MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag were specified.
    Thanks to this and to the fact that we no longer reset
    type of requests for metadata locks between re-executions
    we no longer need to acquire exclusive metadata lock on
    table to be created in a special way. This lock is acquired
    by code handling acquiring of upgradable locks.
    Also changed signature/calling convention for this function
    to simplify its usage.
  - Accordingly special lock strategy for table list elements
    which was used for such locks became unnecessary and was
    removed. Other strategies were renamed.
  - Since we no longer have guarantee that MDL_request object
    which were not satisfied due to lock conflict belongs to
    table list element Open_table_context class and its methods
    were extended to remember pointer to MDL_request which has
    caused problem at request_backoff_action() time and use it
    in recover_from_failed_open(). Similar approach is used
    for cases when problem from which we need to recover is
    not related to MDL but to the table itself. In this case
    we store pointer to the element of table list.
  - Changed open_tables()/open_tables_check_upgradable_mdl()/
    open_tables_acquire_upgradable_mdl() not to rely on
    MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to understand when
    upgradable metadata locks should be acquired and not to
    infer type of MDL lock from type of table-level lock.
    Instead we assume that type of MDL to be acquired was set
    in the parser (we can do this as type of MDL_request is
    no longer reset between re-executions).
sql/sql_class.h:
  Since we no longer have guarantee that MDL_request object
  which were not satisfied due to lock conflict belongs to
  table list element Open_table_context class and its methods
  were extended to remember pointer to MDL_request which has
  caused problem at request_backoff_action() time and use it
  in recover_from_failed_open(). Similar approach is used
  for cases when problem from which we need to recover is
  not related to MDL but to the table itself. In this case
  we store pointer to the element of table list.
sql/sql_db.cc:
  Now one needs to properly initialize table list element's
  MDL_request object before calling mysql_rm_table_part2()
  or mysql_rename_tables().
sql/sql_lex.cc:
  st_select_lex/st_select_lex_node::add_table_to_list() method
  now has argument which allows specify type of metadata lock
  to be requested for table list element being added.
sql/sql_lex.h:
  - st_select_lex/st_select_lex_node::add_table_to_list()
    method now has argument which specifies type of metadata
    lock to be requested for table list element being added.
    This allows to explicitly set type of MDL lock to be
    acquired for a DDL statement in parser. It is also more
    future-proof than inferring type of MDL request from type
    of table-level lock.
  - Added Yacc_state::m_mdl_type member which specifies which
    type of metadata lock should be requested for tables to be
    added to table list by a grammar rule in cases when the same
    rule is used in several statements requiring different kinds
    of metadata locks.
sql/sql_parse.cc:
  - st_select_lex::add_table_to_list() method now has argument
    which specifies type of metadata lock to be requested for
    table list element being added. This allows to explicitly
    set type of MDL lock to be acquired for a DDL statement in
    parser. It is also more future-proof than inferring type of
    MDL request from type of table-level lock.
  - EXCLUSIVE_DOWNGRADABLE_MDL lock strategy has a new name -
    OTLS_DOWNGRADE_IF_EXISTS.
  - Adjusted LOCK TABLES implementation to the fact that we no
    longer infer type of metadata lock to be acquired from table
    level lock and that type of MDL request is set at parsing.
    And thus MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag became
    unnecessary.
sql/sql_prepare.cc:
  TABLE_LIST's lock strategy SHARED_MDL was renamed to OTLS_NONE
  as now it means that metadata lock should not be changed during
  call to open_table() (if it has been already acquired) and is
  also used for exclusive metadata lock.
sql/sql_show.cc:
  st_select_lex::add_table_to_list() method now has argument
  which specifies type of metadata lock to be requested for
  table list element being added.
sql/sql_table.cc:
  - Adjusted mysql_admin_table()'s code to the fact that
    open_tables() no longer determines what kind of metadata
    lock should be obtained basing on type of table-level
    lock and flags. Instead type of metadata lock for table
    to be open should be set before calling open_tables().
  - Changed mysql_alter_table() code to the facts:
    a) that now it is responsibility of caller to properly
    initalize MDL_request in table list elements before calling
    lock_table_names()
    b) and that MYSQL_OPEN_TAKE_UPGRADABLE_MDL is no longer
    necessary since type of metadata lock to be obtained
    at open_tables() time is set during parsing.
  - Changed code of mysql_recreate_table() to properly set
    type of metadata and table-level lock to be obtained
    by mysql_alter_table() which it calls.
sql/sql_trigger.cc:
  Instead of relying on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to
  force open_tables() to take an upgradable lock we now specify
  exact type of lock to be taken when constructing table list
  element for table to be open for CREATE/DROP TRIGGER.
sql/sql_view.cc:
  We no longer use TABLE_LIST::EXCLUSIVE_MDL strategy to force
  open_tables() to take an exclusive metadata lock on view to
  be created. Instead we rely on parser setting proper type of
  metadata lock to request and open_tables() acquiring it.
  This became possible thanks to the fact that we no longer
  reset type of MDL_request between statement re-executions.
sql/sql_yacc.yy:
  Instead of inferring type of MDL_request for table to be
  open from type of table-level lock and flags passed to
  open_tables() we now explicitly specify them at parsing.
  This became possible thanks to the fact that we no longer
  reset type of MDL_request between statement re-executions.
  In future this should allow to decouple type of metadata
  lock from type of table-level lock.
  The only exception to this approach is statements implemented
  through mysql_admin_table() which re-uses same table list
  element several times with different types of table-level
  and metadata locks.
  We now also properly initialize MDL_request objects for table
  list elements which are later passed to lock_table_names()
  function.
sql/table.cc:
  Do not reset type of MDL_request between statement
  re-executions. This became unnecessesary as we no longer
  change type of MDL_request residing in table list element.
  In its turn this change allows to set type of MDL_request
  only once - at parsing time.
sql/table.h:
  Got rid of TABLE_LIST::EXCLUSIVE_MDL lock strategy.
  Now we can specify that we need to acquire exclusive lock
  on table to be processed by open_tables() through setting
  an appropriate type of MDL_request at parsing time (this
  became possible thanks to the fact that we no longer reset
  types of MDL_request's belonging to table list elements
  between statement re-execution).
  Strategy SHARED_MDL was renamed to OTLS_NONE as now it
  means that metadata lock should not be changed during call
  to open_table() (if it has been already acquired) and is
  also used for exclusive metadata lock.
  Strategy EXCLUSIVE_DOWNGRADABLE_MDL was renamed to
  OTLS_DOWNGRADE_IF_EXISTS.
 2010-05-25 16:35:01 +04:00
Dmitry Lenev
705b98dff7 Follow-up for the fix for bug #46947 "Embedded SELECT without 
FOR UPDATE is causing a lock".
 
This patch tries to address problems which were exposed 
during backporting of original patch to 5.1 tree.
 
- It ensures that we don't change locking behavior of simple
  SELECT statements on InnoDB tables when they are executed
  under LOCK TABLES ... READ and with @@innodb_table_locks=0.
  Also we no longer pass TL_READ_DEFAULT/TL_WRITE_DEFAULT 
  lock types, which are supposed to be parser-only, to 
  handler::start_stmt() method.
- It makes check_/no_concurrent_insert.inc auxiliary scripts 
  more robust against changes in test cases that use them
  and also ensures that they don't unnecessarily change 
  environment of caller.

mysql-test/include/check_concurrent_insert.inc:
  Reset DEBUG_SYNC facility before and after using it in
  auxiliary script. This makes this script more robust against
  changes in test cases calling it. It also ensures that script
  does not unnecessarily change environment of caller.
mysql-test/include/check_no_concurrent_insert.inc:
  Reset DEBUG_SYNC facility before and after using it in
  auxiliary script. This makes this script more robust against
  changes in test cases calling it. It also ensures that script
  does not unnecessarily change environment of caller.
mysql-test/r/innodb-lock.result:
  Added coverage for LOCK TABLES ... READ behavior in
  @@innodb_table_locks = 0 mode. This test also checks
  that an appropriate type of lock is passed to
  handler::start_stmt() method.
mysql-test/t/innodb-lock.test:
  Added coverage for LOCK TABLES ... READ behavior in
  @@innodb_table_locks = 0 mode. This test also checks
  that an appropriate type of lock is passed to
  handler::start_stmt() method.
sql/sql_base.cc:
  Since we no longer set TL_READ as lock type for tables used  
  in simple SELECT right in the parser, in order to preserve  
  behavior for such statements on InnoDB tables when in  
  LOCK TABLES mode with @innodb_table_locks = 0,  
  check_lock_and_start_stmt() had to be changed to convert  
  TL_READ_DEFAULT to an appropriate type of read lock before  
  passing it to handler::start_stmt() method.  
  We do similar thing for TL_WRITE_DEFAULT as this lock type  
  is also supposed to be parser-only type.  
  As consequence read_lock_type_for_table() had to be  
  adjusted to behave properly when it is called from  
  check_lock_and_start_stmt() in prelocked mode.
 2010-05-21 16:41:24 +04:00
Alfranio Correia
3404d83083 BUG#49019 Mixing self-logging eng. and regular eng. does not switch to row in mixed 
mode
      
Post-push fix after backporting the patch to 5.1-bugteam:

  1 - changed the name of some variables to be equivalent to pe.
  2 - fixed that patch to mark a statement as unsafe when both a
  self-logging eng. and regular eng. are accessed and one of them
  is updated.
 2010-05-16 15:37:44 +01:00
Alexey Kopytov
e118ec409f Manual merge from mysql-5.1-bugteam to mysql-trunk-merge. 
Conflicts:

Text conflict in sql/sql_base.cc
Text conflict in sql/sql_partition.cc
Text conflict in sql/sql_priv.h
Text conflict in sql/sql_show.cc
 2010-05-11 20:02:50 +04:00
Mattias Jonsson
6fa04cad8f merge 2010-05-10 15:17:06 +02:00
Alfranio Correia
209cccf55d BUG#49019 Mixing self-logging eng. and regular eng. does not switch to row in mixed mode 
Backport of the patch for 5.1-bugteam.
 2010-05-09 23:45:25 +01:00
Konstantin Osipov
9e62cf67b3 Clean-up, give better names, add comments to 
thd->in_multi_stmt_transaction() and thd->active_transaction().


include/mysql_com.h:
  Comment SERVER_STATUS_IN_TRANS flag.
sql/ha_ndbcluster.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
sql/handler.cc:
  Add comments.
sql/log.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
sql/log_event.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
sql/sql_base.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
sql/sql_cache.cc:
  Rename: thd->active_transaction() ->
  thd->in_active_multi_stmt_transaction().
sql/sql_class.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
sql/sql_class.h:
  Rename and comment two transaction processing- related methods.
sql/sql_parse.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
  Rename: thd->active_transaction() ->
  thd->in_active_multi_stmt_transaction().
sql/sql_prepare.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
sql/sql_rename.cc:
  Rename: thd->active_transaction() ->
  thd->in_active_multi_stmt_transaction().
sql/sql_table.cc:
  Rename: thd->active_transaction() ->
  thd->in_active_multi_stmt_transaction().
sql/sys_vars.cc:
  Rename: thd->active_transaction() ->
  thd->in_active_multi_stmt_transaction().
sql/transaction.cc:
  Rename: thd->in_multi_stmt_transaction() -> 
  thd->in_multi_stmt_transaction_mode().
  Rename: thd->active_transaction() ->
  thd->in_active_multi_stmt_transaction().
 2010-05-06 02:02:08 +04:00
Konstantin Osipov
1ab519d91f Committing on behalf or Dmitry Lenev: 
Fix for bug #46947 "Embedded SELECT without FOR UPDATE is
causing a lock", with after-review fixes.

SELECT statements with subqueries referencing InnoDB tables
were acquiring shared locks on rows in these tables when they
were executed in REPEATABLE-READ mode and with statement or
mixed mode binary logging turned on.

This was a regression which were introduced when fixing
bug 39843.

The problem was that for tables belonging to subqueries
parser set TL_READ_DEFAULT as a lock type. In cases when
statement/mixed binary logging at open_tables() time this
type of lock was converted to TL_READ_NO_INSERT lock at
open_tables() time and caused InnoDB engine to acquire
shared locks on reads from these tables. Although in some
cases such behavior was correct (e.g. for subqueries in
DELETE) in case of SELECT it has caused unnecessary locking.

This patch tries to solve this problem by rethinking our
approach to how we handle locking for SELECT and subqueries.
Now we always set TL_READ_DEFAULT lock type for all cases
when we read data. When at open_tables() time this lock
is interpreted as TL_READ_NO_INSERT or TL_READ depending
on whether this statement as a whole or call to function
which uses particular table should be written to the
binary log or not (if yes then statement should be properly
serialized with concurrent statements and stronger lock
should be acquired).

Test coverage is added for both InnoDB and MyISAM.

This patch introduces an "incompatible" change in locking
scheme for subqueries used in SELECT ... FOR UPDATE and
SELECT .. IN SHARE MODE.
In 4.1 the server would use a snapshot InnoDB read for 
subqueries in SELECT FOR UPDATE and SELECT .. IN SHARE MODE
statements, regardless of whether the binary log is on or off.
If the user required a different type of read (i.e. locking read),
he/she could request so explicitly by providing FOR UPDATE/IN SHARE MODE
clause for each individual subquery.
On of the patches for 5.0 broke this behaviour (which was not documented
or tested), and started to use locking reads fora all subqueries in SELECT ... 
FOR UPDATE/IN SHARE MODE. This patch restored 4.1 behaviour.

mysql-test/include/check_concurrent_insert.inc:
  Added auxiliary script which allows to check if statement
  reading table allows concurrent inserts in it.
mysql-test/include/check_no_concurrent_insert.inc:
  Added auxiliary script which allows to check that statement
  reading table doesn't allow concurrent inserts in it.
mysql-test/include/check_no_row_lock.inc:
  Added auxiliary script which allows to check if statement
  reading table doesn't take locks on its rows.
mysql-test/include/check_shared_row_lock.inc:
  Added auxiliary script which allows to check if statement
  reading table takes shared locks on some of its rows.
mysql-test/r/bug39022.result:
  After bug #46947 'Embedded SELECT without FOR UPDATE is
  causing a lock' was fixed test case for bug 39022 has to
  be adjusted in order to trigger execution path on which
  original problem was encountered.
mysql-test/r/innodb_mysql_lock2.result:
  Added coverage for handling of locking in various cases when
  we read data from InnoDB tables (includes test case for
  bug #46947 'Embedded SELECT without FOR UPDATE is causing a
  lock').
mysql-test/r/lock_sync.result:
  Added coverage for handling of locking in various cases when
  we read data from MyISAM tables.
mysql-test/t/bug39022.test:
  After bug #46947 'Embedded SELECT without FOR UPDATE is
  causing a lock' was fixed test case for bug 39022 has to
  be adjusted in order to trigger execution path on which
  original problem was encountered.
mysql-test/t/innodb_mysql_lock2.test:
  Added coverage for handling of locking in various cases when
  we read data from InnoDB tables (includes test case for
  bug #46947 'Embedded SELECT without FOR UPDATE is causing a
  lock').
mysql-test/t/lock_sync.test:
  Added coverage for handling of locking in various cases when
  we read data from MyISAM tables.
sql/log_event.cc:
  Since LEX::lock_option member was removed we no longer can
  rely on its value in Load_log_event::print_query() to
  determine that log event correponds to LOAD DATA CONCURRENT
  statement (this was not correct in all situations anyway).
  A new Load_log_event's member was introduced as a replacement.
  It is initialized at event object construction time and
  explicitly indicates whether LOAD DATA was concurrent.
sql/log_event.h:
  Since LEX::lock_option member was removed we no longer can
  rely on its value in Load_log_event::print_query() to
  determine that log event correponds to LOAD DATA CONCURRENT
  statement (this was not correct in all situations anyway).
  A new Load_log_event's member was introduced as a replacement.
  It is initialized at event object construction time and
  explicitly indicates whether LOAD DATA was concurrent.
sql/sp_head.cc:
  sp_head::reset_lex():
    Before parsing substatement reset part of parser state
    which needs this (e.g. set Yacc_state::m_lock_type to
    default value).
sql/sql_acl.cc:
  Since LEX::reset_n_backup_query_tables_list() now also
  resets LEX::sql_command member (as it became part of
  Query_tables_list class) we have to restore it in cases
  when while working with proxy Query_table_list we assume
  that LEX::sql_command still corresponds to original SQL
  command being executed (for example, when we are logging
  statement to the binary log while having Query_tables_list
  reset and backed up).
sql/sql_base.cc:
  Changed read_lock_type_for_table() to return a weak TL_READ
  type of lock in cases when we are executing statement which
  won't update tables directly and table doesn't belong to
  statement's prelocking list and thus can't be used by a
  stored function. It is OK to do so since in this case table
  won't be used by statement or function call which will be
  written to the binary log, so serializability requirements
  for it can be relaxed.
  One of results from this change is that SELECTs on InnoDB
  tables no longer takes shared row locks for tables which
  are used in subqueries (i.e. bug #46947 is fixed).
  Another result is that for similar SELECTs on MyISAM tables
  concurrent inserts are allowed.
  In order to implement this change signature of
  read_lock_type_for_table() function was changed to take
  pointers to Query_tables_list and TABLE_LIST objects.
sql/sql_base.h:
  - Function read_lock_type_for_table() now takes pointers
    to Query_tables_list and TABLE_LIST elements as its
    arguments since to correctly determine lock type it needs
    to know what statement is being performed and whether table
    element for which lock type to be determined belongs to
    prelocking list.
sql/sql_lex.cc:
  - Removed LEX::lock_option and st_select_lex::lock_option
    members. Places in parser that were using them now use
    Yacc_state::m_lock_type instead.
  - To emphasize that LEX::sql_command member is used during
    process of opening and locking of tables it was moved to
    Query_tables_list class. It is now reset by
    Query_tables_list::reset_query_tables_list() method.
sql/sql_lex.h:
  - Removed st_select_lex::lock_option member as there is no
    real need for per-SELECT lock type (HIGH_PRIORITY option
    should apply to the whole statement. FOR UPDATE/LOCK IN
    SHARE MODE clauses can be handled without this member).
    The main effect which was achieved by introduction of this
    member, i.e. using TL_READ_DEFAULT lock type for
    subqueries, is now achieved by setting LEX::lock_option
    (or rather its replacement - Yacc_state::m_lock_type) to
    TL_READ_DEFAULT in almost all cases.
  - To emphasize that LEX::sql_command member is used during
    process of opening and locking of tables it was moved to
    Query_tables_list class.
  - Replaced LEX::lock_option with Yacc_state::m_lock_type
    in order to emphasize that this value is relevant only
    during parsing. Unlike for LEX::lock_option the default
    value for Yacc_state::m_lock_type is TL_READ_DEFAULT.
    Note that for cases when it is OK to take a "weak" read
    lock (e.g. simple SELECT) this lock type will be converted
    to TL_READ at open_tables() time. So this change won't
    cause negative change in behavior for such statements.
    OTOH this change ensures that, for example, for SELECTs
    which are used in stored functions TL_READ_NO_INSERT lock
    is taken when necessary and as result calls to such stored
    functions can be written to the binary log with correct
    serialization.
sql/sql_load.cc:
  Load_log_event constructor now requires a parameter that
  indicates whether LOAD DATA is concurrent.
sql/sql_parse.cc:
  LEX::lock_option was replaced with Yacc_state::m_lock_type.
  And instead of resetting the latter implicitly in
  mysql_init_multi_delete() we do it explicitly in the
  places in parser which call this function.
sql/sql_priv.h:
  - To be able more easily distinguish high-priority SELECTs
    in st_select_lex::print() method added flag for
    HIGH_PRIORITY option.
sql/sql_select.cc:
  Changed code not to rely on LEX::lock_option to determine
  that it is high-priority SELECT. It was replaced with
  Yacc_state::m_lock_type which is accessible only at
  parse time. So instead of LEX::lock_option we now rely
  on a newly introduced flag for st_select_lex::options -
  SELECT_HIGH_PRIORITY.
sql/sql_show.cc:
  Since LEX::reset_n_backup_query_tables_list() now also
  resets LEX::sql_command member (as it became part of
  Query_tables_list class) we have to restore it in cases
  when while working with proxy Query_table_list we assume
  that LEX::sql_command still corresponds to original SQL
  command being executed.
sql/sql_table.cc:
  Since LEX::reset_query_tables_list() now also resets
  LEX::sql_command member (as it became part of
  Query_tables_list class) we have to restore value of this
  member when this method is called by mysql_admin_table(),
  to make this code safe for re-execution.
sql/sql_trigger.cc:
  Since LEX::reset_n_backup_query_tables_list() now also
  resets LEX::sql_command member (as it became part of
  Query_tables_list class) we have to restore it in cases
  when while working with proxy Query_table_list we assume
  that LEX::sql_command still corresponds to original SQL
  command being executed (for example, when we are logging
  statement to the binary log while having Query_tables_list
  reset and backed up).
sql/sql_update.cc:
  Function read_lock_type_for_table() now takes pointers
  to Query_tables_list and TABLE_LIST elements as its
  arguments since to correctly determine lock type it needs
  to know what statement is being performed and whether table
  element for which lock type to be determined belongs to
  prelocking list.
sql/sql_yacc.yy:
  - Removed st_select_lex::lock_option member as there is no
    real need for per-SELECT lock type (HIGH_PRIORITY option
    should apply to the whole statement. FOR UPDATE/LOCK IN
    SHARE MODE clauses can be handled without this member).
    The main effect which was achieved by introduction of this
    member, i.e. using TL_READ_DEFAULT lock type for
    subqueries, is now achieved by setting LEX::lock_option
    (or rather its replacement - Yacc_state::m_lock_type) to
    TL_READ_DEFAULT in almost all cases.
  - Replaced LEX::lock_option with Yacc_state::m_lock_type
    in order to emphasize that this value is relevant only
    during parsing. Unlike for LEX::lock_option the default
    value for Yacc_state::m_lock_type is TL_READ_DEFAULT.
    Note that for cases when it is OK to take a "weak" read
    lock (e.g. simple SELECT) this lock type will be converted
    to TL_READ at open_tables() time. So this change won't
    cause negative change in behavior for such statements.
    OTOH this change ensures that, for example, for SELECTs
    which are used in stored functions TL_READ_NO_INSERT lock
    is taken when necessary and as result calls to such stored
    functions can be written to the binary log with correct
    serialization.
  - To be able more easily distinguish high-priority SELECTs
    in st_select_lex::print() method we now use new flag
    in st_select_lex::options bit-field.
 2010-04-28 14:04:11 +04:00
Jon Olav Hauglid
ac78736638 merge from mysql-trunk-bugfixing 2010-04-20 10:51:50 +02:00