Bug#41112: crash in mysql_ha_close_table/get_lock_data with alter table
The problem is that the server wasn't handling robustly failures
to re-open a table during a HANDLER .. READ statement. If the
table needed to be re-opened due to it's storage engine being
altered to one that doesn't support HANDLER, a reference (dangling
pointer) to a closed table could be left in place and accessed in
later attempts to fetch from the table using the handler. Also,
if the server failed to set a error message if the re-open
failed. These problems could lead to server crashes or hangs.
The solution is to remove any references to a closed table and
to set a error if reopening a table during a HANDLER .. READ
statement fails.
mysql-test/include/handler.inc:
Add test case for Bug#41110 and Bug#41112
mysql-test/r/handler_innodb.result:
Add test case result for Bug#41110 and Bug#41112
mysql-test/r/handler_myisam.result:
Add test case result for Bug#41110 and Bug#41112
sql/sql_handler.cc:
Remove redundant reopen check.
Set errors even if reopening table.
Reset TABLE_LIST::table reference when the table is closed.
Bug#41112: crash in mysql_ha_close_table/get_lock_data with alter table
The problem is that the server wasn't handling robustly failures
to re-open a table during a HANDLER .. READ statement. If the
table needed to be re-opened due to it's storage engine being
altered to one that doesn't support HANDLER, a reference (dangling
pointer) to a closed table could be left in place and accessed in
later attempts to fetch from the table using the handler. Also,
if the server failed to set a error message if the re-open
failed. These problems could lead to server crashes or hangs.
The solution is to remove any references to a closed table and
to set a error if reopening a table during a HANDLER .. READ
statement fails.
Bug#41112: crash in mysql_ha_close_table/get_lock_data with alter table
The problem is that the server wasn't handling robustly failures
to re-open a table during a HANDLER .. READ statement. If the
table needed to be re-opened due to it's storage engine being
altered to one that doesn't support HANDLER, a reference (dangling
pointer) to a closed table could be left in place and accessed in
later attempts to fetch from the table using the handler. Also,
if the server failed to set a error message if the re-open
failed. These problems could lead to server crashes or hangs.
The solution is to remove any references to a closed table and
to set a error if reopening a table during a HANDLER .. READ
statement fails.
There is no test case in this change set as the test depends on
a testing feature only available on 5.1 and later.
sql/sql_handler.cc:
Remove redundant reopen check.
Set errors even if reopening table.
Reset TABLE_LIST::table reference when the table is closed.
Bug#41112: crash in mysql_ha_close_table/get_lock_data with alter table
The problem is that the server wasn't handling robustly failures
to re-open a table during a HANDLER .. READ statement. If the
table needed to be re-opened due to it's storage engine being
altered to one that doesn't support HANDLER, a reference (dangling
pointer) to a closed table could be left in place and accessed in
later attempts to fetch from the table using the handler. Also,
if the server failed to set a error message if the re-open
failed. These problems could lead to server crashes or hangs.
The solution is to remove any references to a closed table and
to set a error if reopening a table during a HANDLER .. READ
statement fails.
There is no test case in this change set as the test depends on
a testing feature only available on 5.1 and later.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
This bug is actually two bugs in one, one of which is CREATE TRIGGER under
LOCK TABLES and the other is CREATE TRIGGER under LOCK TABLES simultaneous
to a FLUSH TABLES WITH READ LOCK (global read lock). Both situations could
lead to a server crash or deadlock.
The first problem arises from the fact that when under LOCK TABLES, if the
table is in the set of locked tables, the table is already open and it doesn't
need to be reopened (not a placeholder). Also in this case, if the table is
not write locked, a exclusive lock can't be acquired because of a possible
deadlock with another thread also holding a (read) lock on the table. The
second issue arises from the fact that one should never wait for a global
read lock if it's holding any locked tables, because the global read lock
is waiting for these tables and this leads to a circular wait deadlock.
The solution for the first case is to check if the table is write locked
and upgraded the write lock to a exclusive lock and fail otherwise for non
write locked tables. Grabbin the exclusive lock in this case also means
to ensure that the table is opened only by the calling thread. The second
issue is partly fixed by not waiting for the global read lock if the thread
is holding any locked tables.
The second issue is only partly addressed in this patch because it turned
out to be much wider and also affects other DDL statements. Reported as
Bug#32395
mysql-test/r/trigger.result:
Add test case result for Bug#23713
mysql-test/r/trigger_notembedded.result:
Add test case result for Bug#23713
mysql-test/t/trigger.test:
Add test case for Bug#23713
mysql-test/t/trigger_notembedded.test:
Add test case for Bug#23713
sql/mysql_priv.h:
Locally export wait_while_table_is_used and name_lock_locked_table
and add flag to mysql_ha_rm_tables to signal that LOCK_open is locked.
sql/sql_base.cc:
Introduce name_lock_locked_table function and match
close_old_data_files function declaration and definition.
sql/sql_handler.cc:
Add flag to mysql_ha_rm_tables to signal that LOCK_open is locked.
sql/sql_rename.cc:
Fix mysql_ha_rm_tables caller.
sql/sql_table.cc:
Export wait_while_table_is_used and assert that LOCK_open is locked
and fix mysql_ha_rm_tables caller.
sql/sql_trigger.cc:
Upgrade write locked tables to a exclusive lock and fail if
the table is not write locked. Also, don't wait for the global
read lock if under LOCK TABLES.
This bug is actually two bugs in one, one of which is CREATE TRIGGER under
LOCK TABLES and the other is CREATE TRIGGER under LOCK TABLES simultaneous
to a FLUSH TABLES WITH READ LOCK (global read lock). Both situations could
lead to a server crash or deadlock.
The first problem arises from the fact that when under LOCK TABLES, if the
table is in the set of locked tables, the table is already open and it doesn't
need to be reopened (not a placeholder). Also in this case, if the table is
not write locked, a exclusive lock can't be acquired because of a possible
deadlock with another thread also holding a (read) lock on the table. The
second issue arises from the fact that one should never wait for a global
read lock if it's holding any locked tables, because the global read lock
is waiting for these tables and this leads to a circular wait deadlock.
The solution for the first case is to check if the table is write locked
and upgraded the write lock to a exclusive lock and fail otherwise for non
write locked tables. Grabbin the exclusive lock in this case also means
to ensure that the table is opened only by the calling thread. The second
issue is partly fixed by not waiting for the global read lock if the thread
is holding any locked tables.
The second issue is only partly addressed in this patch because it turned
out to be much wider and also affects other DDL statements. Reported as
Bug#32395
The problem is that DROP TABLE and other DDL statements failed to
automatically close handlers associated with tables that were marked
for reopen (FLUSH TABLES).
The current implementation fails to properly discard handlers of
dropped tables (that were marked for reopen) because it searches
on the open handler tables list and using the current alias of the
table being dropped. The problem is that it must not use the open
handler tables list to search because the table might have been
closed (marked for reopen) by a flush tables command and also it
must not use the current table alias at all since multiple different
aliases may be associated with a single table. This is specially
visible when a user has two open handlers (using alias) of a same
table and a flush tables command is issued before the table is
dropped (see test case). Scanning the handler table list is also
useless for dropping handlers associated with temporary tables,
because temporary tables are not kept in the THD::handler_tables
list.
The solution is to simple scan the handlers hash table searching
for, and deleting all handlers with matching table names if the
reopen flag is not passed to the flush function, indicating that
the handlers should be deleted. All matching handlers are deleted
even if the associated the table is not open.
mysql-test/include/handler.inc:
Add test case for Bug#31397
mysql-test/r/handler_innodb.result:
Add test case result for Bug#31397
mysql-test/r/handler_myisam.result:
Add test case result for Bug#31397
sql/mysql_priv.h:
Rename flush functions to better match the intent of the caller and
update functions prototypes and remove unused flags.
sql/sql_base.cc:
Rename flush functions to better match the intent of the caller.
sql/sql_class.cc:
Rename the flush functions to better match the intent of the caller.
The hash_free function is moved to the cleanup.
sql/sql_handler.cc:
When dropping tables for a final close, scan the handler's hash table since
the table might not be in the handlers open table list because the table was
marked for reopen or because it's a temporary table.
sql/sql_rename.cc:
Drop handlers associated with tables that are being renamed.
sql/sql_table.cc:
Now that temporary tables are properly removed even when opened
by a SQL HANDLER, enable the assert since this branch can't be taken
outside of SF/trigger/prelocked mode.
The problem is that DROP TABLE and other DDL statements failed to
automatically close handlers associated with tables that were marked
for reopen (FLUSH TABLES).
The current implementation fails to properly discard handlers of
dropped tables (that were marked for reopen) because it searches
on the open handler tables list and using the current alias of the
table being dropped. The problem is that it must not use the open
handler tables list to search because the table might have been
closed (marked for reopen) by a flush tables command and also it
must not use the current table alias at all since multiple different
aliases may be associated with a single table. This is specially
visible when a user has two open handlers (using alias) of a same
table and a flush tables command is issued before the table is
dropped (see test case). Scanning the handler table list is also
useless for dropping handlers associated with temporary tables,
because temporary tables are not kept in the THD::handler_tables
list.
The solution is to simple scan the handlers hash table searching
for, and deleting all handlers with matching table names if the
reopen flag is not passed to the flush function, indicating that
the handlers should be deleted. All matching handlers are deleted
even if the associated the table is not open.
If a stored function that contains a drop temporary table statement
is invoked by a create temporary table of the same name may cause
a server crash. The problem is that when dropping a table no check
is done to ensure that table is not being used by some outer query
(or outer statement), potentially leaving the outer query with a
reference to a stale (freed) table.
The solution is when dropping a temporary table, always check if
the table is being used by some outer statement as a temporary
table can be dropped inside stored procedures.
The check is performed by looking at the TABLE::query_id value for
temporary tables. To simplify this check and to solve a bug related
to handling of temporary tables in prelocked mode, this patch changes
the way in which this member is used to track the fact that table is
used/unused. Now we ensure that TABLE::query_id is zero for unused
temporary tables (which means that all temporary tables which were
used by a statement should be marked as free for reuse after it's
execution has been completed).
mysql-test/include/handler.inc:
Add test case for side effect of Bug#30882
mysql-test/r/handler_innodb.result:
Add test case result for side effect of Bug#30882
mysql-test/r/handler_myisam.result:
Add test case result for side effect of Bug#30882
mysql-test/r/sp-error.result:
Add test case result for Bug#30882
mysql-test/t/sp-error.test:
Add test case for Bug#30882
sql/event_db_repository.cc:
Update close_thread_tables call, no more default values.
sql/mysql_priv.h:
Remove implicit default parameters values of the close_thread_tables
function as no callers are using it.
sql/slave.cc:
Update close_thread_tables call, no more default values
sql/sp_head.cc:
Update close_thread_tables call, no more default values
sql/sql_base.cc:
Changed the approach to distinguishing currently unused temporary tables.
Now we ensure that such tables always have TABLE::query_id set to 0 and
use this fact to perform checks during opening and dropping of temporary
tables. This means that we have to call close_thread_tables() even for
statements which use only temporary tables. To make this call cheaper,
we re-factored close_thread_tables() to not take LOCK_open unless there
are open base tables.
sql/sql_handler.cc:
Properly close temporary tables associated with a handler.
sql/sql_insert.cc:
close_temporary_table is now merged into drop_temporary_table.
sql/sql_parse.cc:
Now the condition doesn't cover all cases because close_thread_tables()
must be called even for statements that use only temporary tables.
sql/sql_table.cc:
Use drop_temporary_table which perform checks to verify if
the table is not being used. Error path problem is due to
a handler tables issue and is going to be addressed in bug
31397.
sql/table.h:
Rename previously unused clear_query_id and document the usage of
query_id and open_by_handler.
If a stored function that contains a drop temporary table statement
is invoked by a create temporary table of the same name may cause
a server crash. The problem is that when dropping a table no check
is done to ensure that table is not being used by some outer query
(or outer statement), potentially leaving the outer query with a
reference to a stale (freed) table.
The solution is when dropping a temporary table, always check if
the table is being used by some outer statement as a temporary
table can be dropped inside stored procedures.
The check is performed by looking at the TABLE::query_id value for
temporary tables. To simplify this check and to solve a bug related
to handling of temporary tables in prelocked mode, this patch changes
the way in which this member is used to track the fact that table is
used/unused. Now we ensure that TABLE::query_id is zero for unused
temporary tables (which means that all temporary tables which were
used by a statement should be marked as free for reuse after it's
execution has been completed).
If mysql_lock_tables fails because the lock was aborted, we need to
reset thd->some_tables_delete, otherwise we might loop indefinitely
because handler's tables are not closed in a standard way, meaning
that close_thread_tables() (which resets some_tables_deleted) is not
used.
This patch fixes sporadical failures of handler_myisam/innodb tests
which were introduced by previous fix for this bug.
sql/sql_handler.cc:
Properly reset thd->some_tables_deleted if mysql_lock_tables
fails for some reason.
If mysql_lock_tables fails because the lock was aborted, we need to
reset thd->some_tables_delete, otherwise we might loop indefinitely
because handler's tables are not closed in a standard way, meaning
that close_thread_tables() (which resets some_tables_deleted) is not
used.
This patch fixes sporadical failures of handler_myisam/innodb tests
which were introduced by previous fix for this bug.
This deadlock occurs when a client issues a HANDLER ... OPEN statement
that tries to open a table that has a pending name-lock on it by another
client that also needs a name-lock on some other table which is already
open and associated to a HANDLER instance owned by the first client.
The deadlock happens because the open_table() function will back-off
and wait until the name-lock goes away, causing a circular wait if some
other name-lock is also pending for one of the open HANDLER tables.
Such situation, for example, can be easily repeated by issuing a RENAME
TABLE command in such a way that the existing table is already open
as a HANDLER table by another client and this client tries to open
a HANDLER to the new table name.
The solution is to allow handler tables with older versions (marked for
flush) to be closed before waiting for the name-lock completion. This is
safe because no other name-lock can be issued between the flush and the
check for pending name-locks.
The test case for this bug is going to be committed into 5.1 because it
requires a test feature only avaiable in 5.1 (wait_condition).
sql/sql_base.cc:
Improve comments in the open_table() function, stating the importance
of the handler tables flushing for the back-off process.
sql/sql_handler.cc:
Allows handler tables flushes when opening new tables in order to avoid
potential deadlocks. Add comments explaining the importance of the flush.
This deadlock occurs when a client issues a HANDLER ... OPEN statement
that tries to open a table that has a pending name-lock on it by another
client that also needs a name-lock on some other table which is already
open and associated to a HANDLER instance owned by the first client.
The deadlock happens because the open_table() function will back-off
and wait until the name-lock goes away, causing a circular wait if some
other name-lock is also pending for one of the open HANDLER tables.
Such situation, for example, can be easily repeated by issuing a RENAME
TABLE command in such a way that the existing table is already open
as a HANDLER table by another client and this client tries to open
a HANDLER to the new table name.
The solution is to allow handler tables with older versions (marked for
flush) to be closed before waiting for the name-lock completion. This is
safe because no other name-lock can be issued between the flush and the
check for pending name-locks.
The test case for this bug is going to be committed into 5.1 because it
requires a test feature only avaiable in 5.1 (wait_condition).
This bug is a symptom of the way handler's tables are managed. The
most different aspect, compared to the conventional behavior, is that
the handler's tables are long lived, meaning that their lifetimes are
not bounded by the duration of the command that opened them. For this
effect the handler code uses its own list (handler_tables instead of
open_tables) to hold open handler tables so that the tables won't be
closed at the end of the command/statement. Besides the handler_tables
list, there is a hash (handler_tables_hash) which is used to associate
handler aliases to tables and to refresh the tables upon demand (flush
tables).
The current implementation doesn't work properly with refreshed tables
-- more precisely when flush commands are issued by other initiators.
This happens because when a handler open or read statement is being
processed, the associated table has to be opened or locked and, for this
matter, the open_tables and handler_tables lists are swapped so that the
new table being opened is inserted into the handler_tables list. But when
opening or locking the table, if the refresh version is different from the
thread refresh version then all used tables in the open_tables list (now
handler_tables) are refreshed. In the "refreshing" process the handler
tables are flushed (closed) without being properly unlinked from the
handler hash.
The current implementation also fails to properly discard handlers of
dropped tables, but this and other problems are going to be addressed
in the fixes for bugs 31397 and 31409.
The chosen approach tries to properly save and restore the table state
so that no table is flushed during the table open and lock operations.
The logic is almost the same as before with the list swapping, but with
a working glue code.
The test case for this bug is going to be committed into 5.1 because it
requires a test feature only avaiable in 5.1 (wait_condition).
sql/sql_handler.cc:
Backup and reset the open_tables and handler_table lists when opening
a new handler table and merge the opened table into the handler_tables
list afterwards. When locking, do the same but possibly close the table
if a refresh is pending.
This bug is a symptom of the way handler's tables are managed. The
most different aspect, compared to the conventional behavior, is that
the handler's tables are long lived, meaning that their lifetimes are
not bounded by the duration of the command that opened them. For this
effect the handler code uses its own list (handler_tables instead of
open_tables) to hold open handler tables so that the tables won't be
closed at the end of the command/statement. Besides the handler_tables
list, there is a hash (handler_tables_hash) which is used to associate
handler aliases to tables and to refresh the tables upon demand (flush
tables).
The current implementation doesn't work properly with refreshed tables
-- more precisely when flush commands are issued by other initiators.
This happens because when a handler open or read statement is being
processed, the associated table has to be opened or locked and, for this
matter, the open_tables and handler_tables lists are swapped so that the
new table being opened is inserted into the handler_tables list. But when
opening or locking the table, if the refresh version is different from the
thread refresh version then all used tables in the open_tables list (now
handler_tables) are refreshed. In the "refreshing" process the handler
tables are flushed (closed) without being properly unlinked from the
handler hash.
The current implementation also fails to properly discard handlers of
dropped tables, but this and other problems are going to be addressed
in the fixes for bugs 31397 and 31409.
The chosen approach tries to properly save and restore the table state
so that no table is flushed during the table open and lock operations.
The logic is almost the same as before with the list swapping, but with
a working glue code.
The test case for this bug is going to be committed into 5.1 because it
requires a test feature only avaiable in 5.1 (wait_condition).
into weblab.(none):/home/marcsql/TREE/mysql-5.1-rt50-merge
client/mysql.cc:
Auto merged
mysql-test/r/query_cache.result:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_yacc.yy:
Auto merged
into moksha.local:/Users/davi/mysql/push/bugs/30632-5.1
sql/sql_handler.cc:
Auto merged
mysql-test/include/handler.inc:
Auto merged
mysql-test/r/handler_myisam.result:
Auto merged
If, after the tables are locked, one of the conditions to read from a
HANDLER table is not met, the handler code wrongly jumps to a error path
that won't unlock the tables.
The user-visible effect is that after a error in a handler read command,
all subsequent handler operations on the same table will hang.
The fix is simply to correct the code to jump to the (same) error path that
unlocks the tables.
mysql-test/r/handler.result:
Bug#30632 test case result
mysql-test/t/handler.test:
Bug#30632 test case
sql/sql_handler.cc:
Always unlock the internal and external table level locks if any of the conditions
(including errors) to read from a HANDLER table are not met.
If, after the tables are locked, one of the conditions to read from a
HANDLER table is not met, the handler code wrongly jumps to a error path
that won't unlock the tables.
The user-visible effect is that after a error in a handler read command,
all subsequent handler operations on the same table will hang.
The fix is simply to correct the code to jump to the (same) error path that
unlocks the tables.
into hynda.mysql.fi:/home/my/mysql-5.1-marvel
client/mysqldump.c:
Auto merged
sql/event_db_repository.cc:
Auto merged
sql/event_queue.cc:
Auto merged
sql/field.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/table.h:
Auto merged
sql/sql_yacc.yy:
Manual merge with 5.1 main tree.
into hynda.mysql.fi:/home/my/mysql-5.1-marvel
mysql-test/mysql-test-run.pl:
Auto merged
sql/field.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_string.cc:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/r/status.result:
Manual merge with main 5.0 tree.
mysql-test/t/status.test:
Manual merge with main 5.0 tree.
sql/log_event.cc:
Manual merge with main 5.0 tree.
sql/log_event.h:
Manual merge with main 5.0 tree.
into weblab.(none):/home/marcsql/TREE/mysql-5.1-rt-merge
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/log_event.h:
Auto merged
sql/rpl_record.cc:
Auto merged
sql/slave.h:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
mysql_ha_open calls mysql_ha_close on the error path (unsupported) to close the (opened) table before inserting it into the tables hash list handler_tables_hash) but mysql_ha_close only closes tables which are on the hash list, causing the table to be left open and locked.
This change moves the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close (mysql_ha_close).
mysql-test/r/lock_multi.result:
Bug#25856 test result
mysql-test/t/lock_multi.test:
Bug#25856 test case
sql/sql_handler.cc:
Move the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close.
mysql_ha_open calls mysql_ha_close on the error path (unsupported) to close the (opened) table before inserting it into the tables hash list handler_tables_hash) but mysql_ha_close only closes tables which are on the hash list, causing the table to be left open and locked.
This change moves the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close (mysql_ha_close).
