In RBR, DDL statement will change binlog format to non row-based
format before it is binlogged, but the binlog format was not be
restored, and then manipulating a temporary table can not reset binlog
format to row-based format rightly. So that the manipulated statement
is binlogged with statement-based format.
To fix the problem, restore the state of binlog format after the DDL
statement is binlogged.
mysql-test/extra/rpl_tests/rpl_tmp_table_and_DDL.test:
Added the test file to verify if executing DDL statement before
trying to manipulate a temporary table causes row-based replication
to break with error 'table does not exist'.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Correct the test result, all the above binlog event
should be row-based after the bug49132 is fixed IN RBR.
mysql-test/suite/ndb/r/ndb_tmp_table_and_DDL.result:
Test result for bug#49132 base on ndb engine.
mysql-test/suite/ndb/t/ndb_tmp_table_and_DDL.test:
Added the test file to verify if executing DDL statement before
trying to manipulate a temporary table causes row-based replication
to break with error 'table does not exist' base on ndb engine.
mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result:
Test result for bug#49132 base on myisam engine.
mysql-test/suite/rpl/t/rpl_tmp_table_and_DDL.test:
Added the test file to verify if executing DDL statement before
trying to manipulate a temporary table causes row-based replication
to break with error 'table does not exist' base on myisam engine.
sql/event_db_repository.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/events.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/sp.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/sql_acl.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
sql/sql_udf.cc:
Added code to restore the state of binlog format after the DDL
statement is binlogged.
In RBR, DDL statement will change binlog format to non row-based
format before it is binlogged, but the binlog format was not be
restored, and then manipulating a temporary table can not reset binlog
format to row-based format rightly. So that the manipulated statement
is binlogged with statement-based format.
To fix the problem, restore the state of binlog format after the DDL
statement is binlogged.
condition variable per context instead of one mutex and one conditional
variable for the whole subsystem.
This should increase concurrency in this subsystem.
It also opens the way for further changes which are necessary to solve
such bugs as bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock"
and bug #37346 "innodb does not detect deadlock between update and alter
table".
Two other notable changes done by this patch:
- MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object metadata lock is acquired. Now this has
to be done by explicit calls outside of MDL subsystem.
- Instead of using separate MDL_context for opening system tables/tables
for purposes of I_S we now create MDL savepoint in the main context
before opening tables and rollback to this savepoint after closing
them. This means that it is now possible to get ER_LOCK_DEADLOCK error
even not inside a transaction. This might happen in unlikely case when
one runs DDL on one of system tables while also running DDL on some
other tables. Cases when this ER_LOCK_DEADLOCK error is not justified
will be addressed by advanced deadlock detector for MDL subsystem which
we plan to implement.
mysql-test/include/handler.inc:
Adjusted handler_myisam.test and handler_innodb.test to the fact that
exclusive metadata locks on tables are now acquired according to
alphabetical order of fully qualified table names instead of order
in which tables are mentioned in statement.
mysql-test/r/handler_innodb.result:
Adjusted handler_myisam.test and handler_innodb.test to the fact that
exclusive metadata locks on tables are now acquired according to
alphabetical order of fully qualified table names instead of order
in which tables are mentioned in statement.
mysql-test/r/handler_myisam.result:
Adjusted handler_myisam.test and handler_innodb.test to the fact that
exclusive metadata locks on tables are now acquired according to
alphabetical order of fully qualified table names instead of order
in which tables are mentioned in statement.
mysql-test/r/mdl_sync.result:
Adjusted mdl_sync.test to the fact that exclusive metadata locks on
tables are now acquired according to alphabetical order of fully
qualified table names instead of order in which tables are mentioned
in statement.
mysql-test/t/mdl_sync.test:
Adjusted mdl_sync.test to the fact that exclusive metadata locks on
tables are now acquired according to alphabetical order of fully
qualified table names instead of order in which tables are mentioned
in statement.
sql/events.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/ha_ndbcluster.cc:
Since manipulations with open table state no longer install proxy
MDL_context it does not make sense to perform them in order to
satisfy assert in mysql_rm_tables_part2(). Removed them per agreement
with Cluster team. This has not broken test suite since scenario in
which deadlock can occur and assertion fails is not covered by tests.
sql/lock.cc:
MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object exclusive metadata lock is acquired.
Now this has to be done by explicit calls outside of MDL subsystem.
sql/log.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/mdl.cc:
Changed metadata locking subsystem to use mutex per lock and condition
variable per context instead of one mutex and one conditional variable
for the whole subsystem.
Changed approach to handling of global metadata locks. Instead of
implicitly acquiring intention exclusive locks when user requests
per-object upgradeable or exclusive locks now we require them to be
acquired explicitly in the same way as ordinary metadata locks.
In fact global lock are now ordinary metadata locks in new GLOBAL
namespace.
To implement these changes:
- Removed LOCK_mdl mutex and COND_mdl condition variable.
- Introduced MDL_lock::m_mutex mutexes which protect individual lock
objects.
- Replaced mdl_locks hash with MDL_map class, which has hash for
MDL_lock objects as a member and separate mutex which protects this
hash. Methods of this class allow to find(), find_or_create() or
remove() MDL_lock objects in concurrency-friendly fashion (i.e.
for most common operation, find_or_create(), we don't acquire
MDL_lock::m_mutex while holding MDL_map::m_mutex. Thanks to MikaelR
for this idea and benchmarks!). Added three auxiliary members to
MDL_lock class (m_is_destroyed, m_ref_usage, m_ref_release) to
support this concurrency-friendly behavior.
- Introduced MDL_context::m_ctx_wakeup_cond condition variable to be
used for waiting until this context's pending request can be
satisfied or its thread has to perform actions to resolve potential
deadlock. Context which want to wait add ticket corresponding to the
request to an appropriate queue of waiters in MDL_lock object so
they can be noticed when other contexts change state of lock and be
awaken by them by signalling on MDL_context::m_ctx_wakeup_cond.
As consequence MDL_ticket objects has to be used for any waiting
in metadata locking subsystem including one which happens in
MDL_context::wait_for_locks() method.
Another consequence is that MDL_context is no longer copyable and
can't be saved/restored when working with system tables.
- Made MDL_lock an abstract class, which delegates specifying exact
compatibility matrix to its descendants. Added MDL_global_lock child
class for global lock (The old is_lock_type_compatible() method
became can_grant_lock() method of this class). Added MDL_object_lock
class to represent per-object lock (The old MDL_lock::can_grant_lock()
became its method). Choice between two classes happens based on MDL
namespace in MDL_lock::create() method.
- Got rid of MDL_lock::type member as its meaning became ambigous for
global locks.
- To simplify waking up of contexts waiting for lock split waiting queue
in MDL_lock class in two queues. One for pending requests for exclusive
(including intention exclusive) locks and another for requests for
shared locks.
- Added virtual wake_up_waiters() method to MDL_lock, MDL_global_lock and
MDL_object_lock classes which allows to wake up waiting contexts after
state of lock changes. Replaced old duplicated code with calls to this
method.
- Adjusted MDL_context::try_acquire_shared_lock()/exclusive_lock()/
global_shared_lock(), MDL_ticket::upgrade_shared_lock_to_exclusive_lock()
and MDL_context::release_ticket() methods to use MDL_map and
MDL_lock::m_mutex instead of single LOCK_mdl mutex and wake up
waiters according to the approach described above. The latter method
also was renamed to MDL_context::release_lock().
- Changed MDL_context::try_acquire_shared_lock()/exclusive_lock() and
release_lock() not to handle global locks. They are now supposed to
be taken explicitly like ordinary metadata locks.
- Added helper MDL_context::try_acquire_global_intention_exclusive_lock()
and acquire_global_intention_exclusive_lock() methods.
- Moved common code from MDL_context::acquire_global_shared_lock() and
acquire_global_intention_exclusive_lock() to new method -
MDL_context::acquire_lock_impl().
- Moved common code from MDL_context::try_acquire_shared_lock(),
try_acquire_global_intention_exclusive_lock()/exclusive_lock()
to MDL_context::try_acquire_lock_impl().
- Since acquiring of several exclusive locks can no longer happen under
single LOCK_mdl mutex the approach to it had to be changed. Now we do
it in one by one fashion. This is done in alphabetical order to avoid
deadlocks. Changed MDL_context::acquire_exclusive_locks() accordingly
(as part of this change moved code responsible for acquiring single
exclusive lock to new MDL_context::acquire_exclusive_lock_impl()
method).
- Since we no longer have single LOCK_mdl mutex which protects all
MDL_context::m_is_waiting_in_mdl members using these members to
determine if we have really awaken context holding conflicting
shared lock became inconvinient. Got rid of this member and changed
notify_shared_lock() helper function and process of acquiring
of/upgrading to exclusive lock not to rely on such information.
Now in MDL_context::acquire_exclusive_lock_impl() and
MDL_ticket::upgrade_shared_lock_to_exclusive_lock() we simply
re-try to wake up threads holding conflicting shared locks after
small time out.
- Adjusted MDL_context::can_wait_lead_to_deadlock() and
MDL_ticket::has_pending_conflicting_lock() to use per-lock
mutexes instead of LOCK_mdl. To do this introduced
MDL_lock::has_pending_exclusive_lock() method.
sql/mdl.h:
Changed metadata locking subsystem to use mutex per lock and condition
variable per context instead of one mutex and one conditional variable
for the whole subsystem. In order to implement this change:
- Added MDL_key::cmp() method to be able to sort MDL_key objects
alphabetically. Changed length fields in MDL_key class to uint16
as 16-bit is enough for length of any key.
- Changed MDL_ticket::get_ctx() to return pointer to non-const
object in order to be able to use MDL_context::awake() method
for such contexts.
- Got rid of unlocked versions of can_wait_lead_to_deadlock()/
has_pending_conflicting_lock() methods in MDL_context and
MDL_ticket. We no longer has single mutex which protects all
locks. Thus one always has to use versions of these methods
which acquire per-lock mutexes.
- MDL_request_list type of list now counts its elements.
- Added MDL_context::m_ctx_wakeup_cond condition variable to be used
for waiting until this context's pending request can be satisfied
or its thread has to perform actions to resolve potential deadlock.
Added awake() method to wake up context from such wait.
Addition of condition variable made MDL_context uncopyable.
As result we no longer can save/restore MDL_context when working
with system tables. Instead we create MDL savepoint before opening
those tables and rollback to it once they are closed.
- MDL_context::release_ticket() became release_lock() method.
- Added auxiliary MDL_context::acquire_exclusive_lock_impl() method
which does all necessary work to acquire exclusive lock on one object
but should not be used directly as it does not enforce any asserts
ensuring that no deadlocks are possible.
- Since we no longer need to know if thread trying to acquire exclusive
lock managed to wake up any threads having conflicting shared locks
(as, anyway, we will try to wake up such threads again shortly)
- MDL_context::m_is_waiting_in_mdl member became unnecessary and
notify_shared_lock() no longer needs to be friend of MDL_context.
Changed approach to handling of global metadata locks. Instead of
implicitly acquiring intention exclusive locks when user requests
per-object upgradeable or exclusive locks now we require them to be
acquired explicitly in the same way as ordinary metadata locks.
- Added new GLOBAL namespace for such locks.
- Added new type of lock to be requested MDL_INTENTION_EXCLISIVE.
- Added MDL_context::try_acquire_global_intention_exclusive_lock()
and acquire_global_intention_exclusive_lock() methods.
- Moved common code from MDL_context::acquire_global_shared_lock()
and acquire_global_intention_exclusive_lock() to new method -
MDL_context::acquire_lock_impl().
- Moved common code from MDL_context::try_acquire_shared_lock(),
try_acquire_global_intention_exclusive_lock()/exclusive_lock()
to MDL_context::try_acquire_lock_impl().
- Added helper MDL_context::is_global_lock_owner() method to be
able easily to find what kind of global lock this context holds.
- MDL_context::m_has_global_shared_lock became unnecessary as
global read lock is now represented by ordinary ticket.
- Removed assert in MDL_context::set_lt_or_ha_sentinel() which became
false for cases when we execute LOCK TABLES under global read lock
mode.
sql/mysql_priv.h:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result calls
opening/closing system tables were changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sp.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sp.h:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sql_base.cc:
close_thread_tables():
Since we no longer use separate MDL_context for opening system
tables we need to avoid releasing all transaction locks when
closing system table. Releasing metadata lock on system table
is now responsibility of THD::restore_backup_open_tables_state().
open_table_get_mdl_lock(),
Open_table_context::recover_from_failed_open():
MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object upgradable or exclusive metadata lock is
acquired. So this have to be done explicitly from these calls.
Changed Open_table_context class to store MDL_request object for
global intention exclusive lock acquired when opening tables.
open_table():
Do not release metadata lock if we have failed to open table as
this lock might have been acquired by one of previous statements
in transaction, and therefore should not be released.
open_system_tables_for_read()/close_system_tables()/
open_performance_schema_table():
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
close_performance_schema_table():
Got rid of duplicated code.
sql/sql_class.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. Also releasing
metadata lock on system table is now responsibility of
THD::restore_backup_open_tables_state().
Adjusted assert in THD::cleanup() to take into account fact that now
we also use MDL sentinel for global read lock.
sql/sql_class.h:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. As result:
- 'mdl_context' member was moved out of Open_tables_state to THD class.
enter_locked_tables_mode()/leave_locked_tables_mode() had to follow.
- Methods of THD responsible for saving/restoring open table state were
changed to use Open_tables_backup class which in addition to
Open_table_state has a member for this savepoint.
Changed Open_table_context class to store MDL_request object for
global intention exclusive lock acquired when opening tables.
sql/sql_delete.cc:
MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object exclusive metadata lock is acquired.
Now this has to be done by explicit calls outside of MDL subsystem.
sql/sql_help.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sql_parse.cc:
Adjusted assert reload_acl_and_cache() to the fact that global read
lock now takes full-blown metadata lock.
sql/sql_plist.h:
Added support for element counting to I_P_List list template.
One can use policy classes to specify if such counting is needed
or not needed for particular list.
sql/sql_show.cc:
Instead of using separate MDL_context for opening tables for I_S
purposes we now create MDL savepoint in the main context before
opening tables and rollback to this savepoint after closing them.
To support this and similar change for system tables methods of
THD responsible for saving/restoring open table state were changed
to use Open_tables_backup class which in addition to Open_table_state
has a member for this savepoint. As result code opening/closing tables
for I_S purposes was changed to use Open_tables_backup instead of
Open_table_state class as well.
sql/sql_table.cc:
mysql_rm_tables_part2():
Since now global intention exclusive metadata lock is ordinary
metadata lock we no longer can rely that by releasing MDL locks
on all tables we will release all locks acquired by this routine.
So in non-LOCK-TABLES mode we have to release all locks acquired
explicitly.
prepare_for_repair(), mysql_alter_table():
MDL subsystem no longer implicitly acquires global intention
exclusive metadata lock when per-object exclusive metadata lock
is acquired. Now this has to be done by explicit calls outside of
MDL subsystem.
sql/tztime.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
Also changed code not to use special mechanism for open system tables
when it is not really necessary.
condition variable per context instead of one mutex and one conditional
variable for the whole subsystem.
This should increase concurrency in this subsystem.
It also opens the way for further changes which are necessary to solve
such bugs as bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock"
and bug #37346 "innodb does not detect deadlock between update and alter
table".
Two other notable changes done by this patch:
- MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object metadata lock is acquired. Now this has
to be done by explicit calls outside of MDL subsystem.
- Instead of using separate MDL_context for opening system tables/tables
for purposes of I_S we now create MDL savepoint in the main context
before opening tables and rollback to this savepoint after closing
them. This means that it is now possible to get ER_LOCK_DEADLOCK error
even not inside a transaction. This might happen in unlikely case when
one runs DDL on one of system tables while also running DDL on some
other tables. Cases when this ER_LOCK_DEADLOCK error is not justified
will be addressed by advanced deadlock detector for MDL subsystem which
we plan to implement.
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase. Created directory.
Conflict because storage/innobase is not versioned, but has versioned children. Versioned directory.
Conflict adding file storage/innobase. Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler. Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children. Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase. Created directory.
Conflict because storage/innobase is not versioned, but has versioned children. Versioned directory.
Conflict adding file storage/innobase. Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler. Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children. Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
3655 Jon Olav Hauglid 2009-10-19
Bug #30977 Concurrent statement using stored function and DROP FUNCTION
breaks SBR
Bug #48246 assert in close_thread_table
Implement a fix for:
Bug #41804 purge stored procedure cache causes mysterious hang for many
minutes
Bug #49972 Crash in prepared statements
The problem was that concurrent execution of DML statements that
use stored functions and DDL statements that drop/modify the same
function might result in incorrect binary log in statement (and
mixed) mode and therefore break replication.
This patch fixes the problem by introducing metadata locking for
stored procedures and functions. This is similar to what is done
in Bug#25144 for views. Procedures and functions now are
locked using metadata locks until the transaction is either
committed or rolled back. This prevents other statements from
modifying the procedure/function while it is being executed. This
provides commit ordering - guaranteeing serializability across
multiple transactions and thus fixes the reported binlog problem.
Note that we do not take locks for top-level CALLs. This means
that procedures called directly are not protected from changes by
simultaneous DDL operations so they are executed at the state they
had at the time of the CALL. By not taking locks for top-level
CALLs, we still allow transactions to be started inside
procedures.
This patch also changes stored procedure cache invalidation.
Upon a change of cache version, we no longer invalidate the entire
cache, but only those routines which we use, only when a statement
is executed that uses them.
This patch also changes the logic of prepared statement validation.
A stored procedure used by a prepared statement is now validated
only once a metadata lock has been acquired. A version mismatch
causes a flush of the obsolete routine from the cache and
statement reprepare.
Incompatible changes:
1) ER_LOCK_DEADLOCK is reported for a transaction trying to access
a procedure/function that is locked by a DDL operation in
another connection.
2) Procedure/function DDL operations are now prohibited in LOCK
TABLES mode as exclusive locks must be taken all at once and
LOCK TABLES provides no way to specifiy procedures/functions to
be locked.
Test cases have been added to sp-lock.test and rpl_sp.test.
Work on this bug has very much been a team effort and this patch
includes and is based on contributions from Davi Arnaut, Dmitry
Lenev, Magne Mæhre and Konstantin Osipov.
mysql-test/r/ps_ddl.result:
Update results (Bug#30977).
mysql-test/r/ps_ddl1.result:
Update results (Bug#30977).
mysql-test/r/sp-error.result:
Update results (Bug#30977).
mysql-test/r/sp-lock.result:
Update results (Bug#30977).
mysql-test/suite/rpl/r/rpl_sp.result:
Update results (Bug#30977).
mysql-test/suite/rpl/t/rpl_sp.test:
Add a test case for Bug#30977.
mysql-test/t/ps_ddl.test:
Update comments. We no longer re-prepare a prepared statement
when a stored procedure used in top-level CALL is changed.
mysql-test/t/ps_ddl1.test:
Modifying stored procedure p1 no longer invalidates prepared
statement "call p1" -- we can re-use the prepared statement
without invalidation.
mysql-test/t/sp-error.test:
Use a constant for an error value.
mysql-test/t/sp-lock.test:
Add test coverage for Bug#30977.
sql/lock.cc:
Implement lock_routine_name() - a way to acquire an
exclusive metadata lock (ex- name-lock) on
stored procedure/function.
sql/sp.cc:
Change semantics of sp_cache_routine() -- now it has an option
to make sure that the routine that is cached is up to date (has
the latest sp cache version).
Add sp_cache_invalidate() to sp_drop_routine(), where it was
missing (a bug!).
Acquire metadata locks for SP DDL (ALTER/CREATE/DROP). This is
the core of the fix for Bug#30977.
Since caching and cache invalidation scheme was changed, make
sure we don't invalidate the SP cache in the middle of a stored
routine execution. At the same time, make sure we don't access
stale data due to lack of invalidation.
For that, change ALTER FUNCTION/PROCEDURE to not use the cache,
and SHOW PROCEDURE CODE/SHOW CREATE PROCEDURE/FUNCTION to always
read an up to date version of the routine from the cache.
sql/sp.h:
Add a helper wrapper around sp_cache_routine().
sql/sp_cache.cc:
Implement new sp_cache_version() and sp_cache_flush_obsolete().
Now we flush stale routines individually, rather than all at once.
sql/sp_cache.h:
Update signatures of sp_cache_version() and sp_cache_flush_obsolete().
sql/sp_head.cc:
Add a default initialization of sp_head::m_sp_cache_version.
Remove a redundant sp_head::create().
sql/sp_head.h:
Add m_sp_cache_version to sp_head class - we now
keep track of every routine in the stored procedure cache, rather than
of the entire cache.
sql/sql_base.cc:
Implement prelocking for stored routines. Validate stored
routines after they were locked.
Flush obsolete routines upon next access, one by one, not all at once
(Bug#41804).
Style fixes.
sql/sql_class.h:
Rename a Open_table_context method.
sql/sql_parse.cc:
Make sure stored procedures DDL commits the active transaction
(issues an implicit commit before and after).
Remove sp_head::create(), a pure redundancy.
Move the semantical check during alter routine inside sp_update_routine() code in order to:
- avoid using SP cache during update, it may be obsolete.
- speed up and simplify the update procedure.
Remove sp_cache_flush_obsolete() calls, we no longer flush the entire
cache, ever, stale routines are flushed before next use, one at a time.
sql/sql_prepare.cc:
Move routine metadata validation to open_and_process_routine().
Fix Bug#49972 (don't swap flags at reprepare).
Reset Sroutine_hash_entries in reinit_stmt_before_use().
Remove SP cache invalidation, it's now done by open_tables().
sql/sql_show.cc:
Fix a warning: remove an unused label.
sql/sql_table.cc:
Reset mdl_request.ticket for tickets acquired for routines inlined
through a view, in CHECK TABLE statement, to satisfy an MDL assert.
sql/sql_update.cc:
Move the cleanup of "translation items" to close_tables_for_reopen(),
since it's needed in all cases when we back off, not just
the back-off in multi-update. This fixes a bug when the server
would crash on attempt to back off when opening tables
for a statement that uses information_schema tables.
3655 Jon Olav Hauglid 2009-10-19
Bug #30977 Concurrent statement using stored function and DROP FUNCTION
breaks SBR
Bug #48246 assert in close_thread_table
Implement a fix for:
Bug #41804 purge stored procedure cache causes mysterious hang for many
minutes
Bug #49972 Crash in prepared statements
The problem was that concurrent execution of DML statements that
use stored functions and DDL statements that drop/modify the same
function might result in incorrect binary log in statement (and
mixed) mode and therefore break replication.
This patch fixes the problem by introducing metadata locking for
stored procedures and functions. This is similar to what is done
in Bug#25144 for views. Procedures and functions now are
locked using metadata locks until the transaction is either
committed or rolled back. This prevents other statements from
modifying the procedure/function while it is being executed. This
provides commit ordering - guaranteeing serializability across
multiple transactions and thus fixes the reported binlog problem.
Note that we do not take locks for top-level CALLs. This means
that procedures called directly are not protected from changes by
simultaneous DDL operations so they are executed at the state they
had at the time of the CALL. By not taking locks for top-level
CALLs, we still allow transactions to be started inside
procedures.
This patch also changes stored procedure cache invalidation.
Upon a change of cache version, we no longer invalidate the entire
cache, but only those routines which we use, only when a statement
is executed that uses them.
This patch also changes the logic of prepared statement validation.
A stored procedure used by a prepared statement is now validated
only once a metadata lock has been acquired. A version mismatch
causes a flush of the obsolete routine from the cache and
statement reprepare.
Incompatible changes:
1) ER_LOCK_DEADLOCK is reported for a transaction trying to access
a procedure/function that is locked by a DDL operation in
another connection.
2) Procedure/function DDL operations are now prohibited in LOCK
TABLES mode as exclusive locks must be taken all at once and
LOCK TABLES provides no way to specifiy procedures/functions to
be locked.
Test cases have been added to sp-lock.test and rpl_sp.test.
Work on this bug has very much been a team effort and this patch
includes and is based on contributions from Davi Arnaut, Dmitry
Lenev, Magne Mæhre and Konstantin Osipov.
hide_view_error() does not take into account that thread query may be killed.
Added a check for thd->killed.
Addon: backported bug32140 fix from 6.0
mysql-test/r/sp_notembedded.result:
test case
mysql-test/t/sp_notembedded.test:
test case
sql/sp.cc:
backported bug32140 fix from 6.0
sql/table.cc:
Added a check for thd->killed.
------------------------------------------------------------
revno: 2617.68.25
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-next-bg-pre2-2
timestamp: Wed 2009-09-16 18:26:50 +0400
message:
Follow-up for one of pre-requisite patches for fixing bug #30977
"Concurrent statement using stored function and DROP FUNCTION
breaks SBR".
Made enum_mdl_namespace enum part of MDL_key class and removed MDL_
prefix from the names of enum members. In order to do the latter
changed name of PROCEDURE symbol to PROCEDURE_SYM (otherwise macro
which was automatically generated for this symbol conflicted with
MDL_key::PROCEDURE enum member).
------------------------------------------------------------
revno: 2617.68.25
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-next-bg-pre2-2
timestamp: Wed 2009-09-16 18:26:50 +0400
message:
Follow-up for one of pre-requisite patches for fixing bug #30977
"Concurrent statement using stored function and DROP FUNCTION
breaks SBR".
Made enum_mdl_namespace enum part of MDL_key class and removed MDL_
prefix from the names of enum members. In order to do the latter
changed name of PROCEDURE symbol to PROCEDURE_SYM (otherwise macro
which was automatically generated for this symbol conflicted with
MDL_key::PROCEDURE enum member).
------------------------------------------------------------
revno: 2617.68.24
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-next-bg-pre2-2
timestamp: Wed 2009-09-16 17:25:29 +0400
message:
Pre-requisite patch for fixing bug #30977 "Concurrent statement
using stored function and DROP FUNCTION breaks SBR".
Added MDL_request for stored routine as member to Sroutine_hash_entry
in order to be able perform metadata locking for stored routines in
future (Sroutine_hash_entry is an equivalent of TABLE_LIST class for
stored routines).
(WL#4284, follow up fixes).
sql/mdl.cc:
Introduced version of MDL_request::init() method which initializes
lock request using pre-built MDL key.
MDL_key::table_name/table_name_length() getters were
renamed to reflect the fact that MDL_key objects are
now created not only for tables.
sql/mdl.h:
Extended enum_mdl_namespace enum with values which correspond
to namespaces for stored functions and triggers.
Renamed MDL_key::table_name/table_name_length() getters
to MDL_key::name() and name_length() correspondingly to
reflect the fact that MDL_key objects are now created
not only for tables.
Added MDL_key::mdl_namespace() getter.
Also added version of MDL_request::init() method which
initializes lock request using pre-built MDL key.
sql/sp.cc:
Added MDL_request for stored routine as member to Sroutine_hash_entry.
Changed code to use MDL_key from this request as a key for LEX::sroutines
set. Removed separate "key" member from Sroutine_hash_entry as it became
unnecessary.
sql/sp.h:
Added MDL_request for stored routine as member to Sroutine_hash_entry
in order to be able perform metadata locking for stored routines in
future (Sroutine_hash_entry is an equivalent of TABLE_LIST class for
stored routines).
Removed Sroutine_hash_entry::key member as now we can use MDL_key from
this request as a key for LEX::sroutines set.
sql/sp_head.cc:
Removed sp_name::m_sroutines_key member and set_routine_type() method.
Since key for routine in LEX::sroutines set has no longer sp_name::m_qname
as suffix we won't save anything by creating it at sp_name construction
time.
Adjusted sp_name constructor used for creating temporary objects for
lookups in SP-cache to accept MDL_key as parameter and to avoid any
memory allocation.
Finally, removed sp_head::m_soutines_key member for reasons similar
to why sp_name::m_sroutines_key was removed
sql/sp_head.h:
Removed sp_name::m_sroutines_key member and set_routine_type() method.
Since key for routine in LEX::sroutines set has no longer sp_name::m_qname
as suffix we won't save anything by creating it at sp_name construction
time.
Adjusted sp_name constructor used for creating temporary objects for
lookups in SP-cache to accept MDL_key as parameter and to avoid any
memory allocation.
Finally, removed sp_head::m_soutines_key member for reasons similar
to why sp_name::m_sroutines_key was removed.
sql/sql_base.cc:
Adjusted code to the fact that we now use MDL_key from
Sroutine_hash_entry::mdl_request as a key for LEX::sroutines set.
MDL_key::table_name/table_name_length() getters were
renamed to reflect the fact that MDL_key objects are
now created not only for tables.
sql/sql_trigger.cc:
sp_add_used_routine() now takes MDL_key as parameter as now we use
instance of this class as a key for LEX::sroutines set.
------------------------------------------------------------
revno: 2617.68.24
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-next-bg-pre2-2
timestamp: Wed 2009-09-16 17:25:29 +0400
message:
Pre-requisite patch for fixing bug #30977 "Concurrent statement
using stored function and DROP FUNCTION breaks SBR".
Added MDL_request for stored routine as member to Sroutine_hash_entry
in order to be able perform metadata locking for stored routines in
future (Sroutine_hash_entry is an equivalent of TABLE_LIST class for
stored routines).
(WL#4284, follow up fixes).
----------------------------------------------------------
revno: 2617.69.24
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-42546
timestamp: Fri 2009-08-14 19:22:05 +0400
message:
A pre-requisite for a fix for Bug#42546 "Backup: RESTORE fails, thinking it
finds an existing table"
Back-port from WL 148 "Foreign keys" feature tree a patch
that introduced Prelocking_strategy class -- a way to parameterize
open_tables() behaviour, implemented by Dmitry Lenev.
(Part of WL#4284).
sql/sql_base.cc:
Implement different prelocking strategies. Use an instance of
prelocking_strategy in open_tables().
sql/sql_class.h:
Add declarations for class Prelocking_strategy.
sql/sql_lex.h:
Add a helper method to access last table of the global table list
(lex->query_tables).
sql/sql_parse.cc:
Use a special prelocking strategy when locking tables for LOCK TABLES.
sql/sql_table.cc:
Use normal open_and_lock_tables_derived() in ALTER TABLE.
sql/sql_yacc.yy:
Modify the grammar to not pollute the global table list with tables
that should not be opened.
----------------------------------------------------------
revno: 2617.69.24
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-42546
timestamp: Fri 2009-08-14 19:22:05 +0400
message:
A pre-requisite for a fix for Bug#42546 "Backup: RESTORE fails, thinking it
finds an existing table"
Back-port from WL 148 "Foreign keys" feature tree a patch
that introduced Prelocking_strategy class -- a way to parameterize
open_tables() behaviour, implemented by Dmitry Lenev.
(Part of WL#4284).
----------------------------------------------------------
revno: 2617.69.20
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-4284-1-assert
timestamp: Thu 2009-08-13 18:29:55 +0400
message:
WL#4284 "Transactional DDL locking"
A review fix.
Since WL#4284 implementation separated MDL_request and MDL_ticket,
MDL_request becamse a utility object necessary only to get a ticket.
Store it by-value in TABLE_LIST with the intent to merge
MDL_request::key with table_list->table_name and table_list->db
in future.
Change the MDL subsystem to not require MDL_requests to
stay around till close_thread_tables().
Remove the list of requests from the MDL context.
Requests for shared metadata locks acquired in open_tables()
are only used as a list in recover_from_failed_open_table_attempt(),
which calls mdl_context.wait_for_locks() for this list.
To keep such list for recover_from_failed_open_table_attempt(),
introduce a context class (Open_table_context), that collects
all requests.
A lot of minor cleanups and simplications that became possible
with this change.
sql/event_db_repository.cc:
Remove alloc_mdl_requests(). Now MDL_request instance is a member
of TABLE_LIST, and init_one_table() initializes it.
sql/ha_ndbcluster_binlog.cc:
Remove now unnecessary declaration and initialization
of binlog_mdl_request.
sql/lock.cc:
No need to allocate MDL requests in lock_table_names() now.
sql/log.cc:
Use init_one_table() method, remove alloc_mdl_requests(),
which is now unnecessary.
sql/log_event.cc:
No need to allocate mdl_request separately now.
Use init_one_table() method.
sql/log_event_old.cc:
Update to the new signature of close_tables_for_reopen().
sql/mdl.cc:
Update try_acquire_exclusive_lock() to be more easy to use.
Function lock_table_name_if_not_cached() has been removed.
Make acquire_shared_lock() signature consistent with
try_acquire_exclusive_lock() signature.
Remove methods that are no longer used.
Update comments.
sql/mdl.h:
Implement an assignment operator that doesn't
copy MDL_key (MDL_key::operator= is private and
should remain private).
This is a hack to work-around assignment of TABLE_LIST
by value in several places. Such assignments violate
encapsulation, since only perform a shallow copy.
In most cases these assignments are a hack on their own.
sql/mysql_priv.h:
Update signatures of close_thread_tables() and close_tables_for_reopen().
sql/sp.cc:
Allocate TABLE_LIST in thd->mem_root.
Use init_one_table().
sql/sp_head.cc:
Use init_one_table().
Remove thd->locked_tables_root, it's no longer needed.
sql/sql_acl.cc:
Use init_mdl_requests() and init_one_table().
sql/sql_base.cc:
Update to new signatures of try_acquire_shared_lock() and
try_acquire_exclusive_lock().
Remove lock_table_name_if_not_cached().
Fix a bug in open_ltable() that would not return ER_LOCK_DEADLOCK
in case of a failed lock_tables() and a multi-statement
transaction.
Fix a bug in open_and_lock_tables_derived() that would
not return ER_LOCK_DEADLOCK in case of a multi-statement
transaction and a failure of lock_tables().
Move assignment of enum_open_table_action to a method of Open_table_context, a new class that maintains information
for backoff actions.
Minor rearrangements of the code.
Remove alloc_mdl_requests() in functions that work with system
tables: instead the patch ensures that callers always initialize
TABLE_LIST argument.
sql/sql_class.cc:
THD::locked_tables_root is no more.
sql/sql_class.h:
THD::locked_tables_root is no more.
Add a declaration for Open_table_context class.
sql/sql_delete.cc:
Update to use the simplified MDL API.
sql/sql_handler.cc:
TABLE_LIST::mdl_request is stored by-value now.
Ensure that mdl_request.ticket is NULL for every request
that is passed into MDL, to satisfy MDL asserts.
@ sql/sql_help.cc
Function open_system_tables_for_read() no longer initializes
mdl_requests.
Move TABLE_LIST::mdl_request initialization closer to
TABLE_LIST initialization.
sql/sql_help.cc:
Function open_system_tables_for_read() no longer initializes
mdl_requests.
Move TABLE_LIST::mdl_request initialization closer to
TABLE_LIST initialization.
sql/sql_insert.cc:
Remove assignment by-value of TABLE_LIST in
TABLEOP_HOOKS. We can't carry over a granted
MDL ticket from one table list to another.
sql/sql_parse.cc:
Change alloc_mdl_requests() -> init_mdl_requests().
@todo We can remove init_mdl_requests() altogether
in some places: all places that call add_table_to_list()
already have mdl requests initialized.
sql/sql_plugin.cc:
Use init_one_table().
THD::locked_tables_root is no more.
sql/sql_servers.cc:
Use init_one_table().
sql/sql_show.cc:
Update acquire_high_priority_shared_lock() to use
TABLE_LIST::mdl_request rather than allocate an own.
Fix get_trigger_table_impl() to use init_one_table(),
check for out of memory, follow the coding style.
sql/sql_table.cc:
Update to work with TABLE_LIST::mdl_request by-value.
Remove lock_table_name_if_not_cached().
The code that used to delegate to it is quite simple and
concise without it now.
sql/sql_udf.cc:
Use init_one_table().
sql/sql_update.cc:
Update to use the new signature of close_tables_for_reopen().
sql/table.cc:
Move re-setting of mdl_requests for prepared statements
and stored procedures from close_thread_tables() to
reinit_stmt_before_use().
Change alloc_mdl_requests() to init_mdl_requests().
init_mdl_requests() is a hack that can't be deleted
until we don't have a list-aware TABLE_LIST constructor.
Hopefully its use will be minimal
sql/table.h:
Change alloc_mdl_requests() to init_mdl_requests()
TABLE_LIST::mdl_request is stored by value.
sql/tztime.cc:
We no longer initialize mdl requests in open_system_tables_for*()
functions. Move this initialization closer to initialization
of the rest of TABLE_LIST members.
storage/myisammrg/ha_myisammrg.cc:
Simplify mdl_request initialization.
----------------------------------------------------------
revno: 2617.69.20
committer: Konstantin Osipov <kostja@sun.com>
branch nick: 5.4-4284-1-assert
timestamp: Thu 2009-08-13 18:29:55 +0400
message:
WL#4284 "Transactional DDL locking"
A review fix.
Since WL#4284 implementation separated MDL_request and MDL_ticket,
MDL_request becamse a utility object necessary only to get a ticket.
Store it by-value in TABLE_LIST with the intent to merge
MDL_request::key with table_list->table_name and table_list->db
in future.
Change the MDL subsystem to not require MDL_requests to
stay around till close_thread_tables().
Remove the list of requests from the MDL context.
Requests for shared metadata locks acquired in open_tables()
are only used as a list in recover_from_failed_open_table_attempt(),
which calls mdl_context.wait_for_locks() for this list.
To keep such list for recover_from_failed_open_table_attempt(),
introduce a context class (Open_table_context), that collects
all requests.
A lot of minor cleanups and simplications that became possible
with this change.
----------------------------------------------------------
revno: 2630.2.7
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-runtime
timestamp: Wed 2008-06-04 15:18:52 +0400
message:
Fix a code regression (not observable externally) that I introduced
in the fix for Bug#26141
(backporting as part of all patches related to WL#3726)
----------------------------------------------------------
revno: 2630.2.7
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-runtime
timestamp: Wed 2008-06-04 15:18:52 +0400
message:
Fix a code regression (not observable externally) that I introduced
in the fix for Bug#26141
(backporting as part of all patches related to WL#3726)
Problem: SHOW CREATE FUNCTION and SELECT DTD_IDENTIFIER FROM I_S.ROUTINES
returned wrong values in case of ENUM return data type and UCS2
character set.
Fix: the string to collect returned data type was incorrectly set to
"binary" character set, therefore UCS2 values where returned with
extra '\0' characters.
Setting string character set to creation_ctx->get_client_cs()
in sp_find_routine(), and to system_charset_info in sp_create_routine
fixes the problem.
Adding tests:
- the original test with Latin letters
- an extra test with non-Latin letters
Problem: SHOW CREATE FUNCTION and SELECT DTD_IDENTIFIER FROM I_S.ROUTINES
returned wrong values in case of ENUM return data type and UCS2
character set.
Fix: the string to collect returned data type was incorrectly set to
"binary" character set, therefore UCS2 values where returned with
extra '\0' characters.
Setting string character set to creation_ctx->get_client_cs()
in sp_find_routine(), and to system_charset_info in sp_create_routine
fixes the problem.
Adding tests:
- the original test with Latin letters
- an extra test with non-Latin letters
The problem is that the server could crash when attempting
to access a non-conformant proc system table. One such case
was a crash when invoking stored procedure related statements
on a 5.1 server with a proc system table in the 5.0 format.
The solution is to validate the proc system table format
before attempts to access it are made. If the table is not
in the format that the server expects, a message is written
to the error log and the statement that caused the table to
be accessed fails.
mysql-test/r/sp-destruct.result:
Add test case result for Bug#41726
mysql-test/t/sp-destruct.test:
Add test case for Bug#41726
sql/event_db_repository.cc:
Update code to use new structures.
sql/sp.cc:
Describe the proc table format and use it to validate when
opening a instance of the table.
Add a check to insure that a error message is written to
the error log only once.
sql/sql_acl.cc:
Remove unused variable and use new structure.
sql/sql_acl.h:
Export field definition.
sql/table.cc:
Accept the field count and definition in a single structure.
sql/table.h:
Combine the field count and definition in a single structure.
Transform function into a class in order to support different
ways of reporting a error.
Add a pointer cache to TABLE_SHARE.
The problem is that the server could crash when attempting
to access a non-conformant proc system table. One such case
was a crash when invoking stored procedure related statements
on a 5.1 server with a proc system table in the 5.0 format.
The solution is to validate the proc system table format
before attempts to access it are made. If the table is not
in the format that the server expects, a message is written
to the error log and the statement that caused the table to
be accessed fails.
This is the non-ndb part of the patch.
The return value of mysql_bin_log.write was ignored by most callers,
which may lead to inconsistent on master and slave if the transaction
was committed while the binlog was not correctly written. If
my_error() is call in mysql_bin_log.write, this could also lead to
assertion issue if my_ok() or my_error() is called after.
This fixed the problem by let the caller to check and handle the
return value of mysql_bin_log.write. This patch only adresses the
simple cases.
mysql-test/include/binlog_inject_error.inc:
inject binlog write error when doing a query
mysql-test/suite/binlog/t/binlog_write_error.test:
Simple test case to check if proper error is reported when injecting binlog write errors.
sql/events.cc:
check return value of mysql_bin_log.write
sql/log.cc:
check return value of mysql_bin_log.write
sql/log_event.cc:
check return value of mysql_bin_log.write
sql/log_event_old.cc:
check return value of mysql_bin_log.write
sql/mysql_priv.h:
Change write_bin_log to return int instead of void
sql/rpl_injector.cc:
check return value of writing binlog
sql/sp.cc:
check return value of writing binlog
sql/sp_head.cc:
return 1 if writing binlog failed
sql/sql_acl.cc:
check return value of writing binlog
sql/sql_base.cc:
check return value of writing binlog
sql/sql_class.h:
Change binlog_show_create_table to return int
sql/sql_db.cc:
Change write_to_binlog to return int
check return value of writing binlog
sql/sql_delete.cc:
check return value of writing binlog
sql/sql_insert.cc:
check return value of writing binlog
sql/sql_load.cc:
check return value of writing binlog
sql/sql_parse.cc:
check return value of writing binlog
sql/sql_partition.cc:
check return value of writing binlog
sql/sql_rename.cc:
check return value of writing binlog
sql/sql_repl.cc:
check return value of writing binlog
sql/sql_table.cc:
Change write_bin_log to return int, and return 1 if there was error writing binlog
sql/sql_tablespace.cc:
check return value of writing binlog
sql/sql_trigger.cc:
check return value of writing binlog
sql/sql_udf.cc:
check return value of writing binlog
sql/sql_update.cc:
check return value of writing binlog
sql/sql_view.cc:
check return value of writing binlog
This is the non-ndb part of the patch.
The return value of mysql_bin_log.write was ignored by most callers,
which may lead to inconsistent on master and slave if the transaction
was committed while the binlog was not correctly written. If
my_error() is call in mysql_bin_log.write, this could also lead to
assertion issue if my_ok() or my_error() is called after.
This fixed the problem by let the caller to check and handle the
return value of mysql_bin_log.write. This patch only adresses the
simple cases.
Not all my_hash_insert() calls are checked for return value.
This patch adds appropriate checks and failure responses
where needed.
mysys/hash.c:
* Debug hook for testing failures in my_hash_insert()
------------------------------------------------------------
revno: 2618
revision-id: sp1r-davi@mysql.com/endora.local-20080418131946-26951
parent: sp1r-davi@mysql.com/endora.local-20080417190810-26185
committer: davi@mysql.com/endora.local
timestamp: Fri 2008-04-18 10:19:46 -0300
message:
Bug#32140: wrong error code caught when an SF() call is interruped with KILL query
The problem is that killing a query which calls a stored function
could return a wrong error (table corrupt) instead of the query
interrupted error message.
The solution is to not set the table corrupt error if the query
is killed, the query interrupted error message will be set later
when the query is finished.
sql/sp.cc:
Don't set a error if the thread was killed, the query
interrupted error will be set later.
------------------------------------------------------------
revno: 2618
revision-id: sp1r-davi@mysql.com/endora.local-20080418131946-26951
parent: sp1r-davi@mysql.com/endora.local-20080417190810-26185
committer: davi@mysql.com/endora.local
timestamp: Fri 2008-04-18 10:19:46 -0300
message:
Bug#32140: wrong error code caught when an SF() call is interruped with KILL query
The problem is that killing a query which calls a stored function
could return a wrong error (table corrupt) instead of the query
interrupted error message.
The solution is to not set the table corrupt error if the query
is killed, the query interrupted error message will be set later
when the query is finished.
# Bug#24690 Stored functions: RETURNing UTF8 strings
# do not return UTF8_UNICODE_CI collation
#
# Bug#17903: cast to char results in binary
# Regression. The character set was not being properly initialized
# for CAST() with a type like CHAR(2) BINARY, which resulted in
# incorrect results or even a server crash.
#
Backporting from mysql-6.0-codebase.
mysql-test/r/sp-ucs2.result:
mysql-test/t/sp-ucs2.test:
Adding tests
sql/mysql_priv.h:
Adding prototype
sql/sp.cc
Remember COLLATE clause for non-default collations
sql/sql_parse.cc
Adding a new helper function
sql/sql_yacc.yy
- Allow "CHARACTER SET cs COLLATE cl" in
SP parameters, RETURNS, DECLARE
- Minor reorganization for "ASCII" and "UNICODE"
related rules, to make the code more readable,
also to allow these aliases:
* "VARCHAR(10) ASCII BINARY" -> CHARACTER SET latin1 COLLATE latin1_bin
* "VARCHAR(10) BINARY ASCII" -> CHARACTER SET latin1 COLLATE latin1_bin
* "VARCHAR(10) UNICODE BINARY" -> CHARACTER SET ucs2 COLLATE ucs2_bin
* "VARCHAR(10) BINARY UNICODE" -> CHARACTER SET ucs2 COLLATE ucs2_bin
Previously these four aliases returned the error
"This version of MySQL does not yet support return value collation".
Note:
This patch allows "VARCHAR(10) CHARACTER SET cs COLLATE cl"
and the above four aliases.
"VARCHAR(10) COLLATE cl" is still not allowed
i.e. when COLLATE is given without CHARACTER SET.
If we want to support this, we need an architecture decision
which character set to use by default.
# Bug#24690 Stored functions: RETURNing UTF8 strings
# do not return UTF8_UNICODE_CI collation
#
# Bug#17903: cast to char results in binary
# Regression. The character set was not being properly initialized
# for CAST() with a type like CHAR(2) BINARY, which resulted in
# incorrect results or even a server crash.
#
Backporting from mysql-6.0-codebase.
mysql-test/r/sp-ucs2.result:
mysql-test/t/sp-ucs2.test:
Adding tests
sql/mysql_priv.h:
Adding prototype
sql/sp.cc
Remember COLLATE clause for non-default collations
sql/sql_parse.cc
Adding a new helper function
sql/sql_yacc.yy
- Allow "CHARACTER SET cs COLLATE cl" in
SP parameters, RETURNS, DECLARE
- Minor reorganization for "ASCII" and "UNICODE"
related rules, to make the code more readable,
also to allow these aliases:
* "VARCHAR(10) ASCII BINARY" -> CHARACTER SET latin1 COLLATE latin1_bin
* "VARCHAR(10) BINARY ASCII" -> CHARACTER SET latin1 COLLATE latin1_bin
* "VARCHAR(10) UNICODE BINARY" -> CHARACTER SET ucs2 COLLATE ucs2_bin
* "VARCHAR(10) BINARY UNICODE" -> CHARACTER SET ucs2 COLLATE ucs2_bin
Previously these four aliases returned the error
"This version of MySQL does not yet support return value collation".
Note:
This patch allows "VARCHAR(10) CHARACTER SET cs COLLATE cl"
and the above four aliases.
"VARCHAR(10) COLLATE cl" is still not allowed
i.e. when COLLATE is given without CHARACTER SET.
If we want to support this, we need an architecture decision
which character set to use by default.
Non-transactional updates that take place inside a transaction present problems
for logging because they are visible to other clients before the transaction
is committed, and they are not rolled back even if the transaction is rolled
back. It is not always possible to log correctly in statement format when both
transactional and non-transactional tables are used in the same transaction.
In the current patch, we ensure that such scenario is completely safe under the
ROW and MIXED modes.
Non-transactional updates that take place inside a transaction present problems
for logging because they are visible to other clients before the transaction
is committed, and they are not rolled back even if the transaction is rolled
back. It is not always possible to log correctly in statement format when both
transactional and non-transactional tables are used in the same transaction.
In the current patch, we ensure that such scenario is completely safe under the
ROW and MIXED modes.
The original code comes, as far as I know, from Google (Mark Callaghan's team) with additional work from Percona, Ourdelta and Weldon Whipple.
This code provides the same functionallity, but with a lot of changes to make it faster and better fit the MariaDB infrastucture.
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_sent, Rows_read
Added new variable / startup option 'userstat' to control if user statistics should be enabled or not
Added my_getcputime(); Returns cpu time used by this thread.
New FLUSH commands:
- FLUSH SLOW QUERY LOG
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
New SHOW commands:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
New Information schemas:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Added support for all new flush commands to mysqladmin
Added handler::ha_... wrappers for all handler read calls to do statistics counting
- Changed all code to use new ha_... calls
- Count number of read rows, changed rows and rows read trough an index
Added counting of number of bytes sent to binary log (status variable Binlog_bytes_written)
Added counting of access denied errors (status variable Access_denied_erors)
Bugs fixed:
- Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
- CLOCK_GETTIME was not propely working on Linuxm
client/mysqladmin.cc:
Added support for all new flush commmands and some common combinations:
flush-slow-log
flush-table-statistics
flush-index-statistics
flush-user-statistics
flush-client-statistics
flush-all-status
flush-all-statistics
configure.in:
Added checking if clock_gettime needs the librt.
(Fixes Bug #37639 clock_gettime is never used/enabled in Linux/Unix)
include/my_sys.h:
Added my_getcputime()
include/mysql_com.h:
Added LIST_PROCESS_HOST_LEN & new REFRESH target defines
mysql-test/r/information_schema.result:
New information schema tables added
mysql-test/r/information_schema_all_engines.result:
New information schema tables added
mysql-test/r/information_schema_db.result:
New information schema tables added
mysql-test/r/log_slow.result:
Added testing that flosh slow query logs is accepted
mysql-test/r/status_user.result:
Basic testing of user, client, table and index statistics
mysql-test/t/log_slow.test:
Added testing that flosh slow query logs is accepted
mysql-test/t/status_user-master.opt:
Ensure that we get a fresh restart before running status_user.test
mysql-test/t/status_user.test:
Basic testing of user, client, table and index statistics
mysys/my_getsystime.c:
Added my_getcputime()
Returns cpu time used by this thread.
sql/authors.h:
Updated authors to have core and original MySQL developers first.
sql/event_data_objects.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/event_db_repository.cc:
Changed to use new ha_... calls
sql/filesort.cc:
Changed to use new ha_... calls
sql/ha_partition.cc:
Changed to use new ha_... calls
Fixed comment syntax
sql/handler.cc:
Changed to use new ha_... calls
Reset table statistics
Added code to update global table and index status
Added counting of rows changed
sql/handler.h:
Added table and index statistics variables
Added function reset_statistics()
Added handler::ha_... wrappers for all handler read calls to do statistics counting
Protected all normal read calls to ensure we use the new calls in the server.
Made ha_partition a friend class so that partition code can call the old read functions
sql/item_subselect.cc:
Changed to use new ha_... calls
sql/lex.h:
Added keywords for new information schema tables and flush commands
sql/log.cc:
Added flush_slow_log()
Added counting of number of bytes sent to binary log
Removed not needed test of thd (It's used before, so it's safe to use)
Added THD object to MYSQL_BIN_LOG::write_cache() to simplify statistics counting
sql/log.h:
Added new parameter to write_cache()
Added flush_slow_log() functions.
sql/log_event.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/log_event_old.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/mysql_priv.h:
Updated call to mysql_reset_thd_for_next_command()
Added new statistics functions and variables needed by these.
sql/mysqld.cc:
Added new statistics variables and structures to handle these
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_set, Rows_read
Added new option 'userstat' to control if user statistics should be enabled or not
sql/opt_range.cc:
Changed to use new ha_... calls
sql/opt_range.h:
Changed to use new ha_... calls
sql/opt_sum.cc:
Changed to use new ha_... calls
sql/records.cc:
Changed to use new ha_... calls
sql/set_var.cc:
Added variable 'userstat'
sql/sp.cc:
Changed to use new ha_... calls
sql/sql_acl.cc:
Changed to use new ha_... calls
Added counting of access_denied_errors
sql/sql_base.cc:
Added call to statistics functions
sql/sql_class.cc:
Added usage of org_status_var, to store status variables at start of command
Added functions THD::update_stats(), THD::update_all_stats()
Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
sql/sql_class.h:
Added new status variables to status_var
Moved variables that was not ulong in status_var last.
Added variables to THD for storing temporary values during statistics counting
sql/sql_connect.cc:
Variables and functions to calculate user and client statistics
Added counting of access_denied_errors and lost_connections
sql/sql_cursor.cc:
Changed to use new ha_... calls
sql/sql_handler.cc:
Changed to use new ha_... calls
sql/sql_help.cc:
Changed to use new ha_... calls
sql/sql_insert.cc:
Changed to use new ha_... calls
sql/sql_lex.h:
Added SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, SQLCOM_SHOW_CLIENT_STATS
sql/sql_parse.cc:
Added handling of:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
Added handling of new FLUSH commands:
- FLUSH SLOW QUERY LOGS
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
Added THD parameter to mysql_reset_thd_for_next_command()
Added initialization and calls to user statistics functions
Added increment of statistics variables empty_queries, rows_sent and access_denied_errors.
Added counting of cpu time per query
sql/sql_plugin.cc:
Changed to use new ha_... calls
sql/sql_prepare.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/sql_select.cc:
Changed to use new ha_... calls
Indentation changes
sql/sql_servers.cc:
Changed to use new ha_... calls
sql/sql_show.cc:
Added counting of access denied errors
Added function for new information schema tables:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Changed to use new ha_... calls
sql/sql_table.cc:
Changed to use new ha_... calls
sql/sql_udf.cc:
Changed to use new ha_... calls
sql/sql_update.cc:
Changed to use new ha_... calls
sql/sql_yacc.yy:
Add new show and flush commands
sql/structs.h:
Add name_length to KEY to avoid some strlen
Added cache_name to KEY for fast storage of keyvalue in cache
Added structs USER_STATS, TABLE_STATS, INDEX_STATS
Added function prototypes for statistics functions
sql/table.cc:
Store db+table+index name into keyinfo->cache_name
sql/table.h:
Added new information schema tables
sql/tztime.cc:
Changed to use new ha_... calls
Implemented the server infrastructure for the fix:
1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of
thd_query()
2. Did some encapsulation in THD : aggregated thd_query and
thd_query_length into a LEX_STRING and made accessor and mutator
methods for easy code updating.
3. Updated the server code to use the new methods where applicable.
Implemented the server infrastructure for the fix:
1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of
thd_query()
2. Did some encapsulation in THD : aggregated thd_query and
thd_query_length into a LEX_STRING and made accessor and mutator
methods for easy code updating.
3. Updated the server code to use the new methods where applicable.
If the log_bin_trust_function_creators option is not defined, creating a stored
function requires either one of the modifiers DETERMINISTIC, NO SQL, or READS
SQL DATA. Executing a stored function should also follows the same rules if in
STATEMENT mode. However, this was not happening and a wrong error was being
printed out: ER_BINLOG_ROW_RBR_TO_SBR.
The patch makes the creation and execution compatible and prints out the correct
error ER_BINLOG_UNSAFE_ROUTINE when a stored function without one of the modifiers
above is executed in STATEMENT mode.
If the log_bin_trust_function_creators option is not defined, creating a stored
function requires either one of the modifiers DETERMINISTIC, NO SQL, or READS
SQL DATA. Executing a stored function should also follows the same rules if in
STATEMENT mode. However, this was not happening and a wrong error was being
printed out: ER_BINLOG_ROW_RBR_TO_SBR.
The patch makes the creation and execution compatible and prints out the correct
error ER_BINLOG_UNSAFE_ROUTINE when a stored function without one of the modifiers
above is executed in STATEMENT mode.
General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
mysql-test/extra/rpl_tests/create_recursive_construct.inc:
Added auxiliary file used by binlog_unsafe.test to create and
execute recursive constructs
(functions/procedures/triggers/views/prepared statements).
mysql-test/extra/rpl_tests/rpl_foreign_key.test:
removed unnecessary set @@session.binlog_format
mysql-test/extra/rpl_tests/rpl_insert_delayed.test:
Filter out table id from table map events in binlog listing.
Got rid of $binlog_format_statement.
mysql-test/extra/rpl_tests/rpl_ndb_apply_status.test:
disable warnings around call to unsafe procedure
mysql-test/include/rpl_udf.inc:
Disabled warnings for code that generates warnings
for some binlog formats. That would otherwise cause
inconsistencies in the result file.
mysql-test/r/mysqldump.result:
Views are now unsafe if they contain a LIMIT clause.
That fixed BUG#45831. Due to BUG#45832, a warning is
printed for the CREATE VIEW statement.
mysql-test/r/sp_trans.result:
Unsafe statements in stored procedures did not give a warning if
binlog_format=statement. This is BUG#45824. Now they do, so this
result file gets a new warning.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Error message changed.
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
INSERT DELAYED didn't generate a warning when binlog_format=STATEMENT.
That was BUG#45825. Now there is a warning, so result file needs to be
updated.
mysql-test/suite/binlog/r/binlog_stm_ps.result:
Changed error message.
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file:
- error message changed
- added test for most combinations of unsafe constructs invoked
from recursive constructs
- INSERT DELAYED now gives a warning (because BUG#45826 is fixed)
- INSERT..SELECT..LIMIT now gives a warning from inside recursive
constructs (because BUG#45785 was fixed)
- When a recursive construct (e.g., stored proc or function)
contains more than one statement, at least one of which is
unsafe, then all statements in the recursive construct give
warnings. This is a new bug introduced by this changeset.
It will be addressed in a post-push fix.
mysql-test/suite/binlog/t/binlog_innodb.test:
Changed error code for innodb updates with READ COMMITTED or
READ UNCOMMITTED transaction isolation level and
binlog_format=statement.
mysql-test/suite/binlog/t/binlog_multi_engine.test:
The error code has changed for statements where more than one
engine is involved and one of them is self-logging.
mysql-test/suite/binlog/t/binlog_unsafe-master.opt:
Since binlog_unsafe now tests unsafe-ness of UDF's, we need an extra
flag in the .opt file.
mysql-test/suite/binlog/t/binlog_unsafe.test:
- Clarified comment.
- Rewrote first part of test. Now it tests not only unsafe variables
and functions, but also unsafe-ness due to INSERT..SELECT..LIMIT,
INSERT DELAYED, insert into two autoinc columns, use of UDF's, and
access to log tables in the mysql database.
Also, in addition to functions, procedures, triggers, and prepared
statements, it now also tests views; and it constructs recursive
calls in two levels by combining these recursive constructs.
Part of the logic is in extra/rpl_tests/create_recursive_construct.inc.
- added tests for all special system variables that should not be unsafe.
- added specific tests for BUG#45785 and BUG#45825
mysql-test/suite/rpl/r/rpl_events.result:
updated result file
mysql-test/suite/rpl/r/rpl_extraColmaster_innodb.result:
updated result file
mysql-test/suite/rpl/r/rpl_extraColmaster_myisam.result:
updated result file
mysql-test/suite/rpl/r/rpl_foreign_key_innodb.result:
updated result file
mysql-test/suite/rpl/r/rpl_idempotency.result:
updated result file
mysql-test/suite/rpl/r/rpl_mix_found_rows.result:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). This file equals
the second half of the old rpl_found_rows.result, with the following
modifications:
- minor formatting changes
- additional initialization
mysql-test/suite/rpl/r/rpl_mix_insert_delayed.result:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/r/rpl_rbr_to_sbr.result:
updated result file
mysql-test/suite/rpl/r/rpl_row_idempotency.result:
Moved the second half of rpl_idempotency.test, which only
executed in row mode, to rpl_row_idempotency.test. This is
the new result file.
mysql-test/suite/rpl/r/rpl_row_insert_delayed.result:
Got rid of unnecessary explicit setting of binlog format.
mysql-test/suite/rpl/r/rpl_stm_found_rows.result:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). Changes in
this file:
- minor formatting changes
- warning is now issued for unsafe statements inside procedures
(since BUG#45824 is fixed)
- second half of file is moved to rpl_mix_found_rows.result
mysql-test/suite/rpl/r/rpl_stm_insert_delayed.result:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
error message changed
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
updated result file
mysql-test/suite/rpl/r/rpl_udf.result:
Remove explicit set of binlog format (and triplicate test execution)
and rely on test system executing the test in all binlog formats.
mysql-test/suite/rpl/t/rpl_bug31076.test:
Test is only valid in mixed or row mode since it generates row events.
mysql-test/suite/rpl/t/rpl_events.test:
Removed explicit set of binlog_format and removed duplicate testing.
Instead, we rely on the test system to try all binlog formats.
mysql-test/suite/rpl/t/rpl_extraColmaster_innodb.test:
Removed triplicate testing and instead relying on test system.
Test is only relevant for row format since statement-based replication
cannot handle extra columns on master.
mysql-test/suite/rpl/t/rpl_extraColmaster_myisam.test:
Removed triplicate testing and instead relying on test system.
Test is only relevant for row format since statement-based replication
cannot handle extra columns on master.
mysql-test/suite/rpl/t/rpl_idempotency-slave.opt:
Removed .opt file to avoid server restarts.
mysql-test/suite/rpl/t/rpl_idempotency.test:
- Moved out row-only tests to a new test file, rpl_row_idempotency.test.
rpl_idempotency now only contains tests that execute in all
binlog_formats.
- While I was here, also removed .opt file to avoid server restarts.
The slave_exec_mode is now set inside the test instead.
mysql-test/suite/rpl/t/rpl_mix_found_rows.test:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). This file
contains the second half of the original rpl_found_rows.test with the
follwing changes:
- initialization
- removed SET_BINLOG_FORMAT and added have_binlog_format_mixed.inc
- minor formatting changes
mysql-test/suite/rpl/t/rpl_mix_insert_delayed.test:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/t/rpl_rbr_to_sbr.test:
Test cannot execute in statement mode, since we no longer
switch to row format when binlog_format=statement.
Enforced mixed mode throughout the test.
mysql-test/suite/rpl/t/rpl_row_idempotency.test:
Moved the second half of rpl_idempotency.test, which only
executed in row mode, to this new file. We now rely on the
test system to set binlog format.
mysql-test/suite/rpl/t/rpl_row_insert_delayed.test:
- Got rid of unnecessary explicit setting of binlog format.
- extra/rpl_tests/rpl_insert_delayed.test does not need the
$binlog_format_statement variable any more, so that was
removed.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
The test switches binlog_format internally and master generates both
row and statement events. Hence, the slave must be able to log in both
statement and row format. Hence test was changed to only execute in
mixed mode.
mysql-test/suite/rpl/t/rpl_stm_found_rows.test:
Split rpl_found_rows.test into rpl_mix_found_rows.test (a new file) and
rpl_stm_found_rows.test (renamed rpl_found_rows.test). Changes in
this file:
- minor formatting changes
- added have_binlog_format_statement and removed SET BINLOG_FORMAT.
- second half of file is moved to rpl_mix_found_rows.test
- added cleanup code
mysql-test/suite/rpl/t/rpl_stm_insert_delayed.test:
Moved out code operating in mixed mode from rpl_stm_insert_delayed
(into rpl_mix_insert_delayed) and got rid of explicit setting of
binlog format.
mysql-test/suite/rpl/t/rpl_switch_stm_row_mixed.test:
The test switches binlog_format internally and master generates both
row and statement events. Hence, the slave must be able to log in both
statement and row format. Hence test was changed to only execute in
mixed mode on slave.
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
Removed explicit set of binlog format. Instead, the test now only
executes in row mode.
mysql-test/suite/rpl/t/rpl_udf.test:
Remove explicit set of binlog format (and triplicate test execution)
and rely on test system executing the test in all binlog formats.
mysql-test/suite/rpl_ndb/combinations:
Added combinations file for rpl_ndb.
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
new result file
mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result:
updated result file
mysql-test/suite/rpl_ndb/t/rpl_ndb_2innodb.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_2myisam.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_basic.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-master.opt:
new option file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors-slave.opt:
new option file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
New test case to verify all errors and warnings generated by
decide_logging_format.
mysql-test/suite/rpl_ndb/t/rpl_ndb_blob.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_blob2.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_circular.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_circular_simplex.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
While I was here, also made the test clean up after itself.
mysql-test/suite/rpl_ndb/t/rpl_ndb_commit_afterflush.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_ctype_ucs2_def.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_delete_nowhere.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_do_db.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_do_table.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_func003.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_innodb_trans.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_insert_ignore.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_mixed_engines_transactions.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_multi_update3.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_rep_ignore.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_row_001.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_sp003.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_sp006.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/suite/rpl_ndb/t/rpl_ndb_trig004.test:
The test needs slave to be able to switch to row mode, so the
test was changed to only execute in mixed and row mode.
mysql-test/t/partition_innodb_stmt.test:
Changed error code for innodb updates with READ COMMITTED or
READ UNCOMMITTED transaction isolation level and
binlog_format=statement.
sql/event_db_repository.cc:
Use member function to read current_stmt_binlog_row_based.
sql/events.cc:
Use member function to read current_stmt_binlog_row_based.
sql/ha_ndbcluster_binlog.cc:
reset_current_stmt_binlog_row_based() is not a no-op for the ndb_binlog
thread any more. Instead, the ndb_binlog thread now forces row mode both
initially and just after calling mysql_parse. (mysql_parse() is the only
place where reset_current_stmt_binlog_row_based() may be called from
the ndb_binlog thread, so these are the only two places that need to
change.)
sql/ha_partition.cc:
Use member function to read current_stmt_binlog_row_based.
sql/handler.cc:
Use member function to read current_stmt_binlog_row_based.
sql/item_create.cc:
Added DBUG_ENTER to some functions, to be able to trace when
set_stmt_unsafe is called.
sql/log.cc:
Use member function to read current_stmt_binlog_row_based.
sql/log_event.cc:
- Moved logic for changing to row format out of do_apply_event (and into
decide_logging_format).
- Added @todo comment for post-push cleanup.
sql/log_event_old.cc:
Move logic for changing to row format out of do_apply_event (and into
decide_logging_format).
sql/mysql_priv.h:
Make decide_logging_format() a member of the THD class, for two reasons:
- It is natural from an object-oriented perspective.
- decide_logging_format() needs to access private members of THD
(specifically, the new binlog_warning_flags field).
sql/rpl_injector.cc:
Removed call to set_current_stmt_binlog_row_based().
From now on, only decide_logging_fromat is allowed to modify
current_stmt_binlog_row_based. This call is from the ndb_binlog
thread, mostly executing code in ha_ndbcluster_binlog.cc.
This call can be safely removed, because:
- current_stmt_binlog_row_based is initialized for the ndb_binlog
thread's THD object when the THD object is created. So we're
not going to read uninitialized memory.
- The behavior of ndb_binlog thread does not use the state of the
current_stmt_binlog_row_based. It is conceivable that the
ndb_binlog thread would rely on the current_stmt_binlog_format
in two situations:
(1) when it calls mysql_parse;
(2) when it calls THD::binlog_query.
In case (1), it always clears THD::options&OPTION_BIN_LOG (because
run_query() in ha_ndbcluster_binlog.cc is only called with
disable_binlogging = TRUE).
In case (2), it always uses qtype=STMT_QUERY_TYPE.
sql/set_var.cc:
Added @todo comment for post-push cleanup.
sql/share/errmsg.txt:
Added new error messages and clarified ER_BINLOG_UNSAFE_STATEMENT.
sql/sp.cc:
Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
Got rid of MYSQL_QUERY_TYPE: it was equivalent to STMT_QUERY_TYPE.
sql/sp_head.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sp_head.h:
Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
sql/sql_acl.cc:
Got rid of MYSQL_QUERY_TYPE: it was equivalent to STMT_QUERY_TYPE.
sql/sql_base.cc:
- Made decide_logging_format take care of all logic for deciding the
logging format, and for determining the related warnings and errors.
See comment above decide_logging_format for details.
- Made decide_logging_format a member function of THD, since it needs
to access private members of THD and since its purpose is to update
the state of a THD object.
- Added DBUG_ENTER, to be able to trace when set_stmt_unsafe is called.
sql/sql_class.cc:
- Moved logic for determining unsafe warnings away from THD::binlog_query
(and into decide_logging_format()). Now, it works like this:
1. decide_logging_format detects that the current statement shall
produce a warning, if it ever makes it to the binlog
2. decide_logging_format sets a flag of THD::binlog_warning_flags.
3. THD::binlog_query reads the flag. If the flag is set, it generates
a warning.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_class.h:
- Added THD::binlog_warning_flags (see sql_class.cc for explanation).
- Made decide_logging_format() and reset_for_next_command() member
functions of THD (instead of standalone functions). This was needed
for two reasons: (1) the functions need to access the private member
THD::binlog_warning_flags; (2) the purpose of these functions is to
update the staet of a THD object, so from an object-oriented point
of view they should be member functions.
- Encapsulated current_stmt_binlog_row_based, so it is now private and
can only be accessed from a member function. Also changed the
data type to an enumeration instead of a bool.
- Removed MYSQL_QUERY_TYPE, because it was equivalent to
STMT_QUERY_TYPE anyways.
- When reset_current_stmt_binlog_row_based was called from the
ndb_binlog thread, it would behave as a no-op. This special
case has been removed, and the behavior of
reset_current_stmt_binlog_row_based does not depend on which thread
calls it any more. The special case did not serve any purpose,
since the ndb binlog thread did not take the
current_stmt_binlog_row_based flag into account anyways.
sql/sql_delete.cc:
- Moved logic for setting row format for DELETE..LIMIT away from
mysql_prepare_delete.
(Instead, we mark the statement as unsafe at parse time (sql_yacc.yy)
and rely on decide_logging_format() (sql_class.cc) to set row format.)
This is part of the fix for BUG#45831.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_insert.cc:
- Removed unnecessary calls to thd->lex->set_stmt_unsafe() and
thd->set_current_stmt_binlog_row_based_if_mixed() from
handle_delayed_insert(). The calls are unnecessary because they
have already been made; they were made in the constructor of
the `di' object.
- Since decide_logging_format() is now a member function of THD, code
that calls decide_logging_format() had to be updated.
- Added DBUG_ENTER call, to be able to trace when set_stmt_unsafe is
called.
- Moved call to set_stmt_unsafe() for INSERT..SELECT..LIMIT away from
mysql_insert_select_prepare() (and into decide_logging_format).
This is part of the fix for BUG#45831.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_lex.h:
- Added the flag BINLOG_STMT_FLAG_ROW_INJECTION to enum_binlog_stmt_flag.
This was necessary so that a statement can identify itself as a row
injection.
- Added appropriate setter and getter functions for the new flag.
- Added or clarified some comments.
- Added DBUG_ENTER()
sql/sql_load.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_parse.cc:
- Made mysql_reset_thd_for_next_command() clear thd->binlog_warning_flags.
- Since thd->binlog_warning_flags is private, it must be set in a
member function of THD. Hence, moved the body of
mysql_reset_thd_for_next_command() to the new member function
THD::reset_thd_for_next_command(), and made
mysql_reset_thd_for_next_command() call
THD::reset_thd_for_next_command().
- Removed confusing comment.
- Use member function to read current_stmt_binlog_row_based.
sql/sql_repl.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_table.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_udf.cc:
Use member function to read current_stmt_binlog_row_based.
sql/sql_update.cc:
Moved logic for setting row format for UPDATE..LIMIT away from
mysql_prepare_update.
(Instead, we mark the statement as unsafe at parse time (sql_yacc.yy)
and rely on decide_logging_format() (sql_class.cc) to set row format.)
This is part of the fix for BUG#45831.
sql/sql_yacc.yy:
Made INSERT DELAYED, INSERT..SELECT..LIMIT, UPDATE..LIMIT, and
DELETE..LIMIT mark themselves as unsafe at parse time (instead
of at execution time).
This is part of the fixes BUG#45831 and BUG#45825.
storage/example/ha_example.cc:
Made exampledb accept inserts. This was needed by the new test case
rpl_ndb_binlog_format_errors, because it needs an engine that
is statement-only (and accepts inserts).
storage/example/ha_example.h:
Made exampledb a statement-only engine instead of a row-only engine.
No existing test relied exampledb's row-only capabilities. The new
test case rpl_ndb_binlog_format_errors needs an engine that is
statement-only.
storage/innobase/handler/ha_innodb.cc:
- Changed error error code and message given by innodb when
binlog_format=STATEMENT and transaction isolation level is
READ COMMITTED or READ UNCOMMITTED.
- While I was here, also simplified the condition for
checking when to give the error.
General overview:
The logic for switching to row format when binlog_format=MIXED had
numerous flaws. The underlying problem was the lack of a consistent
architecture.
General purpose of this changeset:
This changeset introduces an architecture for switching to row format
when binlog_format=MIXED. It enforces the architecture where it has
to. It leaves some bugs to be fixed later. It adds extensive tests to
verify that unsafe statements work as expected and that appropriate
errors are produced by problems with the selection of binlog format.
It was not practical to split this into smaller pieces of work.
Problem 1:
To determine the logging mode, the code has to take several parameters
into account (namely: (1) the value of binlog_format; (2) the
capabilities of the engines; (3) the type of the current statement:
normal, unsafe, or row injection). These parameters may conflict in
several ways, namely:
- binlog_format=STATEMENT for a row injection
- binlog_format=STATEMENT for an unsafe statement
- binlog_format=STATEMENT for an engine only supporting row logging
- binlog_format=ROW for an engine only supporting statement logging
- statement is unsafe and engine does not support row logging
- row injection in a table that does not support statement logging
- statement modifies one table that does not support row logging and
one that does not support statement logging
Several of these conflicts were not detected, or were detected with
an inappropriate error message. The problem of BUG#39934 was that no
appropriate error message was written for the case when an engine
only supporting row logging executed a row injection with
binlog_format=ROW. However, all above cases must be handled.
Fix 1:
Introduce new error codes (sql/share/errmsg.txt). Ensure that all
conditions are detected and handled in decide_logging_format()
Problem 2:
The binlog format shall be determined once per statement, in
decide_logging_format(). It shall not be changed before or after that.
Before decide_logging_format() is called, all information necessary to
determine the logging format must be available. This principle ensures
that all unsafe statements are handled in a consistent way.
However, this principle is not followed:
thd->set_current_stmt_binlog_row_based_if_mixed() is called in several
places, including from code executing UPDATE..LIMIT,
INSERT..SELECT..LIMIT, DELETE..LIMIT, INSERT DELAYED, and
SET @@binlog_format. After Problem 1 was fixed, that caused
inconsistencies where these unsafe statements would not print the
appropriate warnings or errors for some of the conflicts.
Fix 2:
Remove calls to THD::set_current_stmt_binlog_row_based_if_mixed() from
code executed after decide_logging_format(). Compensate by calling the
set_current_stmt_unsafe() at parse time. This way, all unsafe statements
are detected by decide_logging_format().
Problem 3:
INSERT DELAYED is not unsafe: it is logged in statement format even if
binlog_format=MIXED, and no warning is printed even if
binlog_format=STATEMENT. This is BUG#45825.
Fix 3:
Made INSERT DELAYED set itself to unsafe at parse time. This allows
decide_logging_format() to detect that a warning should be printed or
the binlog_format changed.
Problem 4:
LIMIT clause were not marked as unsafe when executed inside stored
functions/triggers/views/prepared statements. This is
BUG#45785.
Fix 4:
Make statements containing the LIMIT clause marked as unsafe at
parse time, instead of at execution time. This allows propagating
unsafe-ness to the view.
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
sql/log_event.cc:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument instead of figuring it out by itself
sql/log_event.h:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
MySQL crashes if a user without proper privileges attempts to create a procedure.
The crash happens because more than one error state is pushed onto the Diagnostic
area. In this particular case the user is denied to implicitly create a new user
account with the implicitly granted privileges ALTER- and EXECUTE ROUTINE.
The new account is needed if the original user account contained a host mask.
A user account with a host mask is a distinct user account in this context.
An alternative would be to first get the most permissive user account which
include the current user connection and then assign privileges to that
account. This behavior change is considered out of scope for this bug patch.
The implicit assignment of privileges when a user creates a stored routine is a
considered to be a feature for user convenience and as such it is not
a critical operation. Any failure to complete this operation is thus considered
non-fatal (an error becomes a warning).
The patch back ports a stack implementation of the internal error handler interface.
This enables the use of multiple error handlers so that it is possible to intercept
and cancel errors thrown by lower layers. This is needed as a error handler already
is used in the call stack emitting the errors which needs to be converted.
mysql-test/r/grant.result:
* Added test case for bug44658
mysql-test/t/grant.test:
* Added test case for bug44658
sql/sp.cc:
* Removed non functional parameter no_error and my_error calls as all errors
from this function will be converted to a warning anyway.
* Change function return type from int to bool.
sql/sp.h:
* Removed non functional parameter no_error and my_error calls as all errors
from this function will be converted to a warning anyway.
* Changed function return value from int to bool
sql/sql_acl.cc:
* Removed the non functional no_error parameter from the function prototype.
The function is called from two places and in one of the places we now
ignore errors through error handlers.
* Introduced the parameter write_to_binlog
* Introduced an error handler to cancel any error state from mysql_routine_grant.
* Moved my_ok() signal from mysql_routine_grant to make it easier to avoid
setting the wrong state in the Diagnostic area.
* Changed the broken error state in sp_grant_privileges() to a warning
so that if "CREATE PROCEDURE" fails because "Password hash isn't a hexidecimal
number" it is still clear what happened.
sql/sql_acl.h:
* Removed the non functional no_error parameter from the function prototype.
The function is called from two places and in one of the places we now
ignore errors through error handlers.
* Introduced the parameter write_to_binlog
* Changed return type for sp_grant_privileges() from int to bool
sql/sql_class.cc:
* Back ported implementation of internal error handler from 6.0 branch
sql/sql_class.h:
* Back ported implementation of internal error handler from 6.0 branch
sql/sql_parse.cc:
* Moved my_ok() signal from mysql_routine_grant() to make it easier to avoid
setting the wrong state in the Diagnostic area.
MySQL crashes if a user without proper privileges attempts to create a procedure.
The crash happens because more than one error state is pushed onto the Diagnostic
area. In this particular case the user is denied to implicitly create a new user
account with the implicitly granted privileges ALTER- and EXECUTE ROUTINE.
The new account is needed if the original user account contained a host mask.
A user account with a host mask is a distinct user account in this context.
An alternative would be to first get the most permissive user account which
include the current user connection and then assign privileges to that
account. This behavior change is considered out of scope for this bug patch.
The implicit assignment of privileges when a user creates a stored routine is a
considered to be a feature for user convenience and as such it is not
a critical operation. Any failure to complete this operation is thus considered
non-fatal (an error becomes a warning).
The patch back ports a stack implementation of the internal error handler interface.
This enables the use of multiple error handlers so that it is possible to intercept
and cancel errors thrown by lower layers. This is needed as a error handler already
is used in the call stack emitting the errors which needs to be converted.
When the thread executing a DDL was killed after finished its
execution but before writing the binlog event, the error code in
the binlog event could be set wrongly to ER_SERVER_SHUTDOWN or
ER_QUERY_INTERRUPTED.
This patch fixed the problem by ignoring the kill status when
constructing the event for DDL statements.
This patch also included the following changes in order to
provide the test case.
1) modified mysqltest to support variable for connection command
2) modified mysql-test-run.pl, add new variable MYSQL_SLAVE to
run mysql client against the slave mysqld.
When the thread executing a DDL was killed after finished its
execution but before writing the binlog event, the error code in
the binlog event could be set wrongly to ER_SERVER_SHUTDOWN or
ER_QUERY_INTERRUPTED.
This patch fixed the problem by ignoring the kill status when
constructing the event for DDL statements.
This patch also included the following changes in order to
provide the test case.
1) modified mysqltest to support variable for connection command
2) modified mysql-test-run.pl, add new variable MYSQL_SLAVE to
run mysql client against the slave mysqld.
Set wrong sql_mode when creating a procedure.
So that the sql_mode can't be writen into binary log correctly.
Restore the current session sql_mode right before generating the binlog event
when creating a procedure.
mysql-test/suite/binlog/r/binlog_sql_mode.result:
Test result
mysql-test/suite/binlog/t/binlog_sql_mode.test:
Test file for sql_mode testing
sql/sp.cc:
Restore the current session sql_mode right before generating the binlog event.
Set wrong sql_mode when creating a procedure.
So that the sql_mode can't be writen into binary log correctly.
Restore the current session sql_mode right before generating the binlog event
when creating a procedure.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
build)
The crash was caused by freeing the internal parser stack during the parser
execution.
This occured only for complex stored procedures, after reallocating the parser
stack using my_yyoverflow(), with the following C call stack:
- MYSQLparse()
- any rule calling sp_head::restore_lex()
- lex_end()
- x_free(lex->yacc_yyss), xfree(lex->yacc_yyvs)
The root cause is the implementation of stored procedures, which breaks the
assumption from 4.1 that there is only one LEX structure per parser call.
The solution is to separate the LEX structure into:
- attributes that represent a statement (the current LEX structure),
- attributes that relate to the syntax parser itself (Yacc_state),
so that parsing multiple statements in stored programs can create multiple
LEX structures while not changing the unique Yacc_state.
Now, Yacc_state and the existing Lex_input_stream are aggregated into
Parser_state, a structure that represent the complete state of the (Lexical +
Syntax) parser.
mysql-test/r/parser_stack.result:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
mysql-test/t/parser_stack.test:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sp.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sp_head.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_class.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_class.h:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_lex.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_lex.h:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_parse.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_prepare.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_trigger.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_view.cc:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
sql/sql_yacc.yy:
Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
build)
build)
The crash was caused by freeing the internal parser stack during the parser
execution.
This occured only for complex stored procedures, after reallocating the parser
stack using my_yyoverflow(), with the following C call stack:
- MYSQLparse()
- any rule calling sp_head::restore_lex()
- lex_end()
- x_free(lex->yacc_yyss), xfree(lex->yacc_yyvs)
The root cause is the implementation of stored procedures, which breaks the
assumption from 4.1 that there is only one LEX structure per parser call.
The solution is to separate the LEX structure into:
- attributes that represent a statement (the current LEX structure),
- attributes that relate to the syntax parser itself (Yacc_state),
so that parsing multiple statements in stored programs can create multiple
LEX structures while not changing the unique Yacc_state.
Now, Yacc_state and the existing Lex_input_stream are aggregated into
Parser_state, a structure that represent the complete state of the (Lexical +
Syntax) parser.
into bodhi.(none):/opt/local/work/mysql-5.1-27430
Makefile.am:
Auto merged
include/my_global.h:
Auto merged
mysql-test/include/mix1.inc:
Auto merged
sql/item.cc:
Auto merged
sql/my_decimal.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/set_var.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/share/errmsg.txt:
Auto merged
sql/sql_yacc.yy:
Auto merged
libmysqld/CMakeLists.txt:
Manual merge.
libmysqld/lib_sql.cc:
Manual merge.
mysql-test/t/disabled.def:
Manual merge.
into zippy.cornsilk.net:/home/cmiller/work/mysql/bug36570/my51-bug36570
BitKeeper/deleted/.del-binlog_innodb.result:
Auto merged
sql/sp_head.cc:
Auto merged
mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result:
need to re-record.
mysql-test/suite/rpl/r/rpl_sp.result:
need to re-record.
mysql-test/r/mysqlbinlog.result:
manual merge.
mysql-test/suite/rpl/t/rpl_sp.test:
manual merge.
sql/sp.cc:
manual merge.
sql/sp_head.h:
manual merge.
slave
The stored-routine code took the contents of the (lowest) parser
and copied it directly to the binlog, which causes problems if there
is a special case of interpretation at the parser level -- which
there is, in the "/*!VER */" comments. The trailing "*/" caused
errors on the slave, naturally.
Now, since by that point we have /properly/ created parse-tree (as
the rest of the server should do!) for the stored-routine CREATE, we
can construct a perfect statement from that information, instead of
writing uncertain information from an unknown parser state.
Fortunately, there's already a function nearby that does exactly
that.
---
Update for Bug#36570. Qualify routine names with db name when
writing to the binlog ONLY if the source text is qualified.
mysql-test/r/binlog_innodb.result:
Offsets changed due to quoting.
---
New offset to account for db-qualified names.
mysql-test/r/ctype_cp932_binlog.result:
Offsets changed due to quoting.
---
Qualify routine names with DB. Offsets change also.
mysql-test/r/mysqlbinlog.result:
Case changed in result due to interpretation of data instead of
literal recitation.
---
Qualify procedure name with db.
mysql-test/r/rpl_sp.result:
Offsets changed due to quoting. Added tests.
---
Qualify routine names with DB if qualified in query. Offsets change also.
mysql-test/t/rpl_sp.test:
Add version-limiting quotes to exercise bug#36570. Test that
backtick-quoted identifiers and labels work also.
---
Use different db to show qualification works. Qualify routine names
with DB if qualified in query.
sql/sp.cc:
In create_string, we may not have a sp_name parameter yet, so
instead pass the char* and length of the only member we'd get out
of it.
Having done that, we can use the same function to write the
CREATE (FUNC|TRIG|PROC) statement to the binlog as we always used
to display the statement to the user.
---
Make the db name part of the CREATE string if it is specified.
Specify it in part of writing to the binlog when creating a new
routine.
sql/sp_head.cc:
Set the sp_head m_explicit_name member as the sp_name member is set.
We can not peek at this later, as the sp_name is gone by then.
sql/sp_head.h:
Add a member to track whether the name is qualified with the
database.
slave
The stored-routine code took the contents of the (lowest) parser
and copied it directly to the binlog, which causes problems if there
is a special case of interpretation at the parser level -- which
there is, in the "/*!VER */" comments. The trailing "*/" caused
errors on the slave, naturally.
Now, since by that point we have /properly/ created parse-tree (as
the rest of the server should do!) for the stored-routine CREATE, we
can construct a perfect statement from that information, instead of
writing uncertain information from an unknown parser state.
Fortunately, there's already a function nearby that does exactly
that.
---
Update for Bug#36570. Qualify routine names with db name when
writing to the binlog ONLY if the source text is qualified.
into zippy.cornsilk.net:/home/cmiller/work/mysql/bug36570/my51-bug36570
BitKeeper/deleted/.del-binlog_innodb.result:
Auto merged
mysql-test/r/mysqlbinlog.result:
need to record.
mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result:
need to record.
mysql-test/suite/rpl/r/rpl_sp.result:
need to record.
mysql-test/suite/rpl/t/rpl_sp.test:
manual merge.
sql/sp.cc:
Manual merge
slave
The stored-routine code took the contents of the (lowest) parser
and copied it directly to the binlog, which causes problems if there
is a special case of interpretation at the parser level -- which
there is, in the "/*!VER */" comments. The trailing "*/" caused
errors on the slave, naturally.
Now, since by that point we have /properly/ created parse-tree (as
the rest of the server should do!) for the stored-routine CREATE, we
can construct a perfect statement from that information, instead of
writing uncertain information from an unknown parser state.
Fortunately, there's already a function nearby that does exactly
that.
mysql-test/r/binlog_innodb.result:
Offsets changed due to quoting.
mysql-test/r/ctype_cp932_binlog.result:
Offsets changed due to quoting.
mysql-test/r/mysqlbinlog.result:
Case changed in result due to interpretation of data instead of
literal recitation.
mysql-test/r/rpl_sp.result:
Offsets changed due to quoting. Added tests.
mysql-test/t/rpl_sp.test:
Add version-limiting quotes to exercise bug#36570. Test that
backtick-quoted identifiers and labels work also.
sql/sp.cc:
In create_string, we may not have a sp_name parameter yet, so
instead pass the char* and length of the only member we'd get out
of it.
Having done that, we can use the same function to write the
CREATE (FUNC|TRIG|PROC) statement to the binlog as we always used
to display the statement to the user.
slave
The stored-routine code took the contents of the (lowest) parser
and copied it directly to the binlog, which causes problems if there
is a special case of interpretation at the parser level -- which
there is, in the "/*!VER */" comments. The trailing "*/" caused
errors on the slave, naturally.
Now, since by that point we have /properly/ created parse-tree (as
the rest of the server should do!) for the stored-routine CREATE, we
can construct a perfect statement from that information, instead of
writing uncertain information from an unknown parser state.
Fortunately, there's already a function nearby that does exactly
that.
into kaamos.(none):/data/src/opt/mysql-5.1-opt
client/mysql.cc:
Auto merged
client/mysqldump.c:
Auto merged
configure.in:
Auto merged
include/my_global.h:
Auto merged
libmysql/libmysql.c:
Auto merged
libmysqld/lib_sql.cc:
Auto merged
mysql-test/include/mix1.inc:
Auto merged
mysql-test/r/create.result:
Auto merged
mysql-test/r/func_str.result:
Auto merged
mysql-test/r/innodb.result:
Auto merged
mysql-test/r/innodb_mysql.result:
Auto merged
mysql-test/r/select.result:
Auto merged
mysql-test/r/subselect.result:
Auto merged
mysql-test/t/create.test:
Auto merged
mysql-test/t/disabled.def:
Auto merged
sql/filesort.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_sum.cc:
Auto merged
sql/log.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/ndb/src/kernel/blocks/backup/Backup.hpp:
Auto merged
tests/mysql_client_test.c:
Auto merged
mysql-test/r/func_time.result:
Manual merge.
mysql-test/r/view.result:
Manual merge.
mysql-test/t/view.test:
Manual merge.
scripts/mysql_config.sh:
Manual merge.
sql-common/client.c:
Manual merge.
sql/sql_parse.cc:
Manual merge.
into magare.gmz:/home/kgeorge/mysql/work/B30604-5.1-opt
BitKeeper/deleted/.del-binlog_innodb.result:
Auto merged
BitKeeper/deleted/.del-binlog_innodb.test:
Auto merged
mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result:
merged bug 30604 to 5.1-opt
sql/sp.cc:
merged bug 30604 to 5.1-opt
and ps-protocol
Finding a routine should be a transparent operation as
far as the binary log is concerned.
But it was influencing the binary log because of the TIMESTAMP
column in the proc table.
Fixed by preserving and restoring the time_zone usage flag when
searching for a stored routine in the proc table.
mysql-test/r/binlog_innodb.result:
Bug #30604: test case
mysql-test/r/ctype_cp932_binlog.result:
Bug #30604: updated test results (a procedure call before that)
mysql-test/t/binlog_innodb.test:
Bug #30604: test case
sql/sp.cc:
Bug #30604: finding a routine should be a transparent operation as
far as the binary log is concerned.
Fixed by preserving and restoring the time_zone usage flag.
and ps-protocol
Finding a routine should be a transparent operation as
far as the binary log is concerned.
But it was influencing the binary log because of the TIMESTAMP
column in the proc table.
Fixed by preserving and restoring the time_zone usage flag when
searching for a stored routine in the proc table.
into dipika.(none):/opt/local/work/mysql-5.1-2pc-opt-merge-push
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Auto merged
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Auto merged
sql/ha_ndbcluster_binlog.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/suite/rpl_ndb/t/disabled.def:
Manual merge.
a SELECT doesn't cause ROLLBACK of statem".
The idea of the fix is to ensure that we always commit the current
statement at the end of dispatch_command(). In order to not issue
redundant disc syncs, an optimization of the two-phase commit
protocol is implemented to bypass the two phase commit if
the transaction is read-only.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Update test results.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Update test results.
mysql-test/suite/rpl_ndb/t/disabled.def:
Disable the tests, for which this changeset reveals a bug:
the injector thread does not always add 'statement commit' to the
rows injected in circular replication set up.
To be investigated separately.
sql/ha_ndbcluster_binlog.cc:
Add close_thread_tables() to run_query: this ensures
that all tables are closed and there is no pending statement transaction.
sql/handler.cc:
Implement optimisation of read-only transactions.
If a transaction consists only of DML statements that do not change
data, we do not perform a two-phase commit for it
(run one phase commit only).
sql/handler.h:
Implement optimisation of read-only transactions.
If a transaction consists only of DML statements that do not change
data, we do not perform a two-phase commit for it
(run one phase commit only).
sql/log.cc:
Mark the binlog transaction read-write whenever it's started.
We never read from binlog, so it's safe and least intrusive to add
this mark up here.
sql/log_event.cc:
Update to the new layout of thd->transaction.
sql/rpl_injector.cc:
Always commit statement transaction before committing the global one.
sql/sp.cc:
Ad comments.
sql/sp_head.cc:
Add comments.
sql/sql_base.cc:
Commit transaction at the end of the statement. Always.
sql/sql_class.cc:
Update thd_ha_data to return the right pointer in the new layout.
Fix select_dumpvar::send_data to properly return operation status.
A test case from commit.inc would lead to an assertion failure in the
diagnostics area (double assignment). Not test otherwise by the test suite.
sql/sql_class.h:
Implement a new layout of storage engine transaction info in which
it is easy to access all members related to the handlerton only
based on ht->slot.
sql/sql_cursor.cc:
Update to the new layout of thd->transaction.
sql/sql_delete.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
sql/sql_do.cc:
Add DO always clears the error, we must rollback the current
statement before this happens. Otherwise the statement will be committed,
and not rolled back in the end.
sql/sql_insert.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
sql/sql_load.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
sql/sql_parse.cc:
Implement optimisation of read-only transactions: bypass 2-phase
commit for them.
Always commit statement transaction before commiting the global one.
Fix an unrelated crash in check_table_access, when called from
information_schema.
sql/sql_partition.cc:
Partitions commit at the end of a DDL operation.
Make sure that send_ok() is done only if the commit has succeeded.
sql/sql_table.cc:
Use ha_autocommit_or_rollback and end_active_trans everywhere.
Add end_trans to mysql_admin_table, so that it leaves no pending
transaction.
sql/sql_udf.cc:
Remvove a redundant call to close_thread_tables()
sql/sql_update.cc:
Remove wrong and now redundant calls to ha_autocommit_or_rollback.
The transaction is committed in one place, at the end of the statement.
Remove calls to mysql_unlock_tables, since some engines count locks
and commit statement transaction in unlock_tables(), which essentially
equates mysql_unlock_tables to ha_autocommit_or_rollback.
Previously it was necessary to unlock tables soon because we wanted
to avoid sending of 'ok' packet to the client under locked tables.
This is no longer necessary, since OK packet is also sent from one place
at the end of transaction.
mysql-test/include/commit.inc:
New BitKeeper file ``mysql-test/include/commit.inc''
mysql-test/r/commit_1innodb.result:
New BitKeeper file ``mysql-test/r/commit_1innodb.result''
mysql-test/t/commit_1innodb.test:
New BitKeeper file ``mysql-test/t/commit_1innodb.test''
a SELECT doesn't cause ROLLBACK of statem".
The idea of the fix is to ensure that we always commit the current
statement at the end of dispatch_command(). In order to not issue
redundant disc syncs, an optimization of the two-phase commit
protocol is implemented to bypass the two phase commit if
the transaction is read-only.
The problem is that one can not create a stored routine if sql_mode
contains NO_ENGINE_SUBSTITUTION or PAD_CHAR_TO_FULL_LENGTH. Also when
a event is created, the mode is silently lost if sql_mode contains one
of the aforementioned. This was happening because the table definitions
which stored sql_mode values weren't being updated to accept new values
of sql_mode.
The solution is to update, in a backwards compatible manner, the various
table definitions (columns) that store the sql_mode value to take into
account the new possible values. One incompatible change is that if a event
that is being created can't be stored to the mysql.event table, an error
will be raised.
The tests case also ensure that new SQL modes will be added to the mysql.proc
and mysql.event tables, otherwise the tests will fail.
mysql-test/r/events_bugs.result:
Add test case result for Bug#32633
mysql-test/r/information_schema.result:
Update the sql_mode column definition.
mysql-test/r/sp.result:
Add test case result for Bug#32633
mysql-test/r/system_mysql_db.result:
Update the sql_mode column definition.
mysql-test/t/events_bugs.test:
Add test case for Bug#32633
mysql-test/t/sp.test:
Add test case for Bug#32633
mysql-test/t/system_mysql_db_fix50117.test:
Update the sql_mode column definition.
scripts/mysql_system_tables.sql:
Update the sql_mode column definition.
scripts/mysql_system_tables_fix.sql:
Update the sql_mode column definition.
sql/event_db_repository.cc:
Reset and restore SQL modes when storing and loading a
event from the data dictionary. Also throw out a error
if a store fails.
sql/mysqld.cc:
Add warning to avoid this problem in the future.
sql-common/my_user.c:
Truncate length if user name or host name does not fit
in the buffer.
sql/sp.cc:
SQL mode of the thread must not effect data dictionary operations.
The problem is that one can not create a stored routine if sql_mode
contains NO_ENGINE_SUBSTITUTION or PAD_CHAR_TO_FULL_LENGTH. Also when
a event is created, the mode is silently lost if sql_mode contains one
of the aforementioned. This was happening because the table definitions
which stored sql_mode values weren't being updated to accept new values
of sql_mode.
The solution is to update, in a backwards compatible manner, the various
table definitions (columns) that store the sql_mode value to take into
account the new possible values. One incompatible change is that if a event
that is being created can't be stored to the mysql.event table, an error
will be raised.
The tests case also ensure that new SQL modes will be added to the mysql.proc
and mysql.event tables, otherwise the tests will fail.
The problem is that deprecated syntax warnings were not being
suppressed when the stored routine is being parsed for the first
execution. It's doesn't make sense to print out deprecated
syntax warnings when the routine is being executed because this
kind of warning only matters when the routine is being created.
The solution is to suppress deprecated syntax warnings when
parsing the stored routine for loading into the cache (might
mean that the routine is being executed for the first time).
mysql-test/r/sp-error.result:
Add test case result for Bug#21801
mysql-test/t/sp-error.test:
Add test case for Bug#21801
sql/sp.cc:
Implement a internal error handler to catch deprecated
syntax warnings when loading a stored procedure into the
cache.
The problem is that deprecated syntax warnings were not being
suppressed when the stored routine is being parsed for the first
execution. It's doesn't make sense to print out deprecated
syntax warnings when the routine is being executed because this
kind of warning only matters when the routine is being created.
The solution is to suppress deprecated syntax warnings when
parsing the stored routine for loading into the cache (might
mean that the routine is being executed for the first time).
into mysql.com:/misc/mysql/31752_/50-31752_
client/mysql.cc:
Auto merged
sql/log.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/unireg.cc:
Auto merged
into zippy.cornsilk.net:/home/cmiller/work/mysql/mysql-5.1-maint
sql/field.cc:
Auto merged
sql/filesort.cc:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_create.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_geofunc.cc:
Auto merged
sql/item_strfunc.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/log.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/net_serv.cc:
Auto merged
sql/opt_sum.cc:
Auto merged
sql/protocol.h:
Auto merged
sql/records.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.h:
Auto merged
sql/sql_cache.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_prepare.cc:
Auto merged
sql/sql_select.h:
Auto merged
sql/sql_trigger.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/sql_view.cc:
Auto merged
sql/structs.h:
Auto merged
sql/unireg.h:
Auto merged
sql/item.cc:
manual merge
sql/log_event.cc:
manual merge
sql/protocol.cc:
manual merge
sql/sp_head.cc:
manual merge
sql/sql_base.cc:
manual merge
sql/sql_parse.cc:
manual merge
sql/sql_select.cc:
manual merge
strmake() called with wrong parameters:
5.0-specific fixes.
client/mysql.cc:
In debug-mode, strmake() fills unused part of buffer with
a test-pattern. This overwrites our previous extra '\0'
(from previous bzero()).
sql/sp.cc:
off-by-one buffer-size.
The problem was that the RETURNS column in the mysql.proc was of
CHAR(64). That was not enough for storing long-named datatypes.
The fix is to change CHAR(64) to LONGBLOB, and to throw warnings
at the time a stored routine is created if some data is truncated
during writing into mysql.proc.
mysql-test/r/sp.result:
Update test result.
mysql-test/t/sp.test:
Add a test case for BUG#24923.
scripts/mysql_system_tables.sql:
Change the data type of column 'returns' from char(64) to longblob.
scripts/mysql_system_tables_fix.sql:
Change the data type of column 'returns' from char(64) to longblob.
sql/sp.cc:
Produce warnings if any data was truncated during writing
into mysql.proc.
sql/sp.h:
Add new error code.
sql/share/errmsg.txt:
Add new error message.
sql/sql_parse.cc:
Hande
The problem was that the RETURNS column in the mysql.proc was of
CHAR(64). That was not enough for storing long-named datatypes.
The fix is to change CHAR(64) to LONGBLOB, and to throw warnings
at the time a stored routine is created if some data is truncated
during writing into mysql.proc.
into lambda.hsd1.co.comcast.net.:/home/malff/TREE/mysql-5.1-rt-merge
mysql-test/t/sp-error.test:
Auto merged
mysql-test/t/udf.test:
Auto merged
sql/item.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_udf.cc:
Auto merged
mysql-test/r/sp-error.result:
failed auto merge
mysql-test/r/sp.result:
failed auto merge
mysql-test/r/udf.result:
failed auto merge
mysql-test/t/sp.test:
failed auto merge
sql/sp_head.cc:
failed auto merge
sql/sql_yacc.yy:
failed auto merge
Bug#29816 Syntactically wrong query fails with misleading error message
The core problem is that an SQL-invoked function name can be a <schema
qualified routine name> that contains no <schema name>, but the mysql
parser insists that all stored procedures (function, procedures and
triggers) must have a <schema name>, which is not true for functions.
This problem is especially visible when trying to create a function
or when a query contains a syntax error after a function call (in the
same query), both will fail with a "No database selected" message if
the session is not attached to a particular schema, but the first
one should succeed and the second fail with a "syntax error" message.
Part of the fix is to revamp the sp name handling so that a schema
name may be omitted for functions -- this means that the internal
function name representation may not have a dot, which represents
that the function doesn't have a schema name. The other part is
to place schema checks after the type (function, trigger or procedure)
of the routine is known.
mysql-test/r/sp-error.result:
Add test case result for Bug#29816
mysql-test/r/udf.result:
Add test case result for Bug#28318
mysql-test/t/sp-error.test:
Add test case for Bug#29816
mysql-test/t/udf.test:
Add test case for Bug#28318
sql/sp.cc:
Copy the (last) nul byte of the stored routine key and move name parsing
code to the sp_name class constructor.
sql/sp_head.cc:
Revamp routine name parsing for when no schema is specified and
omit dot from the qualified name if the routine is not associated
with a scheme name.
sql/sp_head.h:
Name parsing got bigger, uninline by moving to a single unit -- the sp_head.cc
file.
sql/sql_yacc.yy:
Only copy the schema name if one is actually set and check for schema
name presence only where it's necessary.
Bug#29816 Syntactically wrong query fails with misleading error message
The core problem is that an SQL-invoked function name can be a <schema
qualified routine name> that contains no <schema name>, but the mysql
parser insists that all stored procedures (function, procedures and
triggers) must have a <schema name>, which is not true for functions.
This problem is especially visible when trying to create a function
or when a query contains a syntax error after a function call (in the
same query), both will fail with a "No database selected" message if
the session is not attached to a particular schema, but the first
one should succeed and the second fail with a "syntax error" message.
Part of the fix is to revamp the sp name handling so that a schema
name may be omitted for functions -- this means that the internal
function name representation may not have a dot, which represents
that the function doesn't have a schema name. The other part is
to place schema checks after the type (function, trigger or procedure)
of the routine is known.
into ibm.opbmk:/home/alik/Documents/MySQL/devel/5.1-rt-bug25843
sql/mysql_priv.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_db.cc:
Auto merged
of statement breaks binlog.
There were two problems discovered by this bug:
1. Default (current) database is not fixed at the creation time.
That leads to wrong output of DATABASE() function.
2. Database attributes (@@collation_database) are not fixed at
the creation time. That leads to wrong resultset.
Binlog breakage and Query Cache wrong output happened because of
the first problem.
The fix is to remember the current database at the PREPARE-time and
set it each time at EXECUTE.
mysql-test/include/query_cache_sql_prepare.inc:
The first part of the test case for BUG#25843.
mysql-test/r/query_cache_ps_no_prot.result:
Update result file.
mysql-test/r/query_cache_ps_ps_prot.result:
Update result file.
mysql-test/suite/rpl/r/rpl_ps.result:
Update result file.
mysql-test/suite/rpl/t/rpl_ps.test:
The second part of the test case for BUG#25843.
sql/mysql_priv.h:
Added mysql_opt_change_db() prototype.
sql/sp.cc:
1. Polishing;
2. sp_use_new_db() has been removed;
3. Use mysql_opt_change_db() instead of sp_use_new_db().
sql/sp.h:
sp_use_new_db() has been removed.
This function has nothing to do with a) sp and b) *new* database.
It was merely "switch the current database if needed".
sql/sp_head.cc:
1. Polishing.
2. Use mysql_opt_change_db() instead of sp_use_new_db().
sql/sql_class.cc:
Move THD::{db, db_length} into Statement.
sql/sql_class.h:
Move THD::{db, db_length} into Statement.
sql/sql_db.cc:
Introduce mysql_opt_change_db() as a replacement (and inspired by)
sp_use_new_db().
sql/sql_prepare.cc:
1. Remember the current database in Prepared_statement::prepare().
2. Switch/restore the current database in Prepared_statement::execute().
of statement breaks binlog.
There were two problems discovered by this bug:
1. Default (current) database is not fixed at the creation time.
That leads to wrong output of DATABASE() function.
2. Database attributes (@@collation_database) are not fixed at
the creation time. That leads to wrong resultset.
Binlog breakage and Query Cache wrong output happened because of
the first problem.
The fix is to remember the current database at the PREPARE-time and
set it each time at EXECUTE.
into weblab.(none):/home/marcsql/TREE/mysql-5.1-rt50-merge
client/mysql.cc:
Auto merged
mysql-test/r/query_cache.result:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_yacc.yy:
Auto merged
PREPARE and EXECUTE of statement breaks binlog.
sql/sp.cc:
- Polishing sp_use_new_db():
- renamed no_access_check to force_switch to be more adequate;
- fixed comment;
sql/sql_class.h:
Polishing: fixed comment.
sql/sql_db.cc:
1. Use mysql_change_db_impl() to reset current database instead of
THD::set_db() in mysql_rm_db(). THD::set_db() does not take care of
THD::db_access and database attributes (@@collation_database).
2. Polishing: add, fix comments.
into hynda.mysql.fi:/home/my/mysql-5.1-marvel
client/mysqldump.c:
Auto merged
sql/event_db_repository.cc:
Auto merged
sql/event_queue.cc:
Auto merged
sql/field.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/table.h:
Auto merged
sql/sql_yacc.yy:
Manual merge with 5.1 main tree.
Faster thr_alarm()
Added 'Opened_files' status variable to track calls to my_open()
Don't give warnings when running mysql_install_db
Added option --source-install to mysql_install_db
I had to do the following renames() as used polymorphism didn't work with Forte compiler on 64 bit systems
index_read() -> index_read_map()
index_read_idx() -> index_read_idx_map()
index_read_last() -> index_read_last_map()
BUILD/compile-solaris-sparc-forte:
Updated script to current Solaris installations
Now we compile by default for 64 bits
client/mysql.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
client/mysql_upgrade.c:
Fixed compiler warning (on Forte)
client/mysqladmin.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
client/mysqlcheck.c:
Fixed compiler warning (on Forte)
client/mysqldump.c:
Fixed compiler warning (on Forte)
client/mysqlslap.c:
Fixed compiler warning (on Forte)
client/mysqltest.c:
Fixed compiler warning (on Forte)
client/sql_string.cc:
Avoid compiler warnings when using C function pointers in C++
configure.in:
Added detection of mtmalloc and ieeefp.h
extra/replace.c:
Fixed compiler warning (on Forte)
include/m_ctype.h:
Added some typedef's to make it easy to use C function pointers in C++
include/my_sys.h:
Added my_file_total_opened (counter for calls to my_open())
include/myisam.h:
Fixed compiler warning (on Forte)
libmysql/libmysql.c:
Fixed compiler warning (on Forte) by adding casts and change types
libmysql/manager.c:
Fixed compiler warning (on Forte) by adding casts and change types
mysql-test/r/ctype_cp932_binlog_stm.result:
Updated positions
(Needed because we didn't before correctly restore collation_database after running stored procedure
mysys/my_fopen.c:
Count number of opened files
mysys/my_open.c:
Count number of opened files
mysys/my_static.c:
Count number of opened files
mysys/thr_alarm.c:
Optimization to do less alarm() and pthread_sigmask() calls.
Idea is to remember time for next pending alarm and not reschedule a new alarm if it's after the current one.
Before we only did this if there was other pending alarms.
We don't have to use pthread_sigmask() in case of 'USE_ONE_SIGNAL_HAND' as the alarm()
signal will be blocked for the calling thread anyway and no other thread will have the alarm() signal enabled to call process_alarm()
regex/regcomp.c:
Fixed compiler warning (on Forte) by adding casts and change types
scripts/mysql_install_db.sh:
Added option --source-install to allow one to create a mysql database from the source tree without installing MySQL
Don't give (unnecessary) warnings
server-tools/instance-manager/angel.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
server-tools/instance-manager/thread_registry.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/event_db_repository.cc:
index_read() -> index_read_map()
sql/event_queue.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/field.cc:
Fixed compiler warnings about hidden fields
sql/ha_partition.cc:
Fixed compiler warnings about hidden fields
index_read() -> index_read_map()
sql/ha_partition.h:
index_read() -> index_read_map()
sql/handler.cc:
Added PAGE option to row types (to prepare for future)
index_read() -> index_read_map()
sql/handler.h:
Added ROW_TYPE_PAGE (for future)
Added flag to signal if table was to be created transactionally
I had to do the following renames() as used polymorphism didn't work with Forte compiler on 64 bit systems
index_read() -> index_read_map()
index_read_idx() -> index_read_idx_map()
index_read_last() -> index_read_last_map()
sql/item.cc:
Fixed indentation
Renamed local variable to avoid hiding class variable
sql/item_cmpfunc.cc:
Renamed local variable to avoid hiding class variable
sql/item_cmpfunc.h:
Removed not used variable
sql/item_func.cc:
Renamed local variable to avoid hiding class variable
sql/item_strfunc.cc:
Moved functions from Item_strfunc.cc
sql/item_strfunc.h:
Move functions to item_strfunc.cc
Use C function pointer type to avoid compiler warnings (with Forte)
sql/item_subselect.cc:
index_read() -> index_read_map()
sql/item_xmlfunc.cc:
Renamed local variable to avoid hiding class variable
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/key.cc:
Fixed indentation
sql/log.cc:
Renamed local variable to avoid hiding class variable
sql/log_event.cc:
Removed call to my_time() when creating class instance of Log_event() as this may have static instances.
(One can't call my_time() before my_init())
index_read() -> index_read_map()
Renamed local variable to avoid hiding class variable
sql/log_event_old.cc:
Renamed local variable to avoid hiding class variable
sql/mysql_priv.h:
Made all create_backup_ctx() declarations identical.
This lifted up a bug where wrong create_backup_ctx() was called in some cases.
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/mysqld.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
Fixed indentation
Don't call end_thr_alarm() when calling unireg_abort() as unireg_abort() already calls end_thr_alarm()
Added variable 'Opened_files' (number of calls to my_open() or my_fopen())
Don't print 'loose' warnings when using --bootstrap (to avoid warnings when running mysql_install_db)
Fixed compiler warnings
sql/opt_range.cc:
index_read() -> index_read_map()
sql/opt_sum.cc:
index_read() -> index_read_map()
sql/partition_info.cc:
Renamed local variable to avoid hiding class variable
sql/rpl_filter.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/set_var.cc:
Renamed local variable to avoid hiding class variable
Added 'process_key_cache_t' type to avoid compiler warning (on Forte)
sql/set_var.h:
Added 'process_key_cache_t' type to avoid compiler warning (on Forte)
sql/sp.cc:
More debugging
index_read() -> index_read_map()
sql/sp_cache.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/sp_head.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
Moved 'saved_creation_ctx' higher up to be able to free objects allocated by create_backup_ctx()
sql/sql_acl.cc:
index_read() -> index_read_map()
sql/sql_class.cc:
Renamed local variable to avoid hiding class variable
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/sql_class.h:
Renamed local variable to avoid hiding class variable
sql/sql_db.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
sql/sql_delete.cc:
Renamed local variable to avoid hiding class variable
sql/sql_handler.cc:
index_read() -> index_read_map()
sql/sql_help.cc:
index_read() -> index_read_map()
sql/sql_insert.cc:
index_read() -> index_read_map()
Renamed local variable to avoid hiding class variable
sql/sql_lex.cc:
Renamed local variable to avoid hiding class variable
sql/sql_plugin.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
index_read() -> index_read_map()
Don't give warnings about not used plugins if we are using --warnings=0
sql/sql_select.cc:
index_read() -> index_read_map()
sql-common/client.c:
Fixed compiler warning (on Forte)
sql-common/my_time.c:
Removed never accessed code
Fixed compiler warning (on Forte)
sql/sql_servers.cc:
index_read() -> index_read_map()
sql/sql_show.cc:
Added TRANSACTIONAL to SHOW CREATE
Fixed ROW_TYPE_PAGE
sql/sql_string.cc:
Avoid compiler warnings when using C function pointers in C++
sql/sql_table.cc:
Set create_info->transactional if we used TRANSACTIONAL=1
sql/sql_udf.cc:
index_read() -> index_read_map()
sql/sql_yacc.yy:
Added TRANSACTIONAL=0|1 to CREATE (for future)
Added row type PAGE (was only partionally handled before)
sql/strfunc.cc:
Avoid compiler warnings when using C function pointers in C++
sql/table.cc:
More DBUG statements
Declare all create_backup_ctx() functions identically
Remember if table was created with TRANSACTIONAL flag or not (future safe)
Renamed local variable to avoid hiding class variable
sql/table.h:
Remember if table was created with TRANSACTIONAL=1
sql/tztime.cc:
index_read() -> index_read_map()
sql-common/pack.c:
Fixed compiler warning (on Forte)
storage/archive/archive_reader.c:
Fixed compiler warning (on Forte)
storage/archive/azio.c:
Fixed compiler warning (on Forte)
storage/blackhole/ha_blackhole.cc:
index_read() -> index_read_map()
storage/blackhole/ha_blackhole.h:
index_read() -> index_read_map()
storage/csv/ha_tina.cc:
Declare functions sent to C code with extern "C" to avoid compiler warnings (on Forte)
storage/example/ha_example.cc:
index_read() -> index_read_map()
storage/example/ha_example.h:
index_read() -> index_read_map()
storage/heap/ha_heap.cc:
index_read() -> index_read_map()
storage/heap/ha_heap.h:
index_read() -> index_read_map()
storage/heap/hp_test1.c:
Fixed compiler warning (on Forte)
storage/heap/hp_test2.c:
Fixed compiler warning (on Forte)
storage/myisam/ft_boolean_search.c:
Fixed compiler warning (on Forte)
storage/myisam/ft_nlq_search.c:
Fixed compiler warning (on Forte)
storage/myisam/ft_parser.c:
Fixed compiler warning (on Forte)
storage/myisam/ft_stopwords.c:
Fixed compiler warning (on Forte)
storage/myisam/ha_myisam.cc:
index_read() -> index_read_map()
storage/myisam/ha_myisam.h:
index_read() -> index_read_map()
storage/myisam/mi_check.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_delete.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_dynrec.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_extra.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_key.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_keycache.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_locking.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_log.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_open.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_packrec.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_page.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_rkey.c:
Added comment
storage/myisam/mi_search.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_statrec.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_test1.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_test2.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_test3.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_update.c:
Fixed compiler warning (on Forte)
storage/myisam/mi_write.c:
Fixed compiler warning (on Forte)
storage/myisam/myisamdef.h:
Fixed that file_read/file_write returns type size_t
Changed some functions to use uchar * as argument/return value instead of char*
This fixed some compiler warnings on Forte
storage/myisam/myisamlog.c:
Fixed compiler warning (on Forte)
storage/myisam/myisampack.c:
Fixed compiler warning (on Forte)
storage/myisam/rt_test.c:
Fixed compiler warning (on Forte)
storage/myisam/sort.c:
Fixed compiler warning (on Forte) by adding casts or changing variables to uchar*
storage/myisam/sp_test.c:
Fixed compiler warning (on Forte) by adding casts or changing variables to uchar*
storage/myisammrg/ha_myisammrg.cc:
index_read() -> index_read_map()
storage/myisammrg/ha_myisammrg.h:
index_read() -> index_read_map()
storage/myisammrg/myrg_create.c:
Fixed compiler warning (on Forte) by adding casts or changing variable types
storage/ndb/src/kernel/blocks/dbtup/DbtupExecQuery.cpp:
Tdummy -> align (as in other part of cluster code)
storage/ndb/src/kernel/vm/DynArr256.cpp:
Removed not used variable
storage/ndb/src/ndbapi/Ndb.cpp:
Removed not used variable
strings/strtod.c:
Include ieeefp.h to avoid compiler warning
tests/bug25714.c:
Fixed compiler warning
tests/mysql_client_test.c:
Remove not used variable
Fixed indentation
Removed never reached code
Fixed compiler warning (on Forte) by adding casts or changing variable types
vio/viosocket.c:
Fixed compiler warning (on Forte) by adding casts or changing variable types
Faster thr_alarm()
Added 'Opened_files' status variable to track calls to my_open()
Don't give warnings when running mysql_install_db
Added option --source-install to mysql_install_db
I had to do the following renames() as used polymorphism didn't work with Forte compiler on 64 bit systems
index_read() -> index_read_map()
index_read_idx() -> index_read_idx_map()
index_read_last() -> index_read_last_map()
into weblab.(none):/home/marcsql/TREE/mysql-5.1-25422-d
client/mysqldump.c:
Auto merged
mysql-test/r/log_state.result:
Auto merged
mysql-test/r/show_check.result:
Auto merged
mysql-test/t/show_check.test:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/share/errmsg.txt:
Auto merged
sql/sp.cc:
Auto merged
into ibm.:/home/alik/Documents/MySQL/devel/5.1-rt-merge
mysql-test/r/mysqldump-max.result:
Auto merged
mysql-test/r/openssl_1.result:
Auto merged
mysql-test/t/show_check.test:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
client/mysqldump.c:
Manual merge.
mysql-test/r/mysqldump.result:
Manual merge + use local.
Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
a deadlock)
Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.
The root cause traces to the following code:
in sql_base.cc, open_table()
if (table->in_use != thd)
{
/* wait_for_condition will unlock LOCK_open for us */
wait_for_condition(thd, &LOCK_open, &COND_refresh);
}
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.
In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY
With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.
See WL#3984 for details related to the new locking design.
Additional changes (misc bugs exposed and fixed):
1)
mysqldump which would ignore some tables in dump_all_tables_in_db(),
but forget to ignore the same in dump_all_views_in_db().
2)
mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.
3)
Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).
4)
Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.
sql/handler.cc:
Moved logic for system / log tables in the SQL layer.
sql/handler.h:
Moved logic for system / log tables in the SQL layer.
sql/lock.cc:
Revised locking of log tables
sql/log.cc:
Major cleanup: changed how log tables are locked / written to.
sql/log.h:
Major cleanup: changed how log tables are locked / written to.
sql/mysql_priv.h:
performance schema helpers
sql/slave.cc:
open_ltable() lock flags
sql/sp.cc:
open_ltable() lock flags
sql/sql_acl.cc:
open_ltable() lock flags
sql/sql_class.h:
performance schema helpers
sql/sql_delete.cc:
log tables cleanup in TRUNCATE
sql/sql_error.cc:
Internal handlers can also intercept warnings
sql/sql_insert.cc:
open_ltable() lock flags
sql/sql_parse.cc:
performance schema helpers
sql/sql_plugin.cc:
open_ltable() lock flags
sql/sql_rename.cc:
log tables cleanup in RENAME
sql/sql_servers.cc:
open_ltable() lock flags
sql/sql_show.cc:
Move INFORMATION_SCHEMA_NAME to table.cc
sql/sql_table.cc:
log tables cleanup (admin operations, ALTER TABLE)
sql/sql_udf.cc:
open_ltable() lock flags
sql/table.cc:
Implemented TABLE_CATEGORY.
sql/share/errmsg.txt:
Changed the wording and name of ER_CANT_READ_LOCK_LOG_TABLE
sql/table.h:
Implemented TABLE_CATEGORY.
storage/csv/ha_tina.cc:
Moved logic for system / log tables in the SQL layer.
storage/csv/ha_tina.h:
Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.cc:
Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.h:
Moved logic for system / log tables in the SQL layer.
client/mysqldump.c:
Don't lock tables in the ignore list.
Don't issue empty LOCK TABLES queries.
sql/sql_base.cc:
log tables cleanup
performance schema helpers
mysql-test/r/ps.result:
Adjust test results
mysql-test/r/show_check.result:
Adjust test results
mysql-test/r/status.result:
Adjust test results
mysql-test/t/log_state.test:
Added tests for Bug#29129
mysql-test/t/ps.test:
Make the test output deterministic
mysql-test/t/show_check.test:
Make the test output deterministic
mysql-test/r/log_state.result:
Changed the default location of the log output to LOG_FILE,
for backward compatibility with MySQL 5.0
---
Adjust test results
mysql-test/r/log_tables.result:
cleanup for -ps-protocol
mysql-test/t/log_tables.test:
cleanup for -ps-protocol
sql/set_var.cc:
Changed the default location of the log output to LOG_FILE,
for backward compatibility with MySQL 5.0
---
log tables cleanup
Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
a deadlock)
Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.
The root cause traces to the following code:
in sql_base.cc, open_table()
if (table->in_use != thd)
{
/* wait_for_condition will unlock LOCK_open for us */
wait_for_condition(thd, &LOCK_open, &COND_refresh);
}
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.
In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY
With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.
See WL#3984 for details related to the new locking design.
Additional changes (misc bugs exposed and fixed):
1)
mysqldump which would ignore some tables in dump_all_tables_in_db(),
but forget to ignore the same in dump_all_views_in_db().
2)
mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.
3)
Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).
4)
Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.
mysql-test/r/innodb_mysql.result:
Update test results (merge from the main tree).
mysql-test/r/query_cache.result:
Update test results (merge from the main tree).
mysql-test/r/sp.result:
Update test results (merge from the main tree).
mysql-test/t/query_cache.test:
Use --echo End of to simplify future merges.
sql/handler.h:
st_table_list -> TABLE_LIST
sql/item_create.cc:
A post-merge fix (this code is in sql_yacc.yy in 5.0)
sql/rpl_utility.h:
st_table_list -> TABLE_LIST
sql/sp.cc:
A post-merge fix.
sql/sp_head.cc:
In 5.1 memdup_root returns void*.
sql/sql_show.cc:
st_table_list -> TABLE_LIST
sql/sql_show.h:
st_table_list -> TABLE_LIST
sql/sql_yacc.yy:
A post-merge fix.
sql/table.cc:
st_table_list -> TABLE_LIST
sql/table.h:
st_table_list -> TABLE_LIST
causes full table lock on innodb table.
Also fixes Bug#28502 Triggers that update another innodb table
will block on X lock unnecessarily (duplciate).
Code review fixes.
Both bugs' synopses are misleading: InnoDB table is
not X locked. The statements, however, cannot proceed concurrently,
but this happens due to lock conflicts for tables used in triggers,
not for the InnoDB table.
If a user had an InnoDB table, and two triggers, AFTER UPDATE and
AFTER INSERT, competing for different resources (e.g. two distinct
MyISAM tables), then these two triggers would not be able to execute
concurrently. Moreover, INSERTS/UPDATES of the InnoDB table would
not be able to run concurrently.
The problem had other side-effects (see respective bug reports).
This behavior was a consequence of a shortcoming of the pre-locking
algorithm, which would not distinguish between different DML operations
(e.g. INSERT and DELETE) and pre-lock all the tables
that are used by any trigger defined on the subject table.
The idea of the fix is to extend the pre-locking algorithm to keep track,
for each table, what DML operation it is used for and not
load triggers that are known to never be fired.
mysql-test/r/trigger-trans.result:
Update results (Bug#26141)
mysql-test/r/trigger.result:
Update results (Bug#28502)
mysql-test/t/trigger-trans.test:
Add a test case for Bug#26141 mixing table types in trigger causes
full table lock on innodb table.
mysql-test/t/trigger.test:
Add a test case for Bug#28502 Triggers that update another innodb
table will block echo on X lock unnecessarily. Add more test
coverage for triggers.
sql/item.h:
enum trg_event_type is needed in table.h
sql/sp.cc:
Take into account table_list->trg_event_map when determining
what tables to pre-lock.
After this change, if we attempt to fire a
trigger for which we had not pre-locked any tables, error
'Table was not locked with LOCK TABLES' will be printed.
This, however, should never happen, provided the pre-locking
algorithm has no programming bugs.
Previously a trigger key in the sroutines hash was based on the name
of the table the trigger belongs to. This was possible because we would
always add to the pre-locking list all the triggers defined for a table when
handling this table.
Now the key is based on the name of the trigger, owing
to the fact that a trigger name must be unique in the database it
belongs to.
sql/sp_head.cc:
Generate sroutines hash key in init_spname(). This is a convenient
place since there we have all the necessary information and can
avoid an extra alloc.
Maintain and merge trg_event_map when adding and merging elements
of the pre-locking list.
sql/sp_head.h:
Add ,m_sroutines_key member, used when inserting the sphead for a
trigger into the cache of routines used by a statement.
Previously the key was based on the table name the trigger belonged
to, since for a given table we would add to the sroutines list
all the triggers defined on it.
sql/sql_lex.cc:
Introduce a new lex step: set_trg_event_type_for_tables().
It is called when we have finished parsing but before opening
and locking tables. Now this step is used to evaluate for each
TABLE_LIST instance which INSERT/UPDATE/DELETE operation, if any,
it is used in.
In future this method could be extended to aggregate other information
that is hard to aggregate during parsing.
sql/sql_lex.h:
Add declaration for set_trg_event_type_for_tables().
sql/sql_parse.cc:
Call set_trg_event_type_for_tables() after MYSQLparse(). Remove tabs.
sql/sql_prepare.cc:
Call set_trg_event_type_for_tables() after MYSQLparse().
sql/sql_trigger.cc:
Call set_trg_event_type_for_tables() after MYSQLparse().
sql/sql_trigger.h:
Remove an obsolete member.
sql/sql_view.cc:
Call set_trg_event_type_for_tables() after MYSQLparse().
sql/sql_yacc.yy:
Move assignment of sp_head::m_type before calling sp_head::init_spname(),
one is now used inside another.
sql/table.cc:
Implement TABLE_LIST::set_trg_event_map() - a method that calculates
wh triggers may be fired on this table when executing a statement.
sql/table.h:
Add missing declarations.
Move declaration of trg_event_type from item.h (it will be needed for
trg_event_map bitmap when we start using Bitmap template instead
of uint8).
causes full table lock on innodb table.
Also fixes Bug#28502 Triggers that update another innodb table
will block on X lock unnecessarily (duplciate).
Code review fixes.
Both bugs' synopses are misleading: InnoDB table is
not X locked. The statements, however, cannot proceed concurrently,
but this happens due to lock conflicts for tables used in triggers,
not for the InnoDB table.
If a user had an InnoDB table, and two triggers, AFTER UPDATE and
AFTER INSERT, competing for different resources (e.g. two distinct
MyISAM tables), then these two triggers would not be able to execute
concurrently. Moreover, INSERTS/UPDATES of the InnoDB table would
not be able to run concurrently.
The problem had other side-effects (see respective bug reports).
This behavior was a consequence of a shortcoming of the pre-locking
algorithm, which would not distinguish between different DML operations
(e.g. INSERT and DELETE) and pre-lock all the tables
that are used by any trigger defined on the subject table.
The idea of the fix is to extend the pre-locking algorithm to keep track,
for each table, what DML operation it is used for and not
load triggers that are known to never be fired.
'No database selected' is reported when calling stored procedures
Remove the offending warning introduced by the fix for Bug
25082
This minimal patch relies on the intrinsic knowledge of the fact that
mysql_change_db is never called with 'force_switch' set to TRUE
when such a warning may be needed:
* every stored routine belongs to a database (unlike, e.g., a
user defined function, which does not), so if we're activating the
database of a stored routine, it can never be NULL.
Therefore, this branch is never called for activation.
* if we're restoring the 'old' current database after routine
execution is complete, we should not issue a warning, since it's OK to
call a routine without having previously selected the current database.
TODO: 'force_switch' is an ambiguous flag, since we do not actually
have to 'force' the switch in case of stored routines at all.
When we activate the routine's database, we should perform
all the checks as in case of 'use db', and so we already do (in this
case 'force_switch' is unused).
When we load a routine into cache, we should not use mysql_change_db
at all, since there it's enough to call thd->reset_db(). We
do it this way for triggers, but code for routines is different (wrongly).
TODO: bugs are lurking in replication, since it bypasses mysql_change_db
and calls thd->[re_]set_db to set the current database.
The latter does not change thd->db_charset, thd->sctx->db_access
and thd->variables.collation_database (and this may have nasty side
effects).
These todo items are to be addressed in a separate patch, if at all.
mysql-test/r/sp.result:
Update results (Bug#28551)
mysql-test/t/sp.test:
Add a test case (Bug#28551)
sql/sp.cc:
Remove an obsolete comment.
Replace a check with an assert.
sql/sql_db.cc:
Remove the offending warning introduced by the fix for Bug
25082
This minimal patch relies on the intrinsic knowledge of the fact that
mysql_change_db is never called with 'force_switch' set to TRUE
when such a warning may be needed.
'No database selected' is reported when calling stored procedures
Remove the offending warning introduced by the fix for Bug
25082
This minimal patch relies on the intrinsic knowledge of the fact that
mysql_change_db is never called with 'force_switch' set to TRUE
when such a warning may be needed:
* every stored routine belongs to a database (unlike, e.g., a
user defined function, which does not), so if we're activating the
database of a stored routine, it can never be NULL.
Therefore, this branch is never called for activation.
* if we're restoring the 'old' current database after routine
execution is complete, we should not issue a warning, since it's OK to
call a routine without having previously selected the current database.
TODO: 'force_switch' is an ambiguous flag, since we do not actually
have to 'force' the switch in case of stored routines at all.
When we activate the routine's database, we should perform
all the checks as in case of 'use db', and so we already do (in this
case 'force_switch' is unused).
When we load a routine into cache, we should not use mysql_change_db
at all, since there it's enough to call thd->reset_db(). We
do it this way for triggers, but code for routines is different (wrongly).
TODO: bugs are lurking in replication, since it bypasses mysql_change_db
and calls thd->[re_]set_db to set the current database.
The latter does not change thd->db_charset, thd->sctx->db_access
and thd->variables.collation_database (and this may have nasty side
effects).
These todo items are to be addressed in a separate patch, if at all.
into bodhi.(none):/opt/local/work/mysql-5.1-runtime
mysql-test/r/create.result:
Auto merged
mysql-test/r/events_bugs.result:
Auto merged
sql/event_data_objects.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/table.cc:
Auto merged
- BUG#11986: Stored routines and triggers can fail if the code
has a non-ascii symbol
- BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
- BUG#19443: INFORMATION_SCHEMA does not support charsets properly
- BUG#21249: Character set of SP-var can be ignored
- BUG#25212: Character set of string constant is ignored (stored routines)
- BUG#25221: Character set of string constant is ignored (triggers)
There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
definition;
1. No query-definition-character set.
In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.
The context contains the following data:
- client character set;
- connection collation (character set and collation);
- collation of the owner database;
The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).
2. Wrong mysqldump-output.
The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.
Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).
The solution is
- to store definition queries in the original character set;
- to change SHOW CREATE statement to output definition query in the
binary character set (i.e. without any conversion);
- introduce SHOW CREATE TRIGGER statement;
- to dump special statements to switch the context to the original one
before dumping and restore it afterwards.
Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.
3. INFORMATION_SCHEMA showed non-UTF8 strings
The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.
Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.
This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object. Specialized SHOW CREATE statements should be
used for this.
The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).
Example:
- original query:
CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;
- UTF8 query (for INFORMATION_SCHEMA):
CREATE VIEW v1 AS SELECT 'Hello' AS c1;
client/mysqldump.c:
Set original character set and collation before dumping definition query.
include/my_sys.h:
Move out-parameter to the end of list.
mysql-test/lib/mtr_report.pl:
Ignore server-warnings during the test case.
mysql-test/r/create.result:
Update result file.
mysql-test/r/ctype_cp932_binlog_stm.result:
Update result file.
mysql-test/r/events.result:
Update result file.
mysql-test/r/events_bugs.result:
Update result file.
mysql-test/r/events_grant.result:
Update result file.
mysql-test/r/func_in.result:
Update result file.
mysql-test/r/gis.result:
Update result file.
mysql-test/r/grant.result:
Update result file.
mysql-test/r/information_schema.result:
Update result file.
mysql-test/r/information_schema_db.result:
Update result file.
mysql-test/r/lowercase_view.result:
Update result file.
mysql-test/r/mysqldump.result:
Update result file.
mysql-test/r/ndb_sp.result:
Update result file.
mysql-test/r/ps.result:
Update result file.
mysql-test/r/rpl_replicate_do.result:
Update result file.
mysql-test/r/rpl_sp.result:
Update result file.
mysql-test/r/rpl_trigger.result:
Update result file.
mysql-test/r/rpl_view.result:
Update result file.
mysql-test/r/show_check.result:
Update result file.
mysql-test/r/skip_grants.result:
Update result file.
mysql-test/r/sp-destruct.result:
Update result file.
mysql-test/r/sp-error.result:
Update result file.
mysql-test/r/sp-security.result:
Update result file.
mysql-test/r/sp.result:
Update result file.
mysql-test/r/sql_mode.result:
Update result file.
mysql-test/r/system_mysql_db.result:
Update result file.
mysql-test/r/temp_table.result:
Update result file.
mysql-test/r/trigger-compat.result:
Update result file.
mysql-test/r/trigger-grant.result:
Update result file.
mysql-test/r/trigger.result:
Update result file.
mysql-test/r/view.result:
Update result file.
mysql-test/r/view_grant.result:
Update result file.
mysql-test/t/events.test:
Update test case (new columns added).
mysql-test/t/information_schema.test:
Update test case (new columns added).
mysql-test/t/show_check.test:
Test case for SHOW CREATE TRIGGER in prepared statements and
stored routines.
mysql-test/t/sp-destruct.test:
Update test case (new columns added).
mysql-test/t/sp.test:
Update test case (new columns added).
mysql-test/t/view.test:
Update test.
mysys/charset.c:
Move out-parameter to the end of list.
scripts/mysql_system_tables.sql:
Add new columns to mysql.proc and mysql.event.
scripts/mysql_system_tables_fix.sql:
Add new columns to mysql.proc and mysql.event.
sql/event_data_objects.cc:
Support new attributes for events.
sql/event_data_objects.h:
Support new attributes for events.
sql/event_db_repository.cc:
Support new attributes for events.
sql/event_db_repository.h:
Support new attributes for events.
sql/events.cc:
Add new columns to SHOW CREATE event resultset.
sql/mysql_priv.h:
1. Introduce Object_creation_ctx;
2. Introduce SHOW CREATE TRIGGER;
3. Introduce auxilary functions.
sql/sp.cc:
Add support for new store routines attributes.
sql/sp_head.cc:
Add support for new store routines attributes.
sql/sp_head.h:
Add support for new store routines attributes.
sql/sql_lex.cc:
Generate UTF8-body on parsing/lexing.
sql/sql_lex.h:
1. Generate UTF8-body on parsing/lexing.
2. Introduce SHOW CREATE TRIGGER.
sql/sql_parse.cc:
Introduce SHOW CREATE TRIGGER.
sql/sql_partition.cc:
Update parse_sql().
sql/sql_prepare.cc:
Update parse_sql().
sql/sql_show.cc:
Support new attributes for views
sql/sql_trigger.cc:
Support new attributes for views
sql/sql_trigger.h:
Support new attributes for views
sql/sql_view.cc:
Support new attributes for views
sql/sql_yacc.yy:
1. Add SHOW CREATE TRIGGER statement.
2. Generate UTF8-body for views, stored routines, triggers and events.
sql/table.cc:
Introduce Object_creation_ctx.
sql/table.h:
Introduce Object_creation_ctx.
sql/share/errmsg.txt:
Add new errors.
mysql-test/include/ddl_i18n.check_events.inc:
Aux file for test suite.
mysql-test/include/ddl_i18n.check_sp.inc:
Aux file for test suite.
mysql-test/include/ddl_i18n.check_triggers.inc:
Aux file for test suite.
mysql-test/include/ddl_i18n.check_views.inc:
Aux file for test suite.
mysql-test/include/have_cp1251.inc:
Aux file for test suite.
mysql-test/include/have_cp866.inc:
Aux file for test suite.
mysql-test/include/have_koi8r.inc:
Aux file for test suite.
mysql-test/include/have_utf8.inc:
Aux file for test suite.
mysql-test/r/ddl_i18n_koi8r.result:
Result file.
mysql-test/r/ddl_i18n_utf8.result:
Result file.
mysql-test/r/have_cp1251.require:
Aux file for test suite.
mysql-test/r/have_cp866.require:
Aux file for test suite.
mysql-test/r/have_koi8r.require:
Aux file for test suite.
mysql-test/r/have_utf8.require:
Aux file for test suite.
mysql-test/t/ddl_i18n_koi8r.test:
Complete koi8r test case for the CS patch.
mysql-test/t/ddl_i18n_utf8.test:
Complete utf8 test case for the CS patch.
- BUG#11986: Stored routines and triggers can fail if the code
has a non-ascii symbol
- BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
- BUG#19443: INFORMATION_SCHEMA does not support charsets properly
- BUG#21249: Character set of SP-var can be ignored
- BUG#25212: Character set of string constant is ignored (stored routines)
- BUG#25221: Character set of string constant is ignored (triggers)
There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
definition;
1. No query-definition-character set.
In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.
The context contains the following data:
- client character set;
- connection collation (character set and collation);
- collation of the owner database;
The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).
2. Wrong mysqldump-output.
The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.
Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).
The solution is
- to store definition queries in the original character set;
- to change SHOW CREATE statement to output definition query in the
binary character set (i.e. without any conversion);
- introduce SHOW CREATE TRIGGER statement;
- to dump special statements to switch the context to the original one
before dumping and restore it afterwards.
Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.
3. INFORMATION_SCHEMA showed non-UTF8 strings
The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.
Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.
This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object. Specialized SHOW CREATE statements should be
used for this.
The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).
Example:
- original query:
CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;
- UTF8 query (for INFORMATION_SCHEMA):
CREATE VIEW v1 AS SELECT 'Hello' AS c1;
Sometimes the number of really updated rows (with changed
column values) cannot be determined at the server level
alone (e.g. if the storage engine does not return enough
column values to verify that). So the only dependable way
in such cases is to let the storage engine return that
information if possible.
Fixed the bug at server level by providing a way for the
storage engine to return information about wether it
actually updated the row or the old and the new column
values are the same. It can do that by returning
HA_ERR_RECORD_IS_THE_SAME in ha_update_row().
Note that each storage engine may choose not to try to
return this status code, so this behaviour remains
storage engine specific.
include/my_base.h:
Bug #29157: handle the row not updated special return value
sql/log_event.cc:
Bug #29157: handle the row not updated special return value
sql/sp.cc:
Bug #29157: handle the row not updated special return value
sql/sql_acl.cc:
Bug #29157: handle the row not updated special return value
sql/sql_insert.cc:
Bug #29157: handle the row not updated special return value
sql/sql_servers.cc:
Bug #29157: handle the row not updated special return value
sql/sql_update.cc:
Bug #29157: handle the row not updated special return value
Sometimes the number of really updated rows (with changed
column values) cannot be determined at the server level
alone (e.g. if the storage engine does not return enough
column values to verify that). So the only dependable way
in such cases is to let the storage engine return that
information if possible.
Fixed the bug at server level by providing a way for the
storage engine to return information about wether it
actually updated the row or the old and the new column
values are the same. It can do that by returning
HA_ERR_RECORD_IS_THE_SAME in ha_update_row().
Note that each storage engine may choose not to try to
return this status code, so this behaviour remains
storage engine specific.
into gleb.loc:/home/uchum/work/bk/5.1-opt
sql/item.h:
Auto merged
sql/log_event.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_trigger.cc:
Auto merged
sql/sql_view.cc:
Auto merged
mysql-test/r/rpl_change_master.result:
Merge with 5.1.
mysql-test/t/rpl_change_master.test:
Merge with 5.1.
sql/sql_acl.cc:
Merge with 5.1.
CREATE/DROP TEMPORARY TABLE + ROLLBACK on master
The transaction ability of the storage engines of
the tables on the replication master and the replication
slave must generally be the same.
When the storage engine type of the slave is
non-transactional then transactions on the master that
mix update of transactional and non-transactional tables
should be avoided because they will cause inconsistency of
the data between the master's transactional table and the
slave's non-transactional table.
The effect described by this bug is actually expected.
A detailed test case is added (to be merged later to
the updated rpl_ddl.test), as there was no coverage
by the existing tests.
Some code cleanup is also added by this change.
mysql-test/r/rpl_innodb.result:
Bug #26418: test case
mysql-test/t/rpl_innodb.test:
Bug #26418: test case
sql/events.cc:
Bug #26418: replace repeating code with a function call
sql/sp.cc:
Bug #26418: replace repeating code with a function call
sql/sql_acl.cc:
Bug #26418: replace repeating code with a function call
sql/sql_class.cc:
Bug #26418: remove dead code
sql/sql_class.h:
Bug #26418: remove dead code
sql/sql_delete.cc:
Bug #26418: replace repeating code with a function call
sql/sql_parse.cc:
Bug #26418: replace repeating code with a function call
sql/sql_rename.cc:
Bug #26418: replace repeating code with a function call
sql/sql_tablespace.cc:
Bug #26418: replace repeating code with a function call
sql/sql_trigger.cc:
Bug #26418: replace repeating code with a function call
sql/sql_udf.cc:
Bug #26418: replace repeating code with a function call
sql/sql_view.cc:
Bug #26418: replace repeating code with a function call
CREATE/DROP TEMPORARY TABLE + ROLLBACK on master
The transaction ability of the storage engines of
the tables on the replication master and the replication
slave must generally be the same.
When the storage engine type of the slave is
non-transactional then transactions on the master that
mix update of transactional and non-transactional tables
should be avoided because they will cause inconsistency of
the data between the master's transactional table and the
slave's non-transactional table.
The effect described by this bug is actually expected.
A detailed test case is added (to be merged later to
the updated rpl_ddl.test), as there was no coverage
by the existing tests.
Some code cleanup is also added by this change.
sql/handler.cc:
Polishing to have the consistent code.
sql/sp_head.cc:
Polishing to have the consistent code.
sql/sql_error.cc:
Polishing to have the consistent code.
sql/sql_trigger.cc:
Polishing to have the consistent code.
1. Introduce parse_sql() as a high-level replacement for MYSQLparse().
parse_sql() is responsible to switch and restore "parser context"
(THD::m_lip for now).
2. Fix typo in sp.cc: THD::spcont should be reset *before* calling
the parser.
sql/event_data_objects.cc:
Use parse_sql() instead of MYSQLparse().
sql/mysql_priv.h:
Introduce parse_sql() instead of auto-generated MYSQLparse.
sql/sp.cc:
1. Use parse_sql() instead of MYSQLparse().
2. THD::spcont should be reset before calling the parser.
sql/sql_class.cc:
Reset THD::m_lip.
sql/sql_parse.cc:
1. Introduce parse_sql() instead of auto-generated MYSQLparse().
2. Backup, switch and restore THD::m_lip inside parse_sql().
3. Use parse_sql() instead of MYSQLparse().
sql/sql_partition.cc:
Use parse_sql() instead of MYSQLparse().
sql/sql_prepare.cc:
Use parse_sql() instead of MYSQLparse().
sql/sql_trigger.cc:
Use parse_sql() instead of MYSQLparse().
sql/sql_view.cc:
Use parse_sql() instead of MYSQLparse().
1. Introduce parse_sql() as a high-level replacement for MYSQLparse().
parse_sql() is responsible to switch and restore "parser context"
(THD::m_lip for now).
2. Fix typo in sp.cc: THD::spcont should be reset *before* calling
the parser.
Bug 28127 (Some valid identifiers names are not parsed correctly)
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)
This patch is the second part of a major cleanup, required to fix
Bug 25411 (trigger code truncated).
The root cause of the issue stems from the function skip_rear_comments,
which was a work around to remove "extra" "*/" characters from the query
text, when parsing a query and reusing the text fragments to represent a
view, trigger, function or stored procedure.
The reason for this work around is that "special comments",
like /*!50002 XXX */, were not parsed properly, so that a query like:
AAA /*!50002 BBB */ CCC
would be seen by the parser as "AAA BBB */ CCC" when the current version
is greater or equal to 5.0.2
The root cause of this stems from how special comments are parsed.
Special comments are really out-of-bound text that appear inside a query,
that affects how the parser behave.
In nature, /*!50002 XXX */ in MySQL is similar to the C concept
of preprocessing :
#if VERSION >= 50002
XXX
#endif
Depending on the current VERSION of the server, either the special comment
should be expanded or it should be ignored, but in all cases the "text" of
the query should be re-written to strip the "/*!50002" and "*/" markers,
which does not belong to the SQL language itself.
Prior to this fix, these markers would leak into :
- the storage format for VIEW,
- the storage format for FUNCTION,
- the storage format for FUNCTION parameters, in mysql.proc (param_list),
- the storage format for PROCEDURE,
- the storage format for PROCEDURE parameters, in mysql.proc (param_list),
- the storage format for TRIGGER,
- the binary log used for replication.
In all cases, not only this cause format corruption, but also provide a vector
for dormant security issues, by allowing to tunnel code that will be activated
after an upgrade.
The proper solution is to deal with special comments strictly during parsing,
when accepting a query from the outside world.
Once a query is parsed and an object is created with a persistant
representation, this object should not arbitrarily mutate after an upgrade.
In short, special comments are a useful but limited feature for MYSQLdump,
when used at an *interface* level to facilitate import/export,
but bloating the server *internal* storage format is *not* the proper way
to deal with configuration management of the user logic.
With this fix:
- the Lex_input_stream class now acts as a comment pre-processor,
and either expands or ignore special comments on the fly.
- MYSQLlex and sql_yacc.yy have been cleaned up to strictly use the
public interface of Lex_input_stream. In particular, how the input stream
accepts or rejects a character is private to Lex_input_stream, and the
internal buffer pointers of that class are strictly private, and should not
be tempered with during parsing.
This caused many changes mostly in sql_lex.cc.
During the code cleanup in case MY_LEX_NUMBER_IDENT,
Bug 28127 (Some valid identifiers names are not parsed correctly)
was found and fixed.
By parsing special comments properly, and removing the function
'skip_rear_comments' [sic],
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)
has been fixed as well.
sql/event_data_objects.cc:
Cleanup of the code that extracts the query text
sql/sp.cc:
Cleanup of the code that extracts the query text
sql/sp_head.cc:
Cleanup of the code that extracts the query text
sql/sql_trigger.cc:
Cleanup of the code that extracts the query text
sql/sql_view.cc:
Cleanup of the code that extracts the query text
mysql-test/r/comments.result:
Bug#25411 (trigger code truncated)
mysql-test/r/sp.result:
Bug#25411 (trigger code truncated)
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)
mysql-test/r/trigger.result:
Bug#25411 (trigger code truncated)
mysql-test/r/varbinary.result:
Bug 28127 (Some valid identifiers names are not parsed correctly)
mysql-test/t/comments.test:
Bug#25411 (trigger code truncated)
mysql-test/t/sp.test:
Bug#25411 (trigger code truncated)
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)
mysql-test/t/trigger.test:
Bug#25411 (trigger code truncated)
mysql-test/t/varbinary.test:
Bug 28127 (Some valid identifiers names are not parsed correctly)
sql/sql_lex.cc:
Implemented comment pre-processing in Lex_input_stream,
major cleanup of the lex/yacc code to not use Lex_input_stream private members.
sql/sql_lex.h:
Implemented comment pre-processing in Lex_input_stream,
major cleanup of the lex/yacc code to not use Lex_input_stream private members.
sql/sql_yacc.yy:
post merge fix : view_check_options must be parsed before signaling the end of the query
Bug 28127 (Some valid identifiers names are not parsed correctly)
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)
This patch is the second part of a major cleanup, required to fix
Bug 25411 (trigger code truncated).
The root cause of the issue stems from the function skip_rear_comments,
which was a work around to remove "extra" "*/" characters from the query
text, when parsing a query and reusing the text fragments to represent a
view, trigger, function or stored procedure.
The reason for this work around is that "special comments",
like /*!50002 XXX */, were not parsed properly, so that a query like:
AAA /*!50002 BBB */ CCC
would be seen by the parser as "AAA BBB */ CCC" when the current version
is greater or equal to 5.0.2
The root cause of this stems from how special comments are parsed.
Special comments are really out-of-bound text that appear inside a query,
that affects how the parser behave.
In nature, /*!50002 XXX */ in MySQL is similar to the C concept
of preprocessing :
#if VERSION >= 50002
XXX
#endif
Depending on the current VERSION of the server, either the special comment
should be expanded or it should be ignored, but in all cases the "text" of
the query should be re-written to strip the "/*!50002" and "*/" markers,
which does not belong to the SQL language itself.
Prior to this fix, these markers would leak into :
- the storage format for VIEW,
- the storage format for FUNCTION,
- the storage format for FUNCTION parameters, in mysql.proc (param_list),
- the storage format for PROCEDURE,
- the storage format for PROCEDURE parameters, in mysql.proc (param_list),
- the storage format for TRIGGER,
- the binary log used for replication.
In all cases, not only this cause format corruption, but also provide a vector
for dormant security issues, by allowing to tunnel code that will be activated
after an upgrade.
The proper solution is to deal with special comments strictly during parsing,
when accepting a query from the outside world.
Once a query is parsed and an object is created with a persistant
representation, this object should not arbitrarily mutate after an upgrade.
In short, special comments are a useful but limited feature for MYSQLdump,
when used at an *interface* level to facilitate import/export,
but bloating the server *internal* storage format is *not* the proper way
to deal with configuration management of the user logic.
With this fix:
- the Lex_input_stream class now acts as a comment pre-processor,
and either expands or ignore special comments on the fly.
- MYSQLlex and sql_yacc.yy have been cleaned up to strictly use the
public interface of Lex_input_stream. In particular, how the input stream
accepts or rejects a character is private to Lex_input_stream, and the
internal buffer pointers of that class are strictly private, and should not
be tempered with during parsing.
This caused many changes mostly in sql_lex.cc.
During the code cleanup in case MY_LEX_NUMBER_IDENT,
Bug 28127 (Some valid identifiers names are not parsed correctly)
was found and fixed.
By parsing special comments properly, and removing the function
'skip_rear_comments' [sic],
Bug 26302 (MySQL server cuts off trailing "*/" from comments in SP/func)
has been fixed as well.
into bodhi.(none):/opt/local/work/mysql-5.1-runtime
client/mysqlbinlog.cc:
Auto merged
include/config-win.h:
Auto merged
include/my_global.h:
Auto merged
include/my_pthread.h:
Auto merged
mysql-test/lib/mtr_report.pl:
Auto merged
mysql-test/t/disabled.def:
Auto merged
sql/event_data_objects.cc:
Auto merged
sql/event_queue.cc:
Auto merged
sql/field.h:
Auto merged
sql/filesort.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/handler.h:
Auto merged
sql/item_func.h:
Auto merged
sql/item_xmlfunc.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sp.h:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sp_head.h:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_list.h:
Auto merged
sql/sql_load.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_partition.cc:
Auto merged
sql/sql_prepare.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_test.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/sql_view.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/heap/hp_hash.c:
Auto merged
tests/mysql_client_test.c:
Auto merged
sql/item_func.cc:
Manual merge.
sql/sp.cc:
Manual merge.
sql/sql_cache.cc:
Manual merge.
sql/sql_table.cc:
Manual merge.
strings/my_vsnprintf.c:
Manual merge.
