Background:
- as described in MySQL Internals Prepared Stored
(http://forge.mysql.com/wiki/MySQL_Internals_Prepared_Stored),
the Optimizer sometimes does destructive changes to the parsed
LEX-object (Item-tree), which makes it impossible to re-use
that tree for PS/SP re-execution.
- in order to be able to re-use the Item-tree, the destructive
changes are remembered and rolled back after the statement execution.
The problem, discovered by this bug, was that the objects representing
GROUP-BY clause did not restored after query execution. So, the GROUP-BY
part of the statement could not be properly re-initialized for re-execution
after destructive changes.
Those objects do not take part in the Item-tree, so they can not be saved
using the approach for Item-tree.
The fix is as follows:
- introduce a new array in st_select_lex to store the original
ORDER pointers, representing the GROUP-BY clause;
- Initialize this array in fix_prepare_information().
- restore the list of GROUP-BY items in reinit_stmt_before_use().
mysql-test/suite/innodb/t/group_commit_crash.test:
remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
my_vsnprintf() is ok here, in 5.5
Problem: Statements that write to tables with auto_increment columns
based on the selection from another table, may lead to master
and slave going out of sync, as the order in which the rows
are retrieved from the table may differ on master and slave.
Solution: We mark writing to a table with auto_increment table
based on the rows selected from another table as unsafe. This
will cause the execution of such statements to throw a warning
and forces the statement to be logged in ROW if the logging
format is mixed.
Changes:
1. All the statements that writes to a table with auto_increment
column(s) based on the rows fetched from another table, will now
be unsafe.
2. CREATE TABLE with SELECT will now be unsafe.
sql/share/errmsg-utf8.txt:
Added new warning messages.
sql/sql_base.cc:
-Created function to check statements that write to
tables with auto_increment column and has select.
-Marked all the statements that write to a table
with auto_increment column based on rows fetched
from other table(s) as unsafe.
sql/sql_table.cc:
mark CREATE TABLE[with auto_increment column] as unsafe.
Problem: Statements that write to tables with auto_increment columns
based on the selection from another table, may lead to master
and slave going out of sync, as the order in which the rows
are retrived from the table may differ on master and slave.
Solution: We mark writing to a table with auto_increment table
as unsafe. This will cause the execution of such statements to
throw a warning and forces the statement to be logged in ROW if
the logging format is mixed.
Changes:
1. All the statements that writes to a table with auto_increment
column(s) based on the rows fetched from another table, will now
be unsafe.
2. CREATE TABLE with SELECT will now be unsafe.
sql/share/errmsg-utf8.txt:
Added new Warning messages
sql/sql_base.cc:
created a new function that checks for select + write on a autoinc table
made all such statements to be unsafe.
sql/sql_parse.cc:
made create autoincremnet tabble + select unsafe
A non-first execution of a prepared statement missed a call of the
TABLE_LIST::process_index_hints() method in the code of the function
setup_tables().
At some scenarios this could lead to the choice of a quite inefficient
execution plan for the base query of the prepared statement.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
A lot of small fixes and new test cases.
client/mysqlbinlog.cc:
Cast removed
client/mysqltest.cc:
Added missing DBUG_RETURN
include/my_pthread.h:
set_timespec_time_nsec() now only takes one argument
mysql-test/t/date_formats.test:
Remove --disable_ps_protocl as now also ps supports microseconds
mysys/my_uuid.c:
Changed to use my_interval_timer() instead of my_getsystime()
mysys/waiting_threads.c:
Changed to use my_hrtime()
sql/field.h:
Added bool special_const_compare() for fields that may convert values before compare (like year)
sql/field_conv.cc:
Added test to get optimal copying of identical temporal values.
sql/item.cc:
Return that item_int is equal if it's positive, even if unsigned flag is different.
Fixed Item_cache_str::save_in_field() to have identical null check as other similar functions
Added proper NULL check to Item_cache_int::save_in_field()
sql/item_cmpfunc.cc:
Don't call convert_constant_item() if there is nothing that is worth converting.
Simplified test when years should be converted
sql/item_sum.cc:
Mark cache values in Item_sum_hybrid as not constants to ensure they are not replaced by other cache values in compare_datetime()
sql/item_timefunc.cc:
Changed sec_to_time() to take a my_decimal argument to ensure we don't loose any sub seconds.
Added Item_temporal_func::get_time() (This simplifies some things)
sql/mysql_priv.h:
Added Lazy_string_decimal()
sql/mysqld.cc:
Added my_decimal constants max_seconds_for_time_type, time_second_part_factor
sql/table.cc:
Changed expr_arena to be of type CONVENTIONAL_EXECUTION to ensure that we don't loose any items that are created by fix_fields()
sql/tztime.cc:
TIME_to_gmt_sec() now sets *in_dst_time_gap in case of errors
This is needed to be able to detect if timestamp is 0
storage/maria/lockman.c:
Changed from my_getsystime() to set_timespec_time_nsec()
storage/maria/ma_loghandler.c:
Changed from my_getsystime() to my_hrtime()
storage/maria/ma_recovery.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/maria/unittest/trnman-t.c:
Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/xtradb/handler/ha_innodb.cc:
Added support for new time,datetime and timestamp
unittest/mysys/thr_template.c:
my_getsystime() -> my_interval_timer()
unittest/mysys/waiting_threads-t.c:
my_getsystime() -> my_interval_timer()
prepared statements with cursor protocol).
The problem was a bug in Materialized-cursor implementation.
Materialized_cursor::open() called send_result_metadata()
with items pointing to already closed table.
The fix is to send metadata when the table is still open.
NOTE: this is a "partial" fix: metadata are different with
and without --cursor-protocol, but that's a different large
problem, one indication of which is reported as Bug 24176.
and collateral changes.
* introduce my_hrtime_t, my_timediff_t, and conversion macros
* inroduce TIME_RESULT, but it can only be returned from Item::cmp_type(),
never from Item::result_type()
* pack_time/unpack_time function for "packed" representation of
MYSQL_TIME in a longlong that can be compared
* ADDTIME()/SUBTIME()/+- INTERVAL now work with TIME values
* numbers aren't quoted in EXPLAIN EXTENDED
* new column I_S.COLUMNS.DATETIME_PRECISION
* date/time values are compares to anything as date/time, not as strings or numbers.
* old timestamp(X) is no longer supported
* MYSQL_TIME to string conversion functions take precision as an argument
* unified the warnings from Field_timestamp/datetime/time/date/newdate store methods
* Field_timestamp_hires, Field_datetime_hires, Field_time_hires
* Field_temporal
* Lazy_string class to pass a value (string, number, time) polymorphically down the stack
* make_truncated_value_warning and Field::set_datetime_warning use Lazy_string as an argument, removed char*/int/double variants
* removed Field::can_be_compared_as_longlong(). Use Field::cmp_type() == INT_RESULT instead
* introduced Item::cmp_result() instead of Item::is_datetime() and Item::result_as_longlong()
* in many cases date/time types are treated like other types, not as special cases
* greatly simplified Arg_comparator (regarding date/time/year code)
* SEC_TO_TIME is real function, not integer.
* microsecond precision in NOW, CURTIME, etc
* Item_temporal. All items derived from it only provide get_date, but no val* methods
* replication of NOW(6)
* Protocol::store(time) now takes the precision as an argument
* @@TIMESTAMP is a double
client/mysqlbinlog.cc:
remove unneded casts
include/my_sys.h:
introduce my_hrtime_t, my_timediff_t, and conversion macros
include/my_time.h:
pack_time/unpack_time, etc.
convenience functions to work with MYSQL_TIME::second_part
libmysql/libmysql.c:
str_to_time() is gone. str_to_datetime() does it now.
my_TIME_to_str() takes the precision as an argument
mysql-test/include/ps_conv.inc:
time is not equal to datetime anymore
mysql-test/r/distinct.result:
a test for an old MySQL bug
mysql-test/r/explain.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/func_default.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/func_sapdb.result:
when decimals=NOT_FIXED_DEC it means "not fixed" indeed
mysql-test/r/func_test.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/func_time.result:
ADDTIME()/SUBTIME()/+- INTERVAL now work with TIME values
mysql-test/r/having.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/information_schema.result:
new column I_S.COLUMNS.DATETIME_PRECISION
mysql-test/r/join_outer.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/metadata.result:
TIMESTAMP no longer has zerofill flag
mysql-test/r/range.result:
invalid datetime is not compared with as a string
mysql-test/r/select.result:
NO_ZERO_IN_DATE, etc only affect storage - according to the manual
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/subselect.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/sysdate_is_now.result:
when decimals=NOT_FIXED_DEC it means "not fixed" indeed
mysql-test/r/type_blob.result:
TIMESTAMP(N) is not deprecated
mysql-test/r/type_timestamp.result:
old TIMESTAMP(X) semantics is not supported anymore
mysql-test/r/union.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/r/varbinary.result:
numbers aren't quoted in EXPLAIN EXTENDED
mysql-test/t/distinct.test:
test for an old MySQL bug
mysql-test/t/func_time.test:
+- INTERVAL now works with TIME values
mysql-test/t/select.test:
typo
mysql-test/t/subselect.test:
only one error per statement, please
mysql-test/t/system_mysql_db_fix40123.test:
old timestamp(X) is no longer supported
mysql-test/t/system_mysql_db_fix50030.test:
old timestamp(X) is no longer supported
mysql-test/t/system_mysql_db_fix50117.test:
old timestamp(X) is no longer supported
mysql-test/t/type_blob.test:
old timestamp(X) is no longer supported
mysql-test/t/type_timestamp.test:
old timestamp(X) is no longer supported
mysys/my_getsystime.c:
functions to get the time with microsecond precision
mysys/my_init.c:
move the my_getsystime.c initialization code to my_getsystime.c
mysys/my_static.c:
no need to make these variables extern
mysys/my_static.h:
no need to make these variables extern
scripts/mysql_system_tables.sql:
old timestamp(X) is no longer supported
scripts/mysql_system_tables_fix.sql:
old timestamp(X) is no longer supported
scripts/mysqlhotcopy.sh:
old timestamp(X) is no longer supported
sql-common/my_time.c:
* call str_to_time from str_to_datetime, as appropriate
* date/time to string conversions take precision as an argument
* number_to_time()
* TIME_to_double()
* pack_time() and unpack_time()
sql/event_data_objects.cc:
cast is not needed
my_datetime_to_str() takes precision as an argument
sql/event_db_repository.cc:
avoid dangerous downcast (because the pointer is
not always Field_timestamp, see events_1.test)
sql/event_queue.cc:
avoid silly double-work for cond_wait
(having an endpoint of wait, subtract the current time to get the timeout,
and use set_timespec() macro to fill in struct timespec, by adding the current
time to the timeout)
sql/field.cc:
* remove virtual Field::get_time(), everyone should use only Field::get_date()
* remove lots of #ifdef WORDS_BIGENDIAN
* unified the warnings from Field_timestamp/datetime/time/date/newdate store methods
* Field_timestamp_hires, Field_datetime_hires, Field_time_hires
* Field_temporal
* make_truncated_value_warning and Field::set_datetime_warning use Lazy_string as an argument, removed char*/int/double variants
sql/field.h:
* remove virtual Field::get_time(), everyone should use only Field::get_date()
* remove lots of #ifdef WORDS_BIGENDIAN
* unified the warnings from Field_timestamp/datetime/time/date/newdate store methods
* Field_timestamp_hires, Field_datetime_hires, Field_time_hires
* Field_temporal
* make_truncated_value_warning and Field::set_datetime_warning use Lazy_string as an argument, removed char*/int/double variants
* removed Field::can_be_compared_as_longlong(). Use Field::cmp_type() == INT_RESULT instead
sql/filesort.cc:
TIME_RESULT, cmp_time()
sql/item.cc:
* numbers aren't quoted in EXPLAIN EXTENDED
* Item::cmp_result() instead of Item::is_datetime() and Item::result_as_longlong()
* virtual Item::get_time() is gone
* Item_param::field_type() is set correctly
* Item_datetime, for a datetime constant
* time to anything is compared as a time
* Item_cache::print() prints the value is available
* bug fixed in Item_cache_int::val_str()
sql/item.h:
* Item::print_value(), to be used from Item_xxx::print() when needed
* Item::cmp_result() instead of Item::is_datetime() and Item::result_as_longlong()
* virtual Item::get_time() is gone
* Item_datetime, for a datetime constant
* better default for cast_to_int_type()
* Item_cache objects now *always* have the field_type() set
sql/item_cmpfunc.cc:
* get_year_value, get_time_value are gone. get_datetime_value does it all
* get_value_a_func, get_value_b_func are gone
* can_compare_as_dates() is gone too, TIME_RESULT is used instead
* cmp_type() instead or result_type() when doing a comparison
* compare_datetime and compate_e_datetime in the comparator_matrix, is_nulls_eq is gone
* Item::cmp_result() instead of Item::is_datetime() and Item::result_as_longlong()
sql/item_cmpfunc.h:
greatly simplified Arg_comparator
sql/item_create.cc:
* fix a bug in error messages in CAST
sql/item_func.cc:
Item::cmp_result() instead of Item::is_datetime() and Item::result_as_longlong()
mention all possibitiles in switch over Item_result values, or use default:
sql/item_row.h:
overwrite the default cmp_type() for Item_row,
as no MYSQL_TYPE_xxx value corresponds to ROW_RESULT
sql/item_timefunc.cc:
rewrite make_datetime to support precision argument
SEC_TO_TIME is real function, not integer.
many functions that returned temporal values had duplicate code in val_* methods,
some of them did not have get_date() which resulted in unnecessary date->str->date conversions.
Now they all are derived from Item_temporal_func and *only* provide get_date, not val* methods.
many fixes to set decimals (datetime precision) correctly.
sql/item_timefunc.h:
SEC_TO_TIME is real function, not integer.
many functions that returned temporal values had duplicate code in val_* methods,
some of them did not have get_date() which resulted in unnecessary date->str->date conversions.
Now they all are derived from Item_temporal_func and *only* provide get_date, not val* methods.
many fixes to set decimals (datetime precision) correctly.
sql/log_event.cc:
replication of NOW(6)
sql/log_event.h:
replication of NOW(6)
sql/mysql_priv.h:
Lazy_string class to pass a value (string, number, time) polymorphically down the stack.
make_truncated_value_warning() that uses it.
sql/mysqld.cc:
datetime in Arg_comparator::comparator_matrix
sql/opt_range.cc:
cleanup: don't disable warnings before calling save_in_field_no_warnings()
sql/protocol.cc:
Protocol::store(time) now takes the precision as an argument
sql/protocol.h:
Protocol::store(time) now takes the precision as an argument
sql/rpl_rli.cc:
small cleanup
sql/set_var.cc:
SET TIMESTAMP=double
sql/set_var.h:
@@TIMESTAMP is a double
sql/share/errmsg.txt:
precision and scale are unsigned
sql/slave.cc:
replication of NOW(6)
sql/sp_head.cc:
cleanup
sql/sql_class.cc:
support for NOW(6)
sql/sql_class.h:
support for NOW(6)
sql/sql_insert.cc:
support for NOW(6)
sql/sql_select.cc:
use item->cmp_type().
move a comment where it belongs
sql/sql_show.cc:
new column I_S.COLUMNS.DATETIME_PRECISION
sql/sql_yacc.yy:
TIME(X), DATETIME(X), cast, NOW(X), CURTIME(X), etc
sql/time.cc:
fix date_add_interval() to support MYSQL_TIMESTAMP_TIME argument
storage/myisam/ha_myisam.cc:
TIMESTAMP no longer carries ZEROFIELD flag, still we keep MYI file compatible.
strings/my_vsnprintf.c:
warnings
tests/mysql_client_test.c:
old timestamp(X) does not work anymore
datetime is no longer equal to time
- Fixed problem with oqgraph and 'make dist'
Note that after this merge we have a problem show in join_outer where we examine too many rows in one specific case (related to BUG#57024).
This will be fixed when mwl#128 is merged into 5.3.
structure buffer).
This is a follow-up for WL#4435. The bug actually existed not only
MYSQL_TYPE_DATETIME type. The problem was that Item_param::set_value()
was written in an assumption that it's working with expressions, i.e.
with basic data types.
There are two different quick fixes here:
a) Change Item_param::make_field() -- remove setting of
Send_field::length, Send_field::charsetnr, Send_field::flags and
Send_field::type.
That would lead to marshalling all data using basic types to the client
(MYSQL_TYPE_LONGLONG, MYSQL_TYPE_DOUBLE, MYSQL_TYPE_STRING and
MYSQL_TYPE_NEWDECIMAL). In particular, that means, DATETIME would be
sent as MYSQL_TYPE_STRING, TINYINT -- as MYSQL_TYPE_LONGLONG, etc.
That could be Ok for the client, because the client library does
reverse conversion automatically (the client program would see DATETIME
as MYSQL_TIME object). However, there is a problem with metadata --
the metadata would be wrong (misleading): it would say that DATETIME is
marshaled as MYSQL_TYPE_DATETIME, not as MYSQL_TYPE_STRING.
b) Set Item_param::param_type properly to actual underlying field type.
That would lead to double conversion inside the server: for example,
MYSQL_TIME-object would be converted into STRING-object
(in Item_param::set_value()), and then converted back to MYSQL_TIME-object
(in Item_param::send()).
The data however would be marshalled more properly, and also metadata would
be correct.
This patch implements b).
There is also a possibility to avoid double conversion either by clonning
the data field, or by storing a reference to it and using it on Item::send()
time. That requires more work and might be done later.
The crash happens because original join table is replaced with temporary table
at execution stage and later we attempt to use this temporary table in
select_describe. It might happen that
Item_subselect::update_used_tables() method which sets const_item flag
is not called by some reasons (no where/having conditon in subquery for example).
It prevents JOIN::join_tmp creation and breaks original join.
The fix is to call ::update_used_tables() before ::const_item() check.
mysql-test/r/ps.result:
test case
mysql-test/t/ps.test:
test case
sql/item_subselect.cc:
call ::update_used_tables() before ::const_item() check.
In case of outer join and emtpy WHERE conditon
'always true' condition is created for WHERE clasue.
Later in mysql_select() original SELECT_LEX WHERE
condition is overwritten with created cond.
However SELECT_LEX condition is also used as inital
condition in mysql_select()->JOIN::prepare().
On second execution of PS modified SELECT_LEX condition
is taken and it leads to crash.
The fix is to restore original SELECT_LEX condition
(set to NULL if original cond is NULL) in
reinit_stmt_before_use().
HAVING clause is fixed too for safety reason
(no test case as I did not manage to think out
appropriate example).
mysql-test/r/ps.result:
test case
mysql-test/t/ps.test:
test case
sql/sql_prepare.cc:
restore original SELECT_LEX condition
(set to NULL if original cond is NULL) in
reinit_stmt_before_use()
Conflicts:
Text conflict in configure.in
Text conflict in dbug/dbug.c
Text conflict in mysql-test/r/ps.result
Text conflict in mysql-test/t/ps.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/ha_ndbcluster.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_table.cc
Arg_comparator initializes 'comparators' array in case of
ROW comparison and does not free this array on destruction.
It leads to memory leaks.
The fix:
-added Arg_comparator::cleanup() method which frees
'comparators' array.
-added Item_bool_func2::cleanup() method which calls
Arg_comparator::cleanup() method
mysql-test/r/ps.result:
test case
mysql-test/r/row.result:
test case
mysql-test/t/ps.test:
test case
mysql-test/t/row.test:
test case
sql/item_cmpfunc.h:
-added Arg_comparator::cleanup() method which frees
'comparators' array.
-added Item_bool_func2::cleanup() method which calls
Arg_comparator::cleanup() method
This patch does three things:
- It adds the possibility to force the execution of top-level [NOT] IN
subquery predicates via the IN=>EXISTS transformation. This is done by
setting both optimizer switches partial_match_rowid_merge and
partial_match_table_scan to "off".
- It adjusts all test cases where the complete optimizer_switch is
selected because now we have two more switches.
- For those test cases where the plan changes because of the new available
strategies, we switch off both partial match strategies in order to
force the "old" IN=>EXISTS strategy. This is done because most of these
test cases specifically test bugs in this strategy.
sql/opt_subselect.cc:
Adds the possibility to force the execution of top-level [NOT] IN
subquery predicates via the IN=>EXISTS transformation. This is done by
setting both optimizer switches partial_match_rowid_merge and
partial_match_table_scan to "off".
mysql-test/t/disabled.def:
Restore disabled ssl tests: SSL certificates were updated.
Disable sp_sync.test, the test case can't work in next-4284.
mysql-test/t/partition_innodb.test:
Disable parsing of the test case for Bug#47343,
the test can not work in next-4284.
mysql-test/t/ps_ddl.test:
Update results (CREATE TABLE IF NOT EXISTS takes
into account existence of the temporary table).
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_servers.cc
Text conflict in sql/sql_update.cc
Text conflict in support-files/mysql.spec.sh
on re-execution of prepared statement
Problem: some (see eq_ref_table()) ORDER BY/GROUP BY optimization
is called before each PS execution. However, we don't properly
initialize its stucture every time before the call.
Fix: properly initialize the sturture used.
mysql-test/r/ps.result:
Fix for bug#49570: Assertion failed: !(order->used & map)
on re-execution of prepared statement
- test result.
mysql-test/t/ps.test:
Fix for bug#49570: Assertion failed: !(order->used & map)
on re-execution of prepared statement
- test case.
sql/sql_select.cc:
Fix for bug#49570: Assertion failed: !(order->used & map)
on re-execution of prepared statement
- set order->used to 0 before each eq_ref_table() call,
as the function relies on that.
2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1
- initial changeset that introduced the fix for
Bug#989 and follow up fixes for all test suite failures
introduced in the initial changeset.
------------------------------------------------------------
revno: 2617.31.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Fri 2009-03-06 19:17:00 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
Currently the MySQL server does not keep metadata locks on
schema objects for the duration of a transaction, thus failing
to guarantee the integrity of the schema objects being used
during the transaction and to protect then from concurrent
DDL operations. This also poses a problem for replication as
a DDL operation might be replicated even thought there are
active transactions using the object being modified.
The solution is to defer the release of metadata locks until
a active transaction is either committed or rolled back. This
prevents other statements from modifying the table for the
entire duration of the transaction. This provides commitment
ordering for guaranteeing serializability across multiple
transactions.
- Incompatible change:
If MySQL's metadata locking system encounters a lock conflict,
the usual schema is to use the try and back-off technique to
avoid deadlocks -- this schema consists in releasing all locks
and trying to acquire them all in one go.
But in a transactional context this algorithm can't be utilized
as its not possible to release locks acquired during the course
of the transaction without breaking the transaction commitments.
To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be
returned if a lock conflict is encountered during a transaction.
Let's consider an example:
A transaction has two statements that modify table t1, then table
t2, and then commits. The first statement of the transaction will
acquire a shared metadata lock on table t1, and it will be kept
utill COMMIT to ensure serializability.
At the moment when the second statement attempts to acquire a
shared metadata lock on t2, a concurrent ALTER or DROP statement
might have locked t2 exclusively. The prescription of the current
locking protocol is that the acquirer of the shared lock backs off
-- gives up all his current locks and retries. This implies that
the entire multi-statement transaction has to be rolled back.
- Incompatible change:
FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ
LOCK won't cause locked tables to be implicitly unlocked anymore.
mysql-test/extra/binlog_tests/drop_table.test:
Add test case for Bug#989.
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/include/mix1.inc:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/include/mix2.inc:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/r/flush_block_commit.result:
Update test case result (WL#4284).
mysql-test/r/flush_block_commit_notembedded.result:
Update test case result (WL#4284).
mysql-test/r/innodb.result:
Update test case result (WL#4284).
mysql-test/r/innodb_mysql.result:
Update test case result (WL#4284).
mysql-test/r/lock.result:
Add test case result for an effect of WL#4284/Bug#989
(all locks should be released when a connection terminates).
mysql-test/r/mix2_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/not_embedded_server.result:
Update test case result (effects of WL#4284/Bug#989).
Add a test case for interaction of WL#4284 and FLUSH PRIVILEGES.
mysql-test/r/partition_innodb_semi_consistent.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/partition_sync.result:
Temporarily disable the test case for Bug#43867,
which will be fixed by a subsequent backport.
mysql-test/r/ps.result:
Add a test case for effect of PREPARE on transactional
locks: we take a savepoint at beginning of PREAPRE
and release it at the end. Thus PREPARE does not
accumulate metadata locks (Bug#989/WL#4284).
mysql-test/r/read_only_innodb.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_drop_tbl.result:
Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_drop_tbl.result:
Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_unsafe.result:
A side effect of Bug#989 -- slightly different table map ids.
mysql-test/suite/binlog/t/binlog_row_drop_tbl.test:
Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_drop_tbl.test:
Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_row.test:
Update to the new state name. This
is actually a follow up to another patch for WL#4284,
that changes Locked thread state to Table lock.
mysql-test/suite/ndb/r/ndb_index_ordered.result:
Remove result for disabled part of the test case.
mysql-test/suite/ndb/t/disabled.def:
Temporarily disable a test case (Bug#45621).
mysql-test/suite/ndb/t/ndb_index_ordered.test:
Disable a part of a test case (needs update to
reflect semantics of Bug#989).
mysql-test/suite/rpl/t/disabled.def:
Disable tests made meaningless by transactional metadata
locking.
mysql-test/suite/sys_vars/r/autocommit_func.result:
Add a commit (Bug#989).
mysql-test/suite/sys_vars/t/autocommit_func.test:
Add a commit (Bug#989).
mysql-test/t/flush_block_commit.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
mysql-test/t/flush_block_commit_notembedded.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction.
Add a test case for transaction-scope locks and the global
read lock (Bug#989/WL#4284).
mysql-test/t/innodb.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(effects of Bug#989/WL#4284).
mysql-test/t/lock.test:
Add a test case for Bug#989/WL#4284.
mysql-test/t/not_embedded_server.test:
Add a test case for Bug#989/WL#4284.
mysql-test/t/partition_innodb_semi_consistent.test:
Replace TRUNCATE with DELETE, to not issue
an implicit commit of a transaction, and not depend
on metadata locks.
mysql-test/t/partition_sync.test:
Temporarily disable the test case for Bug#43867,
which needs a fix to be backported from 6.0.
mysql-test/t/ps.test:
Add a test case for semantics of PREPARE and transaction-scope
locks: metadata locks on tables used in PREPARE are enclosed into a
temporary savepoint, taken at the beginning of PREPARE,
and released at the end. Thus PREPARE does not effect
what locks a transaction owns.
mysql-test/t/read_only_innodb.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(Bug#989/WL#4284).
Wait for the read_only statement to actually flush tables before
sending other concurrent statements that depend on its state.
mysql-test/t/xa.test:
Fix test case to reflect the fact that transactions now hold
metadata locks for the duration of a transaction
(Bug#989/WL#4284).
sql/ha_ndbcluster_binlog.cc:
Backport bits of changes of ha_ndbcluster_binlog.cc
from 6.0, to fix the failing binlog test suite with
WL#4284. WL#4284 implementation does not work
with 5.1 implementation of ndbcluster binlog index.
sql/log_event.cc:
Release metadata locks after issuing a commit.
sql/mdl.cc:
Style changes (WL#4284).
sql/mysql_priv.h:
Rename parameter to match the name used in the definition (WL#4284).
sql/rpl_injector.cc:
Release metadata locks on commit (WL#4284).
sql/rpl_rli.cc:
Remove assert made meaningless, metadata locks are released
at the end of the transaction.
sql/set_var.cc:
Close tables and release locks if autocommit mode is set.
sql/slave.cc:
Release metadata locks after a rollback.
sql/sql_acl.cc:
Don't implicitly unlock locked tables. Issue a implicit commit
at the end and unlock tables.
sql/sql_base.cc:
Defer the release of metadata locks when closing tables
if not required to.
Issue a deadlock error if the locking protocol requires
that a transaction re-acquire its locks.
Release metadata locks when closing tables for reopen.
sql/sql_class.cc:
Release metadata locks if the thread is killed.
sql/sql_parse.cc:
Release metadata locks after implicitly committing a active
transaction, or after explicit commits or rollbacks.
sql/sql_plugin.cc:
Allocate MDL request on the stack as the use of the table
is contained within the function. It will be removed from
the context once close_thread_tables is called at the end
of the function.
sql/sql_prepare.cc:
The problem is that the prepare phase of the CREATE TABLE
statement takes a exclusive metadata lock lock and this can
cause a self-deadlock the thread already holds a shared lock
on the table being that should be created.
The solution is to make the prepare phase take a shared
metadata lock when preparing a CREATE TABLE statement. The
execution of the statement will still acquire a exclusive
lock, but won't cause any problem as it issues a implicit
commit.
After some discussions with stakeholders it has been decided that
metadata locks acquired during a PREPARE statement must be released
once the statement is prepared even if it is prepared within a multi
statement transaction.
sql/sql_servers.cc:
Don't implicitly unlock locked tables. Issue a implicit commit
at the end and unlock tables.
sql/sql_table.cc:
Close table and release metadata locks after a admin operation.
sql/table.h:
The problem is that the prepare phase of the CREATE TABLE
statement takes a exclusive metadata lock lock and this can
cause a self-deadlock the thread already holds a shared lock
on the table being that should be created.
The solution is to make the prepare phase take a shared
metadata lock when preparing a CREATE TABLE statement. The
execution of the statement will still acquire a exclusive
lock, but won't cause any problem as it issues a implicit
commit.
sql/transaction.cc:
Release metadata locks after the implicitly committed due
to a new transaction being started. Also, release metadata
locks acquired after a savepoint if the transaction is rolled
back to the save point.
The problem is that in some cases transaction-long metadata locks
could be released before the transaction was committed. This could
happen when a active transaction was ended by a "START TRANSACTION"
or "BEGIN" statement, in which case the metadata locks would be
released before the actual commit of the active transaction.
The solution is to defer the release of metadata locks to after the
transaction has been implicitly committed. No test case is provided
as the effort to provide one is too disproportional to the size of
the fix.
The problem was that the multiple evaluations of a ENCODE or
DECODE function within a single statement caused the random
generator to be reinitialized at each evaluation, even though
the parameters were constants.
The solution is to initialize the random generator only once
if the password (seed) parameter is constant.
This patch borrows code and ideas from Georgi Kodinov's patch.
mysql-test/r/func_str.result:
Add test case result.
mysql-test/r/ps.result:
Add test case result.
mysql-test/t/func_str.test:
Add test case for Bug#49141
mysql-test/t/ps.test:
Add test case for Bug#49141
sql/item_strfunc.cc:
Move seed generation code to a separate method.
Seed only once if the password (seed) argument
is constant.
Remove duplicated code and use a transform method
to apply encoding or decoding.
sql/item_strfunc.h:
Add parameter to signal whether the PRNG is already seeded.
Introduce transform method.
Combine val_str methods.
sql/sql_crypt.cc:
Remove method.
sql/sql_crypt.h:
Seed is supplied as two long integers.
Actually there is two different bugs.
The first one caused crash on queries with WHERE condition over views
containing WHERE condition. A wrong check for prepared statement phase led
to items for view fields being allocated in the execution memory and freed
at the end of execution. Thus the optimized WHERE condition refers to
unallocated memory on the second execution and server crashed.
The second one caused by the Item_cond::compile function not saving changes
it made to the item tree. Thus on the next execution changes weren't
reverted and server crashed on dereferencing of unallocated space.
The new helper function called is_stmt_prepare_or_first_stmt_execute
is added to the Query_arena class.
The find_field_in_view function now uses
is_stmt_prepare_or_first_stmt_execute() to check whether
newly created view items should be freed at the end of the query execution.
The Item_cond::compile function now saves changes it makes to item tree.
mysql-test/r/ps.result:
Added a test case for the bug#48508.
mysql-test/t/ps.test:
Added a test case for the bug#48508.
sql/item_cmpfunc.cc:
Bug#48508: Crash on prepared statement re-execution.
The Item_cond::compile function now saves changes it makes to item tree.
sql/sql_base.cc:
Bug#48508: Crash on prepared statement re-execution.
The find_field_in_view function now uses
is_stmt_prepare_or_first_stmt_execute() to check whether
newly created view items should be freed at the end of the query execution.
sql/sql_class.h:
Bug#48508: Crash on prepared statement re-execution.
The Query_arena::is_stmt_prepare_or_first_sp_execute function now correctly
do its check.
2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29,
2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and
some other minor revisions.
This patch implements:
WL#4264 "Backup: Stabilize Service Interface" -- all the
server prerequisites except si_objects.{h,cc} themselves (they can
be just copied over, when needed).
WL#4435: Support OUT-parameters in prepared statements.
(and all issues in the initial patches for these two
tasks, that were discovered in pushbuild and during testing).
Bug#39519: mysql_stmt_close() should flush all data
associated with the statement.
After execution of a prepared statement, send OUT parameters of the invoked
stored procedure, if any, to the client.
When using the binary protocol, send the parameters in an additional result
set over the wire. When using the text protocol, assign out parameters to
the user variables from the CALL(@var1, @var2, ...) specification.
The following refactoring has been made:
- Protocol::send_fields() was renamed to Protocol::send_result_set_metadata();
- A new Protocol::send_result_set_row() was introduced to incapsulate
common functionality for sending row data.
- Signature of Protocol::prepare_for_send() was changed: this operation
does not need a list of items, the number of items is fully sufficient.
The following backward incompatible changes have been made:
- CLIENT_MULTI_RESULTS is now enabled by default in the client;
- CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
include/mysql.h:
Add a new flag to MYSQL_METHODS::flush_use_result
function pointer. This flag determines if all results
should be flushed or only the first one:
- if flush_all_results is TRUE, then cli_flush_use_result()
will read/flush all pending results. I.e. it will read
all packets while server status attribute indicates that
there are more results. This is a new semantic, required
to fix the bug.
- if flush_all_results is FALSE, the old sematic
is preserved -- i.e. cli_flush_use_result() reads data
until first EOF-packet.
include/mysql.h.pp:
Update the ABI with new calls (compatible changes).
include/mysql_com.h:
Add CLIENT_PS_OUT_PARAMS -- a client capability indicating that the client supportsю
libmysql/libmysql.c:
Add mysql_stmt_next_result() -- analogue of mysql_next_result() for binary protocol.
Fix a minor bug in alloc_fields() -- not all members were copied over,
and some only shallow-copied (catalog).
Flush all results in mysql_stmt_close() (Bug#39519).
libmysqld/lib_sql.cc:
Rename send_fields() -> send_result_set_metadata().
Refactoring: change prepare_for_send() so that it accepts only
what it really needs -- a number of elements in the list.
mysql-test/r/ps.result:
Update results: WL#4435.
mysql-test/t/ps.test:
WL#4435: A test case for an SQL-part of the problem.
sql-common/client.c:
Bug#39519.
Implement new functionality in cli_flush_use_result():
if flush_all_delete is TRUE, then it should read/flush
all pending results.
sql/Makefile.am:
Add a new header sql_prepare.h to the list
of build headers.
sql/events.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/handler.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/mysql_priv.h:
Move sql_prepare.cc-specific declarations to a new
header - sql_prepare.h.
sql/procedure.h:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/protocol.cc:
Move the logic responsible for sending of one result
set row to the Protocol class. Define a template
for end-of-statement action.
Refactoring: change prepare_for_send() so that it accepts
only what it really needs -- a number of elements in the list.
Rename send_fields() to send_result_set_metadata().
sql/protocol.h:
Update with new declarations (WL#4435).
Rename send_fields() -> send_result_set_metadata().
prepare_for_send() only needs the number of columns to send,
and doesn't use the item list - update signature to require
only what's needed.
Add a new protocol type -- Protocol_local.
sql/repl_failsafe.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/slave.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_acl.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_base.cc:
Include sql_prepare.h (for Reprepare_observer).
sql/sql_cache.cc:
Extend the query cache flags block to be able
to store a numeric id for the result format,
not just a flag binary/non-binary.
sql/sql_class.cc:
Update to use the rename of Protocol::send_fields()
to Protocol::send_result_set_metadata().
Use Protocol::send_one_result_set_row().
sql/sql_class.h:
Move the declaration of Reprepare_observer to the
new header - sql_prepare.h.
Update to the new signature of class Protocol::send_fields().
sql/sql_connect.cc:
Use a protocol template method instead of
raw NET layer API at the end of a statement.
sql/sql_cursor.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_error.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_handler.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
Use new method Protocol::send_one_result_set_row().
sql/sql_help.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_lex.cc:
Initialize multi_statements variable.
Add a handy constant for empty lex
string.
sql/sql_lex.h:
Add a separate member for a standalone
parsing option - multi-statements support.
sql/sql_list.cc:
sql_list.h is a standalone header now,
no need to include mysql_priv.h.
sql/sql_list.h:
Make sql_list.h a stand-alone header.
sql/sql_parse.cc:
Include sql_prepare.h for prepared
statements- related declarations.
Use a new Protocol template method to end
each statement (send OK, EOF or ERROR to
the client).
sql/sql_prepare.cc:
Implement Execute Direct API (WL#4264),
currently unused. It will be used by the service
interface (Backup).
Use a new header - sql_prepare.h.
Add support for OUT parameters in the
binary and text protocol (prepared statements
only).
sql/sql_prepare.h:
Add a new header to contain (for now)
all prepared statement- external
related declarations.
sql/sql_profile.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_repl.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_select.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_show.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_string.h:
Add a way to convert a String to LEX_STRING.
sql/sql_table.cc:
Rename: Protocol::send_fields() ->
Protocol::send_result_set_metadata().
sql/sql_update.cc:
Remove an extraneous my_error(). The error
is already reported in update_non_unique_table_error().
sql/sql_yacc.yy:
Support for multi-statements is an independent
property of parsing, not derived from
the protocol type.
tests/mysql_client_test.c:
Add tests for WL#4435 (binary protocol).
