Commit graph

931 commits

Author SHA1 Message Date
Alexander Nozdrin
0ae23cd69d Auto-merge from mysql-trunk-merge. 2009-12-24 10:56:13 +03:00
unknown
b71d8c00ba Enable rpl_get_master_version_and clock and rpl_cross_version
as the bug#46931 and bug#43913 have been closed.
2009-12-24 11:01:39 +08:00
Luis Soares
dcf8e11536 BUG#49836: Replication of geometric fields is broken after WL#5151
Metadata for geometric fields was not being properly stored by
the slave in its the table definition. This happened because
MYSQL_TYPE_GEOMETRY was not included in the 'switch... case' that
handles field metadata according to the field type. Therefore, it
would default to 0, leading to always have a mismatch between
master's field and slave fields'. 

We fix this by deploying the missing 'case MYSQL_TYPE_GEOMETRY:'.

mysql-test/extra/rpl_tests/type_conversions.test:
  Added some tests for blob fields and also the particular
  case for replicating from/into BLOB into/from GEOMETRY.
sql/field.h:
  As requested by Mats, reverted function added by him in
  changeset:
  http://lists.mysql.com/commits/95313
2009-12-24 01:07:35 +00:00
Alexey Kopytov
930a08784c Manual merge of mysql-5.1-bugteam into mysql-trunk-merge.
Conflicts:

Conflict adding files to server-tools.  Created directory.
Conflict because server-tools is not versioned, but has versioned children.  Versioned directory.
Conflict adding files to server-tools/instance-manager.  Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children.  Versioned directory.
Contents conflict in server-tools/instance-manager/instance_map.cc
Contents conflict in server-tools/instance-manager/listener.cc
Contents conflict in server-tools/instance-manager/options.cc
Contents conflict in server-tools/instance-manager/user_map.cc
2009-12-23 16:42:05 +03:00
Alfranio Correia
3543244a12 merge mysql-5.1-rep+2 --> mysql-5.1-rep+2-delivery1 2009-12-23 12:02:57 +00:00
unknown
128c9be664 Bug #50061 Test "rpl_loaddata_symlink" uses a Unix-specific "run-slave" script
Bug #49984  	Test 'rpl_loaddata_symlink' fails with "Could not find target log"

Sometimes the symbolic link is available on Windows if
you have some Unix (emulation) layer installed like
Cygwin, MKS or other. But symbolic link is not always
available. It depends on versions, file volume and
system environment of Windows. And the symbolic link
is not typically used on Windows, at least not in the
degree and style they are used on Unix. We can not 
change the test case to do without symbolic link,
because the test case is used to test that if the
symbolic link works fine.

To fix the problem, skip the test on windows.


mysql-test/suite/rpl/t/rpl_loaddata_symlink.test:
  Added code to skip the test on windows.
2010-02-05 14:49:01 +08:00
Luis Soares
67cc50d2a3 PB2 was showing some valgrind warnings after WL#5151 was pushed.
This patch fixes these warnings and some compile time warnings.

On top of that, it also fixes rpl_err_ignoredtable test failure.
This test was failing because the warning suppression text was not
matching the latest text. We fix this by making them match.


mysql-test/suite/rpl/t/rpl_err_ignoredtable.test:
  Replaced the suppression text.
sql/rpl_record.cc:
  - Fixed some compile time warnings (replaced %d --> %ld and 
    removed unused mas mask variable.
  - Fixed valgrind warnings when using c_ptr(). Replaced with 
    c_ptr_safe().
sql/rpl_utility.cc:
  - Fixed valgrind warnings when using c_ptr(). Replaced with 
    c_ptr_safe().
2009-12-22 11:51:46 +00:00
Sergei Golubchik
1ad5bb1a69 WL#4738 streamline/simplify @@variable creation process
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs  
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations 
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies


CMakeLists.txt:
  Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
client/mysql.cc:
  don't crash with --show-warnings when mysqld dies
config/ac-macros/plugins.m4:
  Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
include/my_getopt.h:
  comments
include/my_pthread.h:
  fix double #define
mysql-test/mysql-test-run.pl:
  run sys_vars suite by default
  properly recognize envirinment variables (e.g. MTR_MAX_SAVE_CORE) set to 0
  escape gdb command line arguments
mysql-test/suite/sys_vars/r/rpl_init_slave_func.result:
  init_slave+utf8 bug
mysql-test/suite/sys_vars/t/rpl_init_slave_func.test:
  init_slave+utf8 bug
mysys/my_getopt.c:
  Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
  Bug#46586 When using the plugin interface the type "set" for options caused a crash.
  Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
mysys/typelib.c:
  support for flagset
sql/ha_ndbcluster.cc:
  backport from telco tree
sql/item_func.cc:
  Bug#49644 init_connect and \0
  Bug#49645 init_slave and multi-byte characters
sql/sql_builtin.cc.in:
  Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
sql/sql_plugin.cc:
  Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
  Bug#32902 plugin variables don't know their names
  Bug#44797 plugins w/o command-line options have no disabling option in --help
sql/sys_vars.cc:
  all server variables are defined here
storage/myisam/ft_parser.c:
  remove unnecessary updates of param->quot
storage/myisam/ha_myisam.cc:
  myisam_* variables belong here
strings/my_vsnprintf.c:
  %o and %llx
unittest/mysys/my_vsnprintf-t.c:
  %o and %llx tests
vio/viosocket.c:
  bugfix: fix @@wait_timeout to work with socket timeouts (vs. alarm thread)
2009-12-22 10:35:56 +01:00
Mats Kindahl
6cbd760627 Merging with mysql-5.1-rep+2 2009-12-21 20:55:40 +01:00
Mats Kindahl
060e3ae739 WL#5151: Conversion between different types when replicating
Bug#49836 reports that the geometry type does not work
with WL#5151 applied.

The GEOMETRY type inherits the blob comparison function,
which read the pack length from the metadata. The GEOMETRY
type does not fill in the metadata with anything sensible,
so it is always zero, meaning that the pack length for the
source type is considered zero, rendering it always "smaller"
than the target type which has pack length 4 (without pointer).

This patch fixes the problem by defining
Field_geom::pack_length_from_metadata() to always use the
same as Field_geom::row_pack_length().
2009-12-21 19:50:25 +01:00
Alfranio Correia
56ff29a422 merge mysql-5.1-rep+3 --> mysql-5.1-rep+2-delivery1 2009-12-21 06:42:23 +00:00
Alfranio Correia
c5af8813fd merge mysql-5.1-rep+2 --> mysql-5.1-rep+2-delivery1 2009-12-21 06:33:32 +00:00
Alfranio Correia
03324f6a0a Post-merge fix. 2009-12-21 06:18:11 +00:00
He Zhenxing
27aba1b569 Bug#47638 The rpl_killed_ddl test fails on Windows
When the $diff_statement variable for diff_master_slave.inc was
put in multiple lines, the rear part of the statement would be 
missing when being executed on Windows systems.

Fixed the problem by always putting the value for $diff_statement
in one line.

mysql-test/suite/rpl/t/rpl_killed_ddl.test:
  putting the statement for $diff_statement variable in one line.
2009-12-21 14:14:45 +08:00
Mats Kindahl
437b5338fb Merging with mysql.5.1-rep+2 2009-12-18 13:09:08 +01:00
Alfranio Correia
394b073154 Post-merge fix after BUG##45292
Updated suppressed warning messages.
2009-12-18 11:15:46 +00:00
Mats Kindahl
b001aefde8 Merging with mysql-5.1-rep+2 2009-12-18 11:16:18 +01:00
Mats Kindahl
e4de9401d6 WL#5151: Conversion between different types
when replicating

The function create_virtual_tmp_table does not
set db_low_byte_first in the same way as
create_tmp_table does, causing copying from
the virtual table to a real table to get strange
values for SET types on big-endian machines.


mysql-test/extra/rpl_tests/type_conversions.test:
  Adding tests for converting between different-sized sets.
sql/sql_select.cc:
  Settng dh_low_byte_first for tables created
  with create_virtual_tmp_table.
2009-12-18 11:12:27 +01:00
Alfranio Correia
3dadf9564c merge mysql-5.1-rep+3 --> mysql-5.1-rep+2-delivery1 2009-12-17 21:43:35 +00:00
Alexander Nozdrin
0e3a80e0a8 Manual merge from mysql-trunk-merge.
Conflicts:
  - storage/myisam/mi_packrec.c
2009-12-17 22:16:54 +03:00
Alexey Kopytov
5069a66670 Automerge from mysql-5.1-bugteam to mysql-trunk-merge. 2009-12-17 18:09:04 +03:00
Andrei Elkin
709f49cbf7 merge from 5.0 with bug@49740 fixes 2009-12-17 16:50:45 +02:00
Alfranio Correia
9a51ad2c54 merging from 5.1 to rep+2 starting at gca(5.1, next-mr) == build@mysql.com-20091208092611-pbno5awyb0v38hs7
Fixed conflicts in:

- binlog.binlog_unsafe
- rpl.rpl_slow_query_log
2009-12-17 00:09:52 +00:00
Magne Mahre
4338e06f51 Bug#47017 rpl_timezone fails on PB-2 with mismatch error
The bug is caused by a race condition between the 
INSERT DELAYED thread and the client thread's FLUSH TABLE.  The 
FLUSH TABLE does not guarantee (as is (wrongly) suggested in the 
test case) that the INSERT DELAYED is ever executed.  The 
execution of the test case will thus not be deterministic.

The fix has been to do a deterministic verification that both
threads are complete by checking the content of the table.
2009-12-16 20:53:56 +01:00
Alexander Nozdrin
5f0c09dd72 Manual merge from mysql-trunk-merge.
Conflicts:
  - include/my_no_pthread.h
  - mysql-test/r/sp-ucs2.result
  - sql/log.cc
  - sql/sql_acl.cc
  - sql/sql_yacc.yy
2009-12-16 21:02:21 +03:00
Alexey Kopytov
f1e83a4163 Manual merge of mysql-5.1-bugteam into mysql-trunk-merge. 2009-12-16 16:47:07 +03:00
Mats Kindahl
25ecb31116 WL#5151: Conversion between different types when replicating
Post-merge fixes to update result files.
2009-12-16 14:30:41 +01:00
Konstantin Osipov
980e8b413e Merge next-mr -> next-4284. 2009-12-16 11:33:54 +03:00
unknown
c7d483155b Bug #46827 rpl_circular_for_4_hosts failed on PB2
This test case tests a circular replication of four hosts. 
A--->B--->C--->D--->A
The replicate is slow and needs more time to replicate all data in the circle.
The time it spends to replicate, sometimes, is longer than the time that 
wait_condition.inc spends to wait that all data has been replicated. This
cause sporadical failure of this test case.
  
This patch uses sync_slave_with_master to ensure that all data can be replicated
successfully in the circle.
2009-12-16 12:41:15 +08:00
unknown
0b7768bc8d Postfix
Only relative log events are showed.
2009-12-16 12:25:46 +08:00
Mats Kindahl
f63a9d69b9 WL#5151: Conversion between different types when replicating
Fixing error codes that changed after merge.
2009-12-15 23:13:31 +01:00
Mats Kindahl
f9abd14225 Merging with mysql-5.1-rep+2 2009-12-15 21:21:00 +01:00
Konstantin Osipov
391b5246fe Merge next-mr -> next-4284. 2009-12-15 22:59:07 +03:00
Mats Kindahl
9e980bf79e BUG#49618: Field length stored incorrectly in binary log
for InnoDB

The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.

Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.

To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.

mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
  Adding test to check compatibility for bit field
  replication when using InnoDB.
sql/field.cc:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/field.h:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/log.cc:
  Removing table map flags since they are not used
  outside table map class.
sql/log_event.cc:
  Removing flags parameter from table map constructor
  since it is not used and does not have to be exposed.
sql/log_event.h:
  Adding flag to denote that bit length for bit field type
  is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
  Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
  Removing obsolete comment and adding flags to store
  table map flags from master.
2009-12-15 16:11:44 +01:00
unknown
e813587b40 Bug #34628 LOAD DATA CONCURRENT INFILE drops CONCURRENT in binary log
'LOAD DATA CONCURRENT [LOCAL] INFILE ...' statment only is binlogged as
'LOAD DATA [LOCAL] INFILE ...' in SBR and MBR.  As a result, if replication is on, 
queries on slaves will be blocked by the replication SQL thread.

This patch write code to write 'CONCURRENT' into the log event if 'CONCURRENT' option
is in the original statement in SBR and MBR.
2009-12-15 13:14:14 +08:00
Mats Kindahl
870daf5c01 WL#5151: Conversion between different types when replicating
Fixing minor error when printing SQL types from master and cleaning some code.

Updating result files.
2009-12-14 23:27:06 +01:00
Andrei Elkin
c00bfe59b3 merging 5.0-bt -> 5.1-bt to local branch with bug@47210 2009-12-14 18:32:22 +02:00
Mats Kindahl
571843804c WL#5151: Conversion between different types when replicating
Row-based replication requires the types of columns on the
master and slave to be approximately the same (some safe
conversions between strings are allowed), but does not
allow safe conversions between fields of similar types such
as TINYINT and INT.

This patch implement type conversions between similar fields
on the master and slave.

The conversions are controlled using a new variable
SLAVE_TYPE_CONVERSIONS of type SET('ALL_LOSSY','ALL_NON_LOSSY').

Non-lossy conversions are any conversions that do not run the
risk of losing any information, while lossy conversions can
potentially truncate the value. The column definitions are
checked to decide if the conversion is acceptable.

If neither conversion is enabled, it is required that the
definitions of the columns are identical on master and slave.

Conversion is done by creating an internal conversion table,
unpacking the master data into it, and then copy the data to
the real table on the slave.

.bzrignore:
  New files added
client/Makefile.am:
  New files added
client/mysqlbinlog.cc:
  Functions in rpl_utility.cc is now needed by mysqlbinlog.cc.
libmysqld/Makefile.am:
  New files added
mysql-test/extra/rpl_tests/check_type.inc:
  Test include file to check a single type conversion.
mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
  Switching to use INT instead of TEXT for column that should not have matching types.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
  Adding code to enable type conversions for BIT tests since InnoDB
  cannot handle them properly due to incorrect information stored as
  metadata.
mysql-test/extra/rpl_tests/type_conversions.test:
  Test file to check a set of type conversions
  with current settings of slave_type_conversions.
mysql-test/suite/rpl/t/rpl_typeconv.test:
  Test file to test conversions from master to slave with
  all possible values for slave_type_conversions.
  
  The test also checks that the slave_type_conversions
  variable works as expected.
sql/field.cc:
  Changing definition of compatible_field_size to both check if 
  two field with identical base types are compatible and give an
  order between them if they are compatible.
  
  This only implement checking on the slave, so it will not affect
  replication from an old master to a new slave.
sql/field.h:
  Changing prototypes for functions:
  - compatible_field_size()
  - init_for_tmp_table()
  - row_pack_length()
sql/log_event.cc:
  Changing compability checks to build a conversion table if the fields
  are compatible, but does not have the same base type.
sql/log_event_old.cc:
  Changing compability checks to build a conversion table if the fields
  are compatible, but does not have the same base type.
sql/mysql_priv.h:
  Adding global option variable for SLAVE_TYPE_CONVERSIONS
sql/mysqld.cc:
  Adding SLAVE_TYPE_CONVERSIONS global server variable.
sql/rpl_record.cc:
  Changing unpack_row to use the conversion table if present.
sql/rpl_rli.h:
  Removing function get_tabledef and replacing it with get_table_data().
  This function retrieve data for table opened for replication, not just
  table definition.
sql/rpl_utility.cc:
  Function table_def::compatible_with is changed to compare table on master
  and slave for compatibility and generate a conversions table if they are
  compatible.
  
  Computing real type of fields from metadata for ENUM and SET types.
  Computing pack_length correctly for ENUM, SET, and BLOB types.
  
  Adding optimization to not check compatibility if no
  slave type conversions are enabled.
sql/rpl_utility.h:
  Changing prototypes since implementation has changed.
  
  Modifying table_def::type() to return real type instead of stored type.
sql/set_var.cc:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/set_var.h:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/share/errmsg.txt:
  Adding error messages for slave type conversions.
sql/sql_class.h:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/sql_select.cc:
  Correcting create_virtual_tmp_table() to compute null bit positions
  correctly in the presence of bit fields.
2009-12-14 12:04:55 +01:00
Alfranio Correia
46fe96c11f merge mysql-5.1-rep+2 --> mysql-5.1-rep+3 2009-12-14 10:40:42 +00:00
Alexander Nozdrin
aceea2342f Manual merge from mysql-trunk-merge.
Conflicts:
  - extra/comp_err.c
  - mysql-test/collections/default.experimental
  - mysql-test/r/archive.result
  - mysql-test/r/select.result
  - mysql-test/suite/binlog/r/binlog_unsafe.result
  - mysql-test/suite/binlog/t/binlog_unsafe.test
  - mysql-test/suite/rpl/t/disabled.def
  - mysql-test/t/archive.test
  - mysql-test/t/select.test
  - sql/item.cc
  - sql/item.h
  - sql/item_timefunc.cc
  - sql/sql_base.cc
  - sql/sql_delete.cc
  - sql/sql_load.cc
  - sql/sql_partition.cc
  - sql/sql_table.cc
  - storage/innobase/handler/ha_innodb.cc
  - vio/vio.c
2009-12-12 23:38:59 +03:00
Alexey Kopytov
ebdef570e5 Manual merge of mysql-5.1-bugteam into mysql-trunk-merge. 2009-12-11 19:40:58 +03:00
Alexander Nozdrin
5676713687 Manual merge from mysql-trunk.
Conflicts:
  - client/mysqltest.cc
  - mysql-test/collections/default.experimental
  - mysql-test/suite/rpl/t/disabled.def
  - sql/mysqld.cc
  - sql/opt_range.cc
  - sql/sp.cc
  - sql/sql_acl.cc
  - sql/sql_partition.cc
  - sql/sql_table.cc
2009-12-11 12:39:38 +03:00
He Zhenxing
f8d1633251 Merge from 5.0-bugteam 2009-12-10 11:51:42 +08:00
He Zhenxing
b13541ff31 Merge Bug#45520 fix from 5.0-bugteam 2009-12-09 14:24:54 +08:00
Jon Olav Hauglid
8da6d448b4 Backport of revno: 2617.62.1
Bug #39675 rename tables on innodb tables with pending 
           transactions causes slave data issue

Bug was already fixed as part of patch for Bug#989 
(If DROP TABLE while there's an active transaction, 
wrong binlog order) 

Test case added to rpl_innodb.test.
2009-12-08 14:22:26 +01:00
Luis Soares
a437804ab0 automerge: mysql-5.1-bugteam bug branch --> mysql-5.1-bugteam latest 2009-12-06 23:14:45 +00:00
Luis Soares
c75712caa8 BUG#49119: Master crashes when executing 'REVOKE ... ON
{PROCEDURE|FUNCTION} FROM ...'

The master would hit an assertion when binary log was
active. This was due to the fact that the thread's diagnostics
area was being cleared before writing to the binlog,
independently of mysql_routine_grant returning an error or
not. When mysql_routine_grant was to return an error, the return
value and the diagnostics area contents would
mismatch. Consequently, neither my_ok would be called nor an
error would be signaled in the diagnostics area, eventually
triggering the assertion in net_end_statement.

We fix this by not clearing the diagnostics area at binlogging
time.
2009-12-06 23:12:11 +00:00
Luis Soares
595719280e BUG#49479: Slave stops with syntax error: LOAD DATA event without
escaped field names

When in mixed or statement mode, the master logs LOAD DATA
queries by resorting to an Execute_load_query_log_event. This
event does not contain the original query, but a rewritten
version of it, which includes the table field names. However, the
rewrite does not escape the field names. If these names match a
reserved keyword, then the slave will stop with a syntax error
when executing the event.

We fix this by escaping the fields names as it happens already
for the table name.

mysql-test/extra/rpl_tests/rpl_loaddata.test:
  Added test case for the reported bug.
mysql-test/r/mysqlbinlog.result:
  Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_killed_simulate.result:
  Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
  Changed result to support escaped field name.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Changed result to support escaped field name.
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
  Changed result to support escaped field name.
mysql-test/suite/rpl/r/rpl_loaddata.result:
  Added result for new test. 
  Changed show slave status positions which are now different because of
  extra escape character in field names.
mysql-test/suite/rpl/r/rpl_loaddata_fatal.result:
  Changed show slave status positions which are now different because of
  extra escape character.
mysql-test/suite/rpl/r/rpl_loaddata_map.result:
  Changed result to support escaped field name.
mysql-test/suite/rpl/r/rpl_stm_log.result:
  Changed result to support escaped field name.
mysql-test/t/mysqlbinlog.test:
  Changed positions which is now different because of extra escape 
  character in field names.
sql/sql_load.cc:
  Appended escape characters before and after field names.
2009-12-06 01:11:32 +00:00
He Zhenxing
0dcc854520 Auto merge from 5.1-rep-semisync 2009-12-05 10:28:53 +08:00
Konstantin Osipov
a14bbee5ab Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5,
2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1
- initial changeset that introduced the fix for 
Bug#989 and follow up fixes for all test suite failures
introduced in the initial changeset. 
------------------------------------------------------------
revno: 2617.31.1
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Fri 2009-03-06 19:17:00 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking

Currently the MySQL server does not keep metadata locks on
schema objects for the duration of a transaction, thus failing
to guarantee the integrity of the schema objects being used
during the transaction and to protect then from concurrent
DDL operations. This also poses a problem for replication as
a DDL operation might be replicated even thought there are
active transactions using the object being modified.

The solution is to defer the release of metadata locks until
a active transaction is either committed or rolled back. This
prevents other statements from modifying the table for the
entire duration of the transaction. This provides commitment
ordering for guaranteeing serializability across multiple
transactions.

- Incompatible change:

If MySQL's metadata locking system encounters a lock conflict,
the usual schema is to use the try and back-off technique to
avoid deadlocks -- this schema consists in releasing all locks
and trying to acquire them all in one go.

But in a transactional context this algorithm can't be utilized
as its not possible to release locks acquired during the course
of the transaction without breaking the transaction commitments.
To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be
returned if a lock conflict is encountered during a transaction.

Let's consider an example:

A transaction has two statements that modify table t1, then table
t2, and then commits. The first statement of the transaction will
acquire a shared metadata lock on table t1, and it will be kept
utill COMMIT to ensure serializability.

At the moment when the second statement attempts to acquire a
shared metadata lock on t2, a concurrent ALTER or DROP statement
might have locked t2 exclusively. The prescription of the current
locking protocol is that the acquirer of the shared lock backs off
-- gives up all his current locks and retries. This implies that
the entire multi-statement transaction has to be rolled back.

- Incompatible change:

FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ
LOCK won't cause locked tables to be implicitly unlocked anymore.


mysql-test/extra/binlog_tests/drop_table.test:
  Add test case for Bug#989.
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/include/mix1.inc:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/include/mix2.inc:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/r/flush_block_commit.result:
  Update test case result (WL#4284).
mysql-test/r/flush_block_commit_notembedded.result:
  Update test case result (WL#4284).
mysql-test/r/innodb.result:
  Update test case result (WL#4284).
mysql-test/r/innodb_mysql.result:
  Update test case result (WL#4284).
mysql-test/r/lock.result:
  Add test case result for an effect of WL#4284/Bug#989
  (all locks should be released when a connection terminates).
mysql-test/r/mix2_myisam.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/not_embedded_server.result:
  Update test case result (effects of WL#4284/Bug#989).
  Add a test case for interaction of WL#4284 and FLUSH PRIVILEGES.
mysql-test/r/partition_innodb_semi_consistent.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/r/partition_sync.result:
  Temporarily disable the test case for Bug#43867,
  which will be fixed by a subsequent backport.
mysql-test/r/ps.result:
  Add a test case for effect of PREPARE on transactional
  locks: we take a savepoint at beginning of PREAPRE
  and release it at the end. Thus PREPARE does not 
  accumulate metadata locks (Bug#989/WL#4284).
mysql-test/r/read_only_innodb.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_drop_tbl.result:
  Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_drop_tbl.result:
  Add a test case result (WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Update test case result (effects of WL#4284/Bug#989).
mysql-test/suite/binlog/r/binlog_unsafe.result:
  A side effect of Bug#989 -- slightly different table map ids.
mysql-test/suite/binlog/t/binlog_row_drop_tbl.test:
  Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_drop_tbl.test:
  Add a test case for WL#4284/Bug#989.
mysql-test/suite/binlog/t/binlog_stm_row.test:
  Update to the new state name. This
  is actually a follow up to another patch for WL#4284, 
  that changes Locked thread state to Table lock.
mysql-test/suite/ndb/r/ndb_index_ordered.result:
  Remove result for disabled part of the test case.
mysql-test/suite/ndb/t/disabled.def:
  Temporarily disable a test case (Bug#45621).
mysql-test/suite/ndb/t/ndb_index_ordered.test:
  Disable a part of a test case (needs update to
  reflect semantics of Bug#989).
mysql-test/suite/rpl/t/disabled.def:
  Disable tests made meaningless by transactional metadata
  locking.
mysql-test/suite/sys_vars/r/autocommit_func.result:
  Add a commit (Bug#989).
mysql-test/suite/sys_vars/t/autocommit_func.test:
  Add a commit (Bug#989).
mysql-test/t/flush_block_commit.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
mysql-test/t/flush_block_commit_notembedded.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction.
  Add a test case for transaction-scope locks and the global
  read lock (Bug#989/WL#4284).
mysql-test/t/innodb.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction
  (effects of Bug#989/WL#4284).
mysql-test/t/lock.test:
  Add a test case for Bug#989/WL#4284.
mysql-test/t/not_embedded_server.test:
  Add a test case for Bug#989/WL#4284.
mysql-test/t/partition_innodb_semi_consistent.test:
  Replace TRUNCATE with DELETE, to not issue
  an implicit commit of a transaction, and not depend
  on metadata locks.
mysql-test/t/partition_sync.test:
  Temporarily disable the test case for Bug#43867,
  which needs a fix to be backported from 6.0.
mysql-test/t/ps.test:
  Add a test case for semantics of PREPARE and transaction-scope
  locks: metadata locks on tables used in PREPARE are enclosed into a
  temporary savepoint, taken at the beginning of PREPARE,
  and released at the end. Thus PREPARE does not effect
  what locks a transaction owns.
mysql-test/t/read_only_innodb.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction 
  (Bug#989/WL#4284).
  
  Wait for the read_only statement to actually flush tables before
  sending other concurrent statements that depend on its state.
mysql-test/t/xa.test:
  Fix test case to reflect the fact that transactions now hold
  metadata locks for the duration of a transaction 
  (Bug#989/WL#4284).
sql/ha_ndbcluster_binlog.cc:
  Backport bits of changes of ha_ndbcluster_binlog.cc
  from 6.0, to fix the failing binlog test suite with
  WL#4284. WL#4284 implementation does not work
  with 5.1 implementation of ndbcluster binlog index.
sql/log_event.cc:
  Release metadata locks after issuing a commit.
sql/mdl.cc:
  Style changes (WL#4284).
sql/mysql_priv.h:
  Rename parameter to match the name used in the definition (WL#4284).
sql/rpl_injector.cc:
  Release metadata locks on commit (WL#4284).
sql/rpl_rli.cc:
  Remove assert made meaningless, metadata locks are released
  at the end of the transaction.
sql/set_var.cc:
  Close tables and release locks if autocommit mode is set.
sql/slave.cc:
  Release metadata locks after a rollback.
sql/sql_acl.cc:
  Don't implicitly unlock locked tables. Issue a implicit commit
  at the end and unlock tables.
sql/sql_base.cc:
  Defer the release of metadata locks when closing tables
  if not required to.
  Issue a deadlock error if the locking protocol requires
  that a transaction re-acquire its locks.
  
  Release metadata locks when closing tables for reopen.
sql/sql_class.cc:
  Release metadata locks if the thread is killed.
sql/sql_parse.cc:
  Release metadata locks after implicitly committing a active
  transaction, or after explicit commits or rollbacks.
sql/sql_plugin.cc:
  
  Allocate MDL request on the stack as the use of the table
  is contained within the function. It will be removed from
  the context once close_thread_tables is called at the end
  of the function.
sql/sql_prepare.cc:
  The problem is that the prepare phase of the CREATE TABLE
  statement takes a exclusive metadata lock lock and this can
  cause a self-deadlock the thread already holds a shared lock
  on the table being that should be created.
  
  The solution is to make the prepare phase take a shared
  metadata lock when preparing a CREATE TABLE statement. The
  execution of the statement will still acquire a exclusive
  lock, but won't cause any problem as it issues a implicit
  commit.
  
  After some discussions with stakeholders it has been decided that
  metadata locks acquired during a PREPARE statement must be released
  once the statement is prepared even if it is prepared within a multi
  statement transaction.
sql/sql_servers.cc:
  Don't implicitly unlock locked tables. Issue a implicit commit
  at the end and unlock tables.
sql/sql_table.cc:
  Close table and release metadata locks after a admin operation.
sql/table.h:
  The problem is that the prepare phase of the CREATE TABLE
  statement takes a exclusive metadata lock lock and this can
  cause a self-deadlock the thread already holds a shared lock
  on the table being that should be created.
  
  The solution is to make the prepare phase take a shared
  metadata lock when preparing a CREATE TABLE statement. The
  execution of the statement will still acquire a exclusive
  lock, but won't cause any problem as it issues a implicit
  commit.
sql/transaction.cc:
  Release metadata locks after the implicitly committed due
  to a new transaction being started. Also, release metadata
  locks acquired after a savepoint if the transaction is rolled
  back to the save point.
  
  The problem is that in some cases transaction-long metadata locks
  could be released before the transaction was committed. This could
  happen when a active transaction was ended by a "START TRANSACTION"
  or "BEGIN" statement, in which case the metadata locks would be
  released before the actual commit of the active transaction.
  
  The solution is to defer the release of metadata locks to after the
  transaction has been implicitly committed. No test case is provided
  as the effort to provide one is too disproportional to the size of
  the fix.
2009-12-05 02:02:48 +03:00