cant find record
Some engines return data for the record. Despite the fact that
the null bit is set for some fields, their old value may still in
the row. This can happen when unpacking an AI from the binlog on
top of a previous record in which a field is set to NULL, which
previously contained a value. Ultimately, this may cause the
comparison of records to fail when the slave is doing an index or
range scan.
We fix this by deploying a call to reset() for each field that is
set to null while unpacking a row from the binary log.
Furthermore, we also add mixed mode test case to cover the
scenario where updating and setting a field to null through a
Query event and later searching it through a rows event will
succeed.
Finally, we also change the reset() method, from Field_bit class,
so that it takes into account bits stored among the null bits and
not only the ones stored in the record.
mysql-test/suite/rpl/t/rpl_set_null_innodb.test:
InnoDB test.
mysql-test/suite/rpl/t/rpl_set_null_myisam.test:
MyISAM test.
mysql-test/suite/rpl_ndb/t/rpl_ndb_set_null.test:
NDB test.
sql/field.h:
Changed reset so that it also clears the bits
among the null_bits for the Field_bit class.
sql/rpl_record.cc:
Resetting field after setting it to null when unpacking
row.
cant find record
Some engines return data for the record. Despite the fact that
the null bit is set for some fields, their old value may still in
the row. This can happen when unpacking an AI from the binlog on
top of a previous record in which a field is set to NULL, which
previously contained a value. Ultimately, this may cause the
comparison of records to fail when the slave is doing an index or
range scan.
We fix this by deploying a call to reset() for each field that is
set to null while unpacking a row from the binary log.
Furthermore, we also add mixed mode test case to cover the
scenario where updating and setting a field to null through a
Query event and later searching it through a rows event will
succeed.
Finally, we also change the reset() method, from Field_bit class,
so that it takes into account bits stored among the null bits and
not only the ones stored in the record.
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase. Created directory.
Conflict because storage/innobase is not versioned, but has versioned children. Versioned directory.
Conflict adding file storage/innobase. Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler. Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children. Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase. Created directory.
Conflict because storage/innobase is not versioned, but has versioned children. Versioned directory.
Conflict adding file storage/innobase. Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler. Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children. Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
for InnoDB
The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.
Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.
To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.
mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
Adding test to check compatibility for bit field
replication when using InnoDB.
sql/field.cc:
Extending compatible_field_size() with flags from
table map to allow fields to check master info.
sql/field.h:
Extending compatible_field_size() with flags from
table map to allow fields to check master info.
sql/log.cc:
Removing table map flags since they are not used
outside table map class.
sql/log_event.cc:
Removing flags parameter from table map constructor
since it is not used and does not have to be exposed.
sql/log_event.h:
Adding flag to denote that bit length for bit field type
is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
Removing obsolete comment and adding flags to store
table map flags from master.
for InnoDB
The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.
Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.
To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.
Row-based replication requires the types of columns on the
master and slave to be approximately the same (some safe
conversions between strings are allowed), but does not
allow safe conversions between fields of similar types such
as TINYINT and INT.
This patch implement type conversions between similar fields
on the master and slave.
The conversions are controlled using a new variable
SLAVE_TYPE_CONVERSIONS of type SET('ALL_LOSSY','ALL_NON_LOSSY').
Non-lossy conversions are any conversions that do not run the
risk of losing any information, while lossy conversions can
potentially truncate the value. The column definitions are
checked to decide if the conversion is acceptable.
If neither conversion is enabled, it is required that the
definitions of the columns are identical on master and slave.
Conversion is done by creating an internal conversion table,
unpacking the master data into it, and then copy the data to
the real table on the slave.
.bzrignore:
New files added
client/Makefile.am:
New files added
client/mysqlbinlog.cc:
Functions in rpl_utility.cc is now needed by mysqlbinlog.cc.
libmysqld/Makefile.am:
New files added
mysql-test/extra/rpl_tests/check_type.inc:
Test include file to check a single type conversion.
mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
Switching to use INT instead of TEXT for column that should not have matching types.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Adding code to enable type conversions for BIT tests since InnoDB
cannot handle them properly due to incorrect information stored as
metadata.
mysql-test/extra/rpl_tests/type_conversions.test:
Test file to check a set of type conversions
with current settings of slave_type_conversions.
mysql-test/suite/rpl/t/rpl_typeconv.test:
Test file to test conversions from master to slave with
all possible values for slave_type_conversions.
The test also checks that the slave_type_conversions
variable works as expected.
sql/field.cc:
Changing definition of compatible_field_size to both check if
two field with identical base types are compatible and give an
order between them if they are compatible.
This only implement checking on the slave, so it will not affect
replication from an old master to a new slave.
sql/field.h:
Changing prototypes for functions:
- compatible_field_size()
- init_for_tmp_table()
- row_pack_length()
sql/log_event.cc:
Changing compability checks to build a conversion table if the fields
are compatible, but does not have the same base type.
sql/log_event_old.cc:
Changing compability checks to build a conversion table if the fields
are compatible, but does not have the same base type.
sql/mysql_priv.h:
Adding global option variable for SLAVE_TYPE_CONVERSIONS
sql/mysqld.cc:
Adding SLAVE_TYPE_CONVERSIONS global server variable.
sql/rpl_record.cc:
Changing unpack_row to use the conversion table if present.
sql/rpl_rli.h:
Removing function get_tabledef and replacing it with get_table_data().
This function retrieve data for table opened for replication, not just
table definition.
sql/rpl_utility.cc:
Function table_def::compatible_with is changed to compare table on master
and slave for compatibility and generate a conversions table if they are
compatible.
Computing real type of fields from metadata for ENUM and SET types.
Computing pack_length correctly for ENUM, SET, and BLOB types.
Adding optimization to not check compatibility if no
slave type conversions are enabled.
sql/rpl_utility.h:
Changing prototypes since implementation has changed.
Modifying table_def::type() to return real type instead of stored type.
sql/set_var.cc:
Adding SLAVE_TYPE_CONVERSIONS variable.
sql/set_var.h:
Adding SLAVE_TYPE_CONVERSIONS variable.
sql/share/errmsg.txt:
Adding error messages for slave type conversions.
sql/sql_class.h:
Adding SLAVE_TYPE_CONVERSIONS variable.
sql/sql_select.cc:
Correcting create_virtual_tmp_table() to compute null bit positions
correctly in the presence of bit fields.
Row-based replication requires the types of columns on the
master and slave to be approximately the same (some safe
conversions between strings are allowed), but does not
allow safe conversions between fields of similar types such
as TINYINT and INT.
This patch implement type conversions between similar fields
on the master and slave.
The conversions are controlled using a new variable
SLAVE_TYPE_CONVERSIONS of type SET('ALL_LOSSY','ALL_NON_LOSSY').
Non-lossy conversions are any conversions that do not run the
risk of losing any information, while lossy conversions can
potentially truncate the value. The column definitions are
checked to decide if the conversion is acceptable.
If neither conversion is enabled, it is required that the
definitions of the columns are identical on master and slave.
Conversion is done by creating an internal conversion table,
unpacking the master data into it, and then copy the data to
the real table on the slave.
----------------------------------------------------------
revno: 2630.10.1
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-lock-tables-tidyup
timestamp: Wed 2008-06-11 15:49:58 +0400
message:
WL#3726, review fixes.
Now that we have metadata locks, we don't need to keep a crippled
TABLE instance in the table cache to indicate that a table is locked.
Remove all code that used this technique. Instead, rely on metadata
locks and use the standard open_table() and close_thread_table()
to manipulate with the table cache tables.
Removes a list of functions that have become unused (see the comment
for sql_base.cc for details).
Under LOCK TABLES, keep a TABLE_LIST instance for each table
that may be temporarily closed. For that, implement an own class for
LOCK TABLES mode, Locked_tables_list.
This is a pre-requisite patch for WL#4144.
This is not exactly a backport: there is no new
online ALTER table in Celosia, so the old alter table
code was changed to work with the new table cache API.
mysql-test/r/lock.result:
Update results (WL#3726 post-review patch).
mysql-test/r/trigger-compat.result:
We take the table from the table cache now, thus no warning.
mysql-test/suite/rpl/r/rpl_trigger.result:
We take the table from the table cache now, thus no warning.
mysql-test/t/lock.test:
Additional tests for LOCK TABLES mode (previously not covered by
the test suite (WL#3726).
sql/field.h:
Remove reopen_table().
sql/lock.cc:
Remove an obsolete parameter of mysql_lock_remove().
It's not used anywhere now either.
sql/mysql_priv.h:
Add 4 new open_table() flags.
Remove declarations of removed functions.
sql/sp_head.cc:
Rename thd->mdl_el_root to thd->locked_tables_root.
sql/sql_acl.cc:
Use the new implementation of unlock_locked_tables().
sql/sql_base.cc:
Implement class Locked_tables_list.
Implement close_all_tables_for_name().
Rewrite close_cached_tables() to use the new reopen_tables().
Remove reopen_table(), reopen_tables(), reopen_table_entry()
(ex. open_unireg_entry()), close_data_files_and_leave_as_placeholders(),
close_handle_and_leave_table_as_placeholder(), close_cached_table(),
table_def_change_share(), reattach_merge(), reopen_name_locked_table(),
unlink_open_table().
Move acquisition of a metadata lock into an own function
- open_table_get_mdl_lock().
sql/sql_class.cc:
Deploy class Locked_tables_list.
sql/sql_class.h:
Declare class Locked_tables_list.
Keep one instance of this class in class THD.
Rename mdl_el_root to locked_tables_root.
sql/sql_db.cc:
Update a comment.
sql/sql_insert.cc:
Use the plain open_table() to open a just created table in
CREATE TABLE .. SELECT.
sql/sql_parse.cc:
Use thd->locked_tables_list to enter and leave LTM_LOCK_TABLES mode.
sql/sql_partition.cc:
Deploy the new method of working with partitioned table locks.
sql/sql_servers.cc:
Update to the new signature of unlock_locked_tables().
sql/sql_table.cc:
In mysql_rm_table_part2(), the branch that removes a table under
LOCK TABLES, make sure that the table being dropped
is also removed from THD::locked_tables_list.
Update ALTER TABLE and CREATE TABLE LIKE implementation to use
open_table() and close_all_tables_for_name() instead of
reopen_tables().
sql/sql_trigger.cc:
Use the new locking way.
sql/table.h:
Add TABLE::pos_in_locked_tables, which is used only under
LOCK TABLES.
----------------------------------------------------------
revno: 2630.10.1
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-lock-tables-tidyup
timestamp: Wed 2008-06-11 15:49:58 +0400
message:
WL#3726, review fixes.
Now that we have metadata locks, we don't need to keep a crippled
TABLE instance in the table cache to indicate that a table is locked.
Remove all code that used this technique. Instead, rely on metadata
locks and use the standard open_table() and close_thread_table()
to manipulate with the table cache tables.
Removes a list of functions that have become unused (see the comment
for sql_base.cc for details).
Under LOCK TABLES, keep a TABLE_LIST instance for each table
that may be temporarily closed. For that, implement an own class for
LOCK TABLES mode, Locked_tables_list.
This is a pre-requisite patch for WL#4144.
This is not exactly a backport: there is no new
online ALTER table in Celosia, so the old alter table
code was changed to work with the new table cache API.
Bug #48370 Absolutely wrong calculations with GROUP BY and
decimal fields when using IF
Added the test cases in the above two bugs for regression
testing.
Added additional tests that demonstrate a incomplete fix.
Added a new factory method for Field_new_decimal to
create a field from an (decimal returning) Item.
In the new method made sure that all the precision and
length variables are capped in a proper way.
This is required because Item's can have larger precision
than the decimal fields and thus need to be capped when
creating a field based on an Item type.
Fixed the wrong typecast to Item_decimal.
Bug #48370 Absolutely wrong calculations with GROUP BY and
decimal fields when using IF
Added the test cases in the above two bugs for regression
testing.
Added additional tests that demonstrate a incomplete fix.
Added a new factory method for Field_new_decimal to
create a field from an (decimal returning) Item.
In the new method made sure that all the precision and
length variables are capped in a proper way.
This is required because Item's can have larger precision
than the decimal fields and thus need to be capped when
creating a field based on an Item type.
Fixed the wrong typecast to Item_decimal.
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
STRING_RESULT argument
There is a "magic" number for precision : NOT_FIXED_DEC.
This means that the precision is not a fixed number.
But this constant was re-defined in several files and
was not available to the UDF developers.
Moved the NOT_FIXED_DEC definition to the correct header
and removed the redundant definitions.
Backported to 5.6.0 (mysql-next-mr-runtime)
client/sql_string.h:
moved NOT_FIXED_DEC to the correct header
include/mysql_com.h:
moved NOT_FIXED_DEC to the correct header
libmysql/libmysql.c:
moved NOT_FIXED_DEC to the correct header
sql/field.h:
moved NOT_FIXED_DEC to the correct header
sql/sql_string.h:
moved NOT_FIXED_DEC to the correct header
storage/ndb/include/kernel/signaldata/DictTabInfo.hpp:
moved NOT_FIXED_DEC to the correct header
STRING_RESULT argument
There is a "magic" number for precision : NOT_FIXED_DEC.
This means that the precision is not a fixed number.
But this constant was re-defined in several files and
was not available to the UDF developers.
Moved the NOT_FIXED_DEC definition to the correct header
and removed the redundant definitions.
Backported to 5.6.0 (mysql-next-mr-runtime)
Conflicts
=========
Text conflict in .bzr-mysql/default.conf
Text conflict in libmysqld/CMakeLists.txt
Text conflict in libmysqld/Makefile.am
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/extra/rpl_tests/rpl_row_sp006.test
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_create_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_sp006.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/Makefile.am
Text conflict in sql/log_event_old.cc
Text conflict in sql/rpl_rli.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_binlog.cc
Text conflict in sql/sql_lex.h
21 conflicts encountered.
NOTE
====
mysql-5.1-rpl-merge has been made a mirror of mysql-next-mr:
- "mysql-5.1-rpl-merge$ bzr pull ../mysql-next-mr"
This is the first cset (merge/...) committed after pulling
from mysql-next-mr.
Conflicts
=========
Text conflict in .bzr-mysql/default.conf
Text conflict in libmysqld/CMakeLists.txt
Text conflict in libmysqld/Makefile.am
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/extra/rpl_tests/rpl_row_sp006.test
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_create_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_sp006.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/Makefile.am
Text conflict in sql/log_event_old.cc
Text conflict in sql/rpl_rli.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_binlog.cc
Text conflict in sql/sql_lex.h
21 conflicts encountered.
NOTE
====
mysql-5.1-rpl-merge has been made a mirror of mysql-next-mr:
- "mysql-5.1-rpl-merge$ bzr pull ../mysql-next-mr"
This is the first cset (merge/...) committed after pulling
from mysql-next-mr.
----------------------------------------------------------
revno: 2630.22.8
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-runtime
timestamp: Sun 2008-08-10 18:49:52 +0400
message:
Get rid of typedef struct for the most commonly used types:
TABLE, TABLE_SHARE, LEX. This simplifies use of tags
and forward declarations.
