when replicating
The function create_virtual_tmp_table does not
set db_low_byte_first in the same way as
create_tmp_table does, causing copying from
the virtual table to a real table to get strange
values for SET types on big-endian machines.
mysql-test/extra/rpl_tests/type_conversions.test:
Adding tests for converting between different-sized sets.
sql/sql_select.cc:
Settng dh_low_byte_first for tables created
with create_virtual_tmp_table.
replicating
Fixes to make mysqlbinlog work correctly and updates
to rpl_ndb result files.
sql/log_event.cc:
The processing of the MYSQL_STRING_TYPE is now done
in table_def::type(), so the support functions for
mysqlbinlog are changed to work correctly.
for InnoDB
The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.
Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.
To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.
mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
Adding test to check compatibility for bit field
replication when using InnoDB.
sql/field.cc:
Extending compatible_field_size() with flags from
table map to allow fields to check master info.
sql/field.h:
Extending compatible_field_size() with flags from
table map to allow fields to check master info.
sql/log.cc:
Removing table map flags since they are not used
outside table map class.
sql/log_event.cc:
Removing flags parameter from table map constructor
since it is not used and does not have to be exposed.
sql/log_event.h:
Adding flag to denote that bit length for bit field type
is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
Removing obsolete comment and adding flags to store
table map flags from master.
Row-based replication requires the types of columns on the
master and slave to be approximately the same (some safe
conversions between strings are allowed), but does not
allow safe conversions between fields of similar types such
as TINYINT and INT.
This patch implement type conversions between similar fields
on the master and slave.
The conversions are controlled using a new variable
SLAVE_TYPE_CONVERSIONS of type SET('ALL_LOSSY','ALL_NON_LOSSY').
Non-lossy conversions are any conversions that do not run the
risk of losing any information, while lossy conversions can
potentially truncate the value. The column definitions are
checked to decide if the conversion is acceptable.
If neither conversion is enabled, it is required that the
definitions of the columns are identical on master and slave.
Conversion is done by creating an internal conversion table,
unpacking the master data into it, and then copy the data to
the real table on the slave.
.bzrignore:
New files added
client/Makefile.am:
New files added
client/mysqlbinlog.cc:
Functions in rpl_utility.cc is now needed by mysqlbinlog.cc.
libmysqld/Makefile.am:
New files added
mysql-test/extra/rpl_tests/check_type.inc:
Test include file to check a single type conversion.
mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
Switching to use INT instead of TEXT for column that should not have matching types.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
Adding code to enable type conversions for BIT tests since InnoDB
cannot handle them properly due to incorrect information stored as
metadata.
mysql-test/extra/rpl_tests/type_conversions.test:
Test file to check a set of type conversions
with current settings of slave_type_conversions.
mysql-test/suite/rpl/t/rpl_typeconv.test:
Test file to test conversions from master to slave with
all possible values for slave_type_conversions.
The test also checks that the slave_type_conversions
variable works as expected.
sql/field.cc:
Changing definition of compatible_field_size to both check if
two field with identical base types are compatible and give an
order between them if they are compatible.
This only implement checking on the slave, so it will not affect
replication from an old master to a new slave.
sql/field.h:
Changing prototypes for functions:
- compatible_field_size()
- init_for_tmp_table()
- row_pack_length()
sql/log_event.cc:
Changing compability checks to build a conversion table if the fields
are compatible, but does not have the same base type.
sql/log_event_old.cc:
Changing compability checks to build a conversion table if the fields
are compatible, but does not have the same base type.
sql/mysql_priv.h:
Adding global option variable for SLAVE_TYPE_CONVERSIONS
sql/mysqld.cc:
Adding SLAVE_TYPE_CONVERSIONS global server variable.
sql/rpl_record.cc:
Changing unpack_row to use the conversion table if present.
sql/rpl_rli.h:
Removing function get_tabledef and replacing it with get_table_data().
This function retrieve data for table opened for replication, not just
table definition.
sql/rpl_utility.cc:
Function table_def::compatible_with is changed to compare table on master
and slave for compatibility and generate a conversions table if they are
compatible.
Computing real type of fields from metadata for ENUM and SET types.
Computing pack_length correctly for ENUM, SET, and BLOB types.
Adding optimization to not check compatibility if no
slave type conversions are enabled.
sql/rpl_utility.h:
Changing prototypes since implementation has changed.
Modifying table_def::type() to return real type instead of stored type.
sql/set_var.cc:
Adding SLAVE_TYPE_CONVERSIONS variable.
sql/set_var.h:
Adding SLAVE_TYPE_CONVERSIONS variable.
sql/share/errmsg.txt:
Adding error messages for slave type conversions.
sql/sql_class.h:
Adding SLAVE_TYPE_CONVERSIONS variable.
sql/sql_select.cc:
Correcting create_virtual_tmp_table() to compute null bit positions
correctly in the presence of bit fields.
Support for flushing individual logs, so that the user can
selectively flush a subset of the server logs.
Flush of individual logs is done according to the
following syntax:
FLUSH <log_category> LOGS;
The syntax is extended so that the user is able to flush a
subset of logs:
FLUSH [log_category LOGS,];
where log_category is one of:
SLOW
ERROR
BINARY
ENGINE
GENERAL
RELAY.
mysql-test/suite/rpl/r/rpl_flush_logs.result:
Test result for WL#5142.
mysql-test/suite/rpl/t/rpl_flush_logs.test:
Added the test file to verify if the 'flush individual log'
statement works fine.
sql/log.cc:
Added the two functions to flush slow and general log.
sql/sql_parse.cc:
Added code to flush specified logs against the option.
sql/sql_yacc.yy:
Added code to parse the 'flush * log' statement syntax and
set its option to Lex->type.
Fixes several test cases.
applied revisions: r6181, r6182, r6183, r6184
r6183 - changes are made only to tests in innodb suite which is only
innodb-consistent-master.opt
Detailed revision comments:
r6181 | vasil | 2009-11-17 12:21:41 +0200 (Tue, 17 Nov 2009) | 33 lines
branches/zip:
At the end of innodb-index.test: restore the environment as it was before
the test was started to silence this warning:
MTR's internal check of the test case 'main.innodb-index' failed.
This means that the test case does not preserve the state that existed
before the test case was executed. Most likely the test case did not
do a proper clean-up.
This is the diff of the states of the servers before and after the
test case was executed:
mysqltest: Logging to '/tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.log'.
mysqltest: Results saved in '/tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result'.
mysqltest: Connecting to server localhost:13000 (socket /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
mysqltest: ... Connected.
mysqltest: Start processing test commands from './include/check-testcase.test' ...
mysqltest: ... Done processing test commands.
--- /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result 2009-11-17 13:10:40.000000000 +0300
+++ /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.reject 2009-11-17 13:10:54.000000000 +0300
@@ -84,7 +84,7 @@
INNODB_DOUBLEWRITE ON
INNODB_FAST_SHUTDOWN 1
INNODB_FILE_FORMAT Antelope
-INNODB_FILE_FORMAT_CHECK Antelope
+INNODB_FILE_FORMAT_CHECK Barracuda
INNODB_FILE_PER_TABLE OFF
INNODB_FLUSH_LOG_AT_TRX_COMMIT 1
INNODB_FLUSH_METHOD
mysqltest: Result content mismatch
not ok
r6182 | marko | 2009-11-17 13:49:15 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: Set svn:eol-style on mysql-test files.
r6183 | marko | 2009-11-17 13:51:16 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: Prepend loose_ to plugin-only mysql-test options.
r6184 | marko | 2009-11-17 13:52:01 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: innodb-index.test: Restore innodb_file_format_check.
1. add testcase for BUG#46676
2. Allow CREATE INDEX to be interrupted
3. ha_innobase::change_active_index(): When the history is
missing, report it to the client, not to the error log
4. ChangeLog entries
appplied revisions:r6169, r6170, r6175, r6177, r6179
Detailed revision comments:
r6169 | calvin | 2009-11-12 14:40:43 +0200 (Thu, 12 Nov 2009) | 6 lines
branches/zip: add test case for bug#46676
This crash is reproducible with InnoDB plugin 1.0.4 + MySQL 5.1.37.
But no longer reproducible after MySQL 5.1.38 (with plugin 1.0.5).
Add test case to catch future regression.
r6170 | marko | 2009-11-12 15:49:08 +0200 (Thu, 12 Nov 2009) | 4 lines
branches/zip: Allow CREATE INDEX to be interrupted. (Issue #354)
rb://183 approved by Heikki Tuuri
r6175 | vasil | 2009-11-16 20:07:39 +0200 (Mon, 16 Nov 2009) | 4 lines
branches/zip:
Wrap line at 78th char in the ChangeLog
r6177 | calvin | 2009-11-16 20:20:38 +0200 (Mon, 16 Nov 2009) | 2 lines
branches/zip: add an entry to ChangeLog for r6065
r6179 | marko | 2009-11-17 10:19:34 +0200 (Tue, 17 Nov 2009) | 2 lines
branches/zip: ha_innobase::change_active_index(): When the history is
missing, report it to the client, not to the error log.
applied revisions: r6157
Detailed revision comments:
r6157 | jyang | 2009-11-11 14:27:09 +0200 (Wed, 11 Nov 2009) | 10 lines
branches/zip: Fix an issue that a local variable defined
in innodb_file_format_check_validate() is being referenced
across function in innodb_file_format_check_update().
In addition, fix "set global innodb_file_format_check =
DEFAULT" call.
Bug #47167: "set global innodb_file_format_check" cannot
set value by User-Defined Variable."
rb://169 approved by Sunny Bains and Marko.
The 'slave_patternload_file' is assigned to the real path of the load data file
when initializing the object of Relay_log_info. But the path of the load data
file is not formatted to real path when executing event from relay log. So the
error will be encountered if the path of the load data file is a symbolic link.
Actually the global 'opt_secure_file_priv' is not formatted to real path when
loading data from file. So the same thing will happen too.
To fix these errors, the path of the load data file should be formatted to
real path when executing event from relay log. And the 'opt_secure_file_priv'
should be formatted to real path when loading data infile.
mysql-test/suite/rpl/r/rpl_loaddata_symlink.result:
Test result for bug#43913.
mysql-test/suite/rpl/t/rpl_loaddata_symlink-master.sh:
Added the test file to create a link from $MYSQLTEST_VARDIR/std_data
to $MYSQLTEST_VARDIR/std_data_master_link
mysql-test/suite/rpl/t/rpl_loaddata_symlink-slave.sh:
Added the test file to create a link from $MYSQLTEST_VARDIR/std_data
to $MYSQLTEST_VARDIR/std_data_slave_link
mysql-test/suite/rpl/t/rpl_loaddata_symlink.test:
Added the test file to verify if loading data infile will work fine
if the path of the load data file is a symbolic link.
sql/rpl_rli.cc:
Added call 'my_realpath' function for avoiding sometimes the 'fn_format'
function can't format real path rightly.
<tmp_tbl> with RBL
When binlogging the statement, the server always handle the existing
object as a table, even though it is a view. However a view is
handled differently in other parts of the code thus leading the
statement to crash in RBL if the view exists.
This happens because the underlying tables for the view are not opened
when we try to call store_create_info() on the view in order to build
a CREATE TABLE statement.
This patch will only address the crash problem, other binlogging
problems related to CREATE TABLE IF NOT EXISTS LIKE when the existing
object is a view will be solved by BUG 47442.
In RBR, All statements operating on temporary tables should not be binlogged.
Despite this fact, after executing 'TRUNCATE... ' on a temporary table,
the command is still logged, even if in row-based mode. Consequently, this raises
problems in the slave as the table may not exist, resulting in an
execution failure. Ultimately, this causes the slave to report
an error and abort.
After this patch, 'TRUNCATE ...' statement on a temporary table will not be
binlogged in RBR.
beyond unsigned long.
BUG#44779: binlog.binlog_max_extension may be causing failure on
next test in PB
NOTE1: this is the backport to next-mr.
NOTE2: already includes patch for BUG#44779.
Binlog file extensions would turn into negative numbers once the
variable used to hold the value reached maximum for signed
long. Consequently, incrementing value to the next (negative) number
would lead to .000000 extension, causing the server to fail.
This patch addresses this issue by not allowing negative extensions
and by returning an error on find_uniq_filename, when the limit is
reached. Additionally, warnings are printed to the error log when the
limit is approaching. FLUSH LOGS will also report warnings to the
user, if the extension number has reached the limit. The limit has been
set to 0x7FFFFFFF as the maximum.
mysql-test/suite/binlog/t/binlog_max_extension.test:
Test case added that checks the maximum available number for
binlog extensions.
sql/log.cc:
Changes to find_uniq_filename and test_if_number.
sql/log.h:
Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.
The mentioned on the bug report set of bugs fixes have not be pushed to the main trees.
Fixed with extracting commits done to 6.0-rpl tree and applying them to the main 5.1.
Notes.
1. part of changes - the mtr's specific - were packported to the main 5.0 tree for mtr v1
as http://lists.mysql.com/commits/46562
However, there is no that fix anymore in the mtr v2. (This fact was mailed to mtr maintaining
people).
2. Bug@36929 crash in kill_zombie_dump_threads-> THD::awake() with replication tests
is not backported because the base code of the patch is libevent and that was removed
from the main trees due to its instability.
client/mysqlbinlog.cc:
fixes for BUG#35546
mysql-test/suite/rpl/r/rpl_bug41902.result:
the new tests result file is added.
mysql-test/suite/rpl/t/rpl_bug41902-slave.opt:
conf file for bug41902 testing is added.
mysql-test/suite/rpl/t/rpl_bug41902.test:
regression tests for Bug #41902 is added.
sql/log.cc:
collection of changes due to Bug #48463.
sql/log.h:
collection of changes due to Bug #48463.
sql/rpl_rli.h:
collection of changes due to Bug #48463.
sql/slave.cc:
collection of changes due to Bug #48463.
sql/sql_repl.cc:
collection of changes due to Bug #48463.
Problem: Some system functions that could return different values on
master and slave were not marked unsafe. In particular:
GET_LOCK
IS_FREE_LOCK
IS_USED_LOCK
MASTER_POS_WAIT
RELEASE_LOCK
SLEEP
SYSDATE
VERSION
Fix: Mark these functions unsafe.
mysql-test/extra/rpl_tests/rpl_stm_000001.test:
- The test does not work in mixed mode any more, since it tries to
simulate an error in the sql thread in a query that uses get_lock.
Since get_lock now causes the query to be logged in row format,
the error didn't happen. Hence, we now force statement mode.
- Warnings must be disabled when the unsafe query is issued.
- Replaced some save_master_pos+connection slave+sync_with_master
by sync_slave_with_master.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_row.result:
updated result file
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file
mysql-test/suite/binlog/t/binlog_killed.test:
binlog_killed only works in statement format now, since
it switches to row mode in mixed mode.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test:
suppress warnings for unsafe statements
mysql-test/suite/binlog/t/binlog_stm_row.test:
- Suppress warnings in test that causes warnings.
- The test sets binlog format explicitly, so no need to execute it
twice.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Added test for all unsafe system functions. This test also includes
system functions that were unsafe prior to BUG#47995.
mysql-test/suite/rpl/r/rpl_err_ignoredtable.result:
updated result file
mysql-test/suite/rpl/r/rpl_get_lock.result:
updated result file
mysql-test/suite/rpl/r/rpl_nondeterministic_functions.result:
new result file
mysql-test/suite/rpl/r/rpl_stm_000001.result:
updated result file
mysql-test/suite/rpl/r/rpl_trigger.result:
updated result file
mysql-test/suite/rpl/t/rpl_err_ignoredtable.test:
- suppress warnings for unsafe statement
- replaced save_master_pos+connection slave+sync_with_master
with sync_slave_with_master
mysql-test/suite/rpl/t/rpl_get_lock.test:
update test case that causes new warnings
mysql-test/suite/rpl/t/rpl_nondeterministic_functions.test:
Added new test case for nondeterministic functions.
mysql-test/suite/rpl/t/rpl_trigger.test:
update test case that causes new warnings
sql/item_create.cc:
Marked some system functions unsafe.
sql/item_strfunc.cc:
Clarified comment related to this bug.
sql/sql_yacc.yy:
Marked sysdate unsafe.
One statement that have more than one different tables to update with
autoinc columns just was marked as unsafe in mixed mode, so the unsafe
warning can't be produced in statement mode.
To fix the problem, mark the statement as unsafe in statement mode too.
mysql-test/extra/rpl_tests/rpl_insert_id.test:
The test case is updated due to the patch of bug#45827.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
The test result is updated due to the patch of bug#45827.
mysql-test/suite/binlog/r/binlog_unsafe.result:
Test result for bug#45827.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Added test to verify if stmt that have more than one
different tables to update with autoinc columns will
produce unsafe warning
mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result:
The test result is updated due to the patch of bug#45827.
mysql-test/suite/rpl/t/rpl_trigger.test:
The test case is updated due to the patch of bug#45827.
mysql-test/suite/rpl/t/rpl_variables_stm.test:
The test case is updated due to the patch of bug#45827.
sql/sql_base.cc:
Reomved the 'set_current_stmt_binlog_row_based_if_mixed' function
for producing unsafe warnings by executing 'decide_logging_format'
function later in statement mode
Detailed revision comments:
r6101 | jyang | 2009-10-23 11:45:50 +0300 (Fri, 23 Oct 2009) | 7 lines
branches/zip: Update test result with the WARN_LEVEL_ERROR
to WARN_LEVEL_WARN change. This is the same result as
submitted in rb://172 review, which approved by Sunny Bains
and Marko.
Detailed revision comments:
r6100 | jyang | 2009-10-22 06:51:07 +0300 (Thu, 22 Oct 2009) | 6 lines
branches/zip: As a request from mysql, WARN_LEVEL_ERROR cannot
be used for push_warning_* call any more. Switch to
WARN_LEVEL_WARN. Bug #47233.
rb://172 approved by Sunny Bains and Marko.
Detailed revision comments:
r6095 | vasil | 2009-10-19 16:04:59 +0300 (Mon, 19 Oct 2009) | 7 lines
branches/zip:
Fix Bug#47808 innodb_information_schema.test fails when run under valgrind
by using the wait_until_rows_count macro that loops until the number of
rows becomes 14 instead of sleep 0.1, which is obviously very fragile.
The 'rpl_get_master_version_and_clock' test verifies if the slave I/O
thread tries to reconnect to master when it tries to get the values of
the UNIX_TIMESTAMP, SERVER_ID from master under network disconnection.
So the master server is restarted for making the transient network
disconnection. Restarting master server can bring two problems as following:
1. The time out error is encountered sporadically. The slave I/O thread tries
to reconnect master ten times, which is set in my.cnf. So in the test
framework sporadically the slave I/O thread really stoped when it can't
reconnect to master in the ten times successfully before the master starts,
then the time out error will be encountered while waiting for the slave to
start.
2. These warnings and errors are produced in server log file when
the slave I/O thread tries to get the values of the UNIX_TIMESTAMP,
SERVER_ID from master under the transient network disconnection.
To fix problem 1, increase the master retry count to sixty times,
so that the slave I/O thread has enough time to reconnect master
successfully.
To fix problem 2, suppress these warnings and errors by mtr suppression,
because they are expected.
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock-slave.opt:
Added the *.opt file for increasing master retry count to
sixty times.
mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test:
Added mtr suppression for suppressing warnings and errors
in server log file.
binlog, replication aborts
In SBR or MBR, the schema name is not being written to the binlog
when executing a LOAD DATA statement. This becomes a problem when
the current database (lets call it db1) is different from the
table's schema (lets call it db2). For instance, take the
following statements:
use db1;
load data local infile 'infile.txt' into table db2.t
Should this statement be logged without t's schema (db2), when
replaying it, one can get db1.t populated instead of db2.t (if
db1.t exists). On the other hand, if there is no db1.t at all,
replication will stop.
We fix this by always logging the table (in load file) with fully
qualified name when its schema is different from the current
database or when no default database was selected.
CMakeLists.txt:
Add plugin/semisync subdirectory
mysql-test/mysql-test-run.pl:
Check for semisync dll for Windows
mysql-test/suite/rpl/r/rpl_semi_sync.result:
Update result file
mysql-test/suite/rpl/t/rpl_semi_sync.test:
Test semi-sync on Windows
plugin/semisync/semisync_master.cc:
Define gettimeofday for Windows
