'LOAD DATA CONCURRENT [LOCAL] INFILE ...' statment only is binlogged as
'LOAD DATA [LOCAL] INFILE ...' in SBR and MBR. As a result, if replication is on,
queries on slaves will be blocked by the replication SQL thread.
This patch write code to write 'CONCURRENT' into the log event if 'CONCURRENT' option
is in the original statement in SBR and MBR.
Conflicts
=========
Text conflict in .bzr-mysql/default.conf
Text conflict in libmysqld/CMakeLists.txt
Text conflict in libmysqld/Makefile.am
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/extra/rpl_tests/rpl_row_sp006.test
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_create_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_sp006.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/Makefile.am
Text conflict in sql/log_event_old.cc
Text conflict in sql/rpl_rli.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_binlog.cc
Text conflict in sql/sql_lex.h
21 conflicts encountered.
NOTE
====
mysql-5.1-rpl-merge has been made a mirror of mysql-next-mr:
- "mysql-5.1-rpl-merge$ bzr pull ../mysql-next-mr"
This is the first cset (merge/...) committed after pulling
from mysql-next-mr.
Backporting BUG#43789 to mysql-5.1-bugteam
The replication was generating corrupted data, warning messages on Valgrind
and aborting on debug mode while replicating a "null" to "not null" field.
Specifically the unpack_row routine, was considering the slave's table
definition and trying to retrieve a field value, where there was nothing to be
retrieved, ignoring the fact that the value was defined as "null" by the master.
To fix the problem, we proceed as follows:
1 - If it is not STRICT sql_mode, implicit default values are used, regardless
if it is multi-row or single-row statement.
2 - However, if it is STRICT mode, then a we do what follows:
2.1 If it is a transactional engine, we do a rollback on the first NULL that is
to be set into a NOT NULL column and return an error.
2.2 If it is a non-transactional engine and it is the first row to be inserted
with multi-row, we also return the error. Otherwise, we proceed with the
execution, use implicit default values and print out warning messages.
Unfortunately, the current patch cannot mimic the behavior showed by the master
for updates on multi-tables and multi-row inserts. This happens because such
statements are unfolded in different row events. For instance, considering the
following updates and strict mode:
(master)
create table t1 (a int);
create table t2 (a int not null);
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
t1 would have (10) and t2 would have (0) as this would be handled as a
multi-row update. On the other hand, if we had the following updates:
(master)
create table t1 (a int);
create table t2 (a int);
(slave)
create table t1 (a int);
create table t2 (a int not null);
(master)
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
On the master t1 would have (10) and t2 would have (NULL). On
the slave, t1 would have (10) but the update on t1 would fail.
vs not null
NOTE: Backporting the patch to next-mr.
The replication was generating corrupted data, warning messages on Valgrind
and aborting on debug mode while replicating a "null" to "not null" field.
Specifically the unpack_row routine, was considering the slave's table
definition and trying to retrieve a field value, where there was nothing to be
retrieved, ignoring the fact that the value was defined as "null" by the master.
To fix the problem, we proceed as follows:
1 - If it is not STRICT sql_mode, implicit default values are used, regardless
if it is multi-row or single-row statement.
2 - However, if it is STRICT mode, then a we do what follows:
2.1 If it is a transactional engine, we do a rollback on the first NULL that is
to be set into a NOT NULL column and return an error.
2.2 If it is a non-transactional engine and it is the first row to be inserted
with multi-row, we also return the error. Otherwise, we proceed with the
execution, use implicit default values and print out warning messages.
Unfortunately, the current patch cannot mimic the behavior showed by the master
for updates on multi-tables and multi-row inserts. This happens because such
statements are unfolded in different row events. For instance, considering the
following updates and strict mode:
(master)
create table t1 (a int);
create table t2 (a int not null);
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
t1 would have (10) and t2 would have (0) as this would be handled as a
multi-row update. On the other hand, if we had the following updates:
(master)
create table t1 (a int);
create table t2 (a int);
(slave)
create table t1 (a int);
create table t2 (a int not null);
(master)
insert into t1 values (1);
insert into t2 values (2);
update t1, t2 SET t1.a=10, t2.a=NULL;
On the master t1 would have (10) and t2 would have (NULL). On
the slave, t1 would have (10) but the update on t1 would fail.
"load data" statements were written to the binlog as a mix of the original statement
and bits recreated from parse-info. This relied on implementation details and broke
with IGNORE_SPACES and versioned comments.
We now completely resynthesize the query for LOAD DATA for binlog (which among other
things normalizes them somewhat with regard to case, spaces, etc.).
We have already parsed the query properly, so we make use of that rather
than mix-and-match string literals and parsed items.
This should make us safe with regard to versioned comments, even those
spanning multiple tokens. Also no longer affected by IGNORE_SPACES.
mysql-test/r/mysqlbinlog.result:
LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_killed_simulate.result:
LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_loaddata.result:
LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_loaddata_fatal.result:
LOAD DATA INFILE normalized; offsets adjusted to reflect that
mysql-test/suite/rpl/r/rpl_loaddata_map.result:
LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_loaddatalocal.result:
test for #43746 - trying to break LOAD DATA part of parser
mysql-test/suite/rpl/r/rpl_stm_log.result:
LOAD DATA INFILE normalized
mysql-test/suite/rpl/t/rpl_loaddatalocal.test:
try to break the LOAD DATA part of the parser (test for #43746)
mysql-test/t/mysqlbinlog.test:
LOAD DATA INFILE normalized; adjust offsets to reflect that
sql/log_event.cc:
clean up Load_log_event::print_query and friends so they don't print
excess spaces. add support for printing charset names to print_query.
sql/log_event.h:
We already have three places where we synthesize LOAD DATA queries.
Better use one of those!
sql/sql_lex.h:
When binlogging LOAD DATA statements, we make up the statement to
be logged (from the parse-info, rather than substrings of the
original query) now. Consequently, we no longer need (string-)
pointers into the original query.
sql/sql_load.cc:
Completely rewrote write_execute_load_query_log_event() to synthesize the
LOAD DATA statement wholesale, rather than piece it together from
synthesized bits and literal excerpts from the original query. This
will not only give us a nice, normalized statement (all uppercase,
no excess spaces, etc.), it will also handle comments, including
versioned comments right, which is certainly more than we can say
about the previous incarnation.
sql/sql_yacc.yy:
We're no longer assembling LOAD DATA statements from bodyparts of the
original query, so some bookkeeping in the parser can go.
mysqlbinlog --database parameter was being ignored when processing
row events. As such no event filtering would take place.
This patch addresses this by deploying a call to shall_skip_database
when table_map_events are handled (as these contain also the name of
the database). All other rows events referencing the table id for the
filtered map event, will also be skipped.
client/mysqlbinlog.cc:
Added shall_skip_database call to the part of the code that handles
Table_map_log_events. It inspects the database name and decides whether
to filter the event or not. Furthermore, if table map event is filtered
next events referencing the table id in the table map event, will also
be filtered.
mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_db_filter.test:
Test case that checks if row events are actually filtered out.
sql/log_event.h:
Added a map for holding the currently ignored table map events.
Table map events are inserted when they shall be skipped and removed
once the last row event in the statement is processed.
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
sql/log_event.cc:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument instead of figuring it out by itself
sql/log_event.h:
Changed constructors of Query_log_event and Execute_load_log_event to accept the error code argument
If secure-file-priv was set on slave, it became unable to execute
LOAD DATA INFILE statements sent from master using mixed or
statement-based replication.
This patch fixes the issue by ignoring this security restriction
and checking if the files are created and read by the slave in the
--slave-load-tmpdir while executing the SQL Thread.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
conflicts:
Text conflict in client/mysqltest.cc
Text conflict in mysql-test/include/wait_until_connected_again.inc
Text conflict in mysql-test/lib/mtr_report.pm
Text conflict in mysql-test/mysql-test-run.pl
Text conflict in mysql-test/r/events_bugs.result
Text conflict in mysql-test/r/log_state.result
Text conflict in mysql-test/r/myisam_data_pointer_size_func.result
Text conflict in mysql-test/r/mysqlcheck.result
Text conflict in mysql-test/r/query_cache.result
Text conflict in mysql-test/r/status.result
Text conflict in mysql-test/suite/binlog/r/binlog_index.result
Text conflict in mysql-test/suite/binlog/r/binlog_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_packet.result
Text conflict in mysql-test/suite/rpl/t/rpl_packet.test
Text conflict in mysql-test/t/disabled.def
Text conflict in mysql-test/t/events_bugs.test
Text conflict in mysql-test/t/log_state.test
Text conflict in mysql-test/t/myisam_data_pointer_size_func.test
Text conflict in mysql-test/t/mysqlcheck.test
Text conflict in mysql-test/t/query_cache.test
Text conflict in mysql-test/t/rpl_init_slave_func.test
Text conflict in mysql-test/t/status.test
Adding comments to some of the high-level functions in replication.
sql/log_event.h:
Fixed some mistakes in comments.
sql/repl_failsafe.cc:
Added comment for show_slave_hosts()
sql/slave.cc:
Added comment for show_master_info(), handle_slave_[sql|io](), and next_event()
sql/sql_binlog.cc:
Added @param comment.
sql/sql_lex.h:
Added comment for st_lex_master_info.
sql/sql_repl.cc:
Added comments for functions executing a statement:
PURGE BINARY LOGS
START SLAVE
STOP SLAVE
RESET SLAVE
CHANGE MASTER
RESET MASTER
SHOW BINLOG EVENTS
SHOW MASTER STATUS
SHOW BINARY LOGS
Problem: when the server reads a log_event from file, it should read
the post-header lengths from the format_description_log_event. Some
event types which currently have post-header length 0 did not do this,
and instead had a hard-coded zero length for the post-header. That
means the current server version will not be able to read future
versions of these events.
Fix: make the reader functions read the post-header.
sql/log_event.cc:
- Made Format_description_log_event constructor initialize all
post-header lengths explicitly, to make it easier to find them
in the source code.
- After this, it is no longer necessary to pass the MY_ZEROFILL
flag to my_malloc. I removed the flag and added a sanity-check
that will be executed only in debug-mode.
- Made INTVAR, RAND, USER_VAR, and XID events skip post_header_len
when reading from file.
sql/log_event.h:
Added explicit defines for the lengths of all event types.
Problem: When an Incident_log_event contains a bad incident number on disk,
the server crashes with an assertion.
Fix: Don't validate input with assertions. Use errors.
mysql-test/include/cleanup_fake_relay_log.inc:
Added auxiliary file to restore things that setup_fake_relay_log.inc did.
mysql-test/include/setup_fake_relay_log.inc:
Added auxiliary file to setup replication from an existing relay log.
mysql-test/std_data/bug40482-bin.000001:
Binlog file for rpl.rpl_binlog_corruption
mysql-test/suite/rpl/t/rpl_binlog_corruption.test:
New test file.
sql/log_event.cc:
Check that the incident number is correct at the time the event is constructed.
Do not assert it at the time it is printed.
sql/log_event.h:
Incident_log_event::is_valid() should verify that the incident number is valid.
sql/rpl_constants.h:
Incident numbers should be hard-coded, since they may appear in files.
Rotate event is automatically generated and written when rotating binary
log or relay log. Rotate events for relay logs are usually ignored by slave
SQL thread becuase they have the same server id as that of the slave.
However, if --replicate-same-server-id is enabled, rotate event
for relay log would be treated as if it's a rotate event from master, and
would be executed by slave to update the rli->group_master_log_name and
rli->group_master_log_pos to a wrong value and cause the MASTER_POS_WAIT
function to fail and return NULL.
This patch fixed this problem by setting a flag bit (LOG_EVENT_RELAY_LOG_F)
in the event to tell the SQL thread to ignore these Rotate events generated
for relay logs.
This patch also added another binlog event flag bit (LOG_EVENT_ARTIFICIAL_F)
to distinquish faked events, the method used before this was by checking if
log_pos was zero.
sql/log.h:
Add a member to MYSQL_BIN_LOG to distinguish binary log from relay log.
sql/log_event.cc:
Change artificial_event member to LOG_EVENT_ARTIFICIAL_F flag
If LOG_EVENT_RELAY_LOG_F is set in the event flags for a rotate event, ignore it when updating position
Refactored the code in Rotate_log_event::do_update_pos
sql/log_event.h:
Add LOG_EVENT_RELAY_LOG_F flag to Log_event flags
Add RELAY_LOG flag to Rotate_log_event flags
sql/sql_repl.cc:
Set LOG_EVENT_ARTIFICIAL_F for fake rotate events
- Implementing --base64-format=decode-rows, to display
SQL-alike decoded row events without their BINLOG statements.
- Adding --base64-format=decode-rows into tests when
calling mysqlbinlog to avoid non-deterministic results
- Removing resetting of last_table_id in "RESET MASTER",
which appeared to be dangerous.
Implementing -v command line parameter to mysqlbinlog
to decode and print row events.
mysql-test/include/mysqlbinlog_row_engine.inc
mysql-test/r/mysqlbinlog_row.result
mysql-test/r/mysqlbinlog_row_big.result
mysql-test/r/mysqlbinlog_row_innodb.result
mysql-test/r/mysqlbinlog_row_myisam.result
mysql-test/r/mysqlbinlog_row_trans.result
mysql-test/t/mysqlbinlog_row.test
mysql-test/t/mysqlbinlog_row_big.test
mysql-test/t/mysqlbinlog_row_innodb.test
mysql-test/t/mysqlbinlog_row_myisam.test
mysql-test/t/mysqlbinlog_row_trans.test
Adding tests
client/Makefile.am
Adding new files to symlink
client/mysqlbinlog.cc
Adding -v option
sql/log_event.cc
Impelentations of the new methods
sql/log_event.h
Declaration of the new methods and member
sql/mysql_priv.h
Adding new function prototype
sql/rpl_tblmap.cc
Adding pre-processor conditions
sql/rpl_tblmap.h
Adding pre-processor conditions
sql/rpl_utility.h
Adding pre-processor conditions
sql/sql_base.cc
Adding reset_table_id_sequence() function.
sql/sql_repl.cc
Resetting table_id on "RESET MASTER"
.bzrignore
Ignoring new symlinked files
The problem of this bug is that we need to get the list of tables
to be updated for a multi-table update statement, which requires to
open all the tables referenced by the statement and resolve all
the fields involved in update in order to figure out the list of
tables for update. However if there are replicate filter rules,
some tables might not exist on slave and result in a failure
before we could examine the filter rules.
I think the whole problem can not be solved on slave alone,
the master must record and send the information of tables
involved for update to slave, so that the slave do not need to
open all the tables referenced by the multi-table update statement to
figure out which tables are involved for update.
So a status variable is added to Query_log event to store the
value of table map for update on master. And on slave, it will
try to get the value of this variable and use it to examine
filter rules without opening any tables on slave, if this values
is not available, the old approach is used and thus the bug will
still occur for when replicating from old masters.
sql/sql_class.h:
add member table_map_for_update to THD
sql/sql_parse.cc:
check filter rules by using table_map_for_update value
sql/sql_update.cc:
save the value of table_map_for_update
into mats-laptop.(none):/home/bk/b29020-mysql-5.1-rpl
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Auto merged
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Auto merged
mysql-test/suite/rpl/r/rpl_row_create_table.result:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/sql_insert.cc:
Auto merged
The bug allow multiple executing transactions working with non-transactional
to interfere with each others by interleaving the events of different trans-
actions.
Bug is fixed by writing non-transactional events to the transaction cache and
flushing the cache to the binary log at statement commit. To mimic the behavior
of normal statement-based replication, we flush the transaction cache in row-
based mode when there is no committed statements in the transaction cache,
which means we are committing the first one. This means that it will be written
to the binary log as a "mini-transaction" with just the rows for the statement.
Note that the changes here does not take effect when building the server with
HAVE_TRANSACTIONS set to false, but it is not clear if this was possible before
this patch either.
For row-based logging, we also have that when AUTOCOMMIT=1, the code now always
generates a BEGIN/COMMIT pair for single statements, or BEGIN/ROLLBACK pair in the
case of non-transactional changes in a statement that was rolled back. Note that
for the case where changes to a non-transactional table causes a rollback due
to error, the statement will now be logged with a BEGIN/ROLLBACK pair, even
though some changes has been committed to the non-transactional table.
mysql-test/extra/rpl_tests/rpl_row_delayed_ins.test:
Removing SHOW BINLOG EVENTS causing test to be non-deterministic.
mysql-test/r/ctype_cp932_binlog_row.result:
Result change.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Result change.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Result file change.
mysql-test/suite/binlog/r/binlog_row_binlog.result:
Result file change.
mysql-test/suite/binlog/r/binlog_row_ctype_ucs.result:
Result file change.
mysql-test/suite/binlog/r/binlog_row_insert_select.result:
Result file change.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
Result file change.
mysql-test/suite/binlog/r/binlog_stm_binlog.result:
Result file change.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Removing table that will be used in test to prevent failing if preceeding
tests forgot to drop the table.
mysql-test/suite/rpl/r/rpl_rbr_to_sbr.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_create_table.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_delayed_ins.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_flsh_tbls.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_inexist_tbl.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_log.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_log_innodb.result:
Result file change.
mysql-test/suite/rpl/r/rpl_row_until.result:
Result file change.
mysql-test/suite/rpl/r/rpl_slave_skip.result:
Result file change.
mysql-test/suite/rpl/r/rpl_switch_stm_row_mixed.result:
Result file change.
mysql-test/suite/rpl/r/rpl_truncate_2myisam.result:
Result file change.
mysql-test/suite/rpl/t/rpl_row_create_table.test:
Binlog position change.
mysql-test/suite/rpl/t/rpl_row_flsh_tbls.test:
Binlog position change.
mysql-test/suite/rpl/t/rpl_row_mysqlbinlog.test:
Binlog position change. Added stop position to mysqlbinlog argments to prevent
extreneous output.
mysql-test/suite/rpl/t/rpl_row_until.test:
Binlog position change.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
Binlog position change.
mysql-test/suite/rpl/t/rpl_switch_stm_row_mixed.test:
Removing extreneous SHOW BINLOG EVENTS causing test to be non-deterministic.
mysql-test/suite/rpl_ndb/r/rpl_ndb_log.result:
Result change.
sql/log.cc:
Adding variable at_least_one_stmt to denote that there is at least one
statement committed to the transaction cache (but there might be more).
Removing duplicate checks from binlog_end_trans(). The transaction cache
should always be committed or rolled back when this function is called.
Correcting conditions for binlog_rollback() and binlog_commit() and removing
the previous "invisible commit" in favor of always using explicit commits
in the binary log.
sql/log_event.cc:
Marking table map event to be cached. Removing Muted_query_log_event from code.
sql/log_event.h:
Removing unused class Muted_query_log_event.
sql/sql_insert.cc:
Adding missing call to ha_autocommit_or_rollback() for delayed thread. Marking
CREATE-SELECT statements as transactional, since they don't need to be logged.
Problem: in mixed and statement mode, a query that refers to a
system variable will use the slave's value when replayed on
slave. So if the value of a system variable is inserted into a
table, the slave will differ from the master.
Fix: mark statements that refer to a system variable as "unsafe",
meaning they will be replicated by row in mixed mode and produce a warning
in statement mode. There are some exceptions: some variables are actually
replicated. Those should *not* be marked as unsafe.
BUG#34732: mysqlbinlog does not print default values for auto_increment variables
Problem: mysqlbinlog does not print default values for some variables,
including auto_increment_increment and others. So if a client executing
the output of mysqlbinlog has different default values, replication will
be wrong.
Fix: Always print default values for all variables that are replicated.
I need to fix the two bugs at the same time, because the test cases would
fail if I only fixed one of them.
include/m_ctype.h:
Added definition of ILLEGAL_CHARSET_INFO_NUMBER. We just need a symbol
for a number that will never be used by any charset. ~0U should be safe
since charset numbers are sequential, starting from 0.
mysql-test/include/commit.inc:
Upated test to avoid making statements unsafe.
mysql-test/r/commit_1innodb.result:
Updated test needs updated result file.
mysql-test/r/mysqlbinlog.result:
Updated result file.
mysql-test/r/mysqlbinlog2.result:
Updated result file.
mysql-test/r/user_var-binlog.result:
Updated result file.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
Updated result file.
mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result:
Updated result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
Modified test file needs modified result file.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Need to filter out pseudo_thread_id from result since it is
nondeterministic.
mysql-test/suite/binlog/t/binlog_unsafe.test:
Add tests that using variables is unsafe. The 'CREATE VIEW' tests didn't
make sense, so I removed them. SHOW WARNINGS is not necessary either,
because we get warnings for each statement in the result file.
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file.
mysql-test/suite/rpl/r/rpl_skip_error.result:
Updated result file.
mysql-test/suite/rpl/t/rpl_skip_error.test:
The test used @@server_id, which is not safe to replicate, so it would
have given a warning. The way it used @@server_id was hackish (issue a
query on master that removes rows only on master), so I replaced it by a
more robust way to do the same thing (connect to slave and insert the
rows only there).
Also clarified what the test case does.
mysql-test/t/mysqlbinlog2.test:
Use --short-form instead of manually filtering out nondeterministic stuff
from mysqlbinlog (because we added the nondeterministic @@pseudo_thread_id
to the output).
sql/item_func.cc:
Added method of Item_func_get_system_var that indicates whether the given
system variable will be written to the binlog or not.
sql/item_func.h:
Added method of Item_func_get_system_var that indicates whether the given
system variable will be written to the binlog or not.
sql/log_event.cc:
- auto_increment_offset was not written to the binlog if
auto_increment_increment=1
- mysqlbinlog did not output default values for some variables
(BUG#34732). In st_print_event_info, we remember for each variable whether
it has been printed or not. This is achieved in different ways for
different variables:
- For auto_increment_*, lc_time_names, charset_database_number,
we set the default values in st_print_event_info to something
illegal, so that it will look like they have changed the first time
they are seen.
- For charset, sql_mode, pseudo_thread_id, we add a flag to
st_print_event_info which indicates whether the variable has been
printed.
- Since pseudo_thread_id is now printed more often, and its value is
not guaranteed to be constant across different runs of the same
test script, I replaced it by a constant if --short-form is used.
- Moved st_print_event_info constructor from log_event.h to log_event.cc,
since it now depends on ILLEGAL_CHARSET_NUMBER, which is defined in
m_ctype.h, which is better to include from a .cc file than from a header
file.
sql/log_event.h:
Added fields to st_print_event_info that indicate whether some of the
variables have been written or not. Since the initialization of
charset_database_number now depends on ILLEGAL_CHARSET_INFO_NUMBER, which
is defined in a header file, which we'd better not include from this
header file -- I moved the constructor from here to log_event.cc.
sql/set_var.cc:
System variables now have a flag binlog_status, which indicates if they
are written to the binlog. If nothing is specified, all variables are
marked as not written to the binlog (NOT_IN_BINLOG) when created. In this
file, the variables that are written to the binlog are marked with
SESSION_VARIABLE_IN_BINLOG.
sql/set_var.h:
Added flag binlog_status to class sys_var. Added a getter and a
constructor parameter that sets it.
Since I had to change sys_var_thd_enum constructor anyways, I simplified
it to use default values of arguments instead of three copies of the
constructor.
sql/sql_yacc.yy:
Mark statements that refer to a system variable as "unsafe",
meaning they will be replicated by row in mixed mode. Added comment to
explain strange piece of code just above.
mysql-test/include/diff_tables.inc:
New auxiliary test file that tests whether two tables (possibly one on
master and one on slave) differ.
mysql-test/suite/rpl/r/rpl_variables.result:
New test case needs new result file.
mysql-test/suite/rpl/r/rpl_variables_stm.result:
New test file needs new result file.
mysql-test/suite/rpl/t/rpl_variables.test:
Test that INSERT of @@variables is replicated correctly (by switching to
row-based mode).
mysql-test/suite/rpl/t/rpl_variables_stm.test:
Test that replication of @@variables which are replicated explicitly works
as expected in statement mode (without giving warnings).
Minor update with corrections and notes on the binlog format.
This only affects comments, not code.
sql/log_event.h:
Fixes in documentation of binlog format.
Documented Table_map_log_event and packed integer format. Improved
other documentation. No change outside comments.
sql/log_event.h:
Documented Table_map_log_event and packed integer format. Improved
other documentation. No change outside comments.
into mysql1000.(none):/home/andrei/MySQL/FIXES/5.1/bug32971-rbr_error_prop
mysql-test/extra/rpl_tests/rpl_row_tabledefs.test:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
manual merge use local
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
manual merge use local
The error message due to lack of the default value for an extra field
was not as informative as it should be.
Fixed with improving the scheme of gathering, propagating and reporting
errors in applying rows events.
The scheme is in the following.
Any kind of error of processing of a row event incidents are to be
registered with my_error().
In the end Rows_log_event::do_apply_event() invokes rli->report() with the
message to display consisting of all the errors.
This mimics `show warnings' displaying.
A simple test checks three errors in processing an event.
Two hunks - a user level error and pushing it into the list -
have been devoted to already fixed Bug@31702.
Some open issues relating to this artifact listed on BUG@21842 page and
on WL@3679.
Todo: to synchronize the statement in the tests comments on Update and Delete
events may not stop when an extra field does not have a default with wl@3228 spec.
include/my_base.h:
A new handler level error code that is supposed to be mapped to a set of more
specific ER_ user level errors.
mysql-test/extra/rpl_tests/rpl_row_tabledefs.test:
Adding yet another extra fields to see more than one error in show
slave status' report.
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
results changed (the error message etc)
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
results changed
sql/log_event.cc:
Refining slave_rows_error_report to iterate on the list of gathered errors;
Simplifying signature of prepare_record as the function does not call
rli->report to leave that duty to the event's top level code.
sql/log_event.h:
adding a corrupt event error pushing. The error will be seen with
show slave status.
sql/log_event_old.cc:
similar to log_event.cc changes
sql/rpl_record.cc:
prepare_record only pushes an error to the list
sql/rpl_record.h:
signature changed
sql/share/errmsg.txt:
The user level error code that corresponds to HA_ERR_CORRUPT_EVENT.
The error will be reported in show slave status if such a failure happens.
BitKeeper/deleted/.del-show_binlog_events2.inc:
Delete: mysql-test/include/show_binlog_events2.inc
client/mysqlbinlog.cc:
char -> uchar for raw memory.
sql/item_cmpfunc.cc:
Adding cast to remove warning when converting negative integer
to unsigned type.
sql/log_event.cc:
char -> uchar for raw memory.
sql/log_event.h:
char -> uchar for raw memory.
sql/rpl_utility.cc:
Adding cast to remove warning when converting negative integer
to unsigned type.
sql/slave.cc:
char -> uchar for raw memory.
sql/sql_repl.cc:
char -> uchar for raw memory.
sql-common/client.c:
char -> uchar for raw memory.
Main problem: mysql 5.1 cannot read binlogs from 4.1.
Subproblem 1: There is a mistake in sql_ex_info::init. The read_str()
function updates its first argument to point to the next character to
read. However, it is applied only to a copy of the buffer pointer, so the
real buffer pointer is not updated.
Fix 1: do not take a copy of the buffer pointer. The copy was needed
because sql_ex_info::init does not use the const attribute on some of its
arguments. So we add the const attribute, too.
Subproblem 2: The first BINLOG statement is asserted to be a
FORMAT_DESCRIPTION_LOG_EVENT, but 4.1 binlogs begin with START_EVENT_V3.
Fix 2: allow START_EVENT_V3 too.
mysql-test/suite/binlog/std_data/binlog_old_version_4_1.000001:
New BitKeeper file ``mysql-test/suite/binlog/std_data/binlog_old_version_4_1.000001''
mysql-test/suite/binlog/r/binlog_old_versions.result:
Updated result file.
mysql-test/suite/binlog/t/binlog_old_versions.test:
Added a test reading an old 4.1 binlog.
sql/log_event.cc:
1. Added const keyword at the following places:
- input buffer for pretty_print_str
- input buffer for write_str
- input buffer, end pointer, and return value from sql_ex_info::init
2. Fixed the bug by not taking a copy of buf before calling read_str in
sql_ex_info::init().
sql/log_event.h:
Added const keyword to fields of the sql_ex_info struct.
Added const keyword to arguments and return value of sql_ex_info::init
sql/sql_binlog.cc:
The first BINLOG statement must describe the format for future BINLOG
statements. Otherwise, we do not know how to read the BINLOG statement.
Problem: only FORMAT_DESCRIPTION_EVENT is currently allowed as the first
event. Binlogs from 4.1 begin with a START_EVENT_V3, which serves the
same purpose.
Fix: We now allow the first BINLOG statement to be a START_EVENT_V3, as
well as a FORMAT_DESCRIPTION_EVENT.
Problem: Replication fails when master is mysql-5.1-wl2325-5.0-drop6 and
slave is mysql-5.1-new-rpl. The reason is that, in
mysql-5.1-wl2325-5.0-drop6, the event type id's were different than in
mysql-5.1-new-rpl.
Fix (in mysql-5.1-new-rpl):
(1) detect that the server that generated the events uses the old
format, by checking the server version of the format_description_log_event
This patch recognizes mysql-5.1-wl2325-5.0-drop6p13-alpha,
mysql-5.1-wl2325-5.0-drop6, mysql-5.1-wl2325-5.0, mysql-5.1-wl2325-no-dd.
(2) if the generating server is old, map old event types to new event
types using a permutation array.
I've also added a test case which reads binlogs for four different
versions.
mysql-test/suite/binlog/std_data/binlog_old_version_5_1-telco.000001:
BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1-telco.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_row.000001:
BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_row.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_stm.000001:
BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_stm.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1_17.000001:
BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1_17.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1_23.000001:
BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1_23.000001
sql/log_event.cc:
Added code to read events generated by
mysql-5.1-wl2325-5.0-drop6p13-alpha, mysql-5.1-wl2325-5.0-drop6,
mysql-5.1-wl2325-5.0, mysql-5.1-wl2325-no-dd.
More precisely, the event type id's had different numbers in
those versions. To fix, we add a permutation array which maps old_id to
new_id when the format_description_log_event indicates that the
originating server is of the old type. We also need to permute the
post_header_len array accordingly.
sql/log_event.h:
sql/log_event.h@1.169, 2008-01-09 11:34:37+01:00, sven@riska.(none) +5 -1
Added declaration needed in log_event.cc. Also, the destructor of
Format_description_log_event is sometimes called when post_header_len is
null, so we must pass the MY_ALLOW_ZERO_PTR flag to my_free.
mysql-test/suite/binlog/r/binlog_old_versions.result:
Result file for new test.
mysql-test/suite/binlog/t/binlog_old_versions.test:
New test case that loads binlogs from several old versions.
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
client/client_priv.h:
Declared the new option for base64 output.
client/mysqlbinlog.cc:
- Change from using the two-state command line option
"default/--base64-output" to the three-state
"--base64-output=[never|auto|always]"
- Print the FD event even if it is outside the --start-position range.
- Stop if a row event is about to be printed without a preceding FD
event.
- Minor fixes:
* changed 4 to EVENT_TYPE_OFFSET in some places
* Added comments
* before, "mysqlbinlog -xyz" read from stdin; now it does not
(only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
Had to add explicit Format_description_log_event before other BINLOG
statements
mysql-test/t/mysqlbinlog2.test:
we must suppress base64 output in result file because it contains a
timestamp
sql/log_event.cc:
- Made FD events able to print themselves
- Added check that the current FD event knows about the event type, when
an event is about to be read. (Hint to reviewers: I had to re-indent
a big block because of this; use diff -b)
* To get a decent error message, I also added a class method
const char* Log_event::get_type_str(Log_event_type)
which converts number to event type string without having a
Log_event object.
* Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
- Minor fixes:
* Changed return to DBUG_VOID_RETURN
sql/log_event.h:
- Declared enum to describe the three base64_output modes
- Use the enum instead of a flag
- Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
Added error msg.
sql/slave.cc:
- Factored out part of exec_relay_log_event to the new function
apply_event_and_update_pos, because that code is needed when executing
BINLOG statements. (this is be functionally equivalent to the
previous code, except: (1) skipping events is now optional, controlled
by a parameter to the new function (2) the return value of
exec_relay_log_event has changed; see next item).
- Changed returned error value to always be 1. Before, it would return
the error value from apply_log_event, which was unnecessary. This
change is safe because the exact return value of exec_relay_log_event
is never examined; it is only tested to be ==0 or !=0.
- Added comments describing exec_relay_log_event and
apply_event_and_update_pos.
- Minor fixes:
* Removed duplicate declaration of next_event, made queue_event
static.
* Added doxygen code to include this file.
sql/slave.h:
Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
- Made mysql_binlog_statement set the current FD event when the given
event is an FD event. This entails using the new function
apply_event_and_update_pos from slave.cc instead of just calling the
ev->apply method.
- Made mysql_binlog_statement fail if the first BINLOG statement is not
an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
Added test case to verify that:
- my patch fixes the bug
- the new --base64-output flag works as expected
- base64 events not preceded by an FD event give an error
- an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001
without PK
Bug#31609 Not all RBR slave errors reported as errors
bug#32468 delete rows event on a table with foreign key constraint fails
The first two bugs comprise idempotency issues.
First, there was no error code reported under conditions of the bug
description although the slave sql thread halted.
Second, executions were different with and without presence of prim key in
the table.
Third, there was no way to instruct the slave whether to ignore an error
and skip to the following event or to halt.
Fourth, there are handler errors which might happen due to idempotent
applying of binlog but those were not listed among the "idempotent" error
list.
All the named issues are addressed.
Wrt to the 3rd, there is the new global system variable, changeble at run
time, which controls the slave sql thread behaviour.
The new variable allows further extensions to mimic the sql_mode
session/global variable.
To address the 4th, the new bug#32468 had to be fixed as it was staying
in the way.
include/my_bitmap.h:
basic operations with bits of an integer type are added.
mysql-test/extra/rpl_tests/rpl_foreign_key.test:
regression test for bug#32468
mysql-test/extra/rpl_tests/rpl_row_basic.test:
changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/extra/rpl_tests/rpl_row_tabledefs.test:
changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl/r/rpl_foreign_key_innodb.result:
results changed
mysql-test/suite/rpl/r/rpl_idempotency.result:
results changed
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
results changed
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
results changed
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
results changed
mysql-test/suite/rpl/r/rpl_row_mystery22.result:
results changed
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
results changed
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
results changed
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
results changed
mysql-test/suite/rpl/t/rpl_idempotency.test:
extenstions to the test providing testing of complements to the
idempotent error set and checking how slave halts when it faces an error
from the list when the mode is STRICT.
mysql-test/suite/rpl/t/rpl_row_basic_11bugs.test:
changes due to bug#31552/31609 idempotency is not default any longer.
mysql-test/suite/rpl/t/rpl_row_mystery22.test:
changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl_ndb/r/rpl_ndb_dd_advance.result:
results changed
mysql-test/suite/rpl_ndb/r/rpl_row_basic_7ndb.result:
results changed
sql/log_event.cc:
the fix for bug#32468 delete rows event on a table with foreign key constraint fails
ensures the flags are set at proper time so that their values will be caught
by innodb.
reseting the flags is done along the common error and errorless execution
path.
The list of idempotent error is extended with foreign keys related items.
NDB engine write events are designed with the replace sematics in mind.
Therefore the corrsponding ndb handler's flag are (re)set regardless of
the slave's execution mode.
Rows_log_event::write_row() starts using the bool replace argument as its
caller sets it depending on the event's execution mode.
sql/log_event.h:
adding a new member to hold the slave's mode during execution of the event.
sql/mysql_priv.h:
changes to link the command line option with the new global sys var.
sql/mysqld.cc:
introduction of the new command line option.
providing its initialization to a default.
changes to link the command line option with the new global sys var.
sql/rpl_rli.cc:
rli post-event-execution cleanup restores the default bits.
sql/set_var.cc:
The new "standard" sys_var_set class' and the new global system var related
declarations and definitions.
fix_slave_exec_mode() is used as with the update method of a new class so
as at time of the command line arguments parsing.
sql/set_var.h:
new declarations. The class for the new global sys var is based on
yet another new "standard" one.
sql/share/errmsg.txt:
slave_exec_mode setting error;
slave inconsistency error which may be not an error when the intention
is "idempotent". I.e consisting of row-based events binlog is being
applied for the 2nd (more) time.
sql/sql_class.h:
The names for the bits of the new sever slave_exec_mode_options.
mysql-test/suite/rpl/t/rpl_idempotency-master.opt:
innodb is necessary
mysql-test/suite/rpl/t/rpl_idempotency-slave.opt:
innodb is necessary, as well as the tests start with non-default
IDEMPOTENT slave execution mode.
into koti.dsl.inet.fi:/home/elkin/MySQL/5.1-merge-bug27571
client/mysql.cc:
Auto merged
mysql-test/r/ctype_euckr.result:
Auto merged
mysql-test/r/ctype_uca.result:
Auto merged
mysql-test/suite/binlog/r/binlog_killed.result:
Auto merged
mysql-test/suite/binlog/t/binlog_killed.test:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
strings/ctype-euc_kr.c:
Auto merged
mysql-test/suite/rpl/r/rpl_sp_effects.result:
manual merge ul
mysql-test/suite/rpl/t/rpl_sp_effects.test:
manual merge
sql/slave.cc:
leaving for manual merge
sql/sql_delete.cc:
leaving for manual merge
sql/sql_insert.cc:
leaving for manual merge
sql/sql_load.cc:
leaving for manual merge
sql/sql_update.cc:
leaving for manual merge
Query_log_event::error_code
A query can perform completely having the local var error of mysql_$query
zero, where $query in insert, update, delete, load,
and be binlogged with error_code e.g KILLED_QUERY while there is no
reason do to so.
That can happen because Query_log_event consults thd->killed flag to
evaluate error_code.
Fixed with implementing a scheme suggested and partly implemented at
time of bug@22725 work-on. error_status is cached immediatly after the
control leaves the main rows-loop and that instance always corresponds
to `error' the local of mysql_$query functions. The cached value
is passed to Query_log_event constructor, not the default thd->killed
which can be changed in between of the caching and the constructing.
mysql-test/r/binlog_killed.result:
results changed
mysql-test/t/binlog_killed.test:
Demonstrating that effective killing during rows-loop execution leads to the speficied actions:
binlogging with the error for a query modified a not-transactional table or
rolling back effects for transactional table;
fixing possible non-determinism with ID when query_log_enabled;
leave commented out tests for multi-update,delete due to another bug;
removing an obsolete tests template;
changing system rm to --remove_file.
sql/log_event.cc:
adding killed status arg
sql/log_event.h:
added killed status arg
sql/sql_delete.cc:
deploying the update part patch for delete, multi-delete
sql/sql_insert.cc:
deploying the update-part patch for insert..select
sql/sql_load.cc:
deploying the update-part patch for load data.
simulation added.
sql/sql_update.cc:
Impementing the fix as described in the comments left by bug@22725.
Also simulation of killing after the loop that would affect binlogging in the old code.
mysql-test/t/binlog_killed_bug27571-master.opt:
post rows-loop killing simulation's options
mysql-test/t/binlog_killed_bug27571.test:
Checking that if killing happens inbetween of the end of rows loop and
recording into binlog that will not lead to recording any error incl
the killed error.
mysql-test/t/binlog_killed_simulate-master.opt:
simulation options
mysql-test/t/binlog_killed_simulate.test:
tests for
a query (update is choosen) being killed after the row-loop;
load data killed within the loop - effective killed error in the event is gained.
Documented some binlog events using doxygen. More will be done later.
Also fixed typos in other comments and added remarks about dubious code.
Only comments are affected, there is no change to the actual code.
sql/log_event.cc:
Fixed typos in some comments.
Added remarks (as comments) about questionable code.
sql/log_event.h:
Documented the binary format of following binlog events:
Log_event
Query_log_event
Muted_query_log_event
Slave_log_event (partial)
Load_log_event
Intvar_log_event
Rand_log_event
Rotate_log_event (partial)
sql/sql_class.h:
Fixed typo in comment.
When replicating an update pair (before image, after image) under row-based
replication, and the before image is not found on the slave, the after image
was not discared, and was hence read as a before image for the next row.
Eventually, this lead to an after image being read outside the block of rows
in the event, causing an assertion to fire.
This patch fixes this by reading the after image in the event that the row
was not found on the slave, adds some extra debug assertion to catch future
errors earlier, and also adds a few non-debug checks to prevent reading
outside the block of the event.
include/my_base.h:
Adding error code HA_ERR_CORRUPT_EVENT.
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
Result change.
mysql-test/suite/rpl/t/rpl_row_basic_11bugs.test:
Adding test to try to use row-based replication to replicate an
update of a row that doesn't exist on the slave. We should get
an apropriate error and the slave should stop.
sql/log_event.cc:
Adding debug printouts. Adding code to Update_rows_log_event::do_exec_row()
so that the after image is read (and ignored) in the event of an error in
finding the row. This is necessary so that the second pair of images is
read correctly for the next update pair.
Changing logic for ignoring errors to not include update events, since
a "key not found" error or a "record changed" error is not idempotent
for updates, just for deletes and inserts.
sql/log_event.h:
Adding debug assertions to check that row reading is within the events block of rows.
is possible):
When skipping the beginning of a transaction starting with BEGIN, the OPTION_BEGIN
flag was not set correctly, which caused the slave to not recognize that it was
inside a group. This patch sets the OPTION_BEGIN flag for BEGIN, COMMIT, ROLLBACK,
and XID events. It also adds checks if inside a group before decreasing the
slave skip counter to zero.
Begin_query_log_event was not marked that it could not end a group, which is now
corrected.
mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
Correcting slave skip counter to get the correct behaviour.
mysql-test/suite/rpl/r/rpl_slave_skip.result:
Result change.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
Adding tests to check that skipping works for transactions:
- Skipping one group with BEGIN first
- Skipping two groups with BEGIN first
- Skipping one group without BEGIN first but with AUTOCOMMIT=0
- LOAD DATA INFILE under statement-based replication
mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result:
Result change.
sql/log_event.cc:
Adding checks if we're in a group when the slave skip counter is 1.
In that case, we should keep going.
Adding helping member function Log_event::continue_group() denoting
that this event cannot end a group, and if the skip counter indicates
that the group ends after this event, it should not decrease the skip
counter.
Query_log_event will change the OPTION_BEGIN flag for BEGIN, COMMIT, and
ROLLBACK, even when skipping because of a positive skip count, and
Xid_log_event will also affect the OPTION_BEGIN flag, even when being
skipped.
Begin_load_query_log_event cannot end a group, so it is marked to
continue the group.
sql/log_event.h:
Adding helper function Log_event::continue_group().
sql/rpl_rli.h:
Adding Relay_log_info::get_flag() to get the value of a
replication flag.
sql/slave.cc:
Adding debug output and changing debug message.
mysql-test/suite/rpl/data/rpl_bug28618.dat:
New BitKeeper file ``mysql-test/suite/rpl/data/rpl_bug28618.dat''
mysql-test/suite/rpl/t/rpl_slave_skip-slave.opt:
New BitKeeper file ``mysql-test/suite/rpl/t/rpl_slave_skip-slave.opt''
into mysql.com:/nfsdisk1/lars/MERGE/mysql-5.1-merge
mysql-test/suite/ndb/r/ndb_dd_basic.result:
Auto merged
mysql-test/suite/rpl/include/rpl_mixed_dml.inc:
Auto merged
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
Auto merged
sql/field.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/log_event_old.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/rpl_record.cc:
Auto merged
sql/rpl_record.h:
Auto merged
sql/rpl_utility.cc:
Auto merged
sql/rpl_utility.h:
Auto merged
sql/slave.cc:
Auto merged
sql/slave.h:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_string.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/unireg.h:
Auto merged
mysql-test/suite/rpl/t/disabled.def:
Manual merge
mysql-test/suite/rpl_ndb/t/disabled.def:
Manual merge
sql/log_event.cc:
Manual merge
sql/log_event.h:
Manual merge
sql/sql_yacc.yy:
Manual merge
into sita.local:/Users/tsmith/m/bk/maint/51
sql/field.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/rpl_record.h:
Auto merged
sql/rpl_utility.cc:
Auto merged
sql/rpl_utility.h:
Auto merged
sql/slave.h:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
sql/log_event.cc:
Manual merge
sql/log_event.h:
Manual merge
sql/log_event_old.h:
Manual merge
sql/rpl_record.cc:
Manual merge
sql/slave.cc:
Manual merge
not compiled as a replication server - a fix from rpl clone now applied
to 5.1.22 tree.
sql/log_event.cc:
Exclude Rows_log_event members used in event application if
not compiled as a replication server.
sql/log_event.h:
Don't initialize Rows_log_event members used in event application if
not compiled as a replication server.
sql/log_event.cc:
- Rename RELAY_LOG_INFO -> Relay_log_info.
- Rows_log_event fields which are used for event application not
included when compiling in MYSQL_CLIENT mode.
sql/log_event.h:
- Rename RELAY_LOG_INFO -> Relay_log_info.
- Rows_log_event fields which are used for event application not
included when compiling in MYSQL_CLIENT mode.
sql/log_event_old.cc:
- Rename RELAY_LOG_INFO -> Relay_log_info.
sql/log_event_old.h:
- Rename RELAY_LOG_INFO -> Relay_log_info.
sql/sql_yacc.yy:
Reverting to version used in the rpl tree.
into quant.(none):/ext/mysql/bk/mysql-5.1-bug21842-rpl
mysql-test/suite/ndb/r/ndb_dd_basic.result:
Auto merged
sql/field.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/rpl_record.cc:
Auto merged
sql/rpl_record.h:
Auto merged
sql/rpl_utility.cc:
Auto merged
sql/rpl_utility.h:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_string.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/unireg.h:
Auto merged
sql/log_event.cc:
Manual merge.
sql/log_event.h:
Manual merge.
sql/log_event_old.h:
Manual merge.
using TPC-B):
Problem: A RBR event can contain incomplete row data (only key value and
fields which have been changed). In that case, when the row is unpacked
into record and written to a table, the missing fields get incorrect NULL
values leading to master-slave inconsistency.
Solution: Use values found in slave's table for columns which are not given
in the rows event. The code for writing a single row uses the following
algorithm:
1. unpack row_data into table->record[0],
2. try to insert record,
3. if duplicate record found, fetch it into table->record[0],
4. unpack row_data into table->record[0],
5. write table->record[0] into the table.
Where row_data is the row as stored in the data area of a rows event.
Thus:
a) unpacking of row_data happens at the time when row is written into
a table,
b) when unpacking (in step 4), only columns present in row_data are
overwritten - all other columns remain as they were found in the table.
Since all data needed for the above algorithm is stored inside
Rows_log_event class, functions which locate and write rows are turned
into methods of that class.
replace_record() -> Rows_log_event::write_row()
find_and_fetch_row() -> Rows_log_event::find_row()
Both methods take row data from event's data buffer - the row being
processed is pointed by m_curr_row. They unpack the data as needed into
table's record buffers record[0] or record[1]. When row is unpacked,
m_curr_row_end is set to point at next row in the data buffer.
Other changes introduced in this changeset:
- Change signature of unpack_row(): don't report errors and don't
setup table's rw_set here. Errors can happen only when setting default
values in prepare_record() function and are detected there.
- In Rows_log_event and derived classes, don't pass arguments to
the execution primitives (do_...() member functions) but use class
members instead.
- Move old row handling code into log_event_old.cc to be used by
*_rows_log_event_old classes.
Also, a new test rpl_ndb_2other is added which tests basic replication
from master using ndb tables to slave storing the same tables using
(possibly) different engine (myisam,innodb).
Test is based on existing tests rpl_ndb_2myisam and rpl_ndb_2innodb.
However, these tests doesn't work for various reasons and currently are
disabled (see BUG#19227).
The new test differs from the ones it is based on as follows:
1. Single test tests replication with different storage engines on slave
(myisam, innodb, ndb).
2. Include file extra/rpl_tests/rpl_ndb_2multi_eng.test containing
original tests is replaced by extra/rpl_tests/rpl_ndb_2multi_basic.test
which doesn't contain tests using partitioned tables as these don't work
currently. Instead, it tests replication to a slave which has more or
less columns than master.
3. Include file include/rpl_multi_engine3.inc is replaced with
include/rpl_multi_engine2.inc. The later differs by performing slightly
different operations (updating more than one row in the table) and
clearing table with "TRUNCATE TABLE" statement instead of "DELETE FROM"
as replication of "DELETE" doesn't work well in this setting.
4. Slave must use option --log-slave-updates=0 as otherwise execution of
replication events generated by ndb fails if table uses a different
storage engine on slave (see BUG#29569).
sql/log_event.cc:
- Initialization of new Rows_log_event members.
- Fixing some typos in documentation.
In Rows_log_event::do_apply_event:
- Set COMPLETE_ROWS_F flag (when master and slave have the same number of
columns and all colums are present in the row)
- Move initialization of tables write/read sets here, outside the rows
processing loop (and out of unpack_row() function).
- Remove calls to do_prepare_row() - no longer needed.
- Add code managing m_curr_row and m_curr_row_end pointers.
- Change signatures of row processing methods of Rows_log_event and it
descendants - now most arguments are taken from class members.
- Remove do_prepare_row() methods which are no longer used.
- The auto_afree_ptr template is moved to rpl_utility.h (so that it can
be used in log_event_old.cc).
- Removed copy_extra_fields() function - no longer used.
In Rows_log_event::write_row (former replace_record):
- The old code is moved to log_event_old.cc.
- Use prepare_record() and non-destructive unpack_current_row() to fill record
with data.
- In case a record being inserted already exists on slave and row data is
incomplete use the record found and non-destructive unpack_current_row() to
combine new column values with existing ones.
- More debug info added.
In Rows_log_event::find_row (former find_and_fetch_row function):
- The old code is moved to log_event_old.cc.
- Unpacking of the row is moved here.
- In case of search using PK, the key data is prepared here.
- More debug info added.
- Remove initialization of Rows_log_event::m_after_image buffer which is no
longer used.
- Use new row unpacking methods in Update_rows_log_event::do_exec_row() to
create before and after image.
Note: all existing code used by Rows_log_event::do_apply_event() has been moved
to log_event_old.cc to be used by *_rows_log_event_old classes.
sql/log_event.h:
- Add new COMPLETE_ROWS_F flag in Rows_log_event.
- Add Rows_log_event members describing the row being processed.
- Add a pointer to key buffer which is used in derived classes.
- Add new methods: find__row(), write_row() and unpack_current_row().
- Change signatures of do_...() methods (replace method arguments by
class members).
- Remove do_prepare_row() method which is no longer used.
- Update method documentation.
- Add Old_rows_log_event class, which contains the old row processing code, as
a friend of Rows_log_event so that it can access all members of an event
instance.
sql/log_event_old.cc:
Move here old implementation of Rows_log_event::do_apply_event() and
helper methods.
sql/log_event_old.h:
- Define new class Old_rows_log_event encapsulating old version of
Rows_log_event::do_apply_event() and the helper methods.
- Add the Old_rows_log_event class as a base for *_old versions of RBR event
classes, ensure that the old version of do_apply_event() is called.
- For *_old classes, declare the helper methods used in the old version of
do_apply_event().
sql/rpl_record.cc:
- Make unpack_row non-destructive for columns not present in the row.
- Don't fill read/write set here as it is done outside these functions.
- Move initialization of a record with default values to a separate
function prepare_record().
sql/rpl_record.h:
- Change signature of unpack_row().
- Declare function prepare_record().
sql/rpl_utility.cc:
Make tabe_def::calc_field_size() a const method.
sql/rpl_utility.h:
Make table_def::calc_field_size() a const method.
Move auto_afree_ptr template here so that it can be re-used (currently
in log_event.cc and log_event_old.cc). Similar with DBUG_PRINT_BITSET
macro.
mysql-test/extra/rpl_tests/rpl_ndb_2multi_basic.test:
Modification of rpl_ndb_2multi_eng test. Tests with partitioned tables
are removed and a setup with slave having different number of columns
than master is added.
mysql-test/include/rpl_multi_engine2.inc:
Modification of rpl_multi_engine3.inc which operates on more rows and
replaces "DELETE FROM t1" with "TRUNCATE TABLE t1" as the first form
doesn't replicate in NDB -> non-NDB setting (BUG#28538).
mysql-test/suite/rpl_ndb/r/rpl_ndb_2other.result:
Results of the test.
mysql-test/suite/rpl_ndb/t/rpl_ndb_2other-slave.opt:
Test options. --log-slave-updates=0 is compulsory as otherwise non-NDB
slave applying row events from NDB master will fail when trying to log
them.
mysql-test/suite/rpl_ndb/t/rpl_ndb_2other.test:
Test replication of NDB table to slave using other engine. The main test
is in extra/rpl_tests/rpl_ndb_2multi_basic.test. It is included here
several times with different settings of default storage engine on slave.
