If a temporary error occured inside a group on an event that was not the first
event of the group, the slave could get stuck because the retry counter is reset
whenever an event was executed successfully.
This patch only reset the retry counter when an entire group has been successfully
executed, or failed with a non-transient error.
sql/slave.cc:
Adding debug printouts to every place where Relay_log_info::trans_retries
is changed and to has_temporary_error() to print the error when returning.
Adding debug variable all_errors_are_temporary_errors to make all errors
for slave thread temporary errors.
Adding code so that the Relay_log_info::trans_retries is only reset when
an entire group was sucessfully executed, or a non-temporary error occured.
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
New BitKeeper file ``mysql-test/suite/rpl/r/rpl_temporary_errors.result''
mysql-test/suite/rpl/t/rpl_temporary_errors-slave.opt:
New BitKeeper file ``mysql-test/suite/rpl/t/rpl_temporary_errors-slave.opt''
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
New BitKeeper file ``mysql-test/suite/rpl/t/rpl_temporary_errors.test''
When replicating an update pair (before image, after image) under row-based
replication, and the before image is not found on the slave, the after image
was not discared, and was hence read as a before image for the next row.
Eventually, this lead to an after image being read outside the block of rows
in the event, causing an assertion to fire.
This patch fixes this by reading the after image in the event that the row
was not found on the slave, adds some extra debug assertion to catch future
errors earlier, and also adds a few non-debug checks to prevent reading
outside the block of the event.
include/my_base.h:
Adding error code HA_ERR_CORRUPT_EVENT.
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
Result change.
mysql-test/suite/rpl/t/rpl_row_basic_11bugs.test:
Adding test to try to use row-based replication to replicate an
update of a row that doesn't exist on the slave. We should get
an apropriate error and the slave should stop.
sql/log_event.cc:
Adding debug printouts. Adding code to Update_rows_log_event::do_exec_row()
so that the after image is read (and ignored) in the event of an error in
finding the row. This is necessary so that the second pair of images is
read correctly for the next update pair.
Changing logic for ignoring errors to not include update events, since
a "key not found" error or a "record changed" error is not idempotent
for updates, just for deletes and inserts.
sql/log_event.h:
Adding debug assertions to check that row reading is within the events block of rows.
storage engines do not set the unused null bits (i.e., the
filler bits and the X bit) correctly. Also adding some casts
to debug printouts to eliminate compiler warnings.
sql/ha_ndbcluster.cc:
Changing debug enter names to use full names for member functions.
sql/records.cc:
Emptying records before starting to read records since some engines do
not set the unused null bits, leading to valgrind errors.
sql/rpl_record.cc:
Adding casts to debug printouts to eliminate compiler warnings.
mysql-test/suite/rpl/include/rpl_mixed_check_select.inc:
Added ORDER BY for SELECT
mysql-test/suite/rpl/include/rpl_mixed_check_view.inc:
Added ORDER BY for VIEW
mysql-test/suite/rpl/include/rpl_mixed_dml.inc:
Added ORDER BY for SELECT
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
Updated result file
Adjusting SHOW CREATE TABLE output from 5.0 to 5.1 format
mysql-test/r/ctype_uca.result:
Adjusting SHOW CREATE TABLE output
mysql-test/r/ctype_ucs.result:
Adjusting SHOW CREATE TABLE output
mysql-test/r/ctype_utf8.result:
Adjusting SHOW CREATE TABLE output
mysql-test/r/func_regexp.result:
Adjusting SHOW CREATE TABLE output
into mysql.com:/home/bar/mysql-work/mysql-5.1.b31081
mysql-test/r/ctype_ucs.result:
Auto merged
mysql-test/r/ctype_utf8.result:
Auto merged
mysql-test/r/func_regexp.result:
Auto merged
mysql-test/t/ctype_uca.test:
Auto merged
mysql-test/t/ctype_ucs.test:
Auto merged
mysql-test/t/ctype_utf8.test:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
mysql-test/extra/binlog_tests/mix_innodb_myisam_side_effects.test:
manual merge from 5.0
mysql-test/r/innodb.result:
results changed
mysql-test/r/multi_update.result:
results changed
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
manual merge: results re-recorded
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
results changed
mysql-test/suite/manual/r/rpl_replication_delay.result:
results recorded for 5.1
mysql-test/t/innodb.test:
removing bug27716 regression test as one exists in binlog suite already.
mysql-test/t/multi_update.test:
manual merge plus refining for execution in both binlog_format.
sql/log_event.cc:
bloody manual merge;
the fact that an event is artificial is encode with setting timestamp argument to zero.
rli's last_master_timestamp won't be updated in stmt_done in such case.
sql/rpl_rli.cc:
simulation and the real fixing code.
Don't update last_master_timestamp with zero.
sql/sql_delete.cc:
manual merge
into koti.dsl.inet.fi:/home/elkin/MySQL/merge-5.1
mysql-test/t/multi_update.test:
Auto merged
sql/slave.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_delete.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_update.cc:
Auto merged
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
manual merge use local; another file has to be changed in 5_1.
mysql-test/r/innodb.result:
manual merge use local to re-record the results
mysql-test/r/multi_update.result:
manual merge use local to re-record the results
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
manual merge use local to re-record the results
mysql-test/t/innodb.test:
restoring bug#27716 snippet
sql/log_event.cc:
trasfering simulation to another file: rpl_rli.cc
similar to bug_27716, but it was stressed on in the synopsis on that there is another
side of the artifact affecting behaviour in transaction.
Fixed with deploying multi_delete::send_error() - otherwise never called - and refining its logic
to perform binlogging job if needed.
The changeset includes the following side effects:
- added tests to check bug_23333's scenarios on the mixture of tables for multi_update;
- fixes bug@30763 with two-liner patch and a test coinciding to one added for bug_23333.
mysql-test/r/innodb.result:
results changed
mysql-test/r/mix_innodb_myisam_binlog.result:
results changed
mysql-test/r/multi_update.result:
results changed
mysql-test/t/innodb.test:
trans table specific test added
mysql-test/t/mix_innodb_myisam_binlog.test:
multi-update and multi-delete of mixure of ta and not-ta tables tests added (relates to bug_23333).
mysql-test/t/multi_update.test:
testing another branch of mult-delete: send_eof() (binloggin there), send_error (early return)
sql/sql_class.h:
a new flag to designate the fact the statement's error has been handled.
The flag is checked by ::send_error() methods (multi_update and _delete classes)
sql/sql_delete.cc:
expanding multi_delete::send_error to
1. early return if error_handled == t
2. binlogging locally if there was a non-trans table modified side effect
sql/sql_parse.cc:
adding multi_update::send_error which can perform binlogging and rollback job in needed
sql/sql_update.cc:
issues relating to
1. bug_27716 with zeroing of `updated' to serve as the flag of early return from send_error().
The flag is changed to be a new member error_handled; also moved outside binlogging branch.
The reason for this change is that bug_23333 fixes were pushed after the bug_27716's and they
left this flaw (also no test coverage).
2. bug_30763 with assertion on trans_safe. I decide to make 2 liner fix for that bug here instead of to remove
those two assertions. This new bug test case is the same as for multi-update on the mixure of tables.
The rational for this fix:
presumption for mutli_update::trans_safe to be set to zero at
multi_update::multi_update or multi_update::initialize_tables() is incorrect.
trans_safe := false should happen only when a non-transactional table gets modified.
Therefore, at initialization the member must be be set to true.
Post-merge fixes. Setting write bit before calling Field::store() since the function asserts that
the write bit has been set.
sql/field.cc:
Setting bit in write set to prevent assertion from throwing when calling Field::store().
into kindahl-laptop.dnsalias.net:/home/bkroot/mysql-5.1-rpl
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/log_event.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/rpl_record.cc:
Manual merge.
Refactoring code to add parameter to pack() and unpack() functions with
purpose of indicating if data should be packed in little-endian or
native order. Using new functions to always pack data for binary log
in little-endian order. The purpose of this refactoring is to allow
proper implementation of endian-agnostic pack() and unpack() functions.
Eliminating several versions of virtual pack() and unpack() functions
in favor for one single virtual function which is overridden in
subclasses.
Implementing pack() and unpack() functions for some field types that
packed data in native format regardless of the value of the
st_table_share::db_low_byte_first flag.
The field types that were packed in native format regardless are:
Field_real, Field_decimal, Field_tiny, Field_short, Field_medium,
Field_long, Field_longlong, and Field_blob.
Before the patch, row-based logging wrote the rows incorrectly on
big-endian machines where the storage engine defined its own
low_byte_first() to be FALSE on big-endian machines (the default
is TRUE), while little-endian machines wrote the fields in correct
order. The only known storage engine that does this is NDB. In effect,
this means that row-based replication from or to a big-endian
machine where the table was using NDB as storage engine failed if the
other engine was either non-NDB or on a little-endian machine.
With this patch, row-based logging is now always done in little-endian
order, while ORDER BY uses the native order if the storage engine
defines low_byte_first() to return FALSE for big-endian machines.
In addition, the max_data_length() function available in Field_blob
was generalized to the entire Field hierarchy to give the maximum
number of bytes that Field::pack() will write.
mysql-test/extra/rpl_tests/rpl_extraMaster_Col.test:
Sorting by columns that produces deterministic order.
mysql-test/suite/rpl/r/rpl_extraColmaster_innodb.result:
Result change.
mysql-test/suite/rpl/r/rpl_extraColmaster_myisam.result:
Result change.
mysql-test/suite/rpl/r/rpl_row_extraColmaster_ndb.result:
Result change.
mysql-test/suite/rpl/t/disabled.def:
Enabling tests.
mysql-test/suite/rpl/t/rpl_row_mysqlbinlog.test:
Adding missing sync_slave_with_master causing slave to keep tables
after shutdown.
mysql-test/suite/rpl_ndb/t/disabled.def:
Enabling tests.
mysql-test/suite/rpl_ndb/t/rpl_ndb_innodb2ndb-slave.opt:
Adding --new option
mysql-test/suite/rpl_ndb/t/rpl_ndb_innodb2ndb.test:
Adding have_log_bin.
mysql-test/suite/rpl_ndb/t/rpl_ndb_myisam2ndb-slave.opt:
Adding --new option
mysql-test/suite/rpl_ndb/t/rpl_ndb_myisam2ndb.test:
Adding have_log_bin
mysql-test/t/partition.test:
Adding have_archive, since that is used in the test.
sql/field.cc:
Eliminating all two-argument pack() and unpack() functions and moving
functionality into the four-argument version. The four argument version
is introduced so that it is possible to avoid using the storage engine
default when writing and reading the packed format (the unpacked format
still uses the storage engine's default). This is used by row-based
replication to write the fields in a storage engine- and endian-agnostic
format.
Packing integral and floating-point numbers in little-endian format
(if requested).
Using pad_char for the field instead of spaces (0x20) when unpacking.
Adding some Doxygen documentation.
---
Adding max_data_length() to denote the maximum number of bytes that
pack() will write.
Adding casts to remove warnings for debug printouts.
sql/field.h:
Eliminating all virtual pack() and unpack() functions except the four-
argument version, which now is the function that shall be overridden.
The two-argument versions are convenience functions, to prevent changes
to code that uses these.
Adding code to pack integer numbers and floating-point numbers in
little-endian format, if requested.
---
Adding max_data_length() to denote the maximum number of bytes that
pack() will write.
sql/log.cc:
Removing debug printout causing crash when starting NDB on Solaris.
sql/log_event.cc:
Adding missing #ifndef causing compile failure. Adding debug printouts.
sql/rpl_record.cc:
Debriding code. Using new pack() and unpack() functions to always pack
fields little-endian. Adding debug printouts.
---
Using max_data_length() when packing field into row.
Adding casts to debug printouts.
sql/sql_show.cc:
Adding code that causes crash on Solaris machines since printf() cannot
handle NULL values for strings properly.
mysql-test/suite/rpl_ndb/r/rpl_ndb_innodb2ndb.result:
New BitKeeper file ``mysql-test/suite/rpl_ndb/r/rpl_ndb_innodb2ndb.result''
mysql-test/suite/rpl_ndb/r/rpl_ndb_myisam2ndb.result:
New BitKeeper file ``mysql-test/suite/rpl_ndb/r/rpl_ndb_myisam2ndb.result''
Problem: rpl_stm_mystery22 is unstable.
Reason: At one place, the test case *should* wait until the SQL thread on the
slave receives an error, but instead it waits until the SQL thread stops. The
SQL thread may stop before the error flag is set, so that when the test case
continues to execute, the error flag is not set.
Fix: Introduce the subroutine mysql-test/include/wait_for_slave_sql_error.inc,
which waits until there is an error in the sql thread of the slave.
Re-commit: fixed one logical error and two smaller things noted by Mats.
mysql-test/suite/rpl/t/rpl_stm_mystery22.test:
Use the new wait_for_slave_sql_error.inc instead of wait_for_slave_to_stop.
There may be a delay from when the slave stops to when Last_SQL_Errno is set,
so it is not safe to merely wait until the slave stops.
mysql-test/include/wait_for_slave_sql_error.inc:
New BitKeeper file ``mysql-test/include/wait_for_slave_sql_error.inc''
This is a subroutine that waits until the sql thread on the slave receives an
error, as indicated by Last_SQL_Errno in "SHOW SLAVE STATUS".
Performance improvements made.
ExtractValue for large XML values is now much faster
(about 2000 times faster of 1Mb-long XML values).
sql/item_xmlfunc.cc:
Performance improvement for huge XML values:
1. Avoid reallocs - reserve extra memory:
using String::reserve() + String::q_append()
instead of String::append()
2. Parent is now not searched through the all previous
nodes in backward direction. Instead, we do the following:
- we introduce data->parent - a new member in MY_XML_USER_DATA
- when entering a new tag/attribute node, we remember offset
of the current node in data->parent
- when leaving a tag/attribute node, we change data->parent
to offset of the parent of the current node.
Problem: The "regex" library written by Henry Spencer
does not support tricky character sets like UCS2.
Fix: convert tricky character sets to UTF8 before calling
regex functions.
mysql-test/r/ctype_uca.result:
Adding tests
mysql-test/r/ctype_ucs.result:
Adding tests
mysql-test/r/ctype_utf8.result:
Adding tests
mysql-test/r/func_regexp.result:
Adding tests
mysql-test/t/ctype_uca.test:
Adding tests
mysql-test/t/ctype_ucs.test:
Adding tests
mysql-test/t/ctype_utf8.test:
Adding tests
mysql-test/t/func_regexp.test:
Adding tests
sql/item_cmpfunc.cc:
- Adding new method Item_func_regex::regcomp()
to share more code between fix_fields() and val_int()
- Adding conversion from ASCII-incompatible charsets like UCS2
to UTF8, because the "regexp" does not support these charsets
- Additional optimization: calculate flags for regcomp only
once in fix_fields, instead of every regcomp()
sql/item_cmpfunc.h:
Adding prototypes for new members and methods
mysql-test/include/ctype_regex.inc:
New BitKeeper file ``mysql-test/include/ctype_regex.inc''
Moving common regular expression tests into a separate
file and uncluding it into func_regexp and into many ctype_xxx tests.
Report claims that Seconds_behind_master behaves unexpectedly.
Code analysis shows that there is an evident flaw in that treating of FormatDescription event is wrong
so that after FLUSH LOGS on slave the Seconds_behind_master's calculation slips and incorrect
value can be reported to SHOW SLAVE STATUS.
Even worse is that the gap between the correct and incorrect deltas grows with time.
Fixed with prohibiting changes to rpl->last_master_timestamp by artifical events (any kind of).
suggestion as comments is added how to fight with lack of info on the slave side by means of
new heartbeat feature coming.
The test can not be done ealily fully determistic.
sql/log_event.cc:
changing timestamp is affirmed only by non-artificial events. Artifical FD won't change it anymore.
The simulation code is off unless server is started with the args from the opt-file.
The simulation code assumes that it will execute specific schedule generated by rpl_replication_delay.test.
sql/slave.cc:
Comments are changed to announce a new possibility to cope with
Seconds_behind_master jumping due to EOF special treatment (reset of the timestamp)
mysql-test/suite/manual/r/rpl_replication_delay.result:
result are not deterministic though there are comments saying the most probable expected
value for Seconds_behind_master
mysql-test/suite/manual/t/rpl_replication_delay-slave.opt:
bug emulation
mysql-test/suite/manual/t/rpl_replication_delay.test:
specic test for bug#29309. It's hard to make it reliable as it deals with timestamps.
(a way to automate the test like this is to have I_S table for show slave status' fields)
A possible way to check results is to run
grep -i 'show\|seconds' < suite/manual/r/rpl_replication_delay.reject and to get the lines like these:
show slave status /* Second_behind reports 0 */;;
Seconds_Behind_Master 0
show slave status /* bug emulated: reports slave threads starting time about 3*3 not 3 secs */;;
Seconds_Behind_Master 9
show slave status /* reports the correct diff with master query time about 3+3 secs */;;
Seconds_Behind_Master 6
Due to time discreteness of time the reported numbers may slightly vary. That's why the test is not reliable.
into koti.dsl.inet.fi:/home/elkin/MySQL/TEAM/5.1-merge
client/mysql.cc:
Auto merged
mysql-test/r/mysql.result:
Auto merged
mysql-test/t/mysql.test:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_repl.cc:
Auto merged
sql/sql_class.h:
conflicting hunk is for log.h
Fix: ignore BOM marker in the first line.
client/mysql.cc:
Skip BOM marker in the very first line.
mysql-test/r/mysql.result:
Adding test
mysql-test/t/mysql.test:
Adding test
Problem: one thread could read uninitialized memory from (the stack of) another
thread.
Fix: swapped order of initializing the memory and making it available to the
other thread.
Fix: put lock around the statement that makes the memory available to the other
thread.
Fix: all fields of the struct are now initialized in the constructor, to avoid
future problems.
sql/sql_class.h:
Initialize all members in constructor for more safe future code.
sql/sql_repl.cc:
Swap order so that linfo is first initialized, then assigned, instead of the
other way around.
Put a lock around the assignment. We use LOCK_thread_count since log_in_use
uses it: log_in_use may be running concurrently, called from
MYSQL_LOG::purge_logs.
into mysql.com:/home/bar/mysql-work/mysql-5.1-new-rpl
mysql-test/t/ctype_uca.test:
Auto merged
sql/mysqld.cc:
Auto merged
mysql-test/t/subselect.test:
After merge fix
Problem: some valid euc-kr characters were rejected because
condition checking multi-byte tail didn't allow
multi-byte characters having the second byte in the ranges
[0x41..0x5A] and [0x61..0x7A].
Fix: allow these byte ranges for mb tails
mysql-test/r/ctype_euckr.result:
Adding tests
mysql-test/t/ctype_euckr.test:
Adding tests
strings/ctype-euc_kr.c:
Fixing wrong tail character pattern
into mysql.com:/nfsdisk1/lars/MERGE/mysql-5.1-merge
mysql-test/suite/binlog/t/binlog_row_mix_innodb_myisam.test:
Auto merged
mysql-test/suite/ndb/r/ndb_dd_basic.result:
Auto merged
sql/field.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/rpl_record.cc:
Auto merged
sql/rpl_utility.cc:
Auto merged
sql/rpl_utility.h:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
active_mi has been reset (shutdown) at the time of quering with
SHOW SLAVE STATUS so that
at handling of SHOW an attempt to read its members segfaults.
Fixed with checking the value of active_mi before to call show_master_info()
Merely send_ok() is invoked when active_mi does not exist.
A test can not be easiely written.
Notice, there are more analogical cases in the code which require a similar
treatment (to be reported as a bug separately).
sql/sql_parse.cc:
Ignore reporting and send only OK if master info struct has been destoyed.
As this must be at shutdown merely a warning is sent to the client.
into pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-5.1-build
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
Auto merged
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
Auto merged
cmake list are auto-generated and thus still required. Correct fix
not yet known.
bdb/CMakeLists.txt:
Revert previous change, it is incorrect as those files are automatically
generated.
in following tests.
mysql-test/suite/rpl/r/rpl_bug31076.result:
Result file change.
mysql-test/suite/rpl/t/rpl_bug31076.test:
Dropping database so that following tests don't see it.
