(WL#794). This can be of interest in some recovery-from-backup scenarios, and also when you have
two databases in one mysqld, having a certain similarity and you want one db to be updated when the other is
(some sort of trigger).
Plus small fix for BUG#3568 "MySQL server crashes when built --with-debug and CHANGE MASTER +MASTER_POS_WAIT"
In tables_ok(), when there is no table having "updating==TRUE" in the list,
return that we don't replicate this statement (the slave is supposed to
replicate *changes* only).
In practice, the case can only happen for this statement:
DELETE t FROM t,u WHERE ... ;
tables_ok(t,u) will now return 0, which (check all_tables_not_ok())
will give a chance to tables_ok(t) to run.
as we transform the 3.23 Load_log_event into a 4.0 Create_file_log_event which is one
byte longer, we need to increment event_len. The bug was that we did not increment it,
so later in code the end 0 was not seen so there was for example a segfault in
strlen(fname) because fname was not 0-terminated.
Other problems remain in 3.23->4.0 replication of LOAD DATA INFILE but they are less serious:
Exec_master_log_pos and Relay_log_space are incorrect. I'll document them.
They are not fixable without significant code changes (if you fix those problems in 4.0,
you get assertion failures somewhere else etc), * which are already done in 5.0.0 *.
too big by 6 bytes. So I add code to substract 6 bytes if the master is 3.23.
This is not perfect (because it won't work if the slave I/O thread has not
noticed yet that the master is 3.23), but as long as the slave I/O thread
starts Exec_master_log_pos will be ok.
It must be merged to 4.1 but not to 5.0 (or it can be, because of #if MYSQL_VERSION_ID),
because 5.0 already works if the master is 3.23 (and in a more natural way:
in 5.0 we store the end_log_pos in the binlog and relay log).
I had to move functions from slave.h to slave.cc to satisfy gcc.
Fixed bugs in group_concat with ORDER BY and DISTINCT (Bugs #2695, #3381 and #3319)
Fixed crash when doing rollback in slave and the io thread catched up with the sql thread
Set locked_in_memory properly
We introduce a new function mysql_test_parse_for_slave().
If the slave sees that the query got a really bad error on master
(killed e.g.), then it calls this function to know if this query
can be ignored because of replicate-*-table rules (do not worry
about replicate-*-db rules: they are checked so early that they have
no bug). If the answer is yes, it skips the query and continues. If
it's no, then it stops and say "fix your slave data manually" (like it
did before this change).
re-using unused LOCK_active_mi to serialize all administrative
commands related to replication:
START SLAVE, STOP SLAVE, RESET SLAVE, CHANGE MASTER, init_slave()
(replication autostart at server startup), end_slave() (replication
autostop at server shutdown), LOAD DATA FROM MASTER.
This protects us against a handful of deadlocks (like BUG#2921
when two START SLAVE, but when two STOP SLAVE too).
Removing unused variables.
ChangeSet 1.1620.12.1 and ChangeSet 1.1625.2.1
from 4.1. This makes the slave I/O thread flush the relay log
after every event, which provides additional safety in case
of brutal crash (reduces chances to lose a part of the relay log).
- the one about BUG#2921
- the one about relay log flushing
Both will be rewritten in a next changeset
(this one will not be pushed before the next changeset).
BUG#2826 "Seconds behind master weirdness"
(sometimes this column of SHOW SLAVE STATUS shows a very big value,
in fact a small negative number casted to ulonglong).
This problem was reported by only one user, but which uses
synchronized time between his servers.
As suggested by the user, to hide this I display max(0, the value)
so that it will be less confusing. For a user, seeing 0 is probably
better than seeing -1 (both tell you that the slave is very close
to the master).
we change THD::system_thread from a 'bool' to a bitmap to be able to
distinguish between delayed-insert threads and slave threads.
- Fix for BUG#1701 "Update from multiple tables" (one line in sql_parse.cc,
plus a new test rpl_multi_update.test). That's just adding an initialization.
The problem was that when the slave SQL thread reads a hot relay log (hot = the one being written to by the
slave I/O thread), it must have the LOCK_log. It already took it for read_log_event(), but needs
it also for check_binlog_magic().
This should fix all recently reported failures of the rpl_max_relay_size test in 4.1 and 5.0
(though the bug exists since 4.0, it showed up first in 5.0).
the relay log before flushing master.info.
Doing 'before' leads to duplicate event, doing after leads to missing event.
Both can be as destructive, but 'duplicate' enables us to later add detection
code to catch it. Whereas 'missing' can't be caught (it can't, because
the I/O thread can produce legal position jumps, for example if it has
ignored an event coming from this slave (rememember that starting from 4.1.1,
the I/O thread filters the server id).
Now the I/O thread (in flush_master_info()) flushes the relay log to disk
after reading every event. Slower but provides additionnal safety in case
of brutal crash.
I had to make the flush optional (i.e. add a if(some_bool_argument) in the function)
because sometimes flush_master_info() is called when there is no usable
relay log (the relay log's IO_CACHE is not initialized so can't be flushed).
(Initial caps for each word.) For example, instead of writing
Until_condition, Until_Log_File, and Until_log_pos, write
Until_Condition, Until_Log_File, and Until_Log_pos.
rli->save_temporary_tables and slave_open_temp_tables
(in old 4.0 you could make "SHOW STATUS LIKE 'slave_open_temp_tables'" grow
indefinitely by doing RESET SLAVE and replicating always the same CREATE
TEMPORARY TABLE).
It's critical to reset save_temporary_tables to 0 (otherwise you may later
read memory which has been freed) so this changeset should go into 4.1.
- when we don't have in_addr_t, use uint32.
- a forgotten initialization of slave_proxy_id in sql/log_event.cc (was not really "forgot", was
"we needn't init it there", but there was one case where we needed...).
- made slave_proxy_id always meaningful in THD and Log_event, so we can
rely more on it (no need to test if it's meaningful). THD::slave_proxy_id
is equal to THD::thread_id except for the slave SQL thread.
- clean up the slave's temporary table (i.e. free their memory) when slave
server shuts down.
"If 2 master threads with same-name temp table, slave makes bad binlog"
and (two birds with one stone) for
BUG#1240 "slave of slave breaks when STOP SLAVE was issud on parent slave
and temp tables".
Here is the design change:
in a slave running with --log-slave-updates, events are now logged with the
thread id they had on the master. So no more id conflicts between master threads,
but introduces id conflicts between one master thread and one normal
client thread connected to the slave. This is solved by storing the server id
in the temp table's name.
New test which requires mysql-test-run to be run with --manager,
otherwise it will be skipped.
Undoing a Monty's change (hum, a chill runs down my spine ;) which was
"Cleanup temporary tables when slave ends" in ChangeSet 1.1572.1.1.
a Format_description_log_event (or maybe it will be named
Description_log_event) which is not recognized by 4.0, so
a 4.0 can't be a slave of 5.0. We detect it early to produce
a helpful message instead of "corrupted relay log" later.
"Add a column "Timestamp_of_last_master_event_executed" in SHOW SLAVE STATUS".
Finally this is adding
- Slave_IO_State (a copy of the State column of SHOW PROCESSLIST for the I/O thread,
so that the users, most of the time, has enough info with only SHOW SLAVE STATUS).
- Seconds_behind_master. When the slave connects to the master it does SELECT UNIX_TIMESTAMP()
on the master, computes the absolute difference between the master's and the slave's clock.
It records the timestamp of the last event executed by the SQL thread, and does a
small computation to find the number of seconds by which the slave is late.
Now LOAD DATA FROM MASTER does not drop the database, instead it only tries to
create it, and drops/creates table-by-table.
* replicate_wild_ignore_table='db1.%' is now considered as "ignore the 'db1'
database as a whole", as it already works for CREATE DATABASE and DROP DATABASE.
Added proper options to CHANGE MASTER TO, new fields to SHOW SLAVE STATUS,
Honoring this parameters during connection to master.
Introduced new format of master.info file
For example the Binlog_dump thread (on the master) sometimes showed "Slave:".
And there were confusing messages where "binlog" was employed instead
of "relay log".
when the SQL thread stops, set rli->inside_transaction to 0. This is needed if the user
later restarts replication from a completely different place where there are only autocommit
statements.
* Detect the case where the master died while flushing the binlog cache to the binlog
and stop with error. Cannot add a testcase for this in 4.0 (I tested it manually)
as the slave always runs with --skip-innodb.
Let's say the lack of comments did not help me ;)
Copying it back again and adding comments; now 3.23->4.0
replication of LOAD DATA INFILE works again.
"LOAD DATA INFILE is badly filtered by binlog-*-db rules".
There will probably be a second final one to merge Dmitri's changes
to rpl_log.result and mine.
2 new tests:
rpl_loaddata_rule_m : test of logging of LOAD DATA INFILE when the master has binlog-*-db rules,
rpl_loaddata_rule_s : test of logging of LOAD DATA INFILE when the slave has binlog-*-db rules and --log-slave-updates.
log-slave-updates since this causes unexpected values in
Exec_master_log_pos in A->B->C replication setup, synchronization
problems in master_pos_wait()...
Still this brokes some functionality in sql/repl_failsafe.cc
(but this file is not used now)
- Bug #985: "Between RESET SLAVE and START SLAVE, SHOW SLAVE STATUS is wrong."
Now RESET SLAVE puts correct info in mi->host etc. A new test rpl_reset_slave
for that.
- Bug #986: "CHANGE MASTER & START SLAVE do not reset error columns in SHOW
SLAVE STATUS". Now these reset the errors.
I extended the task to cleaning error messages, making them look nicer,
and making the output of SHOW SLAVE STATUS (column Last_error) be as complete
as what's printed on the .err file;
previously we would have, for a failure of a replicated LOAD DATA INFILE:
- in the .err, 2 lines:
"duplicate entry 2708 for key 1"
"failed loading SQL_LOAD-5-2-2.info"
- and in SHOW SLAVE STATUS, only:
"failed loading SQL_LOAD-5-2-2.info".
Now SHOW SLAVE STATUS will contain the concatenation of the 2 messages.
and other replicate-*-table options in SHOW SLAVE STATUS.
Seems like it had not been done, so I push it now:
there's 4 new columns to SHOW SLAVE STATUS.
so I commit again in a fresh tree.
Fix for bug#763 (Relay_log_space too big by 4 bytes),
plus comments and DBUG_PRINT, and we don't start replication
if --bootstrap.
For now following tasks have been done:
- PASSWORD() function was rewritten. PASSWORD() now returns SHA1
hash_stage2; for new passwords user.password contains '*'hash_stage2; sql_yacc.yy also fixed;
- password.c: new functions were implemented, old rolled back to 4.0 state
- server code was rewritten to use new authorization algorithm (check_user(), change
user, and other stuff in sql/sql_parse.cc)
- client code was rewritten to use new authorization algorithm
(mysql_real_connect, myslq_authenticate in sql-common/client.c)
- now server barks on 45-byte-length 4.1.0 passwords and refuses 4.1.0-style
authentification. Users with 4.1.0 passwords are blocked (sql/sql_acl.cc)
- mysqladmin.c was fixed to work correctly with new passwords
Tests for 4.0-4.1.1, 4.1.1-4.1.1 (with or without db/password) logons was performed;
mysqladmin also was tested. Additional check are nevertheless necessary.
