we store 7 bytes (1 + 2*3) in every Query_log_event.
In the future if users want binlog optimized for small size and less safe,
we could add --binlog-no-charset (and binlog-no-sql-mode etc): charset info
is something by design optional (even if for now we don't offer possibility to disable it):
it's not a binlog format change.
We try to reduce the number of get_charset() calls in the slave SQL thread to a minimum
by caching the charset read from the previous event (which will often be equal to the one of the current event).
We don't use SET ONE_SHOT for charset-aware repl (we still do for timezones, will be fixed later).
No more errors if one changes the global value of charset vars on master or slave
(as we log charset info in all Query_log_event).
Not fixing Load_log_event as it will be rewritten soon by Dmitri.
Testing how mysqlbinlog behaves in rpl_charset.test.
mysqlbinlog needs to know where charset file is (to be able to convert a charset number found
in binlog (e.g. in User_var_log_event) to a charset name); mysql-test-run needs to pass
the correct value for this option to mysqlbinlog.
Many result udpates (adding charset info into every event shifts log_pos in SHOW BINLOG EVENTS).
Roughly the same job is to be done for timezones :)
CREATE DATABASE statement used the current database instead of the
database created when checking conditions for replication.
CREATE/DROP/ALTER DATABASE statements are now replicated based on
the manipulated database.
Replication using replicate-rewrite-db did not work for LOAD DATA INFILE.
Now is does. There was one place in the code that used current database
instead of the rewrite database.
This allows one to setup a master <-> master replication with non conflicting auto-increment series.
Cleaned up binary log code to make it easyer to add new state variables.
Added simpler 'upper level' logic for artificial events (events that should not cause cleanups on slave).
Simplified binary log handling.
Changed how auto_increment works together with to SET INSERT_ID=# to make it more predictable: Now the inserted rows in a multi-row statement are set independent of the existing rows in the table. (Before only InnoDB did this correctly)
This is the main commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
Below 5.0 means 5.0.0.
MySQL 5.0 is able to replicate FOREIGN_KEY_CHECKS, UNIQUE_KEY_CHECKS (for speed),
SQL_AUTO_IS_NULL, SQL_MODE. Not charsets (WL#1062), not some vars (I can only think
of SQL_SELECT_LIMIT, which deserves a special treatment). Note that this
works for queries, except LOAD DATA INFILE (for this it would have to wait
for Dmitri's push of WL#874, which in turns waits for the present push, so...
the deadlock must be broken!). Note that when Dmitri pushes WL#874 in 5.0.1,
5.0.0 won't be able to replicate a LOAD DATA INFILE from 5.0.1.
Apart from that, the new binlog format is designed so that it can tolerate
a little variation in the events (so that a 5.0.0 slave could replicate a
5.0.1 master, except for LOAD DATA INFILE unfortunately); that is, when I
later add replication of charsets it should break nothing. And when I later
add a UID to every event, it should break nothing.
The main change brought by this patch is a new type of event, Format_description_log_event,
which describes some lengthes in other event types. This event is needed for
the master/slave/mysqlbinlog to understand a 5.0 log. Thanks to this event,
we can later add more bytes to the header of every event without breaking compatibility.
Inside Query_log_event, we have some additional dynamic format, as every Query_log_event
can have a different number of status variables, stored as pairs (code, value); that's
how SQL_MODE and session variables and catalog are stored. Like this, we can later
add count of affected rows, charsets... and we can have options --don't-log-count-affected-rows
if we want.
MySQL 5.0 is able to run on 4.x relay logs, 4.x binlogs.
Upgrading a 4.x master to 5.0 is ok (no need to delete binlogs),
upgrading a 4.x slave to 5.0 is ok (no need to delete relay logs);
so both can be "hot" upgrades.
Upgrading a 3.23 master to 5.0 requires as much as upgrading it to 4.0.
3.23 and 4.x can't be slaves of 5.0.
So downgrading from 5.0 to 4.x may be complicated.
Log_event::log_pos is now the position of the end of the event, which is
more useful than the position of the beginning. We take care about compatibility
with <5.0 (in which log_pos is the beginning).
I added a short test for replication of SQL_MODE and some other variables.
TODO:
- after committing this, merge the latest 5.0 into it
- fix all tests
- update the manual with upgrade notes.
The constructor of Rotate_log_event used when we are rotating our binlog or
relay log, should not assume that there is a nonzero THD available.
For example, when we are reacting to SIGHUP, the THD is 0.
In fact we don't need to use the THD in this constructor;
we can do like for Stop_log_event, and use the minimal Log_event
constructor.
If we were allowed to put Unix-specific commands in the testsuite,
I'd add a test for this (<sigh>).
instead of Log_event::Log_event(THD*, ...) when the event is built in the master
to be written in the binlog.
Rand_log_event already used the good constructor, so there really is no reason
for Intvar_log_event to be an exception.
This fixes a test failure of last night (which appeared after I removed a useless
e.server_id=thd->server_id in log.cc; in fact this line was not useless because
it hid the bad constructor).
Replication tests pass, with Valgrind too.
"If 2 master threads with same-name temp table, slave makes bad binlog"
and (two birds with one stone) for
BUG#1240 "slave of slave breaks when STOP SLAVE was issud on parent slave
and temp tables".
Here is the design change:
in a slave running with --log-slave-updates, events are now logged with the
thread id they had on the master. So no more id conflicts between master threads,
but introduces id conflicts between one master thread and one normal
client thread connected to the slave. This is solved by storing the server id
in the temp table's name.
New test which requires mysql-test-run to be run with --manager,
otherwise it will be skipped.
Undoing a Monty's change (hum, a chill runs down my spine ;) which was
"Cleanup temporary tables when slave ends" in ChangeSet 1.1572.1.1.
"LOAD DATA INFILE is badly filtered by binlog-*-db rules".
There will probably be a second final one to merge Dmitri's changes
to rpl_log.result and mine.
2 new tests:
rpl_loaddata_rule_m : test of logging of LOAD DATA INFILE when the master has binlog-*-db rules,
rpl_loaddata_rule_s : test of logging of LOAD DATA INFILE when the slave has binlog-*-db rules and --log-slave-updates.
as the already-stored timestamp. Now 'created' is used only to know if
this is a first binlog or not. And we may re-use the superfluous bytes
in 5.0 when we need room.
- Comments for future devs.
- Start_log_event::exec_event() : when we hit it, do a rollback.
- We don't need LOG_EVENT_FORCED_ROTATE_F.
- Stop_log_event::exec_event() : when we hit it, we needn't clean anything.
- Removed LOG_EVENT_TIME_F and LOG_EVENT_FORCED_ROTATE_F.
- We don't need Stop events in the relay log.
- Now filtering of server id is done in the I/O thread first.
Fix for bug 254 : the first Start_log_event after server startup will
have created=now(), whereas the next ones (FLUSH LOGS, auto rotation)
will have created=0. Before this, it was always now().
This way, slaves >=4.0.14 will know when they must
drop stale temp tables or not. The next task is now modify 4.0.14 to
implement this.
Class for sending data from server to client (Protocol)
This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
Let MySQL 4.0 read 4.1 .frm files without 4.1 specific extensions
New variables @@rand_seed1 and @@rand_seed2 (used by replication)
DROP TEMPORARY TABLE
Improve mysql-test to be more robust.
Fix that GRANT doesn't delete SSL options
Change innobase_flush_log_at_trx_commit to uint.
Don't rotate logs if we read a rotate log entry from the master.
Fixed bugs in my last changeset that made MySQL hard to compile.
Added mutex around some data that could cause table cache corruptions when using OPTIMIZE TABLE / REPAIR TABLE or automatic repair of MyISAM tables.
Added mutex around some data in the slave start/stop code that could cause THD linked list corruptions
Extended my_chsize() to allow one to specify a filler character.
Extend vio_blocking to return the old state (This made some usage of this function much simpler)
Added testing for some functions that they caller have got the required mutexes before calling the function.
Use setrlimit() to ensure that we can write core file if one specifies --core-file.
Added --slave-compressed-protocol
Made 2 the minimum length for ft_min_word_len
Added variables foreign_key_checks & unique_checks.
Less logging from replication code (if not started with --log-warnings)
Changed that SHOW INNODB STATUS requre the SUPER privilege
More DBUG statements and a lot of new code comments
Made a some new buffers thread specific and changeable.
Resize of key_buffer.
AUTO_COMMIT -> AUTOCOMMIT
Fixed mutex bug in DROP DATABASE
Fixed bug when using auto_increment as second part of a key where first part could include NULL.
Split handler->extra() to extra() and extra_opt() to be able to support thread specific buffers.
Don't write message to error log when slave reconnects becasue of timeout.
Fixed possible update problem when using DELETE/UPDATE on small tables
(In some cases we used index even if table scanning would be better)
A lot of minior code cleanups
(All commit emails since 4.0.1 checked)
This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
fixed bug in LOAD DATA FROM MASTER
fixed rpl000001.result
Slave now replicates 3.23 master, with the exception of LOAD DATA INFILE,
which is still buggy. Will push this one after the pull/merge
Test of unsigned BIGINT values
Fixes for queries-per-hour
Cleanup of replication code (comments and portability fixes)
Make most of the binary log code 4G clean
Changed syntax for GRANT ... QUERIES PER HOUR
first pull, merge,test, and get it to work.
The main change is the new replication code - now we have two slave threads
SQL thread and I/O thread. I have also re-written a lot of the code to
prepare for multi-master implementation.
I also documented IO_CACHE quite extensively and to some extend, THD class.
Stop event and bugs the test suite could not catch
Did some big restructuring of binlog event classes - most important
change is that now each event class has exec_event method and one does
not need to modify slave core code to add a new event. Slave code is
now much smaller and easier to read
Removed new error message
Fixed test case for varbinary
Threads are again killable in "merge_buffers"
Cleanup of sql_repl.cc
SHOW OPEN TABLES now works when no tables are opened
fixed log sequence bugs
fixed bugs in handling Slave event
added test case with SHOW BINLOG EVENTS
have not fixed all the bugs - found some that are also in 3.23,
will fix them there first, then do pull and cleanup
will not push this changeset
mark killed partially completed updates with an error code in binlog
stop replication if the master reports a possible partial/killed update
test partially killed update
