This will be pushed only after I fix the testsuite.
This is the main commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
Below 5.0 means 5.0.0.
MySQL 5.0 is able to replicate FOREIGN_KEY_CHECKS, UNIQUE_KEY_CHECKS (for speed),
SQL_AUTO_IS_NULL, SQL_MODE. Not charsets (WL#1062), not some vars (I can only think
of SQL_SELECT_LIMIT, which deserves a special treatment). Note that this
works for queries, except LOAD DATA INFILE (for this it would have to wait
for Dmitri's push of WL#874, which in turns waits for the present push, so...
the deadlock must be broken!). Note that when Dmitri pushes WL#874 in 5.0.1,
5.0.0 won't be able to replicate a LOAD DATA INFILE from 5.0.1.
Apart from that, the new binlog format is designed so that it can tolerate
a little variation in the events (so that a 5.0.0 slave could replicate a
5.0.1 master, except for LOAD DATA INFILE unfortunately); that is, when I
later add replication of charsets it should break nothing. And when I later
add a UID to every event, it should break nothing.
The main change brought by this patch is a new type of event, Format_description_log_event,
which describes some lengthes in other event types. This event is needed for
the master/slave/mysqlbinlog to understand a 5.0 log. Thanks to this event,
we can later add more bytes to the header of every event without breaking compatibility.
Inside Query_log_event, we have some additional dynamic format, as every Query_log_event
can have a different number of status variables, stored as pairs (code, value); that's
how SQL_MODE and session variables and catalog are stored. Like this, we can later
add count of affected rows, charsets... and we can have options --don't-log-count-affected-rows
if we want.
MySQL 5.0 is able to run on 4.x relay logs, 4.x binlogs.
Upgrading a 4.x master to 5.0 is ok (no need to delete binlogs),
upgrading a 4.x slave to 5.0 is ok (no need to delete relay logs);
so both can be "hot" upgrades.
Upgrading a 3.23 master to 5.0 requires as much as upgrading it to 4.0.
3.23 and 4.x can't be slaves of 5.0.
So downgrading from 5.0 to 4.x may be complicated.
Log_event::log_pos is now the position of the end of the event, which is
more useful than the position of the beginning. We take care about compatibility
with <5.0 (in which log_pos is the beginning).
I added a short test for replication of SQL_MODE and some other variables.
TODO:
- after committing this, merge the latest 5.0 into it
- fix all tests
- update the manual with upgrade notes.
client/Makefile.am:
mysqlbinlog.cc depends slightly on sql/mysql_priv.h
client/mysqlbinlog.cc:
Make mysqlbinlog able to read the new binlog format,
by seeking to the start and reading the first few events, to
detect the format of the binlog.
include/my_sys.h:
a correct tell() for SEQ_READ_APPEND caches.
mysys/mf_iocache2.c:
a correct tell() for SEQ_READ_APPEND caches
(my_b_tell() is not working for such caches).
sql/ha_innodb.cc:
we are getting rid of event lengthes here and there, which is good.
sql/log.cc:
Start events will have created==0 if generated by rotation (like in 3.23).
In 5.0 we always write a Format_description_log_event at the beginning of
every master's binary log and of every slave's relay log.
We also add Rotate and Stop to relay logs (like there already was in
master's binary logs).
When we rotate a relay log, we write the previous relay log's Start event
(the one which was sent from the master) to the beginning of the new log,
so that we don't need the previous relay log to understand the new one;
that's the purpose of MYSQL_LOG::description_event_for_queue.
Removed logging of SET FOREIGN_KEY_CHECKS, because we handle it as flags
in the Query event now.
sql/log_event.cc:
New event type: Format_description_log_event, to describe the log's format.
read_log_event() needs to be passed this event to be able to read 5.0 events.
Query_log_event has new members flags2 and sql_mode for replication of session
variables (except charsets which are WL#1062) and SQL_MODE.
flags2 is in fact a kind of copy of thd->options (&'d with a mask).
Now with this replication of FOREIGN_KEY_CHECKS, SQL_AUTO_IS_NULL, UNIQUE_CHECKS
and SQL_MODE work; with mysqlbinlog too.
sql/log_event.h:
Binlog version is changed to 4.
New classes (details in sql/log_event.cc).
Removing some useless #defines.
sql/mysql_priv.h:
Definition of SELECT_DISTINCT and others must be visible in client/mysqlbinlog.cc,
so adding #ifdefs.
sql/mysqld.cc:
update for prototype change
sql/slave.cc:
When the slave opens a relay log, it reads the first few events to know the format.
When slave I/O thread receives a Rotate from the master, it rotates its relay log
(to avoid mixed format in the relay log).
sql/slave.h:
in the slave we avoid lengthes and rely on absolute positions instead;
hence the introduction of future_group_master_log_pos and future_event_relay_log_pos
(explained in code).
sql/sql_class.cc:
catalog in THD
sql/sql_class.h:
catalog, and new members in MYSQL_LOG
sql/sql_repl.cc:
When the master starts sending binlog to slave, it must
first read the first few events to detect the binlog's format.
Same for SHOW BINLOG EVENTS.
2003-12-18 01:09:05 +01:00
|
|
|
# Replication of session variables.
|
|
|
|
|
# FOREIGN_KEY_CHECKS is tested in rpl_insert_id.test
|
|
|
|
|
|
|
|
|
|
source include/master-slave.inc;
|
|
|
|
|
drop table if exists t1;
|
This is the final commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
It contains a few bugfixes (which I made when running the testsuite).
I carefully updated the results of the testsuite (i.e. I checked for every one,
if the difference between .reject and .result could be explained).
Apparently mysql-test-run --manager is broken in 4.1 and 5.0 currently,
so I could neither run the few tests which require --manager, nor check
that they pass nor modify their .result. But for builds, we don't run
with --manager.
Apart from --manager, the full testsuite passes, with Valgrind too (no errors).
I'm going to push in the next minutes. Remains: update the manual.
Note: by chance I saw that (in 4.1, in 5.0) rpl_get_lock fails when run alone;
this is normal at it makes assumptions on thread ids. I will fix this one day
in 4.1.
mysql-test/r/rpl000015.result:
result update
mysql-test/r/rpl_change_master.result:
result update
mysql-test/r/rpl_error_ignored_table.result:
result update
mysql-test/r/rpl_flush_log_loop.result:
result update
mysql-test/r/rpl_flush_tables.result:
result update
mysql-test/r/rpl_loaddata.result:
result update
mysql-test/r/rpl_loaddata_rule_m.result:
result update
mysql-test/r/rpl_loaddata_rule_s.result:
result update
mysql-test/r/rpl_log.result:
result update
mysql-test/r/rpl_log_pos.result:
result update
mysql-test/r/rpl_max_relay_size.result:
result update
mysql-test/r/rpl_relayrotate.result:
result update
mysql-test/r/rpl_replicate_do.result:
result update
mysql-test/r/rpl_reset_slave.result:
result update
mysql-test/r/rpl_rotate_logs.result:
result update
mysql-test/r/rpl_session_var.result:
result update
mysql-test/r/rpl_temporary.result:
result update
mysql-test/r/rpl_trunc_binlog.result:
result update
mysql-test/r/rpl_until.result:
result update
mysql-test/r/rpl_user_variables.result:
result update
mysql-test/t/rpl000010-slave.opt:
need to wait 2 events, because now we receive a Format_desc on top of the Rotate,
when replication starts.
mysql-test/t/rpl000015.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_change_master.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_empty_master_crash.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_error_ignored_table.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_flush_log_loop.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_loaddata.test:
position update
mysql-test/t/rpl_loaddata_rule_m.test:
position update
mysql-test/t/rpl_loaddata_rule_s.test:
position update
mysql-test/t/rpl_log.test:
position update
mysql-test/t/rpl_log_pos.test:
position update
mysql-test/t/rpl_max_relay_size.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_openssl.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_redirect.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_relayrotate-slave.opt:
better options for this test
mysql-test/t/rpl_relayrotate.test:
using max() is better for debugging (it shows at which place the slave
SQL thread resumed)
mysql-test/t/rpl_replicate_do.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_reset_slave.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_rotate_logs.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_session_var.test:
100 because password() is longer than 10 chars
mysql-test/t/rpl_trunc_binlog.test:
relay log information is not repeatable in general (if a reconnection
happens because --sleep=10 for example), so we hide these columns.
mysql-test/t/rpl_until.test:
position update
mysql-test/t/rpl_user_variables.test:
position update
sql/log.cc:
Ensure that the Format_desc propagated on next relay logs does not trigger
undue actions (like incrementing some positions or clearing some files).
sql/log_event.cc:
* When the slave SQL thread finds a Rotate/Format_desc/Stop in the middle of
a transaction (then these were written by the slave itself to its relay log),
it should not increment rli->group* variables, but only rli->event* ones.
* When the slave SQL thread finds a Format_desc not to be ignored (not
the same server id as the slave's), if it has log_pos==0 it must not trigger
"unfinished transaction in master's binlog" (log_pos==0 is always a marker
in the relay log to mean "this event was not at this place in the master's
binlog": it's for fake Rotate events, and for Format_description events which the master had
to send us for replication to start).
* In the Query_log_event on disk, catalog is now terminated by '\0'.
* thd->catalog must be set to 0 when some exec_event() terminate (otherwise
double free).
sql/slave.cc:
* Fixes for a few bugs when ignoring events in the slave SQL thread:
- do not decrement rli->slave_skip_counter if the event is an event
related to the binlog or relay log itself (FORMAT_DESCRIPTION,
ROTATE, STOP) because these events should never be skipped (or the slave
will be confused). Usually the user wants to skip a query, not a Rotate...
- when we (re)connect to the master, we must free description_event_for_queue
(otherwise memory leak when we reconnect).
* Changed a bit the code where we change description_event_for_queue,
to make it look "safer".
* Moved 'created=0' to log.cc where it is safer.
* When the slave SQL thread finds a Rotate/Format_desc/Stop in the middle of
a transaction (then these were written by the slave itself to its relay log),
it should not increment rli->group* variables, but only rli->event* ones.
sql/sql_class.h:
a warning comment
sql/sql_repl.cc:
A mistake: I had passed a char* instead of char**
2003-12-19 22:40:23 +01:00
|
|
|
create table t1(a varchar(100),b int);
|
This will be pushed only after I fix the testsuite.
This is the main commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
Below 5.0 means 5.0.0.
MySQL 5.0 is able to replicate FOREIGN_KEY_CHECKS, UNIQUE_KEY_CHECKS (for speed),
SQL_AUTO_IS_NULL, SQL_MODE. Not charsets (WL#1062), not some vars (I can only think
of SQL_SELECT_LIMIT, which deserves a special treatment). Note that this
works for queries, except LOAD DATA INFILE (for this it would have to wait
for Dmitri's push of WL#874, which in turns waits for the present push, so...
the deadlock must be broken!). Note that when Dmitri pushes WL#874 in 5.0.1,
5.0.0 won't be able to replicate a LOAD DATA INFILE from 5.0.1.
Apart from that, the new binlog format is designed so that it can tolerate
a little variation in the events (so that a 5.0.0 slave could replicate a
5.0.1 master, except for LOAD DATA INFILE unfortunately); that is, when I
later add replication of charsets it should break nothing. And when I later
add a UID to every event, it should break nothing.
The main change brought by this patch is a new type of event, Format_description_log_event,
which describes some lengthes in other event types. This event is needed for
the master/slave/mysqlbinlog to understand a 5.0 log. Thanks to this event,
we can later add more bytes to the header of every event without breaking compatibility.
Inside Query_log_event, we have some additional dynamic format, as every Query_log_event
can have a different number of status variables, stored as pairs (code, value); that's
how SQL_MODE and session variables and catalog are stored. Like this, we can later
add count of affected rows, charsets... and we can have options --don't-log-count-affected-rows
if we want.
MySQL 5.0 is able to run on 4.x relay logs, 4.x binlogs.
Upgrading a 4.x master to 5.0 is ok (no need to delete binlogs),
upgrading a 4.x slave to 5.0 is ok (no need to delete relay logs);
so both can be "hot" upgrades.
Upgrading a 3.23 master to 5.0 requires as much as upgrading it to 4.0.
3.23 and 4.x can't be slaves of 5.0.
So downgrading from 5.0 to 4.x may be complicated.
Log_event::log_pos is now the position of the end of the event, which is
more useful than the position of the beginning. We take care about compatibility
with <5.0 (in which log_pos is the beginning).
I added a short test for replication of SQL_MODE and some other variables.
TODO:
- after committing this, merge the latest 5.0 into it
- fix all tests
- update the manual with upgrade notes.
client/Makefile.am:
mysqlbinlog.cc depends slightly on sql/mysql_priv.h
client/mysqlbinlog.cc:
Make mysqlbinlog able to read the new binlog format,
by seeking to the start and reading the first few events, to
detect the format of the binlog.
include/my_sys.h:
a correct tell() for SEQ_READ_APPEND caches.
mysys/mf_iocache2.c:
a correct tell() for SEQ_READ_APPEND caches
(my_b_tell() is not working for such caches).
sql/ha_innodb.cc:
we are getting rid of event lengthes here and there, which is good.
sql/log.cc:
Start events will have created==0 if generated by rotation (like in 3.23).
In 5.0 we always write a Format_description_log_event at the beginning of
every master's binary log and of every slave's relay log.
We also add Rotate and Stop to relay logs (like there already was in
master's binary logs).
When we rotate a relay log, we write the previous relay log's Start event
(the one which was sent from the master) to the beginning of the new log,
so that we don't need the previous relay log to understand the new one;
that's the purpose of MYSQL_LOG::description_event_for_queue.
Removed logging of SET FOREIGN_KEY_CHECKS, because we handle it as flags
in the Query event now.
sql/log_event.cc:
New event type: Format_description_log_event, to describe the log's format.
read_log_event() needs to be passed this event to be able to read 5.0 events.
Query_log_event has new members flags2 and sql_mode for replication of session
variables (except charsets which are WL#1062) and SQL_MODE.
flags2 is in fact a kind of copy of thd->options (&'d with a mask).
Now with this replication of FOREIGN_KEY_CHECKS, SQL_AUTO_IS_NULL, UNIQUE_CHECKS
and SQL_MODE work; with mysqlbinlog too.
sql/log_event.h:
Binlog version is changed to 4.
New classes (details in sql/log_event.cc).
Removing some useless #defines.
sql/mysql_priv.h:
Definition of SELECT_DISTINCT and others must be visible in client/mysqlbinlog.cc,
so adding #ifdefs.
sql/mysqld.cc:
update for prototype change
sql/slave.cc:
When the slave opens a relay log, it reads the first few events to know the format.
When slave I/O thread receives a Rotate from the master, it rotates its relay log
(to avoid mixed format in the relay log).
sql/slave.h:
in the slave we avoid lengthes and rely on absolute positions instead;
hence the introduction of future_group_master_log_pos and future_event_relay_log_pos
(explained in code).
sql/sql_class.cc:
catalog in THD
sql/sql_class.h:
catalog, and new members in MYSQL_LOG
sql/sql_repl.cc:
When the master starts sending binlog to slave, it must
first read the first few events to detect the binlog's format.
Same for SHOW BINLOG EVENTS.
2003-12-18 01:09:05 +01:00
|
|
|
set @@session.sql_mode=pipes_as_concat;
|
|
|
|
|
insert into t1 values('My'||'SQL', 1);
|
|
|
|
|
set @@session.sql_mode=default;
|
2005-04-01 14:04:50 +02:00
|
|
|
insert into t1 values('1'||'2', 2);
|
This will be pushed only after I fix the testsuite.
This is the main commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
Below 5.0 means 5.0.0.
MySQL 5.0 is able to replicate FOREIGN_KEY_CHECKS, UNIQUE_KEY_CHECKS (for speed),
SQL_AUTO_IS_NULL, SQL_MODE. Not charsets (WL#1062), not some vars (I can only think
of SQL_SELECT_LIMIT, which deserves a special treatment). Note that this
works for queries, except LOAD DATA INFILE (for this it would have to wait
for Dmitri's push of WL#874, which in turns waits for the present push, so...
the deadlock must be broken!). Note that when Dmitri pushes WL#874 in 5.0.1,
5.0.0 won't be able to replicate a LOAD DATA INFILE from 5.0.1.
Apart from that, the new binlog format is designed so that it can tolerate
a little variation in the events (so that a 5.0.0 slave could replicate a
5.0.1 master, except for LOAD DATA INFILE unfortunately); that is, when I
later add replication of charsets it should break nothing. And when I later
add a UID to every event, it should break nothing.
The main change brought by this patch is a new type of event, Format_description_log_event,
which describes some lengthes in other event types. This event is needed for
the master/slave/mysqlbinlog to understand a 5.0 log. Thanks to this event,
we can later add more bytes to the header of every event without breaking compatibility.
Inside Query_log_event, we have some additional dynamic format, as every Query_log_event
can have a different number of status variables, stored as pairs (code, value); that's
how SQL_MODE and session variables and catalog are stored. Like this, we can later
add count of affected rows, charsets... and we can have options --don't-log-count-affected-rows
if we want.
MySQL 5.0 is able to run on 4.x relay logs, 4.x binlogs.
Upgrading a 4.x master to 5.0 is ok (no need to delete binlogs),
upgrading a 4.x slave to 5.0 is ok (no need to delete relay logs);
so both can be "hot" upgrades.
Upgrading a 3.23 master to 5.0 requires as much as upgrading it to 4.0.
3.23 and 4.x can't be slaves of 5.0.
So downgrading from 5.0 to 4.x may be complicated.
Log_event::log_pos is now the position of the end of the event, which is
more useful than the position of the beginning. We take care about compatibility
with <5.0 (in which log_pos is the beginning).
I added a short test for replication of SQL_MODE and some other variables.
TODO:
- after committing this, merge the latest 5.0 into it
- fix all tests
- update the manual with upgrade notes.
client/Makefile.am:
mysqlbinlog.cc depends slightly on sql/mysql_priv.h
client/mysqlbinlog.cc:
Make mysqlbinlog able to read the new binlog format,
by seeking to the start and reading the first few events, to
detect the format of the binlog.
include/my_sys.h:
a correct tell() for SEQ_READ_APPEND caches.
mysys/mf_iocache2.c:
a correct tell() for SEQ_READ_APPEND caches
(my_b_tell() is not working for such caches).
sql/ha_innodb.cc:
we are getting rid of event lengthes here and there, which is good.
sql/log.cc:
Start events will have created==0 if generated by rotation (like in 3.23).
In 5.0 we always write a Format_description_log_event at the beginning of
every master's binary log and of every slave's relay log.
We also add Rotate and Stop to relay logs (like there already was in
master's binary logs).
When we rotate a relay log, we write the previous relay log's Start event
(the one which was sent from the master) to the beginning of the new log,
so that we don't need the previous relay log to understand the new one;
that's the purpose of MYSQL_LOG::description_event_for_queue.
Removed logging of SET FOREIGN_KEY_CHECKS, because we handle it as flags
in the Query event now.
sql/log_event.cc:
New event type: Format_description_log_event, to describe the log's format.
read_log_event() needs to be passed this event to be able to read 5.0 events.
Query_log_event has new members flags2 and sql_mode for replication of session
variables (except charsets which are WL#1062) and SQL_MODE.
flags2 is in fact a kind of copy of thd->options (&'d with a mask).
Now with this replication of FOREIGN_KEY_CHECKS, SQL_AUTO_IS_NULL, UNIQUE_CHECKS
and SQL_MODE work; with mysqlbinlog too.
sql/log_event.h:
Binlog version is changed to 4.
New classes (details in sql/log_event.cc).
Removing some useless #defines.
sql/mysql_priv.h:
Definition of SELECT_DISTINCT and others must be visible in client/mysqlbinlog.cc,
so adding #ifdefs.
sql/mysqld.cc:
update for prototype change
sql/slave.cc:
When the slave opens a relay log, it reads the first few events to know the format.
When slave I/O thread receives a Rotate from the master, it rotates its relay log
(to avoid mixed format in the relay log).
sql/slave.h:
in the slave we avoid lengthes and rely on absolute positions instead;
hence the introduction of future_group_master_log_pos and future_event_relay_log_pos
(explained in code).
sql/sql_class.cc:
catalog in THD
sql/sql_class.h:
catalog, and new members in MYSQL_LOG
sql/sql_repl.cc:
When the master starts sending binlog to slave, it must
first read the first few events to detect the binlog's format.
Same for SHOW BINLOG EVENTS.
2003-12-18 01:09:05 +01:00
|
|
|
select * from t1 where b<3 order by a;
|
|
|
|
|
save_master_pos;
|
|
|
|
|
connection slave;
|
|
|
|
|
sync_with_master;
|
|
|
|
|
select * from t1 where b<3 order by a;
|
|
|
|
|
connection master;
|
|
|
|
|
# if the slave does the next sync_with_master fine, then it means it accepts the
|
|
|
|
|
# two lines of ANSI syntax below, which is what we want to check.
|
|
|
|
|
set @@session.sql_mode=ignore_space;
|
|
|
|
|
insert into t1 values(password ('MySQL'), 3);
|
|
|
|
|
set @@session.sql_mode=ansi_quotes;
|
|
|
|
|
create table "t2" ("a" int);
|
|
|
|
|
drop table t1, t2;
|
|
|
|
|
set @@session.sql_mode=default;
|
|
|
|
|
create table t1(a int auto_increment primary key);
|
|
|
|
|
create table t2(b int, a int);
|
|
|
|
|
set @@session.sql_auto_is_null=1;
|
|
|
|
|
insert into t1 values(null);
|
|
|
|
|
insert into t2 select 1,a from t1 where a is null;
|
|
|
|
|
set @@session.sql_auto_is_null=0;
|
|
|
|
|
insert into t1 values(null);
|
|
|
|
|
insert into t2 select 2,a from t1 where a is null;
|
|
|
|
|
select * from t2 order by b;
|
|
|
|
|
save_master_pos;
|
|
|
|
|
connection slave;
|
|
|
|
|
sync_with_master;
|
|
|
|
|
select * from t2 order by b;
|
|
|
|
|
connection master;
|
|
|
|
|
drop table t1,t2;
|
|
|
|
|
save_master_pos;
|
|
|
|
|
connection slave;
|
|
|
|
|
sync_with_master;
|