tables requires privileges for them if some table or column level grants
present" (with after-review fixes).
We should set SELECT_ACL for implicitly opened tables in
my_tz_check_n_skip_implicit_tables() to be able to bypass privilege
checking in check_grant(). Also we should exclude those tables from
privilege checking in multi-update.
CREATE DATABASE statement used the current database instead of the
database created when checking conditions for replication.
CREATE/DROP/ALTER DATABASE statements are now replicated based on
the manipulated database.
When we are writing a transaction to the binlog, we log BEGIN/COMMIT with zero error code.
Example: all statements of trans succeeded, connection lost and so implicit rollback:
we don't want ER_NET* errors to be logged in the BEGIN/ROLLBACK events, while statement
events have 0. If there was really a serious error code, it's already in the statement events.
the fact that the transaction log is empty does not mean we're not in a transaction
(it could be BEGIN; SELECT * FOR UPDATE FROM ibtable: then we don't want to commit now, even if
the statement is a MyISAM update).
With a testcase.
This allows one to setup a master <-> master replication with non conflicting auto-increment series.
Cleaned up binary log code to make it easyer to add new state variables.
Added simpler 'upper level' logic for artificial events (events that should not cause cleanups on slave).
Simplified binary log handling.
Changed how auto_increment works together with to SET INSERT_ID=# to make it more predictable: Now the inserted rows in a multi-row statement are set independent of the existing rows in the table. (Before only InnoDB did this correctly)
We must not reset the charset in slave after each statement, otherwise the SET CHARACTER SET is cancelled immediately.
Instead, we write a SET CHARACTER SET DEFAULT to the master's binlog when needed (like we already do for SET FOREIGN_KEY_CHECKS);
such writing is not necessary in 4.1 (in 4.1 the bug does not exist, as the SET ONE_SHOT syntax is used).
I have written a test and it works, but I'm not pushing the test as it requires building with all charsets.
I have noticed differences between what is inserted in the master's table in 4.0 and 4.1, and alerted Bar.
Fix for BUG#4971 "CREATE TABLE ... TYPE=HEAP SELECT ... stops slave (wrong DELETE in binlog)":
replacing the no_log argument of mysql_create_table() by some safer method
(temporarily setting OPTION_BIN_LOG to 0) which guarantees that even the automatic
DELETE FROM heap_table does not get into the binlog when a not-yet-existing HEAP table
is opened by mysql_create_table().
mysqld.cc:
Changed LOGLEVEL enum to loglevel
mysql_priv.h, log.cc:
Changed LOGLEVEL to loglevel. Removed startup_ from some of the DBUG_ENTER macros. Removed the print_msg_to_log function as it was unused.
my_getopt.c, my_getopt.h:
Renamed LOGLEVEL to loglevel to match coding standards
replacing the no_log argument of mysql_create_table() by some safer method
(temporarily setting OPTION_BIN_LOG to 0) which guarantees that even the automatic
DELETE FROM heap_table does not get into the binlog when a not-yet-existing HEAP table
is opened by mysql_create_table().
mysqld.cc:
Changed option_error_reporter to match new function header that includes LOGLEVEL enum
mysql_priv.h:
Removed the MY_ERROR style bitmask. Changed function headers to use new LOGLEVEL enum
log.cc:
Changed print_buffer_to_log to print_buffer_to_file. Remove the timestamp bool and now all log entries written to stderr are timestamped. Removed some unused commented code. changed to use the new LOGLEVEL enum.
my_getopt.c:
Changed functions to use the new LOGLEVEL enum and changed the included error reporter to be default_reporter. This reporter is used in handle_options if a reporter is not given
my_getopt.h:
changed typedefs to use better naming convention. Moved error bitmask into the LOGLEVEL enum and included it here.
Added declarations for print_msg_to_log and vprint_msg_to_log. sql_print_error are simple functions that wrap calls to print_msg_to_log. Define the different error types with MY_ERROR_TYPE, MY_WARNING_TYPE, and MY_INFORMATION_TYPE
gen_lex_hash.cc:
Added NULL error reporting parameter to handle_options
log.cc:
Add print_msg_to_log, print_buffer_to_log, and vprint_msg_to_log. Print_msg_to_log will write the message to the windows event log if on NT. We now have error, warning, and information versions of sql_print_xxxx. T his is a variation of a similar changeset WAX did.
mysqld.cc:
Added option_error_reporter callback function and pass that into handle_options
mysql.cc:
Added NULL as error reporter arg to the end of handle_options
Many files:
Added NULL error reporter parameter as the last paramter to handle_options
my_getopt.c:
Added second function pointer to server as an error reporting callback. Added local function report_option_error that will either write the error to stderr or to the error reporting callback. changed all calls in handle_options from fprintf(stderr, ... ) to report_option_error
my_getopt.h:
Changed declaration of handle_options to use typedefs for the two function pointers. added second function pointer to server as an error reporting callback
mysqld.dsp:
Added custom build step for compiling message file and added message resource file (output of mc)
Bug #4810 "deadlock with KILL when the victim was in a wait state"
(I included mutex unlock into exit_cond() for future safety)
and BUG#4827 "KILL while START SLAVE may lead to replication slave crash"
happen on SMP machines, when a thread is going to wait on a condition and it is KILLed at the
same time.
Cleaning code a bit by adding a test in enter_cond() that we have the mutex (was already the case in all places
where it's called except one which is fixed here).
Replication of charsets and timezones should shortly be changed to not need ONE_SHOT
(but 5.0 should still be able to read it, to be able to replicate 4.1 masters).
a limitation of yesterday's implementation:
if there was an unfinished transaction (COMMIT not typed), and some MyISAM tables were
then updated, and then mysqld crashes, then at restart the server would use the too old
binlog offset known by InnoDB to cut the binlog, thus cutting the successful MyISAM
updates. We fix this by reporting the binlog offset into InnoDB even if InnoDB was not
affected at all by the update.
But the feature is still disabled until we decide if it can go into 4.1.3.
Final push. Printing some warnings at startup, as --innodb-safe-binlog requires
some other options, to work as expected.
Adding 6 new tests (3 pairs). If they fail on some platforms (so far they have been run only on my Linux),
they should be ignored for the 4.1.3 build (you can just
rm mysql-test/*/rpl_crash_*).
Now going to update doc.
Won't be pushed as is - separate email sent for internal review.
WL#1717 "binlog-innodb consistency".
Now when mysqld starts, if InnoDB does a crash recovery, we use the binlog name
and position retrieved from InnoDB (corresponding to the last transaction
successfully committed by InnoDB) to cut any rolled back transaction from
the binary log. This is triggered by the --innodb-safe-binlog option.
Provided you configure mysqld to fsync() InnoDB at every commit (using
flush_log_at_trx_commit) and to fsync() the binlog at every write
(using --sync-binlog=1), this behaviour guarantees that a master always has
consistency between binlog and InnoDB, whenever the crash happens.
6 tests to verify that it works.
Added basic per-thread time zone functionality (based on public
domain elsie-code). Now user can select current time zone
(from the list of time zones described in system tables).
All NOW-like functions honor this time zone, values of TIMESTAMP
type are interpreted as values in this time zone, so now
our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH
LOCAL TIME ZONE (or proper PostgresSQL type).
WL#1266 "CONVERT_TZ() - basic time with time zone conversion
function".
Fixed problems described in Bug #2336 (Different number of warnings
when inserting bad datetime as string or as number). This required
reworking of datetime realted warning hadling (they now generated
at Field object level not in conversion functions).
Optimization: Now Field class descendants use table->in_use member
instead of current_thd macro.
New option --sync-binlog=x (and global settable variable) which will fsync the binlog
after every x-th disk write to it. That is, if in autocommit mode, after every x-th statement
written to the binlog; if using transactions, after every x-th transaction written to the binlog.
x==0 means no fsync. x==1 is the slowest.
There is no test added for this, I have just checked that it works as --sync-binlog=1 dramatically
slows down mysqld.
Made sync-frm a global settable variable.
by binlogging some SET ONE_SHOT CHARACTER_SETetc,
which will be enough until we have it more compact and more complete in 5.0. With the present patch,
replication will work ok between 4.1.3 master and slaves, as long as:
- master and slave have the same GLOBAL.COLLATION_SERVER
- COLLATION_DATABASE and CHARACTER_SET_DATABASE are not used
- application does not use the fact that table is created with charset of the USEd db (BUG#2326).
all of which are not too hard to fulfill.
ONE_SHOT is reserved for internal use of mysqlbinlog|mysql and works only for charsets,
so we give error if used for non-charset vars.
Fix for BUG#3875 "mysqlbinlog produces wrong ouput if query uses
variables containing quotes" and BUG#3943 "Queries with non-ASCII literals are not replicated
properly after SET NAMES".
Detecting that master and slave have different global charsets or server ids.
Fixed bugs in group_concat with ORDER BY and DISTINCT (Bugs #2695, #3381 and #3319)
Fixed crash when doing rollback in slave and the io thread catched up with the sql thread
Set locked_in_memory properly
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
It contains a few bugfixes (which I made when running the testsuite).
I carefully updated the results of the testsuite (i.e. I checked for every one,
if the difference between .reject and .result could be explained).
Apparently mysql-test-run --manager is broken in 4.1 and 5.0 currently,
so I could neither run the few tests which require --manager, nor check
that they pass nor modify their .result. But for builds, we don't run
with --manager.
Apart from --manager, the full testsuite passes, with Valgrind too (no errors).
I'm going to push in the next minutes. Remains: update the manual.
Note: by chance I saw that (in 4.1, in 5.0) rpl_get_lock fails when run alone;
this is normal at it makes assumptions on thread ids. I will fix this one day
in 4.1.
This is the main commit for Worklog tasks:
* A more dynamic binlog format which allows small changes (1064)
* Log session variables in Query_log_event (1063)
Below 5.0 means 5.0.0.
MySQL 5.0 is able to replicate FOREIGN_KEY_CHECKS, UNIQUE_KEY_CHECKS (for speed),
SQL_AUTO_IS_NULL, SQL_MODE. Not charsets (WL#1062), not some vars (I can only think
of SQL_SELECT_LIMIT, which deserves a special treatment). Note that this
works for queries, except LOAD DATA INFILE (for this it would have to wait
for Dmitri's push of WL#874, which in turns waits for the present push, so...
the deadlock must be broken!). Note that when Dmitri pushes WL#874 in 5.0.1,
5.0.0 won't be able to replicate a LOAD DATA INFILE from 5.0.1.
Apart from that, the new binlog format is designed so that it can tolerate
a little variation in the events (so that a 5.0.0 slave could replicate a
5.0.1 master, except for LOAD DATA INFILE unfortunately); that is, when I
later add replication of charsets it should break nothing. And when I later
add a UID to every event, it should break nothing.
The main change brought by this patch is a new type of event, Format_description_log_event,
which describes some lengthes in other event types. This event is needed for
the master/slave/mysqlbinlog to understand a 5.0 log. Thanks to this event,
we can later add more bytes to the header of every event without breaking compatibility.
Inside Query_log_event, we have some additional dynamic format, as every Query_log_event
can have a different number of status variables, stored as pairs (code, value); that's
how SQL_MODE and session variables and catalog are stored. Like this, we can later
add count of affected rows, charsets... and we can have options --don't-log-count-affected-rows
if we want.
MySQL 5.0 is able to run on 4.x relay logs, 4.x binlogs.
Upgrading a 4.x master to 5.0 is ok (no need to delete binlogs),
upgrading a 4.x slave to 5.0 is ok (no need to delete relay logs);
so both can be "hot" upgrades.
Upgrading a 3.23 master to 5.0 requires as much as upgrading it to 4.0.
3.23 and 4.x can't be slaves of 5.0.
So downgrading from 5.0 to 4.x may be complicated.
Log_event::log_pos is now the position of the end of the event, which is
more useful than the position of the beginning. We take care about compatibility
with <5.0 (in which log_pos is the beginning).
I added a short test for replication of SQL_MODE and some other variables.
TODO:
- after committing this, merge the latest 5.0 into it
- fix all tests
- update the manual with upgrade notes.
The constructor of Rotate_log_event used when we are rotating our binlog or
relay log, should not assume that there is a nonzero THD available.
For example, when we are reacting to SIGHUP, the THD is 0.
In fact we don't need to use the THD in this constructor;
we can do like for Stop_log_event, and use the minimal Log_event
constructor.
If we were allowed to put Unix-specific commands in the testsuite,
I'd add a test for this (<sigh>).
Fixed compiler warnings (a lot of hidden variables detected by the Forte compiler)
Added a lot of 'version_xxx' strings to 'show variables'
Prevent copying of TMP_TABLE_PARAM (This caused core dump bug on Solaris)
Fixed problem with printing sub selects to debug log
WL#604 Privileges in embedded library
code added to check privileges in embedded library
NO_EMBEDDED_ACCESS_CHECKS macros inserted in code so we can exclude
access-checking parts. Actually we now can exclude these parts from
standalone server as well. Do we need it?
Access checks are disabled in embedded server by default. One should
edit libmysqld/Makefile manually to get this working.
We definitely need the separate configure for embedded server
Bug #1392 "On Win, slave leaves one temp file after successf. replicating LOAD DATA INFILE"
Windows-specific bug (we forgot to close a file before deleting it).
Patch written by me, tested by Miguel (thanks!) and it works.
Add pack_bits to pack_reclength for dynamic rows. This solves buffer a possible buffer overflow on update.
(This will probably solve bug #563)
Fix test for available file descriptors in mysqltest
Fixed core dump bug in replication tests when running without transactional table support
For example the Binlog_dump thread (on the master) sometimes showed "Slave:".
And there were confusing messages where "binlog" was employed instead
of "relay log".
fix for BUG#1113 "INSERT into non-trans table SELECT ; ROLLBACK" does not send warning"
and
fix for BUG#873 "In transaction, INSERT to non-trans table is written too early to binlog".
Now we don't always write the non-trans update immediately to the binlog;
if there is something in the binlog cache we write it to the binlog cache
(because the non-trans update could depend on a trans table which was modified
earlier in the transaction); then in case of ROLLBACK, we write the binlog
cache to the binlog, wrapped with BEGIN/ROLLBACK.
This guarantees that the slave does the same updates.
For ROLLBACK TO SAVEPOINT: when we execute a SAVEPOINT command we write it
to the binlog cache. At ROLLBACK TO SAVEPOINT, if some non-trans table was updated,
we write ROLLBACK TO SAVEPOINT to the binlog cache; when the transaction
terminates (COMMIT/ROLLBACK), the binlog cache will be flushed to the binlog
(because of the non-trans update) so we'll have SAVEPOINT and ROLLBACK TO
SAVEPOINT in the binlog.
Apart from this rare case of updates of mixed table types in transaction, the
usual way is still clear the binlog cache at ROLLBACK, or chop it at
ROLLBACK TO SAVEPOINT (meaning the SAVEPOINT command is also chopped, which
is fine).
Note that BUG#873 encompasses subbugs 1) and 2) of BUG#333 "3 binlogging bugs when doing INSERT with mixed InnoDB/MyISAM".
"LOAD DATA INFILE is badly filtered by binlog-*-db rules".
There will probably be a second final one to merge Dmitri's changes
to rpl_log.result and mine.
2 new tests:
rpl_loaddata_rule_m : test of logging of LOAD DATA INFILE when the master has binlog-*-db rules,
rpl_loaddata_rule_s : test of logging of LOAD DATA INFILE when the slave has binlog-*-db rules and --log-slave-updates.
the first 4 bytes of the relay log. Indeed comments in mysys/mf_iocache.c
say we must always use my_b_append for such a cache.
This *could* avoid a very rare assertion failure which is:
030524 19:32:38 Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log '/
users/gbichot/4.1.1/mysql-test/var/log/slave-relay-bin.000001' position: 4
030524 19:32:38 next log '/users/gbichot/4.1.1/mysql-test/var/log/slave-relay-bin.000002' is currently active
mysqld: mf_iocache.c:701: _my_b_seq_read: Assertion `pos_in_file == info->end_of_file' failed.
and which seemed to happen always when the SQL thread and/or the I/O thread
were at position 4 in a relay log.
