Bug #21785 "Server crashes after rename of the log table" and
Bug #21966 "Strange warnings on create like/repair of the log
tables"
According to the patch, from now on, one should use RENAME to
perform a log table rotation (this should also be reflected in
the manual).
Here is a sample:
use mysql;
CREATE TABLE IF NOT EXISTS general_log2 LIKE general_log;
RENAME TABLE general_log TO general_log_backup, general_log2 TO general_log;
The rules for Rename of the log tables are following:
IF 1. Log tables are enabled
AND 2. Rename operates on the log table and nothing is being
renamed to the log table.
DO 3. Throw an error message.
ELSE 4. Perform rename.
The very RENAME query will go the the old (backup) table. This is
consistent with the behavoiur we have with binlog ROTATE LOGS
statement.
Other problems, which are solved by the patch are:
1) Now REPAIR of the log table is exclusive operation (as it should be), this
also eliminates lock-related warnings. and
2) CREATE LIKE TABLE now usese usual read lock on the source table rather
then name lock, which is too restrictive. This way we get rid of another
log table-related warning, which occured because of the above fact
(as a side-effect, name lock resulted in a warning).
mysql-test/r/log_tables.result:
update result file
mysql-test/t/log_tables.test:
Add tests for the bugs
sql/handler.cc:
update comment
sql/handler.h:
update function to reflect changes in log tables
locking logic.
sql/lock.cc:
Now we allow locking of the log tables for "privileged" threads
Privileged thread must explicitly close and lock log tables. This
is required for admin operations such as REPAIR.
sql/log.cc:
Changes to the file:
1) Add checks for table schema. It's more important now,
as we allow rename of the log tables. Since we should
check for schema when writing to a log table.
E.g. if one created a table with one-only comlumn and
renamed it to general_log, the server should cope with
it.
2) refactor LOGGER::flush(), so that we can now use the same
machinery as we use in FLUSH LOGS in other statements:
whenever we have to perform a serious operation on the log
tables, we have to
(a) lock logger, which blocks other concurrent statements (such
as selects) (b) close logs. Then perform an
exclusive operation, c) reenable logs and d) unlock logger.
3) Add a function to check if a given table is a log table.
4) Add support for "privileged" thread
5) merge is_[general/slow]_log_table_enabled() into one function.
6) Add new function: reopen _log_tables, which reopens the tables,
which were enabled (after temporary close, required for admin
operation)
sql/log.h:
1) add a new call close_n_lock_tables(). Now we use it instead of
LOGGER::flush() in FLUSH LOGS implementation.
2) add a prototype for the function to check if a given
table is a log table;
3) add privileged table flag to table logger
4) merge is_[general/slow]_log_table_enabled()
into one function.
sql/mysql_priv.h:
move log table defines to log.h
sql/sql_delete.cc:
use new function check_if_log_table() instead of direct strcmp
sql/sql_rename.cc:
Traverse the list of tables in mysql_rename_tables
to make sure that log tables are processed correctly
(that is, according to the rules specified in the
main CS comment)
sql/sql_table.cc:
1) mysql_admin_table() should disable logs if it performs
exclusive admin operation on a log table. This way we
also eliminate warning on REPAIR of the log table.
2) mysql_create_like_table should read-lock the source table
instead getting name lock on it. Name lock is too restrictive
in this case.
sql/share/errmsg.txt:
Add a new error message for rename of the log tables
sql/table.cc:
use new function instead of direct strcmp.
change my_strcasecmp() -> strcmp(), when
comparing system db and table names
storage/csv/ha_tina.cc:
update function to reflect changes in log tables
locking logic.
storage/myisam/ha_myisam.cc:
update function to reflect changes in log tables
locking logic.
into mysql.com:/home/bk/MERGE/mysql-5.1-merge
include/my_sys.h:
Auto merged
mysql-test/t/view.test:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/share/errmsg.txt:
Auto merged
sql/slave.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
storage/ndb/src/mgmapi/mgmapi.cpp:
Auto merged
> sizeof(unsigned long).
sql/log.cc:
Replacing enum constant that contained an unsigned long long constant
on some platforms with a macro (an enumeration constant is limited
to the range of values covered by signed or unsigned long).
---
BUG#20265 (Replication of CREATE-SELECT does not work correctly):
Fixing bug by making binary log handle statement transactions.
The binary log transaction cache can now be truncated to remove
events inserted during this statement or transaction. Also, the
binary log participate in XA transaction handling, although not
as a full 2pc resource.
mysql-test/r/binlog_row_mix_innodb_myisam.result:
Result change
sql/log.cc:
Making change to binlog_end_trans() to support that it can be called
for statement transactions as well.
sql/sql_class.h:
Adding function THD::binlog_start_trans_and_stmt() to start a real transaction
(if necessary) and also a statement transaction.
sql/sql_insert.cc:
I had forgotten to delete this line (it was already disabled using //;
this line was not needed because we do the empty() every time
we write to the binlog (in MYSQL_LOG::write());
t/binlog_stm_binlog.test already tests that the empty() indeed happens
for INSERT DELAYED.
---
Changes to use the statement transactions that the binary log now can handle.
include/mysql/plugin.h:
Adding license information.
plugin/fulltext/plugin_example.c:
License
sql/ha_ndbcluster.cc:
License
sql/ha_partition.cc:
License
sql/handler.h:
License
sql/log.cc:
License
sql/sql_show.cc:
Additional PLUIN display information
storage/archive/ha_archive.cc:
License information added
storage/blackhole/ha_blackhole.cc:
License information added
storage/csv/ha_tina.cc:
License Information
storage/example/ha_example.cc:
License information
storage/federated/ha_federated.cc:
License Information
storage/heap/ha_heap.cc:
License Information
storage/innobase/handler/ha_innodb.cc:
License Information
storage/myisam/ha_myisam.cc:
License Information
storage/myisammrg/ha_myisammrg.cc:
License Information
into neptunus.(none):/home/msvensson/mysql/mysql-5.0-maint
sql/item_func.cc:
Auto merged
sql/log.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_class.h:
Auto merged
invocations of LAST_INSERT_ID.
Reding of LAST_INSERT_ID inside stored function wasn't noted by caller,
and no LAST_INSERT_ID_EVENT was issued for binary log.
The solution is to add THD::last_insert_id_used_bin_log, which is much
like THD::last_insert_id_used, but is reset only for upper-level
statements. This new variable is used to issue LAST_INSERT_ID_EVENT.
mysql-test/r/rpl_insert_id.result:
For bug#21726, add result for statement-based replication of function
calls.
mysql-test/t/rpl_insert_id.test:
For bug#21726, add test case for statement-based replication of function
calls.
sql/item_func.cc:
Set THD::last_insert_id_used_bin_log for issuing of LAST_INSERT_ID_EVENT.
sql/log.cc:
Issue LAST_INSERT_ID_EVENT if THD::last_insert_id_used_bin_log is set.
sql/set_var.cc:
Set THD::last_insert_id_used_bin_log for issuing of LAST_INSERT_ID_EVENT.
sql/sql_class.cc:
Initialize THD::last_insert_id_used_bin_log.
Fix typo, add whitespace.
sql/sql_class.h:
Add THD::last_insert_id_used_bin_log.
sql/sql_parse.cc:
Reset THD::last_insert_id_used_bin_log for upper-level statements.
sql/sql_select.cc:
Set THD::last_insert_id_used_bin_log for issuing of LAST_INSERT_ID_EVENT.
into neptunus.(none):/home/msvensson/mysql/mysql-5.1-new-maint
BitKeeper/etc/ignore:
auto-union
mysql-test/r/ctype_ucs.result:
Auto merged
mysql-test/r/func_gconcat.result:
Auto merged
mysql-test/r/func_group.result:
Auto merged
mysql-test/r/group_by.result:
Auto merged
mysql-test/r/partition.result:
Auto merged
mysql-test/r/ps.result:
Auto merged
mysql-test/r/subselect.result:
Auto merged
mysql-test/r/view.result:
Auto merged
mysql-test/t/ctype_ucs.test:
Auto merged
mysql-test/t/func_gconcat.test:
Auto merged
mysql-test/t/ps.test:
Auto merged
mysql-test/t/view.test:
Auto merged
sql/field.h:
Auto merged
sql/filesort.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/item_sum.cc:
Auto merged
sql/log.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_view.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
storage/innobase/handler/ha_innodb.h:
Auto merged
storage/myisam/mi_check.c:
Auto merged
sql/share/errmsg.txt:
Manul merge(conflict with new german text)
sql/unireg.cc:
Manual merge(as it is in 5.1)
Partitioning now has a flag to allow disabling of engines from being compatible with partitioning. Cleaned up heap naming convention on panic call.
include/heap.h:
Added new function
sql/ha_ndbcluster.cc:
Updated hanlderton functions
sql/ha_ndbcluster_binlog.cc:
Updated handlerton functions.
sql/handler.cc:
Updated handlerton functions.
sql/handler.h:
Updated hanlderton functions
sql/log.cc:
Updated handlerton functions
sql/mysql_priv.h:
Updated handlerton functions
sql/mysqld.cc:
Added Legacy handlerton functions
sql/partition_info.cc:
Added flag support for marking engines not compatible with partitioning
sql/sql_cursor.cc:
Updated hanlderton functions
sql/sql_show.cc:
Updated hanlderton functions
sql/sql_tablespace.cc:
Update for handlerton functions
storage/archive/ha_archive.cc:
Update for hanlderton functions
storage/archive/ha_archive.h:
Update for handlerton functions
storage/blackhole/ha_blackhole.cc:
Update for handlerton functions
storage/csv/ha_tina.cc:
Update for handlerton functions
storage/csv/ha_tina.h:
Update for handlerton functions
storage/federated/ha_federated.cc:
Updated for handlerton functions
storage/federated/ha_federated.h:
Updated for handlerton functions
storage/heap/ha_heap.cc:
Update for handlerton functions
storage/heap/hp_panic.c:
Update of function name
storage/heap/hp_test1.c:
Update of function name
storage/heap/hp_test2.c:
Update of function name.
storage/innobase/handler/ha_innodb.cc:
Update of function name
storage/innobase/handler/ha_innodb.h:
Update of function name
storage/myisam/ha_myisam.cc:
Update of function name.
storage/myisammrg/ha_myisammrg.cc:
Update of function name
Print version_comment after server version in:
mysql.cc: Welcome message and 'status' command output
log.cc: Top of log files
client/mysql.cc:
Add server_version_string() function to return a buffer with the server version and version_comment combined.
Use server_version_string() instead of mysql_get_server_info() for Welcome! message and 'status' command
sql/log.cc:
MYSQL_LOG::open():
- print MYSQL_COMPILATION_COMMENT after the server_version at the top of log files
into zim.(none):/home/brian/mysql/arch-5.1
sql/ha_ndbcluster.cc:
Auto merged
sql/log.cc:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
include/mysql/plugin.h:
Field Removed
sql/ha_ndbcluster.cc:
Field Removed
sql/ha_partition.cc:
Field Removed
sql/log.cc:
Removed Field
storage/archive/ha_archive.cc:
Removed Field
storage/blackhole/ha_blackhole.cc:
Removed Field
storage/csv/ha_tina.cc:
Removed Field
storage/example/ha_example.cc:
Removed Field
storage/federated/ha_federated.cc:
Removed Field
storage/heap/ha_heap.cc:
Removed Field
storage/innobase/handler/ha_innodb.cc:
Removed Field
storage/myisam/ha_myisam.cc:
Removed Field
storage/myisammrg/ha_myisammrg.cc:
Removed Field
into mysql.com:/users/lthalmann/bk/MERGE/mysql-5.1-merge
BitKeeper/etc/ignore:
auto-union
Makefile.am:
Auto merged
client/mysqldump.c:
Auto merged
mysql-test/r/information_schema.result:
Auto merged
mysql-test/t/disabled.def:
Auto merged
sql/CMakeLists.txt:
Auto merged
sql/Makefile.am:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.h:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
mysql-test/r/mysqldump.result:
Restoring rpl tree mysqldump test file to main tree test files
mysql-test/r/status.result:
Manual merge
mysql-test/t/mysqldump.test:
Restoring rpl tree mysqldump test file to main tree test files
In practice this means that handlerton is now created by the server and is passed to the engine. Plugin startups can now also control how plugins are inited (and can optionally pass values). Bit more flexibility to those who want to write plugin interfaces to the database.
include/mysql/plugin.h:
Optional to pass parameter now to init and deinit functions
sql/ha_ndbcluster.cc:
Cleanup, handlerton is now a pointer.
sql/ha_ndbcluster_binlog.cc:
Cleanup (handlerton is now a pointer)
sql/ha_ndbcluster_binlog.h:
Cleanup (handlerton is now a pointer)
sql/ha_partition.cc:
Cleaned up handlerton change
sql/handler.cc:
Cheanup of handlerton change
sql/item_sum.cc:
Cleanedup of handlerton change
sql/log.cc:
Cleanup of handlerton change
sql/mysql_priv.h:
Reverted patch for variables (what would have happen previously if a have_ would have been called that was dynamically loaded? boom!)
sql/mysqld.cc:
Cleanup of handlerton changes and reverted have variable patch
sql/partition_info.cc:
Cleanup of handlerton (we need to clean this up a bit more).
sql/set_var.cc:
Cleanup related to handlerton changes
sql/sql_cache.cc:
Handlerton changes cleanup
sql/sql_insert.cc:
Handlerton changes cleanup.
sql/sql_partition.cc:
Handlerton cleanup changes
sql/sql_plugin.cc:
Handlerton changes.
init() can now be controlled by a plugin specific startup.
There was also an issue with how we deinited the status variables. It should have been occuring before we shut down the engines.
sql/sql_select.cc:
Handlerton cleanup changes
sql/sql_show.cc:
Handlerton cleanup changes
sql/sql_table.cc:
Handlerton cleanup changes
sql/table.cc:
Cleanup
storage/archive/ha_archive.cc:
Cleanup
storage/archive/ha_archive.h:
Cleanup
storage/blackhole/ha_blackhole.cc:
Cleanup
storage/csv/ha_tina.cc:
Cleanup
storage/example/ha_example.cc:
Cleanup
storage/federated/ha_federated.cc:
Cleanup
storage/heap/ha_heap.cc:
Cleanup
storage/innobase/handler/ha_innodb.cc:
Cleanup
storage/myisam/ha_myisam.cc:
Cleanup
storage/myisammrg/ha_myisammrg.cc:
Cleanup
into mysql.com:/users/lthalmann/bk/MERGE/mysql-5.1-merge
mysql-test/r/mysqldump.result:
Auto merged
mysql-test/t/variables.test:
Auto merged
sql/field.cc:
Auto merged
sql/log.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
client/mysqldump.c:
Merge main->rpl
mysql-test/r/variables.result:
Merge main->rpl
mysql-test/t/disabled.def:
Merge main->rpl
mysql-test/t/mysqldump.test:
Merge main->rpl
(as part of the auto_increment cleanup of WL#3146; let's not be
sad, that monster push still removed serious bugs):
one problem with INSERT DELAYED (unexpected interval releases),
one with stored functions (wrong auto_inc binlogging).
These bugs were not released.
mysql-test/extra/binlog_tests/binlog_insert_delayed.test:
more tests of binlogging of INSERT DELAYED: with multi-row INSERTs.
I identified why sleeps are needed to get a repeatable row-based
binlogged: because without sleeps rows sometimes get groupped
and so generate different row based events.
mysql-test/extra/rpl_tests/rpl_foreign_key.test:
don't forget to drop tables on slave too, otherwise it leaves
an orphan innodb table leading to rpl_insert_id failing sometimes
(like in pushbuild "sapsrv2 -max").
mysql-test/extra/rpl_tests/rpl_insert_id.test:
testing that if some statement does not update any row, it does
not pollute the auto_inc binlog variables of the next statement;
the test has to use stored procedures because with plain statements,
mysql_reset_thd_for_next_command() does the resetting (and thus
there is no problem); mysql_reset_thd_for_next_command() is not
called inside routines.
mysql-test/r/binlog_row_binlog.result:
result additions
mysql-test/r/binlog_statement_insert_delayed.result:
result additions
mysql-test/r/binlog_stm_binlog.result:
result additions
mysql-test/r/rpl_insert_id.result:
result additions
mysql-test/r/rpl_loaddata.result:
With the change to log.cc reverted, the result changes and is better:
the change to log.cc had caused some INSERT_ID events to disappear
though they were necessary (but testsuite could not catch that because
it's single-threaded).
mysql-test/r/rpl_ndb_insert_ignore.result:
NDB is now like other engines regarding INSERT IGNORE: autoincrement
values which caused a duplicate key are re-used for next row, not lost.
rpl_ndb_insert_ignore.result is now identical to rpl_insert_ignore.result.
sql/log.cc:
LOAD DATA INFILE is binlogged as several events, and the last of them must
have the auto_inc id. So it's wrong to reset the auto_inc id after every
binlog write (because then it's lost after the first event of LOAD
DATA INFILE and so missing for the last one)/
Another problem: MYSQL_LOG::write() is not always called (for example
if no row was updated), so we were missing reset in some cases.
sql/sp_head.cc:
SELECT func1(),func2() generates two binlog events, so needs to
clear auto_increment binlog variables after each binlog event
(it would be more natural to clear them in the log write code,
but LOAD DATA INFILE would suffer from this see the cset comment
for log.cc). Without the clearing, the problem is:
> exec func1()
>> call cleanup_after_query() (which does not clear our vars here)
>> binlog SELECT func1()
<
> exec func2()
and so SELECT func2() is binlogged with the auto_inc of SELECT func1().
sql/sql_class.cc:
after every statement we should clear auto_inc variables used for
binlogging, except if this was a function/trigger (in which case
it may be "INSERT SELECT func()", where the cleanup_after_query()
executed in func() should not reset the auto_inc binlog variables
as they'll be necessary when binlogging the INSERT SELECT later).
sql/sql_insert.cc:
- as INSERT DELAYED uses the same TABLE object as the delayed_insert
system thread, we should not call ha_release_auto_increment()
from INSERT DELAYED (and btw it's logical as we reserve nothing
as we don't perform the insert). Calling the function caused us to
release values being used by the delayed_insert thread.
So I do the call only if this is a non-DELAYED INSERT.
- Assuming two INSERT DELAYED which get grouped by the delayed_insert
thread, the second may use values reserved by the first, which is ok
per se, but is a problem in statement-based binlogging:
the 2nd INSERT gets binlogged with the "interval start" value
of the first INSERT (=> duplicate error in slave).
- no reason to ha_release_auto_increment() after every inserted row
in INSERT SELECT; more efficient to do it only when the statement ends
sql/sql_parse.cc:
a comment
into mysql.com:/users/lthalmann/bk/MERGE/mysql-5.1-merge
BitKeeper/deleted/.del-rpl_heap.test:
Auto merged
Makefile.am:
Auto merged
mysql-test/extra/binlog_tests/binlog.test:
Auto merged
mysql-test/r/binlog_stm_binlog.result:
Auto merged
mysql-test/r/rpl_switch_stm_row_mixed.result:
Auto merged
mysql-test/t/date_formats.test:
Auto merged
mysql-test/t/mysqlbinlog.test:
Auto merged
mysql-test/t/rpl_trigger.test:
Auto merged
sql/CMakeLists.txt:
Auto merged
sql/Makefile.am:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/log.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
unittest/README.txt:
Auto merged
unittest/unit.pl:
Auto merged
mysql-test/t/disabled.def:
Merge (main -> rpl 5.1)
into production.mysql.com:/usersnfs/rburnett/mysql-5.1
libmysqld/libmysqld.def:
Auto merged
mysql-test/t/handler.test:
Auto merged
sql/mysqld.cc:
Auto merged
sql-common/client.c:
Auto merged
sql/sql_select.cc:
Auto merged
fix deadlock if master switches log file in parallell with "show master logs"
sql/log.cc:
add raw_get_current_log which dont take log mutex
sql/sql_class.h:
add raw_get_current_log which dont take log mutex
sql/sql_repl.cc:
Fix lock order i.e first log_lock then index_lock
sql/log.cc:
Auto merged
sql/sql_class.cc:
Auto merged
mysql-test/r/rpl_switch_stm_row_mixed.result:
Merge with mysql-5.1-new-rpl
mysql-test/t/rpl_switch_stm_row_mixed.test:
Merge with mysql-5.1-new-rpl
Fix to allow the binlog format to be changed between the LOCK and
UNLOCK tables, as well as under MIXED mode.
mysql-test/r/rpl_switch_stm_row_mixed.result:
Result change
mysql-test/t/rpl_switch_stm_row_mixed.test:
Adding test to see that binlog format can be changed when using
LOCK/UNLOCK TABLES both under ROW format and MIXED format.
sql/log.cc:
Removing pre-condition assertion since binlog can now be
statement based.
sql/sql_class.cc:
Adding code to always flush pending event regardless of the binlog
format used. The only exception is if we are inside a stored routine,
where the pending event is never flushed.
gets deadlocked when dropping w/ log on"
Log tables rely on concurrent insert machinery to add data.
This means that log tables are always opened and locked by
special (artificial) logger threads. Because of this, the thread
which tries to drop a log table starts to wait for the table
to be unlocked. Which will happen only if the log table is disabled.
Alike situation happens if one tries to alter a log table.
However in addition to the problem above, alter table calls
check_if_locking_is_allowed() routine for the engine. The
routine does not allow alter for the log tables. So, alter
doesn't start waiting forever for logs to be disabled, but
returns with an error.
Another problem is that not all engines could be used for
the log tables. That's because they need concurrent insert.
In this patch we:
(1) Explicitly disallow to drop/alter a log table if it
is currently used by the logger.
(2) Update MyISAM to support log tables
(3) Allow to drop log tables/alter log tables if log is
disabled
At the same time we (4) Disallow to alter log tables to
unsupported engine (after this patch CSV and MyISAM are
alowed)
Recommit with review fixes.
mysql-test/r/log_tables.result:
Update result file.
Note: there are warnings in result file. This is because of CSV
bug (Bug #21328). They should go away after it is fixed.
mysql-test/t/log_tables.test:
Add a test for the bug
sql/ha_myisam.cc:
Add log table handling to myisam: as log tables
use concurrent insert, they are typically
locked with TL_CONCURRERENT_INSERT lock. So,
disallow other threads to attempt locking of
the log tables in incompatible modes. Because
otherwise the threads will wait for the tables
to be unlocked forever.
sql/handler.cc:
Add a function to check if a table we're going to lock
is a log table and if the lock mode we want allowed
sql/handler.h:
Add a new function to check compatibility of the locking
sql/log.cc:
we shouldn't close the log table if and only
if this particular table is already closed
sql/log.h:
add new functions to check if a log is enabled
sql/share/errmsg.txt:
add new error messages
sql/sql_table.cc:
DROP and ALTER TABLE should not work on log
tables if the log tables are enabled
storage/csv/ha_tina.cc:
move function to check if the locking for the log
tables allowed to handler class, so that we can
reuse it in other engines.
storage/myisam/mi_extra.c:
add new ::extra() flag processing to myisam
storage/myisam/mi_open.c:
init log table flag
storage/myisam/mi_write.c:
update status after each write if it's a log table
storage/myisam/myisamdef.h:
Add new log table flag to myisam share.
We need it to distinguish between usual
and log tables, as for the log tables we
should provide concurrent insert in a
different way than for usual tables: we
want new rows to be immediately visible
to other threads.
Here i just disabled STDERR warnings in embedded server
Later we should get more defined about logs in the embedded server
sql/log.cc:
Log messages muted in embedded server
into dl145k.mysql.com:/data0/mkindahl/bk/MERGE/mysql-5.1-merge
include/my_sys.h:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/rpl_row_create_table.result:
Auto merged
mysql-test/t/disabled.def:
Auto merged
mysql-test/t/rpl_row_create_table.test:
Auto merged
mysys/my_malloc.c:
Auto merged
server-tools/instance-manager/parse.h:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/handler.h:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/set_var.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sp_head.h:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_update.cc:
Auto merged
sql/sql_view.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
mysys/safemalloc.c:
Merge of mysql-5.1-new-rpl into mysql-5.1
sql/ha_federated.cc:
d
Merge of mysql-5.1-new-rpl into mysql-5.1
sql/set_var.cc:
Merge of mysql-5.1-new-rpl with mysql-5.1
sql/slave.cc:
Merge of mysql-5.1-new-rpl into mysql-5.1
sql/sql_class.h:
Merge of mysql-5.1-new-rpl into mysql-5.1
We now reset the THD members related to auto_increment+binlog in
MYSQL_LOG::write(). This is better than in THD::cleanup_after_query(),
which was not able to distinguish between SELECT myfunc1(),myfunc2()
and INSERT INTO t SELECT myfunc1(),myfunc2() from a binlogging point
of view.
Rows_log_event::exec_event() now calls lex_start() instead of
mysql_init_query() because the latter now does too much (it resets
the binlog format).
mysql-test/extra/rpl_tests/rpl_insert_id.test:
fix after merge
mysql-test/mysql-test-run.pl:
-v does not bring useful information when running valgrind; I remove it;
if you think it's useful add it back.
mysql-test/r/binlog_stm_mix_innodb_myisam.result:
Position columns of SHOW BINLOG EVENTS are replaced by # (more robust
if the size of an event changes).
mysql-test/r/rpl_insert_id.result:
fix after merge
mysql-test/r/rpl_loaddata.result:
The binlog positions change, because one event disappeared; indeed there
was this in the binlog (in the current 5.1!):
SET INSERT_ID=2;
SET INSERT_ID=1;
SET TIMESTAMP=1152540671;
load data LOCAL INFILE '/tmp/SQL_LOAD_MB-1-2' INTO table t1;
Two INSERT_ID events, useless and a bug. Goes away afer cleaning up
auto_increment handling.
mysql-test/r/rpl_switch_stm_row_mixed.result:
INSERT_ID=5 appears, it's a consequence of having merged the fix
for BUG#20341
"stored function inserting into one auto_increment puts bad data in slave".
In mixed mode, if one substatement of a stored procedure requires row-based,
the entire procedure uses row-based (was already true for stored functions);
this is a consequence of not doing the resetting of binlog format inside
lock_tables() (which didn't work with how the slave thread executes
row-based binlog events).
mysql-test/t/rpl_switch_stm_row_mixed.test:
removing the multi-row delayed insert because in RBR the number of events
which it generates, is not repeatable (probably depends on how the delayed
thread groups rows, i.e. dependent on timing).
sql/ha_partition.cc:
update to new prototype
sql/ha_partition.h:
update to new prototype of the handler:: method.
sql/handler.cc:
after-merge fixes (manually merging part which was hard to merge in fmtool)
sql/log.cc:
When we write to the binary log, THD's parameters which influenced this
write are reset: stmt_depends_on_first_successful_insert_id_in_prev_stmt
and auto_inc_intervals_in_cur_stmt_for_binlog. This is so that future
writes are not influenced by those and can write their own values.
As a consequence, when we don't write to the binlog we do not reset.
This is to abide by the rule that in a complex statement (using triggers etc),
the first top- or substatement to generate auto_increment ids
wins their writing to the binlog (that writing may be done by the statement
itself or by the caller); so for example for
INSERT INTO t SELECT myfunc() where myfunc() inserts into auto_increment
and INSERT INTO t does not, myfunc() will fill
auto_inc_intervals_in_cur_stmt_for_binlog, which will not be reset when
myfunc() ends, then INSERT INTO t will write to the binlog and thus
write the preserved auto_inc_intervals_in_cur_stmt_for_binlog.
sql/log_event.cc:
mysql_init_query() does too much now to be called in Rows_log_event::exec_event
(it call mysql_reset_thd_for_next_command() which may switch
the binlog format now).
It's ok to call it in Table_map_log_event::exec_event() but its call must
be before setting the binlog format to "row".
sql/sql_base.cc:
Resetting the binlog format in lock_tables() was a bad idea of mine;
it causes problems in execution of row-based binlog events, where
the thread sets the binlog format by itself and does not want a next
lock_tables() to reset the binlog format.
It is also misleading, for a function named lock_tables(), to reset
the binlog format.
As a consequence of this change, in mixed binlogging mode, a routine
is logged either entirely statement-based or entirely row-based, we
don't switch in the middle (this was already true for prelocked routines,
now it's also true for stored procedures).
sql/sql_class.cc:
resetting of auto_increment variables used for binlogging is now done
when writing to the binary log, no need to do the resetting at the end
of the statement. It is also more correct this way; consider
SELECT myfunc1(),myfunc2();
where both functions insert into the same auto_increment column.
Binlogging is done in 2 events: "SELECT myfunc1()" and "SELECT myfunc2()".
So each of those needs to have, in binlog, the INSERT_ID which
it inserted. But as the 2 function calls are executed under prelocked mode,
the old code didn't reset auto_inc_intervals_in_cur_stmt_for_binlog
after the first SELECT was binlogged, and so the INSERT_ID of the first
SELECT was binlogged for the first SELECT and (wrong) also for the 2nd
SELECT event.
stmt_depends_on_first_... has the same logic.
sql/sql_class.h:
clearer comment
sql/sql_delete.cc:
unneeded #ifdef. As we temporarily change the binlog format to "statement"
before calling mysql_delete(), we must restore it afterwards.
sql/sql_insert.cc:
after-merge fixes.
No need to reset auto_inc_intervals_in_cur_stmt_for_binlog for every
row in the delayed insert system thread, because we already reset it
when writing to the binlog.
sql/sql_parse.cc:
unneeded #ifdef
this is a cleanup patch for our current auto_increment handling:
new names for auto_increment variables in THD, new methods to manipulate them
(see sql_class.h), some move into handler::, causing less backup/restore
work when executing substatements.
This makes the logic hopefully clearer, less work is is needed in
mysql_insert().
By cleaning up, using different variables for different purposes (instead
of one for 3 things...), we fix those bugs, which someone may want to fix
in 5.0 too:
BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate
statement-based"
BUG#20341 "stored function inserting into one auto_increment puts bad
data in slave"
BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE"
(now if a row is updated, LAST_INSERT_ID() will return its id)
and re-fixes:
BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT"
(already fixed differently by Ramil in 4.1)
Test of documented behaviour of mysql_insert_id() (there was no test).
The behaviour changes introduced are:
- LAST_INSERT_ID() now returns "the first autogenerated auto_increment value
successfully inserted", instead of "the first autogenerated auto_increment
value if any row was successfully inserted", see auto_increment.test.
Same for mysql_insert_id(), see mysql_client_test.c.
- LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY
UPDATE, see auto_increment.test. Same for mysql_insert_id(), see
mysql_client_test.c.
- LAST_INSERT_ID() does not change if no autogenerated value was successfully
inserted (it used to then be 0), see auto_increment.test.
- if in INSERT SELECT no autogenerated value was successfully inserted,
mysql_insert_id() now returns the id of the last inserted row (it already
did this for INSERT VALUES), see mysql_client_test.c.
- if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X
(it already did this for INSERT VALUES), see mysql_client_test.c.
- NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE,
the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID
influences not only the first row now.
Additionally, when unlocking a table we check that the thread is not keeping
a next_insert_id (as the table is unlocked that id is potentially out-of-date);
forgetting about this next_insert_id is done in a new
handler::ha_release_auto_increment().
Finally we prepare for engines capable of reserving finite-length intervals
of auto_increment values: we store such intervals in THD. The next step
(to be done by the replication team in 5.1) is to read those intervals from
THD and actually store them in the statement-based binary log. NDB
will be a good engine to test that.
mysql-test/extra/binlog_tests/binlog.test:
Testing that if INSERT_ID is set to a value too big for the
column's type, the binlogged INSERT_ID is the truncated value
(important if slave has a column of a "wider" numeric type).
Testing binlogging of INSERT_ID with INSERT DELAYED, to be sure that
we binlog an INSERT_ID event only for the delayed rows which use one.
mysql-test/extra/rpl_tests/rpl_insert_id.test:
Testcase for BUG#20339 "stored procedure using
LAST_INSERT_ID() does not replicate statement-based".
Testcase for BUG#20341 "stored function inserting into one
auto_increment puts bad data in slave".
mysql-test/extra/rpl_tests/rpl_loaddata.test:
Test that LOAD DATA INFILE sets a value for a future LAST_INSERT_ID().
mysql-test/r/auto_increment.result:
behaviour change: when INSERT totally fails (not even succeeds
partially and then rolls back), don't change last_insert_id().
Behaviour change: LAST_INSERT_ID() is now the first successfully inserted,
autogenerated, id.
Behaviour change: if INSERT ON DUPLICATE KEY UPDATE, if the table has auto_increment
and a row is updated, then LAST_INSERT_ID() returns the id of this row.
mysql-test/r/binlog_row_binlog.result:
result update
mysql-test/r/binlog_stm_binlog.result:
result update
mysql-test/r/insert.result:
result update
mysql-test/r/rpl_insert_id.result:
result update
mysql-test/r/rpl_loaddata.result:
result update
mysql-test/r/rpl_ndb_auto_inc.result:
ndb's behaviour is now like other engines wrt SET INSERT_ID
in a multi-row INSERT:
- with INSERT IGNORE: the id passed in SET INSERT_ID is re-used until
a row succeeds.
- generally, SET INSERT_ID sets the first value and other values are
simply computed from this first value, instead of previously where
the 2nd and subsequent values where not influenced by SET INSERT_ID;
this good change is due to the removal of "thd->next_insert_id=0"
from ha_ndbcluster.
mysql-test/t/auto_increment.test:
A testcase of BUG#19243: if ON DUPLICATE KEY UPDATE updates a row,
LAST_INSERT_ID() now returns the id of the row.
Test of new behaviour of last_insert_id() when no autogenerated value was
inserted, or when only some autogenerated value (not the first of them) was
inserted.
mysql-test/t/insert.test:
testing INSERT IGNORE re-using generated values
sql/ha_federated.cc:
update for new variables.
sql/ha_ndbcluster.cc:
handler::auto_increment_column_changed not needed, equivalent to
(insert_id_for_cur_row > 0).
thd->next_insert_id=0 not needed anymore; it was used to force
handler::update_auto_increment() to call ha_ndbcluster::get_auto_increment()
for each row of a multi-row INSERT, now this happens naturally
because NDB says "I have reserved you *one* value" in get_auto_increment(),
so handler::update_auto_increment() calls again for next row.
sql/handler.cc:
More comments, use of new methods and variables. Hopes to be clearer
than current code.
thd->prev_insert_id not in THD anymore: it is managed locally by inserters
(like mysql_insert()).
THD::clear_next_insert_id is now equivalent to
handler::next_insert_id > 0.
get_auto_increment() reserves an interval of values from the engine,
uses this interval for next rows of the statement, until interval
is exhausted then it asks for another interval (of a bigger size
than the first one; size doubles until reaching 65535 then it stays constant).
If doing statement-based binlogging, intervals are remembered in a list
for storage in the binlog.
For "forced" insert_id values (SET INSERT_ID or replication slave),
forced_auto_inc_intervals is non-empty and the handler takes its intervals
from there, without calling get_auto_increment().
ha_release_auto_increment() resets the handler's auto_increment variables;
it calls release_auto_increment() which is handler-dependent and
serves to return to the engine any unused tail of the last used
interval.
If ending a statement, next_insert_id>0 means that autoinc values have been
generated or taken from the master's binlog (in a replication slave) so
we clear those values read from binlog, so that next top- or sub-
statement does not use them.
sql/handler.h:
handler::auto_increment_changed can be replaced by
(handler::insert_id_for_cur_row > 0).
THD::next_insert_id moves into handler (more natural, and prepares
for the day when we'll support a single statement inserting into
two tables - "multi-table INSERT" like we have UPDATE - will this
happen?).
This move makes the backup/restore of THD::next_insert_id when entering
a substatement unneeded, as each substatement has its own handler
objects.
sql/item_func.cc:
new names for variables.
For the setting of what mysql_insert_id() will return to the client,
LAST_INSERT_ID(X) used to simply pretend that the generated autoinc
value for the current row was X, but this led to having no reliable
way to know the really generated value, so we now have a bool:
thd->arg_of_last_insert_id_function which enables us to know that
LAST_INSERT_ID(X) was called (and then X can be found in
thd->first_successful_insert_id_in_prev_stmt).
sql/log.cc:
new variable names for insert_ids. Removing some unused variables in the slow
log.
sql/log_event.cc:
new variable names, comments. Preparing for when master's won't binlog
LAST_INSERT_ID if it was 0.
sql/set_var.cc:
new variable names.
The last change repeats how Bar fixed BUG#20392
"INSERT_ID session variable has weird value" in 5.0.
sql/sql_class.cc:
new variables for insert_id. In THD::cleanup_after_query() we fix
BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate
statement-based" (will one want to fix it in 5.0?). Many comments
about what stored functions do to auto_increment.
In reset|restore_sub_statement_state(), we need to backup less
auto_inc variables as some of them have moved to the handler;
we backup/restore those which are about the current top- or sub-
statement, *not* those about the statement-based binlog
(which evolve as the top- and sub-statement execute).
Because we split THD::last_insert_id into
THD::first_successful_insert_id_in_prev_stmt and
THD::auto_inc_intervals_for_binlog (among others), we fix
BUG#20341 "stored function inserting into one auto_increment
puts bad data in slave": indeed we can afford to not backup/restore
THD::auto_inc_intervals_for_binlog (which fixes the bug) while still
backing up / restoring THD::first_successful_insert_id_in_prev_stmt
(ensuring that the top-level LAST_INSERT_ID() is not affected by INSERTs
done by sub-statements, as is desirable and tested in rpl_insert_id.test).
sql/sql_class.h:
new variables and methods for auto_increment.
Some THD members move into handler (those which are really about
the table being inserted), some stay in THD (those which are
about what a future LAST_INSERT_ID() should return, or about
what should be stored into the statement-based binlog).
THD::next_insert_id moves to handler::.
THD::clear_next_insert_id removed (had become equivalent
to next_insert_id > 0).
THD::last_insert_id becomes four:
THD::first_successful_insert_id_in_cur_stmt,
THD::auto_inc_intervals_for_binlog,
handler::insert_id_for_cur_row,
THD::first_successful_insert_id_in_prev_stmt.
THD::current_insert_id becomes:
THD::first_successful_insert_id_in_prev_stmt_for_binlog
THD::prev_insert_id is removed, handler can just use
handler::insert_id_for_cur_row instead (which is more accurate:
for the first row, prev_insert_id was set before get_auto_increment
was called, so was 0, causing a call to
get_auto_increment() for the 2nd row if the 1st row fails;
here we don't need the call as insert_id_for_cur_row has
the value of the first row).
THD::last_insert_id_used becomes: stmt_depends_on_first_row_in_prev_stmt
THD::insert_id_used is removed (equivalent to
auto_inc_intervals_for_binlog non empty).
The interval returned by get_auto_increment() and currently being
consumed is handler::auto_inc_interval_for_cur_row.
Comments to explain each of them.
select_insert::last_insert_id becomes autoinc_value_of_last_inserted_row.
sql/sql_insert.cc:
the "id" variable is not changed for each row now; it used to compensate for
this contradiction:
- thd->last_insert_id supposed job was to keep the id of the first row
- but it was updated for every row
- so mysql_insert() made sure to catch its first value and restore it at the end of stmt.
Now THD keeps the first value in first_successful_insert_id_in_cur_stmt,
and value of the row in insert_id_for_cur_row. So "id" only serves to fill
mysql_insert_id(), as depending on some conditions, "id" must be different
values.
Prev_insert_id moves from THD to write_record().
We now set LAST_INSERT_ID() in ON DUPLICATE KEY UPDATE too (BUG#19243).
In an INSERT DELAYED, we still "reset auto-increment caching" but differently
(by calling ha_release_auto_increment()).
sql/sql_load.cc:
no need to fiddle with "id", THD maintains
THD::first_successful_insert_id_in_cur_stmt by itself and correctly now.
ha_release_auto_increment() is now (logically) called before we unlock
the table.
sql/sql_parse.cc:
update to new variable names.
Assertion that reset_thd_for_next_command() is not called for every
substatement of a routine (I'm not against it, but if we do this change,
statement-based binlogging needs some adjustments).
sql/sql_select.cc:
update for new variable names
sql/sql_table.cc:
next_insert_id not needed in mysql_alter_table(), THD manages.
sql/sql_update.cc:
update for new variable names.
Even though this is UPDATE, an insert id can be generated (by
LAST_INSERT_ID(X)) and should be recorded because mysql_insert_id() wants
to know about it.
sql/structs.h:
A class for "discrete" intervals (intervals of integer numbers with a certain
increment between them): Discrete_interval, and a class for a list of such
intervals: Discrete_intervals_list
tests/mysql_client_test.c:
tests of behaviour of mysql_insert_id(): there were no such tests, while in
our manual we document its behaviour. In comments you'll notice the behaviour
changes introduced (there are 5).
into zippy.(none):/home/cmiller/work/mysql/merge/mysql-5.1
BitKeeper/etc/ignore:
auto-union
client/mysqldump.c:
Auto merged
mysql-test/r/bdb.result:
Auto merged
mysql-test/r/type_timestamp.result:
Auto merged
mysql-test/r/udf.result:
Auto merged
mysql-test/t/bdb.test:
Auto merged
mysql-test/t/type_timestamp.test:
Auto merged
mysys/my_lib.c:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/table.cc:
Auto merged
mysql-test/r/func_time.result:
manual merge
mysql-test/r/mysqldump.result:
manual merge
mysql-test/t/func_time.test:
manual merge
mysql-test/t/mysqldump.test:
manual merge
sql/log.cc:
manual merge
sql/sp_head.cc:
manual merge
sql/sql_table.cc:
manual merge
into mysql.com:/users/lthalmann/bk/mysql-5.1-new-rpl
BitKeeper/deleted/.del-mysys.vcproj~40a49d09c4184822:
Auto merged
mysql-test/t/rpl_openssl.test:
Auto merged
sql/log.cc:
Auto merged
BitKeeper/deleted/.del-mysql.sln~76a9ff1e793b3547:
Deleted file in 5.1 (removed)
BitKeeper/deleted/.del-mysqld.vcproj~6aa7b3f9c3e28fcb:
Deleted file in 5.1 (removed)
include/my_sys.h:
Manual merge
mysql-test/mysql-test-run.pl:
Manual merge
and BUG#19208 "Test 'rpl000017' hangs on Windows".
Both bugs are caused by attempting to delete an opened
file and to create immediatedly a new one with the same
name. On Windows it can be supported only on NT-platforms
(by using FILE_SHARE_DELETE mode and with renaming the
file before deletion). Because deleting not-closed files
is not supported on all platforms (e.g. Win 98|ME) this
is to be considered harmful and should be eliminated by
a "code redesign".
VC++Files/mysys/mysys.vcproj:
To be sure that __NT__ is defined for Win configurations.
Temporary, to be changed in more appropriate way.
include/my_sys.h:
Adding my_delete_allow_opened to be invoked to delete
a (possibly) not closed file on Windows NT-platforms.
mysys/my_delete.c:
Adding nt_share_delete() function implementing
a (possibly) not closed file deletion on Windows NT.
sql/log.cc:
MYSQL_LOG::reset_logs(): Deleting usually not
closed binlog files.
into bk-internal.mysql.com:/data0/bk/mysql-5.1-kt
mysql-test/r/show_check.result:
Auto merged
sql/log.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/set_var.h:
Auto merged
sql/sql_delete.cc:
Auto merged
mysql-test/r/partition_error.result:
result fix
sql/log.cc:
use open_slow_log() instead of open_query_log() for slow query log
sql/log.h:
after merge fix
sql/set_var.cc:
after merge fix
use open_slow_log() instead of open_query_log() for slow query log
into bk-internal.mysql.com:/data0/bk/mysql-5.1-kt
mysql-test/r/join.result:
Auto merged
mysql-test/t/join.test:
Auto merged
sql/log.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_show.cc:
Auto merged
into mysql.com:/users/lthalmann/bk/MERGE/mysql-5.1-merge
mysql-test/t/archive.test:
Auto merged
sql/log.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_table.cc:
Auto merged
