Commit graph

100 commits

Author SHA1 Message Date
Andrei Elkin
a7127418a7 Bug#11763573 - 56299: MUTEX DEADLOCK WITH COM_BINLOG_DUMP, BINLOG PURGE, AND PROCESSLIST/KILL
The bug case is similar to one fixed earlier bug_49536.
Deadlock involving LOCK_log appears to be possible because the purge running thread
is holding LOCK_log whereas there is no sense of doing that and which fact was
exploited by the earlier bug fixes.

Fixed with small reengineering of rotate_and_purge(), adding two new methods and
setting up a policy to execute those instead of the former
rotate_and_purge(RP_LOCK_LOG_IS_ALREADY_LOCKED).
The policy for using rotate(), purge() is that if the caller acquires LOCK_log itself,
it should call rotate(), release the mutex and run purge().

Side effect of this patch is refining error message of bug@11747416 to print
the whole path.
2011-10-27 17:14:41 +03:00
Guilhem Bichot
27fa7876c8 merge from latest 5.5 2011-05-21 10:59:32 +02:00
Guilhem Bichot
12f651ac9d Merge from 5.1. 2011-05-21 10:21:08 +02:00
Mikael Ronström
7b079a3a7e Merge 5.5 2011-04-15 15:46:11 +02:00
Marc Alff
f0577e551e Bug#11766528 PERFORMANCE_SCHEMA TRACKS BOTH BINARY AND RELAY LOGS IN THE SAME EVENTS
Before this fix, all the performance schema instrumentation for both the binary log
and the relay log would use the following instruments:
- wait/io/file/sql/binlog
- wait/io/file/sql/binlog_index
- wait/synch/mutex/sql/MYSQL_BIN_LOG::LOCK_index
- wait/synch/cond/sql/MYSQL_BIN_LOG::update_cond

This instrumentation is too general and can be more specific.

With this fix, the binlog instrumentation is identical,
and the relay log instrumentation is changed to:
- wait/io/file/sql/relaylog
- wait/io/file/sql/relaylog_index
- wait/synch/mutex/sql/MYSQL_RELAY_LOG::LOCK_index
- wait/synch/cond/sql/MYSQL_RELAY_LOG::update_cond

With this change, the performance instrumentation for the binary log and the relay log,
which share the same structure but have different uses, is more detailed.
This is especially important for hosts in the middle of a replication chain,
that are both masters (binlog) and slaves (relaylog).
2011-03-01 17:39:28 +01:00
Mikael Ronstrom
62c99943b1 BUG#59549, fix compiler error on Windows 2011-01-17 13:25:31 +01:00
Luis Soares
5d6e142b2b BUG#46166
Manual merge from mysql-5.1-bugteam into mysql-5.5-bugteam.

Conflicts
=========

Text conflict in sql/log.cc
Text conflict in sql/log.h
Text conflict in sql/slave.cc
Text conflict in sql/sql_parse.cc
Text conflict in sql/sql_priv.h
2010-12-07 16:11:13 +00:00
Alfranio Correia
f077bda606 merge mysql-5.5-bugteam(local) --> mysql-5.5-bugteam 2010-12-03 01:06:56 +00:00
Luis Soares
aaefb52df8 BUG#46166: MYSQL_BIN_LOG::new_file_impl is not propagating error
when generating new name.
      
If find_uniq_filename returns an error, then this error is not
being propagated upwards, and execution does not report error to
the user (although a entry in the error log is generated).
                  
Additionally, some more errors were ignored in new_file_impl:
- when writing the rotate event
- when reopening the index and binary log file
                  
This patch addresses this by propagating the error up in the
execution stack. Furthermore, when rotation of the binary log
fails, an incident event is written, because there may be a
chance that some changes for a given statement, were not properly
logged. For example, in SBR, LOAD DATA INFILE statement requires
more than one event to be logged, should rotation fail while
logging part of the LOAD DATA events, then the logged data would
become inconsistent with the data in the storage engine.
2010-11-30 23:32:51 +00:00
Marc Alff
80589ada50 Bug#58052 Binary log IO not being accounted for properly
Before this fix, file io for the binary log file was not accounted properly,
and showed no io at all.

This bug was due to the following issues:

1) file io for the binlog was instrumented:
- sometime as "wait/io/file/sql/binlog"
- sometime as "wait/io/file/sql/MYSQL_LOG"
leading to inconsistent event_names.

2) the binlog file itself was using an IO_CACHE,
but the IO_CACHE implementation in mysys/mf_iocache.c was
not instrumented to make performance schema calls to record file io.

3) The "wait/io/file/sql/MYSQL_LOG" instrumentation was used
for several log files, such as:
- the binary log
- the slow log
- the query log
which caused file io in these different log files to be accounted
against the same instrument.
The instrumentation needs to have a finer grain and report io
in different event_names, because each file really serves a different purpose.

With this fix:
- the IO_CACHE implementation is now instrumented
- the "wait/io/file/sql/MYSQL_LOG" instrument has been removed
- binlog io is now always instrumented with "wait/io/file/sql/binlog"
- the slow log is instrumented with a new name, "wait/io/file/sql/slow_log"
- the query log is instrumented with a new name, "wait/io/file/sql/query_log"
2010-11-12 07:23:26 +01:00
Alfranio Correia
bf2c66d4a7 BUG#57275 binlog_cache_size affects trx- and stmt-cache and gets twice the expected memory
After the WL#2687, the binlog_cache_size and max_binlog_cache_size affect both the
stmt-cache and the trx-cache. This means that the resource used is twice the amount
expected/defined by the user.
      
The binlog_cache_use is incremented when the stmt-cache or the trx-cache is used
and binlog_cache_disk_use is incremented when the disk space from the stmt-cache or the
trx-cache is used. This behavior does not allow to distinguish which cache may be harming
performance due to the extra disk accesses and needs to have its in-memory cache
increased.
      
To fix the problem, we introduced two new options and status variables related to the
stmt-cache:
      
  Options:
      
    . binlog_stmt_cache_size
    . max_binlog_stmt_cache_size
      
    Status Variables:
      
    . binlog_stmt_cache_use
    . binlog_stmt_cache_disk_use

So there are

  . binlog_cache_size that defines the size of the transactional cache for
  updates to transactional engines for the binary log.

  . binlog_stmt_cache_size that defines the size of the statement cache for
  updates to non-transactional engines for the binary log.

  . max_binlog_cache_size that sets the total size of the transactional
  cache.

  . max_binlog_stmt_cache_size that sets the total size of the statement
  cache.

  . binlog_cache_use that identifies the number of transactions that used the
  temporary transactional binary log cache.

  . binlog_cache_disk_use that identifies the number of transactions that used
  the temporary transactional binary log cache but that exceeded the value of
  binlog_cache_size.

  . binlog_stmt_cache_use that identifies the number of statements that used the
  temporary non-transactional binary log cache.

  . binlog_stmt_cache_disk_use that identifies the number of statements that used
  the temporary non-transactional binary log cache but that exceeded the value of
  binlog_stmt_cache_size.
2010-11-05 17:42:37 +00:00
Georgi Kodinov
216418e7b2 merge 2010-09-29 17:26:32 +03:00
Alfranio Correia
95116114a4 BUG#55415 wait_for_update_bin_log enters a condition but does not leave
In sql/log.c, member function wait_for_update_bin_log, a condition is entered with
THD::enter_cond but is not exited. This might leave dangling references to the
mutex/condition in the per-thread information area.
      
To fix the problem, we call exit_cond to properly remove references to the mutex,
LOCK_log.
2010-09-06 18:18:44 +01:00
Georgi Kodinov
bd62942ec1 merge 2010-08-20 16:10:49 +03:00
Georgi Kodinov
a54c3dd5c0 merge 2010-08-20 14:22:46 +03:00
Alfranio Correia
1feee134fd BUG#55625 RBR breaks on failing 'CREATE TABLE'
A CREATE...SELECT that fails is written to the binary log if a non-transactional
statement is updated. If the logging format is ROW, the CREATE statement and the
changes are written to the binary log as distinct events and by consequence the
created table is not rolled back in the slave.

In this patch, we opted to let the slave goes out of sync by not writting to the
binary log the CREATE statement. We do this by simply reseting the binary log's
cache.
2010-08-02 20:48:56 +01:00
Davi Arnaut
dd14fa18af WL#5498: Remove dead and unused source code
Remove unused variables.
2010-07-23 17:13:36 -03:00
Alfranio Correia
e662b51eef BUG#53259 Unsafe statement binlogged in statement format w/MyIsam temp tables
BUG#54872 MBR: replication failure caused by using tmp table inside transaction 
      
Changed criteria to classify a statement as unsafe in order to reduce the
number of spurious warnings. So a statement is classified as unsafe when
there is on-going transaction at any point of the execution if:

1. The mixed statement is about to update a transactional table and
a non-transactional table.

2. The mixed statement is about to update a temporary transactional
table and a non-transactional table.
      
3. The mixed statement is about to update a transactional table and
read from a non-transactional table.

4. The mixed statement is about to update a temporary transactional
table and read from a non-transactional table.

5. The mixed statement is about to update a non-transactional table
and read from a transactional table when the isolation level is
lower than repeatable read.

After updating a transactional table if:

6. The mixed statement is about to update a non-transactional table
and read from a temporary transactional table.
 
7. The mixed statement is about to update a non-transactional table
 and read from a temporary transactional table.

8. The mixed statement is about to update a non-transactionala table
   and read from a temporary non-transactional table.
     
9. The mixed statement is about to update a temporary non-transactional
table and update a non-transactional table.
     
10. The mixed statement is about to update a temporary non-transactional
table and read from a non-transactional table.
     
11. A statement is about to update a non-transactional table and the
option variables.binlog_direct_non_trans_update is OFF.

The reason for this is that locks acquired may not protected a concurrent
transaction of interfering in the current execution and by consequence in
the result. So the patch reduced the number of spurious unsafe warnings.

Besides we fixed a regression caused by BUG#51894, which makes temporary
tables to go into the trx-cache if there is an on-going transaction. In
MIXED mode, the patch for BUG#51894 ignores that the trx-cache may have
updates to temporary non-transactional tables that must be written to the
binary log while rolling back the transaction.
      
So we fix this problem by writing the content of the trx-cache to the
binary log while rolling back a transaction if a non-transactional
temporary table was updated and the binary logging format is MIXED.
2010-06-30 16:25:13 +01:00
Jon Olav Hauglid
1af11051bf manual merge from mysql-trunk-bugfixing
Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h
2010-06-06 13:19:29 +02:00
Davi Arnaut
a3d61b0bb2 Fix type mismatch. Table names are represented as LEX_STRING
objects whose length is stored in a size_t type.
2010-05-27 18:28:24 -03:00
Alfranio Correia
89850be0f5 BUG#53560 CREATE TEMP./DROP TEMP. are not binglogged correctly after a failed statement
This patch fixes two problems described as follows:

1 - If there is an on-going transaction and a temporary table is created or
dropped, any failed statement that follows the "create" or "drop commands"
triggers a rollback and by consequence the slave will go out sync because
the binary log will have a wrong sequence of events.

To fix the problem, we changed the expression that evaluates when the
cache should be flushed after either the rollback of a statment or
transaction.

2 - When a "CREATE TEMPORARY TABLE SELECT * FROM" was executed the
OPTION_KEEP_LOG was not set into the thd->options. For that reason, if
the transaction had updated only transactional engines and was rolled
back at the end (.e.g due to a deadlock) the changes were not written
to the binary log, including the creation of the temporary table.
      
To fix the problem, we have set the OPTION_KEEP_LOG into the thd->options
when a "CREATE TEMPORARY TABLE SELECT * FROM" is executed.
2010-05-19 18:01:12 +01:00
Alexey Kopytov
e47ab5a604 Manual merge from mysql-5.1-bugteam to mysql-trunk-merge.
Text conflict in sql/log.cc
Text conflict in sql/log.h
Text conflict in sql/sql_class.cc
Text conflict in sql/sql_load.cc
2010-05-11 12:23:57 +04:00
He Zhenxing
455f754e6b Bug#53189 Table map version unused and can be removed
MYSQL_BIN_LOG m_table_map_version member and it's associated 
functions were not used in the logic of binlogging and replication,
this patch removed all related code.
2010-05-08 11:25:33 +08:00
Alfranio Correia
9ba731c299 BUG#51894 Replication failure with SBR on DROP TEMPORARY TABLE inside a
transaction
BUG#52616 Temp table prevents switch binlog format from STATEMENT to ROW

Before the WL#2687 and BUG#46364, every non-transactional change that happened
after a transactional change was written to trx-cache and flushed upon
committing the transaction. WL#2687 and BUG#46364 changed this behavior and
non-transactional changes are now written to the binary log upon committing
the statement.

A binary log event is identified as transactional or non-transactional through
a flag in the Log_event which is set taking into account the underlie storage
engine on what it is stems from. In the current bug, this flag was not being
set properly when the DROP TEMPORARY TABLE was executed.

However, while fixing this bug we figured out that changes to temporary tables
should be always written to the trx-cache if there is an on-going transaction.
Otherwise, binlog events in the reversed order would be produced.

Regarding concurrency, keeping changes to temporary tables in the trx-cache is
also safe as temporary tables are only visible to the owner connection.

In this patch, we classify the following statements as unsafe:
   1 - INSERT INTO t_myisam SELECT * FROM t_myisam_temp

   2 - INSERT INTO t_myisam_temp SELECT * FROM t_myisam

   3 - CREATE TEMPORARY TABLE t_myisam_temp SELECT * FROM t_myisam

On the other hand, the following statements are classified as safe:

   1 - INSERT INTO t_innodb SELECT * FROM t_myisam_temp

   2 - INSERT INTO t_myisam_temp SELECT * FROM t_innodb

The patch also guarantees that transactions that have a DROP TEMPORARY are
always written to the binary log regardless of the mode and the outcome:
commit or rollback. In particular, the DROP TEMPORARY is extended with the
IF EXISTS clause when the current statement logging format is set to row.

Finally, the patch allows to switch from STATEMENT to MIXED/ROW when there
are temporary tables but the contrary is not possible.
2010-04-20 10:10:43 +01:00
Mats Kindahl
e409d6f69c WL#5030: Split and remove mysql_priv.h
This patch:

- Moves all definitions from the mysql_priv.h file into
  header files for the component where the variable is
  defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
2010-03-31 16:05:33 +02:00
Alexander Nozdrin
33e5e125f5 Manual merge of patch for Bug#46364 from mysql-next-mr-bugfixing.
Conflicts:
  - mysql-test/r/mysqld--help-win.result
  - sql/sys_vars.cc

Original revsion (in next-mr-bugfixing):
------------------------------------------------------------
revno: 2971 [merge]
revision-id: alfranio.correia@sun.com-20100121210527-rbuheu5rnsmcakh1
committer: Alfranio Correia <alfranio.correia@sun.com>
branch nick: mysql-next-mr-bugfixing
timestamp: Thu 2010-01-21 21:05:27 +0000
message:
  BUG#46364 MyISAM transbuffer problems (NTM problem)
        
  It is well-known that due to concurrency issues, a slave can become
  inconsistent when a transaction contains updates to both transaction and
  non-transactional tables.
                      
  In a nutshell, the current code-base tries to preserve causality among the
  statements by writing non-transactional statements to the txn-cache which
  is flushed upon commit. However, modifications done to non-transactional
  tables on behalf of a transaction become immediately visible to other
  connections but may not immediately get into the binary log and therefore
  consistency may be broken.
              
  In general, it is impossible to automatically detect causality/dependency
  among statements by just analyzing the statements sent to the server. This
  happen because dependency may be hidden in the application code and it is
  necessary to know a priori all the statements processed in the context of
  a transaction such as in a procedure. Moreover, even for the few cases that
  we could automatically address in the server, the computation effort
  required could make the approach infeasible.
              
  So, in this patch we introduce the option
        - "--binlog-direct-non-transactional-updates" that can be used to bypass
        the current behavior in order to write directly to binary log statements
        that change non-transactional tables.
  
  Besides, it is used to enable the WL#2687 which is disabled by default.
    ------------------------------------------------------------
    revno: 2970.1.1
    revision-id: alfranio.correia@sun.com-20100121131034-183r4qdyld7an5a0
    parent: alik@sun.com-20100121083914-r9rz2myto3tkdya0
    committer: Alfranio Correia <alfranio.correia@sun.com>
    branch nick: mysql-next-mr-bugfixing
    timestamp: Thu 2010-01-21 13:10:34 +0000
    message:
      BUG#46364 MyISAM transbuffer problems (NTM problem)
            
      It is well-known that due to concurrency issues, a slave can become
      inconsistent when a transaction contains updates to both transaction and
      non-transactional tables.
                          
      In a nutshell, the current code-base tries to preserve causality among the
      statements by writing non-transactional statements to the txn-cache which
      is flushed upon commit. However, modifications done to non-transactional
      tables on behalf of a transaction become immediately visible to other
      connections but may not immediately get into the binary log and therefore
      consistency may be broken.
                  
      In general, it is impossible to automatically detect causality/dependency
      among statements by just analyzing the statements sent to the server. This
      happen because dependency may be hidden in the application code and it is
      necessary to know a priori all the statements processed in the context of
      a transaction such as in a procedure. Moreover, even for the few cases that
      we could automatically address in the server, the computation effort
      required could make the approach infeasible.
                  
      So, in this patch we introduce the option
            - "--binlog-direct-non-transactional-updates" that can be used to bypass
            the current behavior in order to write directly to binary log statements
            that change non-transactional tables.
      
      Besides, it is used to enable the WL#2687 which is disabled by default.
2010-02-02 10:56:42 +03:00
Alfranio Correia
46d1689b7c merge mysql-next-mr --> mysql-5.1-rpl-merge
Conflicts:
  Text conflict in sql/log.cc
  Text conflict in sql/slave.cc
  Text conflict in sql/sql_base.cc
2010-01-13 12:22:34 +00:00
b805e3d4d6 Manual merge from next-mr. 2010-01-12 20:07:09 +08:00
Alfranio Correia
6f7929b143 merge mysql-5.1-rep+2-delivery1 --> mysql-5.1-rpl-merge
Conflicts:

Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/extra/rpl_tests/rpl_loaddata.test
Text conflict in mysql-test/r/mysqlbinlog2.result
Text conflict in mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
Text conflict in mysql-test/suite/binlog/r/binlog_unsafe.result
Text conflict in mysql-test/suite/rpl/r/rpl_insert_id.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result
Text conflict in mysql-test/suite/rpl/r/rpl_udf.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Text conflict in sql/field.h
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysql_priv.h
Text conflict in sql/share/errmsg.txt
Text conflict in sql/sp.cc
Text conflict in sql/sql_acl.cc
Text conflict in sql/sql_base.cc
Text conflict in sql/sql_class.h
Text conflict in sql/sql_db.cc
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_insert.cc
Text conflict in sql/sql_lex.cc
Text conflict in sql/sql_lex.h
Text conflict in sql/sql_load.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_update.cc
Text conflict in sql/sql_view.cc
Conflict adding files to storage/innobase.  Created directory.
Conflict because storage/innobase is not versioned, but has versioned children.  Versioned directory.
Conflict adding file storage/innobase.  Moved existing file to storage/innobase.moved.
Conflict adding files to storage/innobase/handler.  Created directory.
Conflict because storage/innobase/handler is not versioned, but has versioned children.  Versioned directory.
Contents conflict in storage/innobase/handler/ha_innodb.cc
2010-01-07 15:39:11 +00:00
Marc Alff
a4c3bc618b WL#2360 Performance schema
Part IV: sql instrumentation
2010-01-06 22:42:07 -07:00
Sergei Golubchik
ae2768ce9c WL#4738 streamline/simplify @@variable creation process
Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables
Bug#20415 Output of mysqld --help --verbose is incomplete
Bug#25430 variable not found in SELECT @@global.ft_max_word_len;
Bug#32902 plugin variables don't know their names
Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting!
Bug#34829 No default value for variable and setting default does not raise error
Bug#34834 ? Is accepted as a valid sql mode
Bug#34878 Few variables have default value according to documentation but error occurs  
Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var.
Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status
Bug#40988 log_output_basic.test succeeded though syntactically false.
Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails)
Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations 
Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled
Bug#44797 plugins w/o command-line options have no disabling option in --help
Bug#46314 string system variables don't support expressions
Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken
Bug#46586 When using the plugin interface the type "set" for options caused a crash.
Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number
Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds
Bug#49417 some complaints about mysqld --help --verbose output
Bug#49540 DEFAULT value of binlog_format isn't the default value
Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix)
Bug#49644 init_connect and \0
Bug#49645 init_slave and multi-byte characters
Bug#49646 mysql --show-warnings crashes when server dies
2009-12-22 10:35:56 +01:00
Alexander Nozdrin
09942fe8e8 Manual merge from mysql-trunk-merge.
Conflicts:
  - include/my_no_pthread.h
  - mysql-test/r/sp-ucs2.result
  - sql/log.cc
  - sql/sql_acl.cc
  - sql/sql_yacc.yy
2009-12-16 21:02:21 +03:00
Alfranio Correia
75f382ea14 merge mysql-5.1-rep+2 --> mysql-5.1-rep+3 2009-12-14 10:40:42 +00:00
Alexey Kopytov
72b2943594 Manual merge of mysql-5.1-bugteam into mysql-trunk-merge. 2009-12-11 19:40:58 +03:00
Alfranio Correia
e362e9a71b BUG#45292 orphan binary log created when starting server twice
This patch fixes three bugs as follows. First, aborting the server while purging
binary logs might generate orphan files due to how the purge operation was
implemented:

          (purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)

      1 - register the files to be removed in a temporary buffer.
      2 - update the log-bin.index.
      3 - flush the log-bin.index.
      4 - erase the files whose names where register in the temporary buffer
      in step 1.

Thus a failure while  executing step 4 would generate an orphan file. Second,
a similar issue might happen while creating a new binary as follows:

          (create routine - sql/log.cc - MYSQL_BIN_LOG::open)

      1 - open the new log-bin.
      2 - update the log-bin.index.

Thus a failure while executing step 1 would generate an orphan file.

To fix these issues, we record the files to be purged or created before really
removing or adding them. So if a failure happens such records can be used to
automatically remove dangling files. The new steps might be outlined as follows:

          (purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)

      1 - register the files to be removed in the log-bin.~rec~ placed in
      the data directory.
      2 - update the log-bin.index.
      3 - flush the log-bin.index.
      4 - delete the log-bin.~rec~.

          (create routine - sql/log.cc - MYSQL_BIN_LOG::open)

      1 - register the file to be created in the log-bin.~rec~
      placed in the data directory.
      2 - open the new log-bin.
      3 - update the log-bin.index.
      4 - delete the log-bin.~rec~.

          (recovery routine - sql/log.cc - MYSQL_BIN_LOG::open_index_file)

      1 - open the log-bin.index.
      2 - open the log-bin.~rec~.
      3 - for each file in log-bin.~rec~.
        3.1 Check if the file is in the log-bin.index and if so ignore it.
        3.2 Otherwise, delete it.

The third issue can be described as follows. The purge operation was allowing
to remove a file in use thus leading to the loss of data and possible
inconsistencies between the master and slave. Roughly, the routine was only
taking into account the dump threads and so if a slave was not connect the
file might be delete even though it was in use.
2009-12-04 14:40:42 +00:00
34ab4521fb WL#5142 FLUSH LOGS should take optional arguments for which log(s) to flush
Support for flushing individual logs, so that the user can
selectively flush a subset of the server logs.

Flush of individual logs is done according to the 
following syntax:

  FLUSH <log_category> LOGS;

The syntax is extended so that the user is able to flush a
subset of logs:

  FLUSH [log_category LOGS,];

where log_category is one of:
  SLOW
  ERROR
  BINARY
  ENGINE
  GENERAL
  RELAY.
2009-12-03 16:59:58 +08:00
Andrei Elkin
070d190906 mergin 5.1 -> rep+2 -> rep+3. create_table_from_dump issue will be merged on the next step 2009-11-30 20:20:26 +02:00
Luis Soares
1d00d426d9 BUG#40611: MySQL cannot make a binary log after sequential number
beyond unsigned long.
BUG#44779: binlog.binlog_max_extension may be causing failure on 
next test in PB
            
NOTE1: this is the backport to next-mr.
NOTE2: already includes patch for BUG#44779.
            
Binlog file extensions would turn into negative numbers once the
variable used to hold the value reached maximum for signed
long. Consequently, incrementing value to the next (negative) number
would lead to .000000 extension, causing the server to fail.
                        
This patch addresses this issue by not allowing negative extensions
and by returning an error on find_uniq_filename, when the limit is
reached. Additionally, warnings are printed to the error log when the
limit is approaching. FLUSH LOGS will also report warnings to the
user, if the extension number has reached the limit. The limit has been
set to 0x7FFFFFFF as the maximum.
2009-11-22 03:59:48 +00:00
Andrei Elkin
3a76c32c05 Bug #48463 backporting from 6.0-rpl to celosia a set of bugs
The mentioned on the bug report set of bugs fixes have not be pushed to the main trees.

Fixed with extracting commits done to 6.0-rpl tree and applying them to the main 5.1.
Notes.
1. part of changes - the mtr's specific - were packported to the main 5.0 tree for mtr v1
   as http://lists.mysql.com/commits/46562
   However, there is no that fix anymore in the mtr v2. (This fact was mailed to mtr maintaining
   people).

2. Bug@36929  crash in kill_zombie_dump_threads-> THD::awake() with replication tests
   is not backported because the base code of the patch is libevent and that was removed
   from the main trees due to its instability.
2009-11-20 15:30:35 +02:00
Alfranio Correia
60d4662496 WL#2687 WL#5072 BUG#40278 BUG#47175
Non-transactional updates that take place inside a transaction present problems
for logging because they are visible to other clients before the transaction
is committed, and they are not rolled back even if the transaction is rolled
back. It is not always possible to log correctly in statement format when both
transactional and non-transactional tables are used in the same transaction.

In the current patch, we ensure that such scenario is completely safe under the
ROW and MIXED modes.
2009-11-03 19:02:56 +00:00
Luis Soares
75a4dd3e6a BUG#40611, BUG#44779: reverted in mysql-5.1-rep+2. 2009-10-07 22:13:07 +01:00
Andrei Elkin
6d1ad12428 merge from 5.1-rpl+2 repo to a local branch with HB and bug@27808 fixes 2009-10-01 20:22:44 +03:00
Luis Soares
19e6d1c596 Automerge: mysql-5.1-rep+2 (local backports) --> mysql-5.1-rep+2 (local latest) 2009-09-30 12:48:22 +01:00
Alfranio Correia
4e0cb6dbb7 BUG#35542 Add option to sync master and relay log to disk after every event
BUG#31665 sync_binlog should cause relay logs to be synchronized

NOTE: Backporting the patch to next-mr.
      
Add sync_relay_log option to server, this option works for relay log 
the same as option sync_binlog for binlog. This option also synchronize
master info to disk when set to non-zero value.
            
Original patches from Sinisa and Mark, with some modifications
2009-09-29 15:27:12 +01:00
Luis Soares
bdacc5622e BUG#40611: MySQL cannot make a binary log after sequential number
beyond unsigned long.
BUG#44779: binlog.binlog_max_extension may be causing failure on 
next test in PB
      
NOTE1: this is the backport to next-mr.
NOTE2: already includes patch for BUG#44779.
      
Binlog file extensions would turn into negative numbers once the
variable used to hold the value reached maximum for signed
long. Consequently, incrementing value to the next (negative) number
would lead to .000000 extension, causing the server to fail.
                  
This patch addresses this issue by not allowing negative extensions
and by returning an error on find_uniq_filename, when the limit is
reached. Additionally, warnings are printed to the error log when the
limit is approaching. FLUSH LOGS will also report warnings to the
user, if the extension number has reached the limit. The limit has been
set to 0x7FFFFFFF as the maximum.
2009-09-29 15:12:07 +01:00
Andrei Elkin
c03549bf05 WL#342 heartbeat
backporting from 6.0 code base to 5.1.
2009-09-29 14:16:23 +03:00
Alfranio Correia
ac1b464a33 BUG#43929 binlog corruption when max_binlog_cache_size is exceeded
Large transactions and statements may corrupt the binary log if the size of the
cache, which is set by the max_binlog_cache_size, is not enough to store the
the changes.

In a nutshell, to fix the bug, we save the position of the next character in the
cache before starting processing a statement. If there is a problem, we simply
restore the position thus removing any effect of the statement from the cache.
Unfortunately, to avoid corrupting the binary log, we may end up loosing changes
on non-transactional tables if they do not fit in the cache. In such cases, we
store an Incident_log_event in order to stop the slave and alert users that some
changes were not logged.

Precisely, for every non-transactional changes that do not fit into the cache,
we do the following:
  a) the statement is *not* logged
  b) an incident event is logged after committing/rolling back the transaction,
  if any. Note that if a failure happens before writing the incident event to
  the binary log, the slave will not stop and the master will not have reported
  any error.
  c) its respective statement gives an error

For transactional changes that do not fit into the cache, we do the following:
  a) the statement is *not* logged
  b) its respective statement gives an error

To work properly, this patch requires two additional things. Firstly, callers to
MYSQL_BIN_LOG::write and THD::binlog_query must handle any error returned and
take the appropriate actions such as undoing the effects of a statement. We
already changed some calls in the sql_insert.cc, sql_update.cc and sql_insert.cc
modules but the remaining calls spread all over the code should be handled in
BUG#37148. Secondly, statements must be either classified as DDL or DML because
DDLs that do not get into the cache must generate an incident event since they
cannot be rolled back.
2009-06-18 14:52:46 +01:00
He Zhenxing
0793eec018 BUG#41948 Query_log_event constructor needlessly contorted
Make the caller of Query_log_event, Execute_load_log_event
constructors and THD::binlog_query to provide the error code
instead of having the constructors to figure out the error code.
2009-05-30 21:32:28 +08:00
Luis Soares
14d4576bfe merge: 5.1 -> 5.1-rpl
conflicts:
  Text conflict in client/mysqltest.cc
  Text conflict in mysql-test/include/wait_until_connected_again.inc
  Text conflict in mysql-test/lib/mtr_report.pm
  Text conflict in mysql-test/mysql-test-run.pl
  Text conflict in mysql-test/r/events_bugs.result
  Text conflict in mysql-test/r/log_state.result
  Text conflict in mysql-test/r/myisam_data_pointer_size_func.result
  Text conflict in mysql-test/r/mysqlcheck.result
  Text conflict in mysql-test/r/query_cache.result
  Text conflict in mysql-test/r/status.result
  Text conflict in mysql-test/suite/binlog/r/binlog_index.result
  Text conflict in mysql-test/suite/binlog/r/binlog_innodb.result
  Text conflict in mysql-test/suite/rpl/r/rpl_packet.result
  Text conflict in mysql-test/suite/rpl/t/rpl_packet.test
  Text conflict in mysql-test/t/disabled.def
  Text conflict in mysql-test/t/events_bugs.test
  Text conflict in mysql-test/t/log_state.test
  Text conflict in mysql-test/t/myisam_data_pointer_size_func.test
  Text conflict in mysql-test/t/mysqlcheck.test
  Text conflict in mysql-test/t/query_cache.test
  Text conflict in mysql-test/t/rpl_init_slave_func.test
  Text conflict in mysql-test/t/status.test
2009-01-23 13:22:05 +01:00
Luis Soares
0c9aaecc7f BUG#38826 Race in MYSQL_LOG::purge_logs is impossible to debug in production
BUG#39325 Server crash inside MYSQL_LOG::purge_first_log halts replicaiton

The patch reverses the order of the purging and updating events for log and
relay-log.info/index files respectively.

This solves the problem of having holes caused by crashes happening between updating
info/index files and purging logs.

This patch also contains an aditional test case for testing the crashing before purge logs.
      
NOTE1: This is a combined patch for BUG#38826 and BUG#39325. This patch is based on
bugteam tree and takes into account reviewers suggestions.

NOTE2: Merge from 5.0-bugteam
2008-12-04 14:07:56 +00:00