Commit graph

176 commits

Author SHA1 Message Date
Sergei Golubchik
c6d30805db 5.5 merge 2013-11-23 00:50:54 +01:00
Sergei Golubchik
fa3f8a18b2 mysql-5.5.34 merge
(some patches reverted, test case added)
2013-11-19 13:16:25 +01:00
Sergei Golubchik
bdd883ed9a 5.5. merge 2013-11-13 14:07:58 +01:00
unknown
7876d53b5f MDEV-4506: Parallel replication.
Delete any left-over deferred_event from rpl_group_info when
deleting the parent object, to protect against memory leaks.
2013-11-08 15:14:18 +01:00
unknown
cb86ce60b9 Merge MDEV-4506: Parallel replication into 10.0-base. 2013-11-01 09:17:06 +01:00
unknown
9c8da4ed76 MDEV-5196: Server hangs or assertion `!thd->wait_for_commit_ptr' fails on MASTER_POS_WAIT with slave-parallel-threads > 0
Fix a couple of issues in MDEV-4506, Parallel replication:

 - Missing mysql_cond_signal(), which could cause hangs.

 - Fix incorrect update of old-style replication position.

 - Change assertion to error handling (can trigger on manipulated/
   corrupt binlog).
2013-10-30 07:52:30 +01:00
unknown
2fbd1c7307 MDEV-4506: Parallel replication.
MDEV-5189: Error handling in parallel replication.

Fix error handling in parallel worker threads when a query fails:

 - Report the error to the error log.

 - Return the error back, and set rli->abort_slave.

 - Stop executing more events after the error.
2013-10-28 13:24:56 +01:00
unknown
80d0dd7bab MDEV-4506: Parallel replication.
Do not update relay-log.info and master.info on disk after every event
when using GTID mode:

 - relay-log.info and master.info are not crash-safe, and are not used
   when slave restarts in GTID mode (slave connects with GTID position
   instead and immediately rewrites the file with the new, correct
   information found).

 - When using GTID and parallel replication, the position in
   relay-log.info is misleading at best and simply wrong at worst.

 - When using parallel replication, the fact that every single
   transaction needs to do a write() syscall to the same file is
   likely to become a serious bottleneck.

The files are still written at normal slave stop.

In non-GTID mode, the files are written as normal (this is needed to
be able to restart after slave crash, even if such restart is then not
crash-safe, no change).
2013-10-25 12:56:12 +02:00
unknown
96a4f1f628 MDEV-4506: Parallel replication: Update some comments. 2013-10-24 08:53:48 +02:00
unknown
a09d2b105f MDEV-4506: Parallel replication.
Fix some more parts of old-style position updates.
Now we save in rgi some coordinates for master log and relay log, so
that in do_update_pos() we can use the right set of coordinates with
the right events.

The Rotate_log_event::do_update_pos() is fixed in the parallel case
to not directly update relay-log.info (as Rotate event runs directly
in the driver SQL thread, ahead of actual event execution). Instead,
group_master_log_file is updated as part of do_update_pos() in each
event execution.

In the parallel case, position updates happen in parallel without
any ordering, but taking care that position is not updated backwards.
Since position update happens only after event execution this leads
to the right result.

Also fix an access-after-free introduced in an earlier commit.
2013-10-23 15:03:03 +02:00
unknown
7681c6aa78 MDEV-4506: Parallel replication: Intermediate commit.
Fix some part of update of old-style coordinates in parallel replication:

 - Ignore XtraDB request for old-style coordinates, not meaningful for
   parallel replication (must use GTID to get crash-safe parallel slave).

 - Only update relay log coordinates forward, not backwards, to ensure
   that parallel threads do not conflict with each other.

 - Move future_event_relay_log_pos to rgi.
2013-10-17 14:11:19 +02:00
Michael Widenius
5748eb3ec6 Moved the remaining variables, that depends on sql execution, from Relay_log_info to rpl_group_info:
-row_stmt_start_timestamp
-last_event_start_time
-long_find_row_note
-trans_retries

Added slave_executed_entries_lock to protect rli->executed_entries
Added primitives for thread safe 64 bit increment
Update rli->executed_entries when event has executed, not when event has been sent to sql execution thread


sql/log_event.cc:
  row_stmt_start and long_find_row_note is now in rpl_group_info
sql/mysqld.cc:
  Added slave_executed_entries_lock to protect rli->executed_entries
sql/mysqld.h:
  Added slave_executed_entries_lock to protect rli->executed_entries
  Added primitives for thread safe 64 bit increment
sql/rpl_parallel.cc:
  Update rli->executed_entries when event has executed, not when event has been sent to sql execution thread
sql/rpl_rli.cc:
  Moved row_stmt_start_timestamp, last_event_start_time and long_find_row_note from Relay_log_info to rpl_group_info
sql/rpl_rli.h:
  Moved trans_retries, row_stmt_start_timestamp, last_event_start_time and long_find_row_note from Relay_log_info to rpl_group_info
sql/slave.cc:
  Use rgi for trans_retries and last_event_start_time
  Update rli->executed_entries when event has executed, not when event has been sent to sql execution thread
  Reset trans_retries when object is created
2013-10-15 00:17:16 +03:00
Michael Widenius
2e100cc5a4 Fixes for parallel slave:
- Made slaves temporary table multi-thread slave safe by adding mutex around save_temporary_table usage.
  - rli->save_temporary_tables is the active list of all used temporary tables
  - This is copied to THD->temporary_tables when temporary tables are opened and updated when temporary tables are closed
  - Added THD->lock_temporary_tables() and THD->unlock_temporary_tables() to simplify this.
- Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code.
- Added is_part_of_group() to mark functions that are part of the next function. This replaces setting IN_STMT when events are executed.
- Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code.
- If slave_skip_counter is set run things in single threaded mode. This simplifies code for skipping events.
- Updating state of relay log (IN_STMT and IN_TRANSACTION) is moved to one single function: update_state_of_relay_log()
  We can't use OPTION_BEGIN to check for the state anymore as the sql_driver and sql execution threads may be different.
  Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts
  is_in_group() is now independent of state of executed transaction.
- Reset thd->transaction.all.modified_non_trans_table() if we did set it for single table row events.
  This was mainly for keeping the flag as documented.
- Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it.
- Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
- Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
- Changed some functions to take rpl_group_info instead of Relay_log_info to make them multi-slave safe and to simplify usage
  - do_shall_skip()
  - continue_group()
  - sql_slave_killed()
  - next_event()
- Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure.
- set_thd_in_use_temporary_tables() removed as in_use is set on usage
- Added information to thd_proc_info() which thread is waiting for slave mutex to exit.
- In open_table() reuse code from find_temporary_table()

Other things:
- More DBUG statements
- Fixed the rpl_incident.test can be run with --debug
- More comments
- Disabled not used function rpl_connect_master()

mysql-test/suite/perfschema/r/all_instances.result:
  Moved sleep_lock and sleep_cond to rpl_group_info
mysql-test/suite/rpl/r/rpl_incident.result:
  Updated result
mysql-test/suite/rpl/t/rpl_incident-master.opt:
  Not needed anymore
mysql-test/suite/rpl/t/rpl_incident.test:
  Fixed that test can be run with --debug
sql/handler.cc:
  More DBUG_PRINT
sql/log.cc:
  More comments
sql/log_event.cc:
  Added DBUG statements
  do_shall_skip(), continue_group() now takes rpl_group_info param
  Use is_begin(), is_commit() and is_rollback() functions instead of inspecting query string
  We don't have set slaves temporary tables 'in_use' as this is now done when tables are opened.
  Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
  Use IN_TRANSACTION flag to test state of relay log.
  In rows_event_stmt_cleanup() reset thd->transaction.all.modified_non_trans_table if we had set this before.
sql/log_event.h:
  do_shall_skip(), continue_group() now takes rpl_group_info param
  Added is_part_of_group() to mark events that are part of the next event. This replaces setting IN_STMT when events are executed.
  Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code.
sql/log_event_old.cc:
  Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
  do_shall_skip(), continue_group() now takes rpl_group_info param
sql/log_event_old.h:
  Added is_part_of_group() to mark events that are part of the next event.
  do_shall_skip(), continue_group() now takes rpl_group_info param
sql/mysqld.cc:
  Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it.
  Relay_log_info::sleep_lock -> Rpl_group_info::sleep_lock
  Relay_log_info::sleep_cond -> Rpl_group_info::sleep_cond
sql/mysqld.h:
  Updated types and names
sql/rpl_gtid.cc:
  More DBUG
sql/rpl_parallel.cc:
  Updated TODO section
  Set thd for event that is execution
  Use new  is_begin(), is_commit() and is_rollback() functions.
  More comments
sql/rpl_rli.cc:
  sql_thd -> sql_driver_thd
  Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
  Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
  Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts.
  Reset table->in_use for temporary tables as the table may have been used by another THD.
  Use IN_TRANSACTION instead of OPTION_BEGIN to check state of relay log.
  Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
sql/rpl_rli.h:
  Changed relay log state flags to bit masks instead of bit positions (most other code we have uses bit masks)
  Added IN_TRANSACTION to mark if we are in a BEGIN ... COMMIT section.
  save_temporary_tables is now thread safe
  Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
  Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
  Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code
  is_in_group() is now independent of state of executed transaction.
sql/slave.cc:
  Simplifed arguments to io_salve_killed(), sql_slave_killed() and check_io_slave_killed(); No reason to supply THD as this is part of the given structure.
  set_thd_in_use_temporary_tables() removed as in_use is set on usage in sql_base.cc
  sql_thd -> sql_driver_thd
  More DBUG
  Added update_state_of_relay_log() which will calculate the IN_STMT and IN_TRANSACTION state of the relay log after the current element is executed.
  If slave_skip_counter is set run things in single threaded mode.
  Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure.
  Added information to thd_proc_info() which thread is waiting for slave mutex to exit.
  Disabled not used function rpl_connect_master()
  Updated argument to next_event()
sql/sql_base.cc:
  Added mutex around usage of slave's temporary tables. The active list is always kept up to date in sql->rgi_slave->save_temporary_tables.
  Clear thd->temporary_tables after query (safety)
  More DBUG
  When using temporary table, set table->in_use to current thd as the THD may be different for slave threads.
  Some code is ifdef:ed with REMOVE_AFTER_MERGE_WITH_10 as the given code in 10.0 is not yet in this tree.
  In open_table() reuse code from find_temporary_table()
sql/sql_binlog.cc:
  rli->sql_thd -> rli->sql_driver_thd
  Remove duplicate setting of rgi->rli
sql/sql_class.cc:
  Added helper functions rgi_lock_temporary_tables() and rgi_unlock_temporary_tables()
  Would have been nicer to have these inline, but there was no easy way to do that
sql/sql_class.h:
  Added functions to protect slaves temporary tables
sql/sql_parse.cc:
  Added DBUG_PRINT
sql/transaction.cc:
  Added comment
2013-10-14 00:24:05 +03:00
unknown
5633dd8227 MDEV-4506: parallel replication.
Add a simple test case.
Fix bugs found.
2013-09-16 14:33:49 +02:00
unknown
d107bdaa01 MDEV-4506, parallel replication.
Some after-review fixes.
2013-09-13 15:09:57 +02:00
unknown
f9c2b402f4 MDEV-26: Global transaction ID.
Implement @@gtid_binlog_state. This is the internal state of the binlog
(most recent GTID logged for every domain_id and server_id). This allows
to save the state before RESET MASTER and restore it afterwards.
2013-08-23 14:02:13 +02:00
Dmitry Lenev
fc2c669297 Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR
STATUS OF ROLLBACKED TRANSACTION" and bug  - "TRANSACTION
IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK".

The problem in the first bug report was that although deadlock involving
metadata locks was reported using the same error code and message as InnoDB
deadlock it didn't rollback transaction like the latter. This caused
confusion to users as in some cases after ER_LOCK_DEADLOCK transaction
could have been restarted immediately and in some cases rollback was
required.

The problem in the second bug report was that although InnoDB deadlock
caused transaction rollback in all storage engines it didn't cause release
of metadata locks. So concurrent DDL on the tables used in transaction was
blocked until implicit or explicit COMMIT or ROLLBACK was issued in the
connection which got InnoDB deadlock.

The former issue has stemmed from the fact that when support for detection
and reporting metadata locks deadlocks was added we erroneously assumed
that InnoDB doesn't rollback transaction on deadlock but only last statement
(while this is what happens on InnoDB lock timeout actually) and so didn't
implement rollback of transactions on MDL deadlocks.

The latter issue was caused by the fact that rollback of transaction due
to deadlock is carried out by setting THD::transaction_rollback_request
flag at the point where deadlock is detected and performing rollback
inside of trans_rollback_stmt() call when this flag is set. And
trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are
released.

This patch solves these two problems in the following way:

- In case when MDL deadlock is detect transaction rollback is requested
  by setting THD::transaction_rollback_request flag.

- Code performing rollback of transaction if THD::transaction_rollback_request
  is moved out from trans_rollback_stmt(). Now we handle rollback request
  on the same level as we call trans_rollback_stmt() and release statement/
  transaction MDL locks.
2013-08-20 13:12:34 +04:00
unknown
13fddb32de MDEV-4506: Parallel replication: Intermediate commit.
Move annotate-event stuff from Relay_log_info to rpl_group_info,
to make it thread safe.
2013-07-12 14:52:05 +02:00
unknown
47f8e0ef6e MDEV-4506: Parallel replication: Intermediate commit
Remove Relay_log_info::group_info. (It is not thread safe).
2013-07-12 14:42:48 +02:00
unknown
ba4b937af2 MDEV-4506: Parallel replication: Intermediate commit
Move the deferred event stuff from Relay_log_info to rpl_group_info
to make it thread safe for parallel replication.
2013-07-12 14:36:20 +02:00
unknown
a99356fbe7 MDEV-4506: Parallel replication: intermediate commit.
Fix a bunch of issues found with locking, ordering, and non-thread-safe stuff
in Relay_log_info.

Now able to do a simple benchmark, showing 4.5 times speedup for applying a
binlog with 10000 REPLACE statements.
2013-07-08 16:47:07 +02:00
unknown
b5a496a777 MDEV-4506: Parallel replication: Intermediate commit.
Fix some bugs around waiting for worker threads to end during SQL slave stop.

Free Log_event after parallel execution (still needs to be made thread-safe by
using rpl_group_info rather than rli).
2013-07-04 13:17:01 +02:00
unknown
592e464a02 MDEV-4506: Parallel replication. Intermediate commit.
Pass down rpl_group_info * to remove one instance of non-threadsafe
use of rli->group_info.
2013-07-03 19:03:21 +02:00
unknown
31a5edb5c2 MDEV-4506: Parallel replication. Intermediate commit.
Hook in the wait-for-prior-commit logic (not really tested yet).
Clean up some resource maintenance around rpl_group_info (may still be some
smaller issues there though).
Add a ToDo list at the top of rpl_parallel.cc
2013-07-03 13:46:33 +02:00
unknown
1b3dc66e31 MDEV-4506: Parallel replication: Intermediate commit.
First step of splitting out part of Relay_log_info, so that different
event groups being applied in parallel can each use their own copy.
2013-06-28 15:19:30 +02:00
unknown
df66ee7fe9 MDEV-4692: mysql.gtid_slave_pos accumulates values for a domain
When we load the slave state from the mysql.gtid_slave_pos at server start, we
need to load all the rows into the in-memory hash, not just the most recent
one in each replication domain. Otherwise we accumulate cruft in the form of
old rows each time the server restarts.
2013-06-21 21:23:24 +02:00
Sergei Golubchik
4749d40c63 5.5 merge 2013-06-06 17:51:28 +02:00
unknown
7ad47ab0e0 MDEV-4605: Failing to load GTID slave position from rpl.gtid_slave_pos
There were several cases where the slave GTID position was not loaded
correctly before being used. This caused various failures such as
corrupting the position at slave start and empty values of
@@gtid_slave_pos and @@gtid_current_pos.

Fixed by adding more checks for loaded position, and by always loading
the position at server startup.
2013-06-03 07:41:38 +02:00
unknown
ee2b7db3f8 MDEV-4478: Implement GTID "strict mode"
When @@GLOBAL.gtid_strict_mode=1, then certain operations result
in error that would otherwise result in out-of-order binlog files
between servers.

GTID sequence numbers are now allocated independently per domain;
this results in less/no holes in GTID sequences, increasing the
likelyhood that diverging binlogs will be caught by the slave when
GTID strict mode is enabled.
2013-05-28 13:28:31 +02:00
unknown
9fae993024 MDEV-26: Global transaction ID.
Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".

Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.
2013-05-15 19:52:21 +02:00
Michael Widenius
8cdb118a0a Fixed: MDEV-4352; LOAD DATA was not multi-source safe
- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name

Other bug fixes:
- Enlarge buffer for connection names with 'special characters' one can't store in filenames

Optimization:
- Don't do 'lower case' of connection names. We can use cmp_connection_name, where we already have the connection name in lower case.


mysql-test/suite/multi_source/load_data.result:
  Test case for MDEV-4352
mysql-test/suite/multi_source/load_data.test:
  Test case for MDEV-4352
sql/log_event.cc:
  Fixed: MDEV-4352
  - Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
  - Concurrent LOAD DATA commands from two master connections could use the same file name
  
  The fix was to add the connection name (if one exists) to all slave temporary files used by LOAD DATA
sql/rpl_mi.cc:
  Enlarge buffer for connection names with 'special characters' one can't store in filenames
  Use mi->cmp_connection_name for connection file names.
sql/rpl_rli.cc:
  Use mi->cmp_connection_name for connection file names.
sql/slave.cc:
  Removed not needed empty line
sql/sql_const.h:
  Added MAX_FILENAME_MBWIDTH to be able to calculate buffer length for connection_names stored in file names
sql/sql_repl.cc:
  Use mi->cmp_connection_name for connection file names.
2013-05-03 01:50:42 +03:00
Michael Widenius
8e52698580 Fixed compiler warnings and a not critical memory leak
sql/keycaches.cc:
  Added free_all_rpl_filters() to be able to free all filters at cleanup
sql/keycaches.h:
  Added prototype
sql/rpl_rli.cc:
  Fixed compiler warning
sql/slave.cc:
  Free all rpl_filters at cleanup
sql/sp.cc:
  Fixed compiler warning when not all struct elements was initialized
sql/sql_acl.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_ews_global_by_event_name.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_performance_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_consumers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_instruments.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_sync_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_threads.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/xtradb/os/os0file.c:
  Fixed compiler warning when not all struct elements was initialized
2013-04-17 22:33:33 +03:00
Sergei Golubchik
07315d3603 strmake_buf(X,Y) helper, equivalent to strmake(X,Y,sizeof(X)-1)
with a bit of lame protection against abuse.
2013-04-17 19:42:34 +02:00
Lixun Peng
82eedf4e97 Makeing rpl_filter for each Master_info.
Users can set different repplication filter rules for each replication connection, in my.cnf or command line.
But the rules set online will not record in master.info, it means if users restart MySQL, these rules will lose.
So if users wantn't their replication filter rules lose, they should write the rules in my.cnf.

Users can set rules by 2 ways:
1. Online SET command, "SET connection_name.replication_filter_settings = rules;".
2. In my.cnf, "connection_name.replication_filter_settings = rules".

If no connection_name in my.cnf, this rule will apply for ALL replication connection.
If no connetion_name in SET statement, this rull will apply for default_connection_name.
2013-04-16 19:43:28 +08:00
unknown
5aaf73fcaa MDEV-26: Global transaction ID.
Add tests crashing the slave in the middle of replication and checking that
replication picks-up again on restart in a crash-safe way.

Fix silly code that causes crash by inserting uninitialised data into a hash.
2013-03-28 13:03:51 +01:00
unknown
b6b84d6825 MDEV-26: Global transaction ID.
Fix error handling when record_gtid() fails to update the
mysql.rpl_slave_state table.
2013-03-21 17:33:29 +01:00
unknown
9d9ddad759 MDEV-26: Global transaction ID.
Fix things so that a master can switch with MASTER_GTID_POS=AUTO to a slave
that was previously running with log_slave_updates=0, by looking into the
slave replication state on the master when the slave requests something not
present in the binlog.

Be a bit more strict about what position the slave can ask for, to avoid some
easy-to-hit misconfiguration errors.

Start over with seq_no counter when RESET MASTER.
2013-03-18 15:09:36 +01:00
unknown
5e414f6b06 MDEV-26: Global Transaction ID.
Move a bunch of GTID specific code into new file rpl_gtid.cc. Make libmysqld build.
2013-03-11 16:02:40 +01:00
unknown
bfec9c64b8 MDEV-26: Global transaction ID, intermediate commit.
- Fix that slave GTID state was updated from the wrong place in the code,
   causing random crashing and other misery.

 - Fix updates to mysql.rpl_slave_state to not go to binlog (this would cause
   duplicate key errors on the slave and is generally the wrong thing to do).
2013-02-19 11:45:29 +01:00
Michael Widenius
edc89f7511 Buildbot fixes and cleanups:
- Added --verbose to BUILD scripts to get make to write out compile commands.
- Detect if AM_EXTRA_MAKEFLAGS=VERBOSE=1 was used with build scripts.
- Don't write warnings about replication variables when doing bootstrap.
- Fixed that mysql_cond_wait() and mysql_cond_timedwait() will report original source file in case of errors.
- Ignore some compiler warnings

BUILD/FINISH.sh:
  Detect if AM_EXTRA_MAKEFLAGS=VERBOSE=1 or --verbose was used
BUILD/SETUP.sh:
  Added --verbose to print out the full compile lines
  Updated help message
client/mysqltest.cc:
  Fixed that one can use 'replace' with cat_file
cmake/configure.pl:
  If --verbose is used, get make to write out compile commands
debian/dist/Debian/rules:
  Added $AM_EXTRA_MAKEFLAGS to get VERBOSE=1 on buildbot builds
debian/dist/Ubuntu/rules:
  Added $AM_EXTRA_MAKEFLAGS to get VERBOSE=1 on buildbot builds
include/my_pthread.h:
  Made set_timespec_time_nsec() more portable.
include/mysql/psi/mysql_thread.h:
  Fixed that mysql_cond_wait() and mysql_cond_timedwait() will report original source file in case of errors.
mysql-test/suite/innodb/r/auto_increment_dup.result:
  Fixed wrong DBUG_SYNC
mysql-test/suite/innodb/t/auto_increment_dup.test:
  Fixed wrong DBUG_SYNC
mysql-test/suite/perfschema/include/upgrade_check.inc:
  Make test more portable for changes in *.sql files
mysql-test/suite/perfschema/r/pfs_upgrade.result:
  Updated test results
mysql-test/valgrind.supp:
  Ignore running Aria checkpoint thread
scripts/mysqlaccess.sh:
  Changed reference of bugs database
  Ensure that also client-server group is read.
sql/handler.cc:
  Added missing syncpoint
sql/mysqld.cc:
  Don't write warnings about replication variables when doing bootstrap
sql/mysqld.h:
  Don't write warnings about replication variables when doing bootstrap
sql/rpl_rli.cc:
  Don't write warnings about replication variables when doing bootstrap
sql/sql_insert.cc:
  Don't mask SERVER_SHUTDOWN in insert_delayed
  This is done to be able to distingush between shutdown and interrupt errors
support-files/compiler_warnings.supp:
  Ignore some compiler warnings in xtradb,innobase, oqgraph, yassl, string3.h
2013-01-11 02:03:43 +02:00
unknown
03f28863e8 MDEV-26: Global transaction commit. Intermediate commit.
Now slave records GTID in mysql.rpl_slave_state when applying XID log event.
2012-11-05 15:01:49 +01:00
unknown
ab8e8f4b27 MDEV-500: Session variable for server_id
MDEV-26: Global transaction id, partial commit

Change server_id to be a session variable.
User with SUPER can set it to binlog with different server_id.
Implement backward-compatible ::server_id mirror for plugins.
2012-10-23 12:46:29 +02:00
Sergei Golubchik
0af4b6c6ee 5.5 merge 2013-01-29 15:10:47 +01:00
unknown
40bbf697aa MDEV-532: Async InnoDB commit checkpoint.
Make the commit checkpoint inside InnoDB be asynchroneous.
Implement a background thread in binlog to do the writing and flushing of
binlog checkpoint events to disk.
2012-12-14 15:38:07 +01:00
Michael Widenius
572560f38c Changed SHOW_FUNC variabels that don't return SHOW_ARRAY to SHOW_SIMPLE_FUNC.
This allows us to avoid calculating variables (including those involving mutex) that doesn't match the given
wildcard in SHOW STATUS LIKE '...'
Removed all references to active_mi that could cause problems for multi-source replication.

Added START|STOP ALL SLAVES
Added SHOW ALL SLAVES STATUS


include/mysql/plugin.h:
  Added SHOW_SIMPLE_FUNC
include/mysql/plugin_audit.h.pp:
  Updated .pp file
include/mysql/plugin_auth.h.pp:
  Updated .pp file
include/mysql/plugin_ftparser.h.pp:
  Updated .pp file
mysql-test/suite/multi_source/info_logs.result:
  New columns in SHOW ALL SLAVES STATUS
mysql-test/suite/multi_source/info_logs.test:
  Test new syntax
mysql-test/suite/multi_source/simple.result:
  New columns in SHOW ALL SLAVES STATUS
mysql-test/suite/multi_source/simple.test:
  test new syntax
mysql-test/suite/multi_source/syntax.result:
  Updated result
mysql-test/suite/multi_source/syntax.test:
  Test new syntax
mysql-test/suite/rpl/r/rpl_skip_replication.result:
  Updated result
plugin/semisync/semisync_master_plugin.cc:
  SHOW_FUNC -> SHOW_SIMPLE_FUNC
sql/item_create.cc:
  Simplify code
sql/lex.h:
  Added SLAVES keyword
sql/log.cc:
  Constant -> define
sql/log_event.cc:
  Added comment
sql/mysqld.cc:
  SHOW_FUNC -> SHOW_SIMPLE_FUNC
  Made slave_retried_trans, slave_received_heartbeats and heartbeat_period multi-source safe
  Clear variable denied_connections and slave_retried_transactions on startup
sql/mysqld.h:
  Added slave_retried_transactions
sql/rpl_mi.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  Added start_all_slaves() and stop_all_slaves()
sql/rpl_mi.h:
  Added prototypes
sql/rpl_rli.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  added executed_entries
sql/rpl_rli.h:
  Added executed_entries
sql/share/errmsg-utf8.txt:
  More and better error messages
sql/slave.cc:
  Added more fields to SHOW ALL SLAVES STATUS
  Added slave_retried_transactions
  Changed constants -> defines
sql/sql_class.h:
  Added comment
sql/sql_insert.cc:
  active_mi.rli -> thd->rli_slave
sql/sql_lex.h:
  Added SQLCOM_SLAVE_ALL_START & SQLCOM_SLAVE_ALL_STOP
sql/sql_load.cc:
  active_mi.rli -> thd->rli_slave
sql/sql_parse.cc:
  Added START|STOP ALL SLAVES
sql/sql_prepare.cc:
  Added SQLCOM_SLAVE_ALL_START & SQLCOM_SLAVE_ALL_STOP
sql/sql_reload.cc:
  Made REFRESH RELAY LOG multi-source safe
sql/sql_repl.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  Don't send my_ok() from start_slave() or stop_slave() so that we can call it for all master connections
sql/sql_show.cc:
  Compare wild cards early for all variables
sql/sql_yacc.yy:
  Added START|STOP ALL SLAVES
  Added SHOW ALL SLAVES STATUS
sql/sys_vars.cc:
  Made replicate_events_marked_for_skip,slave_net_timeout and rpl_filter multi-source safe.
sql/sys_vars.h:
  Simplify Sys_var_rpl_filter
2012-10-03 01:44:54 +03:00
Michael Widenius
8ac1b41cf3 Made max_relay_log_size depending on master connection.
Changed names of multi-source log files so that original suffixes are kept.


include/my_sys.h:
  Added fn_ext2(), which returns pointer to last '.' in file name
mysql-test/extra/rpl_tests/rpl_max_relay_size.test:
  Updated test
mysql-test/suite/multi_source/info_logs-master.opt:
  Test with strange file names
mysql-test/suite/multi_source/info_logs.result:
  Updated results
mysql-test/suite/multi_source/info_logs.test:
  Changed to test with complex names to be able to verify the filename generator code
mysql-test/suite/multi_source/relaylog_events.result:
  Updated results
mysql-test/suite/multi_source/reset_slave.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.test:
  Added testing of max_relay_log_size
mysql-test/suite/rpl/r/rpl_row_max_relay_size.result:
  Updated results
mysql-test/suite/rpl/r/rpl_stm_max_relay_size.result:
  Updated results
mysql-test/suite/sys_vars/r/max_relay_log_size_basic.result:
  Updated results
mysql-test/suite/sys_vars/t/max_relay_log_size_basic.test:
  Updated results
mysys/mf_fn_ext.c:
  Added fn_ext2(), which returns pointer to last '.' in file name
sql/log.cc:
  Removed some wrong casts
sql/log.h:
  Updated comment to reflect new code
sql/log_event.cc:
  Updated DBUG_PRINT
sql/mysqld.cc:
  Added that max_relay_log_size copies it's values from max_binlog_size
sql/mysqld.h:
  Removed max_relay_log_size
sql/rpl_mi.cc:
  Changed names of multi-source log files so that original suffixes are kept.
sql/rpl_mi.h:
  Updated prototype
sql/rpl_rli.cc:
  Updated comment to reflect new code
  Made max_relay_log_size depending on master connection.
sql/rpl_rli.h:
  Made max_relay_log_size depending on master connection.
sql/set_var.h:
  Made option global so that one can check and change min & max values (sorry Sergei)
sql/sql_class.h:
  Made max_relay_log_size depending on master connection.
sql/sql_repl.cc:
  Updated calls to create_signed_file_name()
sql/sys_vars.cc:
  Made max_relay_log_size depending on master connection.
  Made old code more reusable
sql/sys_vars.h:
  Changed Sys_var_multi_source_uint to ulong to be able to handle max_relay_log_size
  Made old code more reusable
2012-10-01 02:30:44 +03:00
Michael Widenius
5a4b5869a0 Fixed issues found by QA (Elena)
- Added parameter to reset_logs() so that one can specify if new logs should be created.

mysql-test/include/setup_fake_relay_log.inc:
  There is no orphan relay log files anymore
mysql-test/mysql-test-run.pl:
  Added multi_source to test suite
mysql-test/suite/multi_source/info_logs.result:
  New test
mysql-test/suite/multi_source/info_logs.test:
  New test
mysql-test/suite/multi_source/my.cnf:
  Added log-warnings to get more information to the log files
mysql-test/suite/multi_source/relaylog_events.result:
  Added cleanup
mysql-test/suite/multi_source/relaylog_events.test:
  Added cleanup
mysql-test/suite/multi_source/reset_slave.result:
  Updated results after improved RESET SLAVE
mysql-test/suite/multi_source/simple.result:
  Updated results after improved RESET SLAVE
mysql-test/suite/multi_source/simple.test:
  Syncronize positions before show full slave status
mysql-test/suite/rpl/r/rpl_row_show_relaylog_events.result:
  Updated results after improved RESET SLAVE (we now use less relay log files)
mysql-test/suite/rpl/r/rpl_stm_mix_show_relaylog_events.result:
  Updated results after improved RESET SLAVE (we now use less relay log files)
sql/log.cc:
  Added parameter to reset_logs() so that one can specify if new logs should be created.
sql/log.h:
  Added parameter to reset_logs()
sql/rpl_mi.cc:
  Create Master_info_index::index_file_names once at init
  More DBUG_PRINT
  Give error if Master_info_index::check_duplicate_master_info fails
sql/rpl_rli.cc:
  If we do a full reset, don't create any new relay log files.
sql/share/errmsg-utf8.txt:
  Improved error message if connection exists
sql/sql_parse.cc:
  Fixed memory leak
sql/sql_repl.cc:
  check_duplicate_master_info() now generates an error
  Added parameter to reset_logs()
2012-09-28 21:22:24 +03:00
Michael Widenius
1864d9596d Implementation of Multi-source replication (MDEV:253)
Documentation of the feature can be found at: http://kb.askmonty.org/en/multi-source-replication/
This code is based on code from Taobao, developed by Plinux

BUILD/SETUP.sh:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
client/mysqltest.cc:
  Added support for error names starting with 'W'
  Added connection_name support to --sync_with_master
cmake/maintainer.cmake:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
mysql-test/r/mysqltest.result:
  Updated results
mysql-test/r/parser.result:
  Updated results
mysql-test/suite/multi_source/my.cnf:
  Setup of multi-master tests
mysql-test/suite/multi_source/simple.result:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/simple.test:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/syntax.result:
  Test of multi-source syntax
mysql-test/suite/multi_source/syntax.test:
  Test of multi-source syntax
mysql-test/suite/rpl/r/rpl_rotate_logs.result:
  Updated results because of new error messages
mysql-test/t/parser.test:
  Updated test as master_pos_wait() now takes more arguments than before
sql/event_scheduler.cc:
  No reason to initialize slave_thread (it's guaranteed to be zero here)
sql/item_create.cc:
  Added connection_name argument to master_pos_wait()
  Simplified code
sql/item_func.cc:
  Added connection_name argument to master_pos_wait()
sql/item_func.h:
  Added connection_name argument to master_pos_wait()
sql/log.cc:
  Added tag "Master 'connection_name'" to slave errors that has a connection name.
sql/mysqld.cc:
  Added variable mysqld_server_initialized so that other functions can test if server is fully initialized.
  Free all slave data in one place (fewer ifdef's)
  Removed not needed call to close_active_mi()
  Initialize slaves() later in startup to ensure that everthing is really initialized when slaves start.
  Made status variable slave_running multi-source safe
sql/mysqld.h:
  Added mysqld_server_initialized
sql/rpl_mi.cc:
  Store connection name and cmp_connection_name (only used for show full slave status) in Master_info
  Added code for Master_info_index, which handles storage of multi-master information
  Don't write the empty "" connection_name to multi-master.info file. This is handled by the original code.
sql/rpl_mi.h:
  Added connection_name and Master_info_index
sql/rpl_rli.cc:
  Added connection_name to relay log files.
sql/rpl_rli.h:
  Fixed type of slave_skip_counter as we now access it directly in sys_vars.cc, so it must be uint
sql/share/errmsg-utf8.txt:
  Added new error messages needed for multi-source
  Added multi-source name to error ER_MASTER_INFO and WARN_NO_MASTER_INFO
sql/slave.cc:
  Moved things a bit around to make it easier to handle error conditions.
  Create a global master_info_index and add the "" connection to it
  Ensure that new Master_info doesn't fail.
  Don't call terminate_slave_threads(active_mi..) on end_slave() as this is now done automaticly when deleting master_info_index.
  Delete not needed function close_active_mi(). One can achive same thing by calling end_slave().
  Added support for SHOW FULL SLAVE STATUS (show status for all master connections with connection_name as first column)
sql/slave.h:
  Added new prototypes
sql/sql_base.cc:
  More DBUG_PRINT
sql/sql_class.cc:
  Reset thd->connection_name and thd-->default_master_connection
sql/sql_class.h:
  Added thd->connection_name and thd-->default_master_connection
  Added slave_skip_count to variables to make changing the @@sql_slave_skip_count variable thread safe
sql/sql_const.h:
  Added MAX_CONNECTION_NAME
sql/sql_lex.cc:
  Reset 'lex->verbose' (to simplify some sql_yacc.yy code)
sql/sql_lex.h:
  Added connection_name
sql/sql_parse.cc:
  Added support for connection_name to all SLAVE commands.
  - Instead of using active_mi, we now get the current Master_info from master_info_index.
  - Create new replication threads with CHANGE MASTER
  - Added support for show_all_master_info()
sql/sql_reload.cc:
  Made reset/full slave use master_info_index->get_master_info() instead of active_mi.
  If one uses 'RESET SLAVE "connection_name" all' the connection is removed from master_info_index.
sql/sql_repl.cc:
  sql_slave_skip_counter is moved to thd->variables to make it thread safe and fix some bugs with it
  Add connection name to relay log files.
  Added connection name to errors.
  Added some logging for multi-master if log_warnings > 1
  stop_slave():
  - Don't check if thd is set. It's guaranteed to always be set.
  change_master():
  - Check for duplicate connection names in change_master()
  - Check for wrong arguments first in file (to simplify error handling)
  - Register new connections in master_info_index
sql/sql_yacc.yy:
  Added optional connection_name to a all relevant master/slave commands
sql/strfunc.cc:
  my_global.h shoud always be included first.
sql/sys_vars.cc:
  Added variable default_master_connection
  Made variable sql_slave_skip_counter multi-source safe
sql/sys_vars.h:
  Added Sys_var_session_lexstring (needed for default_master_connection)
  Added Sys_var_multi_source_uint (needed for sql_slave_skip_counter).
2012-09-28 02:06:56 +03:00
unknown
288eeb3a31 MDEV-232: Remove one fsync() from commit phase.
Introduce a new storage engine API method commit_checkpoint_request().
This is used to replace the fsync() at the end of every storage engine
commit with a single fsync() when a binlog is rotated.

Binlog rotation is now done during group commit instead of being
delayed until unlog(), removing some server stall and avoiding an
expensive lock/unlock of LOCK_log inside unlog().
2012-09-13 14:31:29 +02:00
Sergei Golubchik
0522307ed1 mysql-5.5 merge 2012-06-14 20:05:31 +02:00