Commit graph

152 commits

Author SHA1 Message Date
unknown
f9c2b402f4 MDEV-26: Global transaction ID.
Implement @@gtid_binlog_state. This is the internal state of the binlog
(most recent GTID logged for every domain_id and server_id). This allows
to save the state before RESET MASTER and restore it afterwards.
2013-08-23 14:02:13 +02:00
unknown
df66ee7fe9 MDEV-4692: mysql.gtid_slave_pos accumulates values for a domain
When we load the slave state from the mysql.gtid_slave_pos at server start, we
need to load all the rows into the in-memory hash, not just the most recent
one in each replication domain. Otherwise we accumulate cruft in the form of
old rows each time the server restarts.
2013-06-21 21:23:24 +02:00
Sergei Golubchik
4749d40c63 5.5 merge 2013-06-06 17:51:28 +02:00
unknown
7ad47ab0e0 MDEV-4605: Failing to load GTID slave position from rpl.gtid_slave_pos
There were several cases where the slave GTID position was not loaded
correctly before being used. This caused various failures such as
corrupting the position at slave start and empty values of
@@gtid_slave_pos and @@gtid_current_pos.

Fixed by adding more checks for loaded position, and by always loading
the position at server startup.
2013-06-03 07:41:38 +02:00
unknown
ee2b7db3f8 MDEV-4478: Implement GTID "strict mode"
When @@GLOBAL.gtid_strict_mode=1, then certain operations result
in error that would otherwise result in out-of-order binlog files
between servers.

GTID sequence numbers are now allocated independently per domain;
this results in less/no holes in GTID sequences, increasing the
likelyhood that diverging binlogs will be caught by the slave when
GTID strict mode is enabled.
2013-05-28 13:28:31 +02:00
unknown
9fae993024 MDEV-26: Global transaction ID.
Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".

Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.
2013-05-15 19:52:21 +02:00
Michael Widenius
8cdb118a0a Fixed: MDEV-4352; LOAD DATA was not multi-source safe
- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name

Other bug fixes:
- Enlarge buffer for connection names with 'special characters' one can't store in filenames

Optimization:
- Don't do 'lower case' of connection names. We can use cmp_connection_name, where we already have the connection name in lower case.


mysql-test/suite/multi_source/load_data.result:
  Test case for MDEV-4352
mysql-test/suite/multi_source/load_data.test:
  Test case for MDEV-4352
sql/log_event.cc:
  Fixed: MDEV-4352
  - Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
  - Concurrent LOAD DATA commands from two master connections could use the same file name
  
  The fix was to add the connection name (if one exists) to all slave temporary files used by LOAD DATA
sql/rpl_mi.cc:
  Enlarge buffer for connection names with 'special characters' one can't store in filenames
  Use mi->cmp_connection_name for connection file names.
sql/rpl_rli.cc:
  Use mi->cmp_connection_name for connection file names.
sql/slave.cc:
  Removed not needed empty line
sql/sql_const.h:
  Added MAX_FILENAME_MBWIDTH to be able to calculate buffer length for connection_names stored in file names
sql/sql_repl.cc:
  Use mi->cmp_connection_name for connection file names.
2013-05-03 01:50:42 +03:00
Michael Widenius
8e52698580 Fixed compiler warnings and a not critical memory leak
sql/keycaches.cc:
  Added free_all_rpl_filters() to be able to free all filters at cleanup
sql/keycaches.h:
  Added prototype
sql/rpl_rli.cc:
  Fixed compiler warning
sql/slave.cc:
  Free all rpl_filters at cleanup
sql/sp.cc:
  Fixed compiler warning when not all struct elements was initialized
sql/sql_acl.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_ews_global_by_event_name.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_performance_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_consumers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_instruments.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_sync_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_threads.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/xtradb/os/os0file.c:
  Fixed compiler warning when not all struct elements was initialized
2013-04-17 22:33:33 +03:00
Sergei Golubchik
07315d3603 strmake_buf(X,Y) helper, equivalent to strmake(X,Y,sizeof(X)-1)
with a bit of lame protection against abuse.
2013-04-17 19:42:34 +02:00
Lixun Peng
82eedf4e97 Makeing rpl_filter for each Master_info.
Users can set different repplication filter rules for each replication connection, in my.cnf or command line.
But the rules set online will not record in master.info, it means if users restart MySQL, these rules will lose.
So if users wantn't their replication filter rules lose, they should write the rules in my.cnf.

Users can set rules by 2 ways:
1. Online SET command, "SET connection_name.replication_filter_settings = rules;".
2. In my.cnf, "connection_name.replication_filter_settings = rules".

If no connection_name in my.cnf, this rule will apply for ALL replication connection.
If no connetion_name in SET statement, this rull will apply for default_connection_name.
2013-04-16 19:43:28 +08:00
unknown
5aaf73fcaa MDEV-26: Global transaction ID.
Add tests crashing the slave in the middle of replication and checking that
replication picks-up again on restart in a crash-safe way.

Fix silly code that causes crash by inserting uninitialised data into a hash.
2013-03-28 13:03:51 +01:00
unknown
b6b84d6825 MDEV-26: Global transaction ID.
Fix error handling when record_gtid() fails to update the
mysql.rpl_slave_state table.
2013-03-21 17:33:29 +01:00
unknown
9d9ddad759 MDEV-26: Global transaction ID.
Fix things so that a master can switch with MASTER_GTID_POS=AUTO to a slave
that was previously running with log_slave_updates=0, by looking into the
slave replication state on the master when the slave requests something not
present in the binlog.

Be a bit more strict about what position the slave can ask for, to avoid some
easy-to-hit misconfiguration errors.

Start over with seq_no counter when RESET MASTER.
2013-03-18 15:09:36 +01:00
unknown
5e414f6b06 MDEV-26: Global Transaction ID.
Move a bunch of GTID specific code into new file rpl_gtid.cc. Make libmysqld build.
2013-03-11 16:02:40 +01:00
unknown
bfec9c64b8 MDEV-26: Global transaction ID, intermediate commit.
- Fix that slave GTID state was updated from the wrong place in the code,
   causing random crashing and other misery.

 - Fix updates to mysql.rpl_slave_state to not go to binlog (this would cause
   duplicate key errors on the slave and is generally the wrong thing to do).
2013-02-19 11:45:29 +01:00
Michael Widenius
edc89f7511 Buildbot fixes and cleanups:
- Added --verbose to BUILD scripts to get make to write out compile commands.
- Detect if AM_EXTRA_MAKEFLAGS=VERBOSE=1 was used with build scripts.
- Don't write warnings about replication variables when doing bootstrap.
- Fixed that mysql_cond_wait() and mysql_cond_timedwait() will report original source file in case of errors.
- Ignore some compiler warnings

BUILD/FINISH.sh:
  Detect if AM_EXTRA_MAKEFLAGS=VERBOSE=1 or --verbose was used
BUILD/SETUP.sh:
  Added --verbose to print out the full compile lines
  Updated help message
client/mysqltest.cc:
  Fixed that one can use 'replace' with cat_file
cmake/configure.pl:
  If --verbose is used, get make to write out compile commands
debian/dist/Debian/rules:
  Added $AM_EXTRA_MAKEFLAGS to get VERBOSE=1 on buildbot builds
debian/dist/Ubuntu/rules:
  Added $AM_EXTRA_MAKEFLAGS to get VERBOSE=1 on buildbot builds
include/my_pthread.h:
  Made set_timespec_time_nsec() more portable.
include/mysql/psi/mysql_thread.h:
  Fixed that mysql_cond_wait() and mysql_cond_timedwait() will report original source file in case of errors.
mysql-test/suite/innodb/r/auto_increment_dup.result:
  Fixed wrong DBUG_SYNC
mysql-test/suite/innodb/t/auto_increment_dup.test:
  Fixed wrong DBUG_SYNC
mysql-test/suite/perfschema/include/upgrade_check.inc:
  Make test more portable for changes in *.sql files
mysql-test/suite/perfschema/r/pfs_upgrade.result:
  Updated test results
mysql-test/valgrind.supp:
  Ignore running Aria checkpoint thread
scripts/mysqlaccess.sh:
  Changed reference of bugs database
  Ensure that also client-server group is read.
sql/handler.cc:
  Added missing syncpoint
sql/mysqld.cc:
  Don't write warnings about replication variables when doing bootstrap
sql/mysqld.h:
  Don't write warnings about replication variables when doing bootstrap
sql/rpl_rli.cc:
  Don't write warnings about replication variables when doing bootstrap
sql/sql_insert.cc:
  Don't mask SERVER_SHUTDOWN in insert_delayed
  This is done to be able to distingush between shutdown and interrupt errors
support-files/compiler_warnings.supp:
  Ignore some compiler warnings in xtradb,innobase, oqgraph, yassl, string3.h
2013-01-11 02:03:43 +02:00
unknown
03f28863e8 MDEV-26: Global transaction commit. Intermediate commit.
Now slave records GTID in mysql.rpl_slave_state when applying XID log event.
2012-11-05 15:01:49 +01:00
unknown
ab8e8f4b27 MDEV-500: Session variable for server_id
MDEV-26: Global transaction id, partial commit

Change server_id to be a session variable.
User with SUPER can set it to binlog with different server_id.
Implement backward-compatible ::server_id mirror for plugins.
2012-10-23 12:46:29 +02:00
Sergei Golubchik
0af4b6c6ee 5.5 merge 2013-01-29 15:10:47 +01:00
unknown
40bbf697aa MDEV-532: Async InnoDB commit checkpoint.
Make the commit checkpoint inside InnoDB be asynchroneous.
Implement a background thread in binlog to do the writing and flushing of
binlog checkpoint events to disk.
2012-12-14 15:38:07 +01:00
Michael Widenius
572560f38c Changed SHOW_FUNC variabels that don't return SHOW_ARRAY to SHOW_SIMPLE_FUNC.
This allows us to avoid calculating variables (including those involving mutex) that doesn't match the given
wildcard in SHOW STATUS LIKE '...'
Removed all references to active_mi that could cause problems for multi-source replication.

Added START|STOP ALL SLAVES
Added SHOW ALL SLAVES STATUS


include/mysql/plugin.h:
  Added SHOW_SIMPLE_FUNC
include/mysql/plugin_audit.h.pp:
  Updated .pp file
include/mysql/plugin_auth.h.pp:
  Updated .pp file
include/mysql/plugin_ftparser.h.pp:
  Updated .pp file
mysql-test/suite/multi_source/info_logs.result:
  New columns in SHOW ALL SLAVES STATUS
mysql-test/suite/multi_source/info_logs.test:
  Test new syntax
mysql-test/suite/multi_source/simple.result:
  New columns in SHOW ALL SLAVES STATUS
mysql-test/suite/multi_source/simple.test:
  test new syntax
mysql-test/suite/multi_source/syntax.result:
  Updated result
mysql-test/suite/multi_source/syntax.test:
  Test new syntax
mysql-test/suite/rpl/r/rpl_skip_replication.result:
  Updated result
plugin/semisync/semisync_master_plugin.cc:
  SHOW_FUNC -> SHOW_SIMPLE_FUNC
sql/item_create.cc:
  Simplify code
sql/lex.h:
  Added SLAVES keyword
sql/log.cc:
  Constant -> define
sql/log_event.cc:
  Added comment
sql/mysqld.cc:
  SHOW_FUNC -> SHOW_SIMPLE_FUNC
  Made slave_retried_trans, slave_received_heartbeats and heartbeat_period multi-source safe
  Clear variable denied_connections and slave_retried_transactions on startup
sql/mysqld.h:
  Added slave_retried_transactions
sql/rpl_mi.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  Added start_all_slaves() and stop_all_slaves()
sql/rpl_mi.h:
  Added prototypes
sql/rpl_rli.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  added executed_entries
sql/rpl_rli.h:
  Added executed_entries
sql/share/errmsg-utf8.txt:
  More and better error messages
sql/slave.cc:
  Added more fields to SHOW ALL SLAVES STATUS
  Added slave_retried_transactions
  Changed constants -> defines
sql/sql_class.h:
  Added comment
sql/sql_insert.cc:
  active_mi.rli -> thd->rli_slave
sql/sql_lex.h:
  Added SQLCOM_SLAVE_ALL_START & SQLCOM_SLAVE_ALL_STOP
sql/sql_load.cc:
  active_mi.rli -> thd->rli_slave
sql/sql_parse.cc:
  Added START|STOP ALL SLAVES
sql/sql_prepare.cc:
  Added SQLCOM_SLAVE_ALL_START & SQLCOM_SLAVE_ALL_STOP
sql/sql_reload.cc:
  Made REFRESH RELAY LOG multi-source safe
sql/sql_repl.cc:
  create_signed_file_name -> create_logfile_name_with_suffix
  Don't send my_ok() from start_slave() or stop_slave() so that we can call it for all master connections
sql/sql_show.cc:
  Compare wild cards early for all variables
sql/sql_yacc.yy:
  Added START|STOP ALL SLAVES
  Added SHOW ALL SLAVES STATUS
sql/sys_vars.cc:
  Made replicate_events_marked_for_skip,slave_net_timeout and rpl_filter multi-source safe.
sql/sys_vars.h:
  Simplify Sys_var_rpl_filter
2012-10-03 01:44:54 +03:00
Michael Widenius
8ac1b41cf3 Made max_relay_log_size depending on master connection.
Changed names of multi-source log files so that original suffixes are kept.


include/my_sys.h:
  Added fn_ext2(), which returns pointer to last '.' in file name
mysql-test/extra/rpl_tests/rpl_max_relay_size.test:
  Updated test
mysql-test/suite/multi_source/info_logs-master.opt:
  Test with strange file names
mysql-test/suite/multi_source/info_logs.result:
  Updated results
mysql-test/suite/multi_source/info_logs.test:
  Changed to test with complex names to be able to verify the filename generator code
mysql-test/suite/multi_source/relaylog_events.result:
  Updated results
mysql-test/suite/multi_source/reset_slave.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.result:
  Updated results
mysql-test/suite/multi_source/skip_counter.test:
  Added testing of max_relay_log_size
mysql-test/suite/rpl/r/rpl_row_max_relay_size.result:
  Updated results
mysql-test/suite/rpl/r/rpl_stm_max_relay_size.result:
  Updated results
mysql-test/suite/sys_vars/r/max_relay_log_size_basic.result:
  Updated results
mysql-test/suite/sys_vars/t/max_relay_log_size_basic.test:
  Updated results
mysys/mf_fn_ext.c:
  Added fn_ext2(), which returns pointer to last '.' in file name
sql/log.cc:
  Removed some wrong casts
sql/log.h:
  Updated comment to reflect new code
sql/log_event.cc:
  Updated DBUG_PRINT
sql/mysqld.cc:
  Added that max_relay_log_size copies it's values from max_binlog_size
sql/mysqld.h:
  Removed max_relay_log_size
sql/rpl_mi.cc:
  Changed names of multi-source log files so that original suffixes are kept.
sql/rpl_mi.h:
  Updated prototype
sql/rpl_rli.cc:
  Updated comment to reflect new code
  Made max_relay_log_size depending on master connection.
sql/rpl_rli.h:
  Made max_relay_log_size depending on master connection.
sql/set_var.h:
  Made option global so that one can check and change min & max values (sorry Sergei)
sql/sql_class.h:
  Made max_relay_log_size depending on master connection.
sql/sql_repl.cc:
  Updated calls to create_signed_file_name()
sql/sys_vars.cc:
  Made max_relay_log_size depending on master connection.
  Made old code more reusable
sql/sys_vars.h:
  Changed Sys_var_multi_source_uint to ulong to be able to handle max_relay_log_size
  Made old code more reusable
2012-10-01 02:30:44 +03:00
Michael Widenius
5a4b5869a0 Fixed issues found by QA (Elena)
- Added parameter to reset_logs() so that one can specify if new logs should be created.

mysql-test/include/setup_fake_relay_log.inc:
  There is no orphan relay log files anymore
mysql-test/mysql-test-run.pl:
  Added multi_source to test suite
mysql-test/suite/multi_source/info_logs.result:
  New test
mysql-test/suite/multi_source/info_logs.test:
  New test
mysql-test/suite/multi_source/my.cnf:
  Added log-warnings to get more information to the log files
mysql-test/suite/multi_source/relaylog_events.result:
  Added cleanup
mysql-test/suite/multi_source/relaylog_events.test:
  Added cleanup
mysql-test/suite/multi_source/reset_slave.result:
  Updated results after improved RESET SLAVE
mysql-test/suite/multi_source/simple.result:
  Updated results after improved RESET SLAVE
mysql-test/suite/multi_source/simple.test:
  Syncronize positions before show full slave status
mysql-test/suite/rpl/r/rpl_row_show_relaylog_events.result:
  Updated results after improved RESET SLAVE (we now use less relay log files)
mysql-test/suite/rpl/r/rpl_stm_mix_show_relaylog_events.result:
  Updated results after improved RESET SLAVE (we now use less relay log files)
sql/log.cc:
  Added parameter to reset_logs() so that one can specify if new logs should be created.
sql/log.h:
  Added parameter to reset_logs()
sql/rpl_mi.cc:
  Create Master_info_index::index_file_names once at init
  More DBUG_PRINT
  Give error if Master_info_index::check_duplicate_master_info fails
sql/rpl_rli.cc:
  If we do a full reset, don't create any new relay log files.
sql/share/errmsg-utf8.txt:
  Improved error message if connection exists
sql/sql_parse.cc:
  Fixed memory leak
sql/sql_repl.cc:
  check_duplicate_master_info() now generates an error
  Added parameter to reset_logs()
2012-09-28 21:22:24 +03:00
Michael Widenius
1864d9596d Implementation of Multi-source replication (MDEV:253)
Documentation of the feature can be found at: http://kb.askmonty.org/en/multi-source-replication/
This code is based on code from Taobao, developed by Plinux

BUILD/SETUP.sh:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
client/mysqltest.cc:
  Added support for error names starting with 'W'
  Added connection_name support to --sync_with_master
cmake/maintainer.cmake:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
mysql-test/r/mysqltest.result:
  Updated results
mysql-test/r/parser.result:
  Updated results
mysql-test/suite/multi_source/my.cnf:
  Setup of multi-master tests
mysql-test/suite/multi_source/simple.result:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/simple.test:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/syntax.result:
  Test of multi-source syntax
mysql-test/suite/multi_source/syntax.test:
  Test of multi-source syntax
mysql-test/suite/rpl/r/rpl_rotate_logs.result:
  Updated results because of new error messages
mysql-test/t/parser.test:
  Updated test as master_pos_wait() now takes more arguments than before
sql/event_scheduler.cc:
  No reason to initialize slave_thread (it's guaranteed to be zero here)
sql/item_create.cc:
  Added connection_name argument to master_pos_wait()
  Simplified code
sql/item_func.cc:
  Added connection_name argument to master_pos_wait()
sql/item_func.h:
  Added connection_name argument to master_pos_wait()
sql/log.cc:
  Added tag "Master 'connection_name'" to slave errors that has a connection name.
sql/mysqld.cc:
  Added variable mysqld_server_initialized so that other functions can test if server is fully initialized.
  Free all slave data in one place (fewer ifdef's)
  Removed not needed call to close_active_mi()
  Initialize slaves() later in startup to ensure that everthing is really initialized when slaves start.
  Made status variable slave_running multi-source safe
sql/mysqld.h:
  Added mysqld_server_initialized
sql/rpl_mi.cc:
  Store connection name and cmp_connection_name (only used for show full slave status) in Master_info
  Added code for Master_info_index, which handles storage of multi-master information
  Don't write the empty "" connection_name to multi-master.info file. This is handled by the original code.
sql/rpl_mi.h:
  Added connection_name and Master_info_index
sql/rpl_rli.cc:
  Added connection_name to relay log files.
sql/rpl_rli.h:
  Fixed type of slave_skip_counter as we now access it directly in sys_vars.cc, so it must be uint
sql/share/errmsg-utf8.txt:
  Added new error messages needed for multi-source
  Added multi-source name to error ER_MASTER_INFO and WARN_NO_MASTER_INFO
sql/slave.cc:
  Moved things a bit around to make it easier to handle error conditions.
  Create a global master_info_index and add the "" connection to it
  Ensure that new Master_info doesn't fail.
  Don't call terminate_slave_threads(active_mi..) on end_slave() as this is now done automaticly when deleting master_info_index.
  Delete not needed function close_active_mi(). One can achive same thing by calling end_slave().
  Added support for SHOW FULL SLAVE STATUS (show status for all master connections with connection_name as first column)
sql/slave.h:
  Added new prototypes
sql/sql_base.cc:
  More DBUG_PRINT
sql/sql_class.cc:
  Reset thd->connection_name and thd-->default_master_connection
sql/sql_class.h:
  Added thd->connection_name and thd-->default_master_connection
  Added slave_skip_count to variables to make changing the @@sql_slave_skip_count variable thread safe
sql/sql_const.h:
  Added MAX_CONNECTION_NAME
sql/sql_lex.cc:
  Reset 'lex->verbose' (to simplify some sql_yacc.yy code)
sql/sql_lex.h:
  Added connection_name
sql/sql_parse.cc:
  Added support for connection_name to all SLAVE commands.
  - Instead of using active_mi, we now get the current Master_info from master_info_index.
  - Create new replication threads with CHANGE MASTER
  - Added support for show_all_master_info()
sql/sql_reload.cc:
  Made reset/full slave use master_info_index->get_master_info() instead of active_mi.
  If one uses 'RESET SLAVE "connection_name" all' the connection is removed from master_info_index.
sql/sql_repl.cc:
  sql_slave_skip_counter is moved to thd->variables to make it thread safe and fix some bugs with it
  Add connection name to relay log files.
  Added connection name to errors.
  Added some logging for multi-master if log_warnings > 1
  stop_slave():
  - Don't check if thd is set. It's guaranteed to always be set.
  change_master():
  - Check for duplicate connection names in change_master()
  - Check for wrong arguments first in file (to simplify error handling)
  - Register new connections in master_info_index
sql/sql_yacc.yy:
  Added optional connection_name to a all relevant master/slave commands
sql/strfunc.cc:
  my_global.h shoud always be included first.
sql/sys_vars.cc:
  Added variable default_master_connection
  Made variable sql_slave_skip_counter multi-source safe
sql/sys_vars.h:
  Added Sys_var_session_lexstring (needed for default_master_connection)
  Added Sys_var_multi_source_uint (needed for sql_slave_skip_counter).
2012-09-28 02:06:56 +03:00
unknown
288eeb3a31 MDEV-232: Remove one fsync() from commit phase.
Introduce a new storage engine API method commit_checkpoint_request().
This is used to replace the fsync() at the end of every storage engine
commit with a single fsync() when a binlog is rotated.

Binlog rotation is now done during group commit instead of being
delayed until unlog(), removing some server stall and avoiding an
expensive lock/unlock of LOCK_log inside unlog().
2012-09-13 14:31:29 +02:00
Sergei Golubchik
0522307ed1 mysql-5.5 merge 2012-06-14 20:05:31 +02:00
Sergei Golubchik
431e042b5d c 2012-05-21 15:30:25 +02:00
Joerg Bruehe
5be07ceadd Merge 5.5.24 back into main 5.5.
This is a weave merge, but without any conflicts.
In 14 source files, the copyright year needed to be updated to 2012.
2012-05-07 22:20:42 +02:00
Andrei Elkin
14de6de946 merge bug11754117-45670 fixes from 5.1. 2012-04-21 13:24:39 +03:00
Andrei Elkin
49e484c8cd BUG#11754117 incorrect logging of INSERT into auto-increment
BUG#11761686 insert_id event is not filtered.
  
Two issues are covered.
  
INSERT into autoincrement field which is not the first part in the composed primary key 
is unsafe by autoincrement logging design. The case is specific to MyISAM engine
because Innodb does not allow such table definition.
  
However no warnings and row-format logging in the MIXED mode was done, and
that is fixed.
  
Int-, Rand-, User-var log-events were not filtered along with their parent
query that made possible them to screw up execution context of the following
query.
  
Fixed with deferring their execution until the parent query.

******
Bug#11754117 

Post review fixes.

mysql-test/suite/rpl/r/rpl_auto_increment_bug45679.result:
  a new result file is added.
mysql-test/suite/rpl/r/rpl_filter_tables_not_exist.result:
  results updated.
mysql-test/suite/rpl/t/rpl_auto_increment_bug45679.test:
  regression test for BUG#11754117-45670 is added.
mysql-test/suite/rpl/t/rpl_filter_tables_not_exist.test:
  regression test for filtering issue of BUG#11754117 - 45670 is added.
sql/log_event.cc:
  Logics are added for deferring and executing events associated 
  with the Query event.
sql/log_event.h:
  Interface to deferred events batch execution is added.
sql/rpl_rli.cc:
  initialization for new RLI members is added.
sql/rpl_rli.h:
  New members to RLI are added to facilitate deferred events gathering
  and execution control;
  two general character RLI cleanup methods are constructed.
sql/rpl_utility.cc:
  Deferred_log_events methods are difined.
sql/rpl_utility.h:
  A new class Deferred_log_events is defined to implement
  IRU events gathering, execution and cleanup.
sql/slave.cc:
  Necessary changes to initialize `rli->deferred_events' and prevent
  deferred event deletion in the main read-exec branch.
sql/sql_base.cc:
  A new safe-check function for multi-part pk with auto-increment is defined
  and deployed in lock_tables().
sql/sql_class.cc:
  Initialization for a new member and replication cleanups are added
  to THD class.
sql/sql_class.h:
  THD class receives a new member to hold a specific execution
  context for slave applier.
sql/sql_parse.cc:
  Execution of the deferred event in started prior to its parent query.
2012-04-20 19:41:20 +03:00
Sergei Golubchik
20e706689d mysql-5.5.22 merge
mysql-test/suite/innodb/t/group_commit_crash.test:
  remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
  remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
  a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
  my_vsnprintf() is ok here, in 5.5
2012-03-28 01:04:46 +02:00
Sergei Golubchik
4933d21e5d merge with mysql-5.5.21 2012-03-09 08:06:59 +01:00
Luis Soares
a33079f854 BUG#13693012
Automerged approved cset.
2012-02-24 16:32:46 +00:00
Luis Soares
580664b2c3 Bug#13693012: SLAVE CRASHING ON INSERT STATEMENT WITH MERGE TABLE
PROBLEM: After WL 4144, when using MyISAM Merge tables, the routine
open_and_lock_tables will append to the list of tables to lock, the
base tables that make up the MERGE table. This has two side-effects in
replication: 

  1. On the master side, we log additional table maps for the base
     tables, since they appear in the list of locked tables, even
     though we don't really use them at the slave.

  2. On the slave side, when opening a MERGE table while applying a
     ROW event, additional tables are appended to the list of tables
     to lock.

Side-effect #1 is not harmful. It's just that when using MyISAM Merge
tables a few table maps more may be logged.

Side-effect #2, is harmful, because the list rli->tables_to_lock is an
extended structure from TABLE_LIST in which the extra fields are
filled from the table maps that are processed. Since
open_and_lock_tables appends tables to the list after all table map
events have been processed we end up with entries without
replication/table map data on them. Thus when trying to access that
info for these extra tables, the server will crash.

SOLUTION: We fix side-effect #2 by making sure that we access the
replication part of the structure for those in the list that were
accounted for when processing the correspondent table map events. All
in all, we never go beyond rli->tables_to_lock_count.

We also deploy an assertion when clearing rli->tables_to_lock, making
sure that the base tables are not in the list anymore (were closed in
close_thread_tables).
2012-02-24 16:07:43 +00:00
MySQL Build Team
7a35cb9150 Updated/added copyright headers 2012-02-16 10:48:16 +01:00
Manish Kumar
e69da6dc3e BUG#11752315 - 43460: STOP SLAVE UNABLE TO COMPLETE WHEN SLAVE THREAD IS TRYING TO RECONNECT TO
Problem : The basic problem is the way the thread sleeps in mysql-5.5 and also in mysql-5.1
          when we execute a stop slave on windows platform.
          On windows platform if the stop slave is executed after the master dies, we have 
          this long wait before the stop slave return a value. This is because there is a 
          sleep of the thread. The sleep is uninterruptable in the two above version,
          which was fixed by Davi patch for the BUG#11765860 for mysql-trunk. Backporting 
          his patch for mysql-5.5 fixes the problem. 

Solution : A new pair of mutex and condition variable is introduced to synchronize thread 
           sleep and finalization. A new mutex is required because the slave threads are 
           terminated while holding the slave thread locks (run_lock), which can not be 
           relinquished during termination as this would affect the lock order.

mysql-test/suite/rpl/r/rpl_start_stop_slave.result:
  The result file associated with the test added.
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
  A test to check the new functionality.
sql/rpl_mi.cc:
  The constructor using the new mutex and condition variables for the master_info.
sql/rpl_mi.h:
  The condition variable and mutex have been added for the master_info.
sql/rpl_rli.cc:
  The constructor using the new mutex and condition variables for the realy_log_info.
sql/rpl_rli.h:
  The condition variable and mutex have been added for the relay_log_info.
sql/slave.cc:
  Use a timed wait on a condition variable to implement a interruptible sleep. 
  The wait is registered with the THD object so that the thread will be woken 
  up if killed.
2012-01-23 17:39:37 +05:30
Sergei Golubchik
38e3ae155d mysql-5.5 merge 2012-01-16 20:16:35 +01:00
Sergei Golubchik
4f435bddfd 5.3 merge 2012-01-13 15:50:02 +01:00
Michael Widenius
6d4224a31c Merge with 5.2.
no_error handling for select (used by INSERT ... SELECT) still needs to be fixed, but I will do that in a separate commit
2011-12-11 11:34:44 +02:00
Michael Widenius
6920457142 Merge with MariaDB 5.1 2011-11-24 18:48:58 +02:00
Michael Widenius
a8d03ab235 Initail merge with MySQL 5.1 (XtraDB still needs to be merged)
Fixed up copyright messages.
2011-11-21 19:13:14 +02:00
Luis Soares
8fe4023e51 BUG#11760927
Follow-up patch to fix valgrind warnings.
2011-11-16 01:18:03 +00:00
Luis Soares
8851022f41 BUG#11760927: 53375: RBR + NO PK => HIGH LOAD ON SLAVE (TABLE
SCAN/CPU) => SLAVE FAILURE

When a statement containing a large amount of ROWs to be applied on
the slave, and the slave's table does not contain a PK, it can take a
considerable amount of time to find and change all the rows that are
to be changed.

The proper slave enhancement will be implemented in WL 5597. However,
in this bug we are making it clear to the user what the problem is, by
printing a message to the error log if the execution time, for a given
statement in RBR, takes more than LONG_FIND_ROW_THRESHOLD (set to 60
seconds). This shall help the DBA to diagnose what's happening when
facing a slave server that is quiet for no apparent reason...

The note is only printed to the error log if log_warnings is set to be
greater than 1.


sql/log_event.cc:
  Core of the patch. 
  
  In Rows_log_event::do_apply_event, sets STMT start 
  timestamp.
  
  In Rows_log_event::find_row, if a PK is not used, then the start 
  timestamp is checked to see if the time spent on this STMT is enough
  to justify the printing of a note (only if it was not printed before).
sql/log_event.h:
  Defining LONG_FIND_ROW_THRESHOLD.
sql/rpl_rli.cc:
  Resets long_find_row state in rli->context_cleanup().
sql/rpl_rli.h:
  Two new rli properties that are necessary to control when to
  emit a note in the error log: 1) timestamp that states when the
  ROW statement started; 2) flag indicating whether the note has
  been emitted for the current statement or not. Also deployed
  accessors.
2011-11-11 17:26:56 +00:00
Sergei Golubchik
6edd76785c merge 2011-11-03 23:39:53 +01:00
Sergei Golubchik
0e007344ea mysql-5.5.18 merge 2011-11-03 19:17:05 +01:00
unknown
5da52cd108 Some after-merge fixes for 5.5 merge. 2011-10-25 12:53:40 +02:00
Sergei Golubchik
76f0b94bb0 merge with 5.3
sql/sql_insert.cc:
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
  ******
  CREATE ... IF NOT EXISTS may do nothing, but
  it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
  small cleanup
  ******
  small cleanup
2011-10-19 21:45:18 +02:00
Sergei Golubchik
9809f05199 5.5-merge 2011-07-02 22:08:51 +02:00
Kent Boortz
68f00a5686 Updated/added copyright headers 2011-06-30 17:37:13 +02:00
Michael Widenius
f197991f41 Merge with 5.1-microseconds
A lot of small fixes and new test cases.

client/mysqlbinlog.cc:
  Cast removed
client/mysqltest.cc:
  Added missing DBUG_RETURN
include/my_pthread.h:
  set_timespec_time_nsec() now only takes one argument
mysql-test/t/date_formats.test:
  Remove --disable_ps_protocl as now also ps supports microseconds
mysys/my_uuid.c:
  Changed to use my_interval_timer() instead of my_getsystime()
mysys/waiting_threads.c:
  Changed to use my_hrtime()
sql/field.h:
  Added bool special_const_compare() for fields that may convert values before compare (like year)
sql/field_conv.cc:
  Added test to get optimal copying of identical temporal values.
sql/item.cc:
  Return that item_int is equal if it's positive, even if unsigned flag is different.
  Fixed Item_cache_str::save_in_field() to have identical null check as other similar functions
  Added proper NULL check to Item_cache_int::save_in_field()
sql/item_cmpfunc.cc:
  Don't call convert_constant_item() if there is nothing that is worth converting.
  Simplified test when years should be converted
sql/item_sum.cc:
  Mark cache values in Item_sum_hybrid as not constants to ensure they are not replaced by other cache values in compare_datetime()
sql/item_timefunc.cc:
  Changed sec_to_time() to take a my_decimal argument to ensure we don't loose any sub seconds.
  Added Item_temporal_func::get_time() (This simplifies some things)
sql/mysql_priv.h:
  Added Lazy_string_decimal()
sql/mysqld.cc:
  Added my_decimal constants max_seconds_for_time_type, time_second_part_factor
sql/table.cc:
  Changed expr_arena to be of type CONVENTIONAL_EXECUTION to ensure that we don't loose any items that are created by fix_fields()
sql/tztime.cc:
  TIME_to_gmt_sec() now sets *in_dst_time_gap in case of errors
  This is needed to be able to detect if timestamp is 0
storage/maria/lockman.c:
  Changed from my_getsystime() to set_timespec_time_nsec()
storage/maria/ma_loghandler.c:
  Changed from my_getsystime() to my_hrtime()
storage/maria/ma_recovery.c:
  Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/maria/unittest/trnman-t.c:
  Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/xtradb/handler/ha_innodb.cc:
  Added support for new time,datetime and timestamp
unittest/mysys/thr_template.c:
  my_getsystime() -> my_interval_timer()
unittest/mysys/waiting_threads-t.c:
  my_getsystime() -> my_interval_timer()
2011-05-28 05:11:32 +03:00