Commit graph

22440 commits

Author SHA1 Message Date
Michael Widenius
1864d9596d Implementation of Multi-source replication (MDEV:253)
Documentation of the feature can be found at: http://kb.askmonty.org/en/multi-source-replication/
This code is based on code from Taobao, developed by Plinux

BUILD/SETUP.sh:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
client/mysqltest.cc:
  Added support for error names starting with 'W'
  Added connection_name support to --sync_with_master
cmake/maintainer.cmake:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
mysql-test/r/mysqltest.result:
  Updated results
mysql-test/r/parser.result:
  Updated results
mysql-test/suite/multi_source/my.cnf:
  Setup of multi-master tests
mysql-test/suite/multi_source/simple.result:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/simple.test:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/syntax.result:
  Test of multi-source syntax
mysql-test/suite/multi_source/syntax.test:
  Test of multi-source syntax
mysql-test/suite/rpl/r/rpl_rotate_logs.result:
  Updated results because of new error messages
mysql-test/t/parser.test:
  Updated test as master_pos_wait() now takes more arguments than before
sql/event_scheduler.cc:
  No reason to initialize slave_thread (it's guaranteed to be zero here)
sql/item_create.cc:
  Added connection_name argument to master_pos_wait()
  Simplified code
sql/item_func.cc:
  Added connection_name argument to master_pos_wait()
sql/item_func.h:
  Added connection_name argument to master_pos_wait()
sql/log.cc:
  Added tag "Master 'connection_name'" to slave errors that has a connection name.
sql/mysqld.cc:
  Added variable mysqld_server_initialized so that other functions can test if server is fully initialized.
  Free all slave data in one place (fewer ifdef's)
  Removed not needed call to close_active_mi()
  Initialize slaves() later in startup to ensure that everthing is really initialized when slaves start.
  Made status variable slave_running multi-source safe
sql/mysqld.h:
  Added mysqld_server_initialized
sql/rpl_mi.cc:
  Store connection name and cmp_connection_name (only used for show full slave status) in Master_info
  Added code for Master_info_index, which handles storage of multi-master information
  Don't write the empty "" connection_name to multi-master.info file. This is handled by the original code.
sql/rpl_mi.h:
  Added connection_name and Master_info_index
sql/rpl_rli.cc:
  Added connection_name to relay log files.
sql/rpl_rli.h:
  Fixed type of slave_skip_counter as we now access it directly in sys_vars.cc, so it must be uint
sql/share/errmsg-utf8.txt:
  Added new error messages needed for multi-source
  Added multi-source name to error ER_MASTER_INFO and WARN_NO_MASTER_INFO
sql/slave.cc:
  Moved things a bit around to make it easier to handle error conditions.
  Create a global master_info_index and add the "" connection to it
  Ensure that new Master_info doesn't fail.
  Don't call terminate_slave_threads(active_mi..) on end_slave() as this is now done automaticly when deleting master_info_index.
  Delete not needed function close_active_mi(). One can achive same thing by calling end_slave().
  Added support for SHOW FULL SLAVE STATUS (show status for all master connections with connection_name as first column)
sql/slave.h:
  Added new prototypes
sql/sql_base.cc:
  More DBUG_PRINT
sql/sql_class.cc:
  Reset thd->connection_name and thd-->default_master_connection
sql/sql_class.h:
  Added thd->connection_name and thd-->default_master_connection
  Added slave_skip_count to variables to make changing the @@sql_slave_skip_count variable thread safe
sql/sql_const.h:
  Added MAX_CONNECTION_NAME
sql/sql_lex.cc:
  Reset 'lex->verbose' (to simplify some sql_yacc.yy code)
sql/sql_lex.h:
  Added connection_name
sql/sql_parse.cc:
  Added support for connection_name to all SLAVE commands.
  - Instead of using active_mi, we now get the current Master_info from master_info_index.
  - Create new replication threads with CHANGE MASTER
  - Added support for show_all_master_info()
sql/sql_reload.cc:
  Made reset/full slave use master_info_index->get_master_info() instead of active_mi.
  If one uses 'RESET SLAVE "connection_name" all' the connection is removed from master_info_index.
sql/sql_repl.cc:
  sql_slave_skip_counter is moved to thd->variables to make it thread safe and fix some bugs with it
  Add connection name to relay log files.
  Added connection name to errors.
  Added some logging for multi-master if log_warnings > 1
  stop_slave():
  - Don't check if thd is set. It's guaranteed to always be set.
  change_master():
  - Check for duplicate connection names in change_master()
  - Check for wrong arguments first in file (to simplify error handling)
  - Register new connections in master_info_index
sql/sql_yacc.yy:
  Added optional connection_name to a all relevant master/slave commands
sql/strfunc.cc:
  my_global.h shoud always be included first.
sql/sys_vars.cc:
  Added variable default_master_connection
  Made variable sql_slave_skip_counter multi-source safe
sql/sys_vars.h:
  Added Sys_var_session_lexstring (needed for default_master_connection)
  Added Sys_var_multi_source_uint (needed for sql_slave_skip_counter).
2012-09-28 02:06:56 +03:00
Michael Widenius
620d14f8c3 Automatic merge 2012-09-22 17:11:40 +03:00
Michael Widenius
3e83c4e8f9 Automatic merge 2012-09-22 15:30:24 +03:00
Michael Widenius
6f57fc5054 Automatic merge 2012-09-18 15:32:08 +03:00
Michael Widenius
ae5bc05988 Fix for MDEV-533: Confusing error code when doing auto-increment insert for out-of-range values
create table t1 (a smallint primary key auto_increment);
insert into t1 values(32767);
insert into t1 values(NULL);
ERROR 1062 (23000): Duplicate entry '32767' for key 'PRIMARY

Now on always gets error HA_ERR_AUTOINC_RANGE=167 "Out of range value for column", independent of
store engine, SQL Mode or number of inserted rows. This is an unique error that is easier to test for in replication.

Another bug fix is that we now get an error when trying to insert a too big auto-generated value, even in non-strict mode.
Before one get insted the max column value inserted.
This patch also fixes some issues with inserting negative numbers in an auto-increment column.
Fixed the ER_DUP_ENTRY and HA_ERR_AUTOINC_ERANGE are compared the same between master and slave.
This ensures that replication works between an old server to a new slave for auto-increment overflow errors.
Added SQLSTATE errors for handler errors

Smaller bug fixes:
* Added warnings for duplicate key errors when using INSERT IGNORE
* Fixed bug when using --skip-log-bin followed by --log-bin, which did set log-bin to "0"
* Allow one to see how cmake is called by using --just-print --just-configure


BUILD/FINISH.sh:
  --just-print --just-configure now shows how cmake would be invoked. Good for understanding parameters to cmake.
cmake/configure.pl:
  --just-print --just-configure now shows how cmake would be invoked. Good for understanding parameters to cmake.
include/CMakeLists.txt:
  Added handler_state.h
include/handler_state.h:
  SQLSTATE for handler error messages.
  Required for HA_ERR_AUTOINC_ERANGE, but solves also some other cases.
mysql-test/extra/binlog_tests/binlog.test:
  Fixed old wrong behaviour
  Added more tests
mysql-test/extra/binlog_tests/binlog_insert_delayed.test:
  Reset binary log to only print what's necessary in show_binlog_events
mysql-test/extra/rpl_tests/rpl_auto_increment.test:
  Update to new error codes
mysql-test/extra/rpl_tests/rpl_insert_delayed.test:
  Ignore warnings as this depends on how the test is run
mysql-test/include/strict_autoinc.inc:
  On now gets an error on overflow
mysql-test/r/auto_increment.result:
  Update results after fixing error message
mysql-test/r/auto_increment_ranges_innodb.result:
  Test new behaviour
mysql-test/r/auto_increment_ranges_myisam.result:
  Test new behaviour
mysql-test/r/commit_1innodb.result:
  Added warnings for duplicate key error
mysql-test/r/create.result:
  Added warnings for duplicate key error
mysql-test/r/insert.result:
  Added warnings for duplicate key error
mysql-test/r/insert_select.result:
  Added warnings for duplicate key error
mysql-test/r/insert_update.result:
  Added warnings for duplicate key error
mysql-test/r/mix2_myisam.result:
  Added warnings for duplicate key error
mysql-test/r/myisam_mrr.result:
  Added warnings for duplicate key error
mysql-test/r/null_key.result:
  Added warnings for duplicate key error
mysql-test/r/replace.result:
  Update to new error codes
mysql-test/r/strict_autoinc_1myisam.result:
  Update to new error codes
mysql-test/r/strict_autoinc_2innodb.result:
  Update to new error codes
mysql-test/r/strict_autoinc_3heap.result:
  Update to new error codes
mysql-test/r/trigger.result:
  Added warnings for duplicate key error
mysql-test/r/xtradb_mrr.result:
  Added warnings for duplicate key error
mysql-test/suite/binlog/r/binlog_innodb_row.result:
  Updated result
mysql-test/suite/binlog/r/binlog_row_binlog.result:
  Out of range data for auto-increment is not inserted anymore
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
  Updated result
mysql-test/suite/binlog/r/binlog_stm_binlog.result:
  Out of range data for auto-increment is not inserted anymore
mysql-test/suite/binlog/r/binlog_unsafe.result:
  Updated result
mysql-test/suite/innodb/r/innodb-autoinc.result:
  Update to new error codes
mysql-test/suite/innodb/r/innodb-lock.result:
  Updated results
mysql-test/suite/innodb/r/innodb.result:
  Updated results
mysql-test/suite/innodb/r/innodb_bug56947.result:
  Updated results
mysql-test/suite/innodb/r/innodb_mysql.result:
  Updated results
mysql-test/suite/innodb/t/innodb-autoinc.test:
  Update to new error codes
mysql-test/suite/maria/maria3.result:
  Updated result
mysql-test/suite/maria/mrr.result:
  Updated result
mysql-test/suite/optimizer_unfixed_bugs/r/bug43617.result:
  Updated result
mysql-test/suite/rpl/r/rpl_auto_increment.result:
  Update to new error codes
mysql-test/suite/rpl/r/rpl_insert_delayed,stmt.rdiff:
  Updated results
mysql-test/suite/rpl/r/rpl_loaddatalocal.result:
  Updated results
mysql-test/t/auto_increment.test:
  Update to new error codes
mysql-test/t/auto_increment_ranges.inc:
  Test new behaviour
mysql-test/t/auto_increment_ranges_innodb.test:
  Test new behaviour
mysql-test/t/auto_increment_ranges_myisam.test:
  Test new behaviour
mysql-test/t/replace.test:
  Update to new error codes
mysys/my_getopt.c:
  Fixed bug when using --skip-log-bin followed by --log-bin, which did set log-bin to "0"
sql/handler.cc:
  Ignore negative values for signed auto-increment columns
  Always give an error if we get an overflow for an auto-increment-column (instead of inserting the max value)
  Ensure that the row number is correct for the out-of-range-value error message.
  
  
  ******
  Fixed wrong printing of column namn for "Out of range value" errors
  Fixed that INSERT_ID is correctly replicated also for out-of-range autoincrement values
  Fixed that print_keydup_error() can also be used to generate warnings
  ******
  Return HA_ERR_AUTOINC_ERANGE (167) instead of ER_WARN_DATA_OUT_OF_RANGE for auto-increment overflow
sql/handler.h:
  Allow INSERT IGNORE to continue also after out-of-range inserts.
  Fixed that print_keydup_error() can also be used to generate warnings
sql/log_event.cc:
  Added DBUG_PRINT
  Fixed the ER_AUTOINC_READ_FAILED, ER_DUP_ENTRY and HA_ERR_AUTOINC_ERANGE are compared the same between master and slave.
  This ensures that replication works between an old server to a new slave for auto-increment overflow errors.
sql/sql_insert.cc:
  Add warnings for duplicate key errors when using INSERT IGNORE
sql/sql_state.c:
  Added handler errors
sql/sql_table.cc:
  Update call to print_keydup_error()
storage/innobase/handler/ha_innodb.cc:
  Fixed increment handling of auto-increment columns to be consistent with rest of MariaDB.
storage/xtradb/handler/ha_innodb.cc:
  Fixed increment handling of auto-increment columns to be consistent with rest of MariaDB.
2012-09-18 15:14:19 +03:00
unknown
caf31114c8 Merged the fix for bug lp:1009187, mdev-373 2012-09-18 13:42:06 +03:00
unknown
0bc89929ef - Merged the fix for bug lp:1009187, mdev-373.
- Performed some refactoring and simplification that was enabled and required by the merge.
2012-09-17 11:13:46 +03:00
unknown
b917fb63a6 Fix bug lp:1009187, mdev-373, mysql bug#58628
Analysis:
The queries in question use the [unique | index]_subquery execution methods.
These methods reuse the ref keys constructed by create_ref_for_key(). The
way create_ref_for_key() works is that it doesn't store in ref.key_copy[]
store_key elements that represent constants. In particular it doesn't store
the store_key for NULL constants.

The execution of [unique | index]_subquery calls
subselect_uniquesubquery_engine::copy_ref_key, which in addition to copy
the left IN argument into a index lookup key, is supposed to detect if
the left IN argument contains NULLs. Since the store_key for the NULL
constant is not copied into the key array, the null is not detected, and
execution erroneously proceeds as if it should look for a complete match.

Solution:
The solution (unlike MySQL) is to reuse already computed information about
NULL presence. Item_in_optimizer::val_int already finds out if the left IN
operand contains NULLs. The fix propagates this to the execution methods
subselect_[unique | index]subquery_engine::exec so it knows if there were
NULL values independent of the presence of keys.

In addition the patch siplifies copy_ref_key() and the logic that hanldes
the case of NULLs in the left IN operand.
2012-09-14 11:26:01 +03:00
unknown
288eeb3a31 MDEV-232: Remove one fsync() from commit phase.
Introduce a new storage engine API method commit_checkpoint_request().
This is used to replace the fsync() at the end of every storage engine
commit with a single fsync() when a binlog is rotated.

Binlog rotation is now done during group commit instead of being
delayed until unlog(), removing some server stall and avoiding an
expensive lock/unlock of LOCK_log inside unlog().
2012-09-13 14:31:29 +02:00
Michael Widenius
6f94b5c76d Fixed random test failure
mysql-test/include/index_merge2.inc:
  InnoDB did report 9 or 7 rows in explain
2012-09-10 17:26:54 +03:00
Michael Widenius
3a793b9d4d Added new status variables:
feature_dynamic_columns,feature_fulltext,feature_gis,feature_locale,feature_subquery,feature_timezone,feature_trigger,feature_xml
Opened_views, Executed_triggers, Executed_events
Added new process status 'updating status' as part of 'freeing items'

mysql-test/r/features.result:
  Test of feature_xxx status variables
mysql-test/r/mysqld--help.result:
  Removed duplicated 'language' variable.
mysql-test/r/view.result:
  Test of opened_views
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
  Write more information on failure
mysql-test/t/features.test:
  Test of feature_xxx status variables
mysql-test/t/view.test:
  Test of opened_views
sql/event_scheduler.cc:
  Increment executed_events status variable
sql/field.cc:
  Increment status variable
sql/item_func.cc:
  Increment status variable
sql/item_strfunc.cc:
  Increment status variable
sql/item_subselect.cc:
  Increment status variable
sql/item_xmlfunc.cc:
  Increment status variable
sql/mysqld.cc:
  Add new status variables to 'show status'
sql/mysqld.h:
  Added executed_events
sql/sql_base.cc:
  Increment status variable
sql/sql_class.h:
  Add new status variables
sql/sql_parse.cc:
  Added new process status 'updating status' as part of 'freeing items'
sql/sql_trigger.cc:
  Increment status variable
sql/sys_vars.cc:
  Increment status variable
sql/tztime.cc:
  Increment status variable
2012-09-09 01:22:06 +03:00
Michael Widenius
5161b3ddde Automatic merge 2012-09-09 00:38:15 +03:00
unknown
caedd1992c merge 5.3->5.5 2012-09-06 00:14:33 +03:00
unknown
54bb28d4a1 MDEV-486 LP BUG#1010116 fix.
Link view/derived table fields to a real table to check turning the table record to null row.

Item_direct_view_ref wrapper now checks if table is turned to null row.
2012-09-05 23:23:58 +03:00
Michael Widenius
d618dfe32d Added function last_value() which returns the last value but evalutes all arguments as a side effect.
Original patch by Eric Herman

client/mysql.cc:
  Added LAST_VALUE
mysql-test/r/last_value.result:
  Testing of LAST_VALUE
mysql-test/t/last_value.test:
  Testing of LAST_VALUE
sql/item_func.cc:
  Added LAST_VALUE()
sql/item_func.h:
  Added LAST_VALUE()
sql/lex.h:
  Added LAST_VALUE()
sql/sql_yacc.yy:
  Added LAST_VALUE()
2012-09-05 18:23:51 +03:00
Igor Babaev
bc644de9a9 Merge MDEV-415 -> 10.0-base. 2012-09-01 19:41:38 -07:00
Igor Babaev
a6b88f1431 MDEV-415: Back-port of the WL task #1393 from the mysql-5.6 code line.
The task adds a more efficient handling of the queries with 
ORDER BY order LIMIT n, such that n is small enough and
no indexes are used for order.
2012-09-01 14:21:59 -07:00
Michael Widenius
1999be8d4e Automatic merge with 5.5 2012-09-01 00:54:54 +03:00
Alexey Botchkov
589c62fefe Bug #1043845 st_distance() results are incorrect depending on variable order.
Autointersections of an object were treated as nodes, so the wrong result.

per-file comments:
  mysql-test/r/gis.result
Bug #1043845 st_distance() results are incorrect depending on variable order.
        test result updated.
  mysql-test/t/gis.test
Bug #1043845 st_distance() results are incorrect depending on variable order.
        test case added.
  sql/item.cc
        small fix to make compilers happy.
  sql/item_geofunc.cc
Bug #1043845 st_distance() results are incorrect depending on variable order.
        Skip intersection points when calculate distance.
2012-08-31 19:50:45 +05:00
Sergei Golubchik
59408093d1 5.3 merge 2012-08-31 14:15:52 +02:00
Sergei Golubchik
a1fd37b1fd MDEV-414 Depending on indexes or execution plans, a warning on incorrect or out of range values in WHERE condition is sometimes produced and sometimes not
use the same method that disables warnings in all relevant places, remove redundant function
2012-08-31 13:03:41 +02:00
Sergei Golubchik
0536c506ff MDEV-437 Microseconds: In time functions precision is calculated modulo 256
store the precision in uint, not uint8
2012-08-30 09:05:27 +02:00
Sergei Golubchik
3444e8e925 MDEV-454 Addition of a time interval reduces the resulting value
1. Field_newdate::get_date should refuse to return a date with zeros when
   TIME_NO_ZERO_IN_DATE is set, not when TIME_FUZZY_DATE is unset
2. Item_func_to_days and Item_date_add_interval can only work with valid dates,
   no zeros allowed.
2012-08-29 17:55:59 +02:00
Sergei Golubchik
a44331ab34 MDEV-456 An out-of-range datetime value (with a 5-digit year) can be created and cause troubles
fix Item_func_add_time::get_date() to generate valid dates.
Move the validity check inside get_date_from_daynr()
instead of relying on callers
(5 that had it, and 2 that did not, but should've)
2012-08-29 10:59:51 +02:00
unknown
95ee3fbf30 MDEV-492: fixed incorrect error check. 2012-08-29 11:35:42 +03:00
Sergey Petrunya
b6eccf51c0 Update test results (checked) 2012-08-28 16:03:22 +04:00
Sergey Petrunya
97bd8d8909 MDEV-405: Server crashes in test_if_skip_sort_order on EXPLAIN with GROUP BY and HAVING in EXISTS subquery
- Testcase
2012-08-28 15:40:38 +04:00
Sergey Petrunya
78497dbf5a Merge 2012-08-28 15:20:37 +04:00
Sergey Petrunya
2d99ea454f MDEV-430: Server crashes in select_describe on EXPLAIN with materialization+semijoin, etc
- Don't do early cleanup of uncorrelated subqueries if we're running an EXPLAIN.
2012-08-28 15:15:05 +04:00
Sergei Golubchik
9a64d0794c 5.3 merge 2012-08-27 18:13:17 +02:00
unknown
4d2b05b7d7 fix for MDEV-367
The problem was that was_null and null_value variables was reset in each reexecution of IN subquery, but engine rerun only for non-constant subqueries.

Fixed checking constant in Item_equal sort.
Fix constant reporting in Item_subselect.
2012-08-25 09:15:57 +03:00
unknown
ced3907c02 Merge from 5.3 2012-08-24 15:29:01 +02:00
unknown
4092d08bb8 Merge into latest 5.3 2012-08-24 14:26:23 +02:00
unknown
fc666a0df6 merge from 5.2 2012-08-24 14:02:32 +02:00
unknown
caa535eb9f Merge from 5.3 2012-08-24 13:57:39 +02:00
unknown
e44a800d91 Merge from 5.2 2012-08-24 13:51:16 +02:00
unknown
89e4d23f3b Merge into latest 5.2. 2012-08-24 12:57:19 +02:00
unknown
96703a63da Merge from 5.1. 2012-08-24 12:32:46 +02:00
unknown
cdeabcfd43 MDEV-382: Incorrect quoting
Various places in the server replication code was incorrectly quoting
strings, which could lead to incorrect SQL on the slave/mysqlbinlog.
2012-08-24 10:06:16 +02:00
Sergei Golubchik
f72a765997 5.2 merge.
two tests still fail:
  main.innodb_icp and main.range_vs_index_merge_innodb
  call records_in_range() with both range ends being open
  (which triggers an assert)
2012-08-22 16:45:25 +02:00
Sergei Golubchik
1fd8150a5b 5.1 merge
increase xtradb verson from 13.0 to 13.01
2012-08-22 16:13:54 +02:00
Sergei Golubchik
cefc30b166 merge with MySQL 5.1.65 2012-08-22 11:40:39 +02:00
unknown
80b3f74705 Fix bug mdev-447: Wrong output from the EXPLAIN command of the test case for lp bug #714999
The fix backports from MWL#182: Explain running statements the logic that
saves the original JOIN_TAB array of a query plan after optimization. This
array is later used during EXPLAIN to iterate over the original JOIN plan
nodes in the cases when this plan could be changed by early subquery
execution during the optimization phase of the outer query.
2012-08-21 15:24:43 +03:00
Michael Widenius
fbe5ac4e24 Fixed MDEV-365 "Got assertion when doing alter table on a partition"
mysql-test/r/partition.result:
  Added test case
mysql-test/t/partition.test:
  Added test case
sql/sql_partition.cc:
  Do mysql_trans_prepare_alter_copy_data() after all original tables are locked.
  (We don't want to disable transactions for the original tables, that still may be in the cache)
sql/sql_table.cc:
  Fixed two wrong DBUG_ENTER
2012-08-15 13:33:37 +03:00
Igor Babaev
d07b179fd2 Fixed bug mdev-449.
The bug could caused a crash when the server executed a query with
ORDER by and sort_buffer_size was set to a small enough number.
It happened because the small sort buffer did not allow to allocate
all merge buffers in it.
Made sure that the allocated sort buffer would be big enough
to contain all possible merge buffers.
2012-08-13 21:13:14 -07:00
Sergei Golubchik
7cc2f8deca fix val_str_ascii to return a string in the ascii-compatible charset.
two items didn't do that properly, one was exploitable, the other was not, but fixed anyway.
2012-08-09 18:25:47 +02:00
Sergei Golubchik
d11829654c merge with MySQL 5.5.27
manually checked every change, reverted incorrect or stupid changes.
2012-08-09 17:22:00 +02:00
Sergei Golubchik
42ea25d4c5 MDEV-375 Server crashes in THD::print_aborted_warning with log_warnings > 3
Don't use ER(xxx) in THD::close_connection(), when current_thd is already reset to NULL.
Prefer ER_THD() or ER_DEFAULT() instead.
2012-07-31 18:32:46 +02:00
Sergey Petrunya
55597a4869 MDEV-410: EXPLAIN shows type=range, while SHOW EXPLAIN and userstat show full table scan is used
- Make Item_subselect::fix_fields() ignore UNCACHEABLE_EXPLAIN flag when deciding whether 
  the subquery item should be marked as constant.
2012-07-25 20:41:48 +04:00
Axel Schwenke
1b1b36ab3e merged with maria/5.5 2012-07-19 13:21:53 +02:00