Commit graph

1929 commits

Author SHA1 Message Date
Michael Widenius
9aac546872 MDEV-5930 Server crashes in thd_get_ha_data on CREATE OR REPLACE TABLE
mysql-test/r/create_or_replace.result:
  More tests for create or replace
mysql-test/t/create_or_replace.test:
  More tests for create or replace
sql/log.cc:
  Don't use binlog_hton if binlog is not enabmed
sql/sql_base.cc:
  We have to call restart_trans_for_tables also if tables where not locked with LOCK TABLES.
  If not, we will get a crash in TokuDB
sql/sql_insert.cc:
  Don't call binlog_reset_cache() if we don't have binary log open
sql/sql_table.cc:
  Don't log to binary log if not open
  Better test if we where using create or replace ... select
storage/tokudb/mysql-test/tokudb_mariadb/r/create_or_replace.result:
  More tests for create or replace
storage/tokudb/mysql-test/tokudb_mariadb/t/create_or_replace.test:
  More tests for create or replace
2014-03-23 18:39:10 +02:00
Michael Widenius
b18a1b0e6c MDEV-5850: MySQL Bug#21317: SHOW CREATE DATABASE does not obey to lower_case_table_names
Bug #3329 Incomplete lower_case_table_names=2 implementation

The problem was that check_db_name() converted database names to lower case also in case of lower_case_table_names=2.

Fixed by removing the conversion in check_db_name for lower_case_table_names = 2 and instead converting db name to
lower case at same places as table names are converted.

Fixed bug that SHOW CREATE DATABASE FOO showed information for database 'foo'.

I also removed some checks of lower_case_table_names when it was enough to use table_alias_charset.


mysql-test/mysql-test-run.pl:
  Added --use-copy argument to force mysql-test-run to copy files instead of doing symlinks. This is needed when you run
  with test directory on another file system
mysql-test/r/lowercase_table.result:
  Updated results
mysql-test/r/lowercase_table2.result:
  Updated results
mysql-test/suite/parts/r/partition_mgm_lc2_innodb.result:
  Updated results
mysql-test/suite/parts/r/partition_mgm_lc2_memory.result:
  Updated results
mysql-test/suite/parts/r/partition_mgm_lc2_myisam.result:
  Updated results
mysql-test/t/lowercase_table.test:
  Added tests with mixed case databases
mysql-test/t/lowercase_table2.test:
  Added tests with mixed case databases
sql/log.cc:
  Don't check lower_case_table_names when we can use table_alias_charset
sql/sql_base.cc:
  Don't check lower_case_table_names when we can use table_alias_charset
sql/sql_db.cc:
  Use cmp_db_names() for checking if current database changed.
  mysql_rm_db() now converts db to lower case if lower_case_table_names was used.
  Changed database options cache to use table_alias_charset. This fixed a bug where SHOW CREATE DATABASE showed wrong information.
sql/sql_parse.cc:
  Change also db name to lower case when file names are changed.
  Don't need to story copy of database name anymore when lower_case_table_names == 2 as check_db_name() don't convert in this case.
  Updated arguments to mysqld_show_create_db().
  When adding table to TABLE_LIST also convert db name to lower case if needed (same way as we do with table names).
sql/sql_show.cc:
  mysqld_show_create_db() now also takes original name as argument for output to user.
sql/sql_show.h:
  Updated prototype for mysqld_show_create_db()
sql/sql_table.cc:
  In mysql_rename_table(), do same conversions to database name as we do for the file name
2014-03-23 15:43:57 +02:00
Sergei Golubchik
10740939eb 5.5 merge 2014-03-26 22:25:38 +01:00
Sergei Golubchik
47f438675b MDEV-5894 MySQL BUG#34750: Print database name in Unknown Table error message 2014-03-20 00:44:35 +01:00
Sergey Vojtovich
e4fde57712 MDEV-5864 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.free_tables
Let TABLE_SHARE::tdc.free_tables, TABLE_SHARE::tdc.all_tables,
TABLE_SHARE::tdc.flushed and corresponding invariants be protected by
per-share TABLE_SHARE::tdc.LOCK_table_share instead of global LOCK_open.
2014-03-20 11:11:13 +04:00
Michael Widenius
913d1f199c MDEV-5854 Interrupted CREATE OR REPLACE is written into binlog, and in a wrong format
mysql-test/r/create_or_replace2.result:
  Added test case
mysql-test/t/create_or_replace.test:
  Fixed comment
mysql-test/t/create_or_replace2.test:
  Added test case
sql/sql_base.cc:
  Safety fix:
  Don't let threads with query_id=0 free temporary tables as this may free temporary tables not in use.
  This is mostly the case for the slave io threads, as most other threads has thd->query_id != 0.
sql/sql_table.cc:
  Added comment.
  Ignore kill when opening temporary table for CREATE ... LIKE.
  This fixed the original isue
2014-03-19 15:15:57 +02:00
Sergei Golubchik
5d8c15228e 5.3-merge 2014-03-16 19:21:37 +01:00
Michael Widenius
f320b12ca5 MDEV-5619: CREATE OR REPLACE does not release MDL_EXCLUSIVE upon failure
mysql-test/r/create_or_replace.result:
  Added test of releasing of metadata locks
mysql-test/t/create_or_replace.test:
  Added test of releasing of metadata locks
sql/handler.h:
  Added marker if table was deleted as part of CREATE OR REPLACE
sql/sql_base.cc:
  Added Locked_tables_list::unlock_locked_table()
sql/sql_class.h:
  New prototypes
sql/sql_insert.cc:
  Unlock metadata locks for deleted table in case of error. Also do unlock tables if this was the only locked table.
sql/sql_table.cc:
  Unlock metadata locks for deleted table in case of error. Also do unlock tables if this was the only locked table.
2014-03-12 11:26:40 +02:00
unknown
321ff25f3d MDEV-5740: Assertion `!derived->first_select()->exclude_from_table_unique_test || derived->outer_select()-> exclude_from_table_unique_test' failed on 2nd execution of PS with derived_merge
Do not check tables of executed units.
Debug info about stages of derived tables execution added.
2014-03-07 13:57:07 +02:00
Sergey Vojtovich
b95c8ce530 MDEV-5675 - Performance: my_hash_sort_bin is called too often
Reduced number of my_hash_sort_bin() calls from 4 to 1 per query.
Reduced number of memory accesses done by my_hash_sort_bin().

Details:
- let MDL subsystem use pre-calculated hash value for hash
  inserts and deletes
- let table cache use pre-calculated MDL hash value
- MDL namespace is excluded from hash value calculation, so that
  hash value can be used by table cache as is
- hash value for MDL is calculated as resulting hash value + MDL
  namespace
- extended hash implementation to accept user defined hash function
2014-03-06 16:19:12 +04:00
Michael Widenius
6efa5efa7d Fixed that rpl_row_create_table can be run with --ps-protocol
As part of the fix we don't anymore generate a create table statement when doing a
CREATE TABLE IF NOT EXISTS table_that_exist LiKE temporary_table
if the 'table_that_exist' existed.

This is because it's not self evident if we should generate a create statement
matching the existing table or the temporary_table. 
The old code generated a table like the existing table in row based replication and like the temporary table
in statement based replication.
It's better to ensure that both cases works the same way.

mysql-test/suite/rpl/r/rpl_row_create_table.result:
  Updated results
  (Now we don't anymore CREATE TABLE IF NOT EXISTS LIKE if the table existed)
sql/sql_base.cc:
  More DBUG_PRINT
sql/sql_error.cc:
  More DBUG_PRINT
sql/sql_table.cc:
  Don't generate a create table statement when doing a
  CREATE TABLE IF NOT EXISTS table_that_exist like temporary_table if the table existed.
2014-02-22 03:11:56 +02:00
Sergey Vojtovich
d12c7adf71 MDEV-5314 - Compiling fails on OSX using clang
This is port of fix for MySQL BUG#17647863.

revno: 5572
revision-id: jon.hauglid@oracle.com-20131030232243-b0pw98oy72uka2sj
committer: Jon Olav Hauglid <jon.hauglid@oracle.com>
timestamp: Thu 2013-10-31 00:22:43 +0100
message:
  Bug#17647863: MYSQL DOES NOT COMPILE ON OSX 10.9 GM

  Rename test() macro to MY_TEST() to avoid conflict with libc++.
2014-02-19 14:05:15 +04:00
Sergey Vojtovich
048e9c40a6 MDEV-5492 - Reduce usage of LOCK_open: TABLE::in_use
Move TABLE::in_use out of LOCK_open.

This is done with assumtion that foreign threads accessing TABLE::in_use
will only need consistent value _after_ marking table for flush and purging
unused table instances. In this case TABLE::in_use will always point to a
valid thread object.

Previously FLUSH TABLES thread may wait for tables flushed subsequently by
concurrent threads which breaks the above assumption, e.g.:
open tables: t1 (version= 1)
thr1 (FLUSH TABLES): refresh_version++
thr1 (FLUSH TABLES): purge table cache
open tables: none
thr2 (SELECT * FROM t1): open tables: t1
open tables: t1 (version= 2)
thr2 (FLUSH TABLES): refresh_version++
thr2 (FLUSH TABLES): purge table cache
thr1 (FLUSH TABLES): wait for old tables (including t1 with version 2)

It is fixed so that FLUSH TABLES waits only for tables that were open
heretofore.
2014-02-13 10:44:10 +04:00
Michael Widenius
1695fc4532 MDEV-5602: CREATE OR REPLACE obtains stricter locks than the connection had before
mysql-test/r/create_or_replace.result:
  Added test case
mysql-test/t/create_or_replace.test:
  Added test case
sql/sql_base.cc:
  Downgrade locks in LOCK TABLES + CREATE TABLE OR REPLACE
sql/sql_table.cc:
  Downgrade locks in LOCK TABLES + CREATE TABLE OR REPLACE
2014-02-05 21:36:16 +02:00
Michael Widenius
10001c8e4f Automatic merge 2014-02-05 19:23:11 +02:00
Sergei Golubchik
72c20282db 10.0-base merge 2014-02-03 15:22:39 +01:00
Sergei Golubchik
59d9d08e2b 5.5 merge 2014-02-01 00:54:03 +01:00
Michael Widenius
43f6e118fe Fixes for CREATE_OR_REPLACE
- MDEV-5587 Server crashes in Locked_tables_list::restore_lock on CREATE OR REPLACE .. SELECT under LOCK
- MDEV-5586 Assertion `share->tdc.all_tables.is_empty() || remove_type != TDC_RT_REMOVE_ALL' fails in tdc_remove_table
- MDEV-5588 Strange error on CREATE OR REPLACE table over an existing view

mysql-test/r/create_or_replace.result:
  Added test cases
mysql-test/r/lowercase_view.result:
  New error message
mysql-test/r/merge.result:
  New error message
mysql-test/r/multi_update.result:
  New error message
mysql-test/r/subselect.result:
  New error message
mysql-test/r/subselect_exists_to_in.result:
  New error message
mysql-test/r/subselect_no_mat.result:
  New error message
mysql-test/r/subselect_no_opts.result:
  New error message
mysql-test/r/subselect_no_scache.result:
  New error message
mysql-test/r/subselect_no_semijoin.result:
  New error message
mysql-test/r/view.result:
  New error message
mysql-test/suite/funcs_1/r/myisam_views-big.result:
  New error message
mysql-test/t/create_or_replace.test:
  New tests
mysql-test/t/view.test:
  New error message
sql/share/errmsg-utf8.txt:
  Added new error message
sql/sql_base.cc:
  Updated error message
  Do an automatic UNLOCK TABLES if we don't have any locked tables (safety fix)
sql/sql_db.cc:
  Updated arguments
sql/sql_load.cc:
  New error message
sql/sql_parse.cc:
  Check that we are not using a table we are dropping and re-creating
sql/sql_table.cc:
  Added parameter to mysql_rm_table_no_locks() to not automaticly do UNLOCK TABLES
  Added better error message if trying to drop a view with DROP TABLE
  Don't try to create something we select from
sql/sql_table.h:
  Updated prototypes
2014-01-31 12:06:28 +02:00
Michael Widenius
7ffc9da093 Implementation of MDEV-5491: CREATE OR REPLACE TABLE
Using CREATE OR REPLACE TABLE is be identical to

DROP TABLE IF EXISTS table_name;
CREATE TABLE ...;

Except that:

* CREATE OR REPLACE is be atomic (now one can create the same table between drop and create).
* Temporary tables will not shadow the table name for the DROP as the CREATE TABLE tells us already if we are using a temporary table or not.
* If the table was locked with LOCK TABLES, the new table will be locked with the same lock after it's created.

Implementation details:
- We don't anymore open the to-be-created table during CREATE TABLE, which the original code did.
  - There is no need to open a table we are planning to create. It's enough to check if the table exists or not.
- Removed some of duplicated code for CREATE IF NOT EXISTS.
- Give an error when using CREATE OR REPLACE with IF NOT EXISTS (conflicting options).
- As a side effect of the code changes, we don't anymore have to internally re-prepare prepared statements with CREATE TABLE if the table exists.
- Made one code path for all testing if log table are in use.
- Better error message if one tries to create/drop/alter a log table in use
- Added back disabled rpl_row_create_table test as it now seams to work and includes a lot of interesting tests.
- Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
- Aligned CREATE OR REPLACE parsing code in sql_yacc.yy for TABLE and VIEW
- Changed interface for drop_temporary_table() to make it more reusable
- Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
- Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument. Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
- Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
- Added restart_trans_for_tables() to be able to restart a transaction.
- DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
- Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
- Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()




mysql-test/include/commit.inc:
  With new code we get fewer status increments
mysql-test/r/commit_1innodb.result:
  With new code we get fewer status increments
mysql-test/r/create.result:
  Added testing of create or replace with timeout
mysql-test/r/create_or_replace.result:
  Basic testing of CREATE OR REPLACE TABLE
mysql-test/r/partition_exchange.result:
  New error message
mysql-test/r/ps_ddl.result:
  Fewer reprepares with new code
mysql-test/suite/archive/discover.result:
  Don't rediscover archive tables if the .frm file exists
  (Sergei will look at this if there is a better way...)
mysql-test/suite/archive/discover.test:
  Don't rediscover archive tables if the .frm file exists
  (Sergei will look at this if there is a better way...)
mysql-test/suite/funcs_1/r/innodb_views.result:
  New error message
mysql-test/suite/funcs_1/r/memory_views.result:
  New error message
mysql-test/suite/rpl/disabled.def:
  rpl_row_create_table should now be safe to use
mysql-test/suite/rpl/r/rpl_row_create_table.result:
  Updated results after adding back disabled test
mysql-test/suite/rpl/t/rpl_create_if_not_exists.test:
  Added comment
mysql-test/suite/rpl/t/rpl_row_create_table.test:
  Added CREATE OR REPLACE TABLE test
mysql-test/t/create.test:
  Added CREATE OR REPLACE TABLE test
mysql-test/t/create_or_replace-master.opt:
  Create logs
mysql-test/t/create_or_replace.test:
  Basic testing of CREATE OR REPLACE TABLE
mysql-test/t/partition_exchange.test:
  Error number changed as we are now using same code for all log table change issues
mysql-test/t/ps_ddl.test:
  Fewer reprepares with new code
sql/handler.h:
  Moved things around a bit in a structure to get better alignment.
  Added HA_LEX_CREATE_REPLACE to mark if we are using CREATE OR REPLACE
  Added 3 elements to end of HA_CREATE_INFO to be able to store state to add backs locks in case of LOCK TABLES.
sql/log.cc:
  Reimplemented check_if_log_table():
  - Simpler and faster usage
  - Can give error messages
  
  This gives us one code path for allmost all error messages if log tables are in use
sql/log.h:
  New interface for check_if_log_table()
sql/slave.cc:
  More logging
sql/sql_alter.cc:
  New interface for check_if_log_table()
sql/sql_base.cc:
  More documentation
  Changed interface for drop_temporary_table() to make it more reusable
  Changed Locked_tables_list::init_locked_tables() to work on the table object instead of the table list object. Before this it used a mix of both, which was not good.
  Locked_tables_list::unlock_locked_tables(THD *thd) now requires a valid thd argument.  Old usage of calling this with 0 i changed to instead call Locked_tables_list::reset()
  Added functions Locked_tables_list:restore_lock() and Locked_tables_list::add_back_last_deleted_lock() to be able to easily add back a locked table to the lock list.
  Check for command number instead of open_strategy of CREATE TABLE was used.
  Added restart_trans_for_tables() to be able to restart a transaction.  This was needed in "create or replace ... select" between the drop table and the select.
sql/sql_base.h:
  Added and updated function prototypes
sql/sql_class.h:
  Added new prototypes to Locked_tables_list class
  Added extra argument to select_create to avoid double call to eof() or send_error()
  - I needed this in some edge case where the table was not created against expections.
sql/sql_db.cc:
  New interface for check_if_log_table()
sql/sql_insert.cc:
  Remember position to lock information so that we can reaquire table lock for LOCK TABLES + CREATE OR REPLACE TABLE SELECT. Later add back the lock by calling restore_lock().
  Removed one not needed indentation level in create_table_from_items()
  Ensure we don't call send_eof() or abort_result_set() twice.
sql/sql_lex.h:
  Removed variable that I temporarly added in an earlier changeset
sql/sql_parse.cc:
  Removed old test code (marked with QQ)
  Ensure that we have open_strategy set as TABLE_LIST::OPEN_STUB in CREATE TABLE
  Removed some IF NOT EXISTS code as this is now handled in create_table_table_impl().
  Set OPTION_KEEP_LOGS later. This code had to be moved as the test for IF EXISTS has changed place.
  DROP_ACL is required if one uses CREATE TABLE OR REPLACE.
sql/sql_partition_admin.cc:
  New interface for check_if_log_table()
sql/sql_rename.cc:
  New interface for check_if_log_table()
sql/sql_table.cc:
  New interface for check_if_log_table()
  Moved some code in mysql_rm_table() under a common test.
  - Safe as temporary tables doesn't have statistics.
  - !is_temporary_table(table) test was moved out from drop_temporary_table() and merged with upper level code.
  - Added drop of normal and temporary tables in create_table_imp() if CREATE OR REPLACE was used.
  - Added reacquiring of table locks in mysql_create_table() and mysql_create_like_table()
  - In mysql_create_like_table(), restore table->open_strategy() if it was changed.
  - Re-test if table was a view after opening it.
sql/sql_table.h:
  New prototype for mysql_create_table_no_lock()
sql/sql_yacc.yy:
  Added syntax for CREATE OR REPLACE TABLE
  Reuse new code for CREATE OR REPLACE VIEW
sql/table.h:
  Added name for enum type
sql/table_cache.cc:
  More DBUG
2014-01-29 15:37:17 +02:00
Michael Widenius
fb018850dc Fixed that setup_natural_join_row_types can safely be called twice
sql/item.h:
  Added cache for setup_natural_join_row_types
sql/sql_base.cc:
  Cache old value of first_name_resolution_table for next call.
  (It's not safe to try to recalculate the value as the join structure may have been changed by the optimizer)
2014-01-26 21:48:23 +01:00
unknown
31249744fe merge 5.3->5.5 2014-01-26 16:41:15 +02:00
Michael Widenius
d15b3386db Fix for MDEV-5531: double call procedure in one session - hard shutdown the server
Main fix was to not cache derivied tables as they may be temporary tables that are deleted before the next query.
This was a bit tricky as Item_field::fix_fields depended on cached_tables to be set to resolve some columns.



mysql-test/r/sp-bugs.result:
  Added test case
mysql-test/t/sp-bugs.test:
  Added test case
sql/item.cc:
  Fixed fix_outer_field to handle case where found field did not have in cached_table
  Idea is that if cached_table is not avaliable, use from_field->table->pos_in_table_list instead
sql/records.cc:
  Also accept INTERNAL_TMP_TABLE for memmap
sql/sql_base.cc:
  More DBUG_PRINT
  Fixed that setup_natural_join_row_types() is not run twice.
  Original code modified context->first_name_resolution_table also for second executions.
  This was wrong as this could give wrong results if some joins had been optimized away between calls.
sql/sql_derived.cc:
  Mark derived tables as internal temporary tables (INTERNAL_TMP_TABLE), not as NON_TRANSACTIONAL_TMP_TABLE.
  This is more correct as the tables are not visible by the end user.
sql/sql_insert.cc:
  Reset pos_in_table_list before calling fix_fields.
  One of the consequences of the change of not caching all generated tables in Item_ident is that
  pos_in_table_list needs to be correct in calls to fix_fields.
sql/sql_lex.cc:
  More DBUG_PRINT
sql/sql_parse.cc:
  Don't cache derivied tables as they may be temporary tables that are deleted before the next query
sql/sql_select.cc:
  Reset table_vector. This was required as some code checked the vector to see if temporary tables had already been created.
sql/table.cc:
  Mark tables with field translations as cacheable (as these will not disapper between stmt executions.
2014-01-24 14:50:18 +02:00
unknown
d9cb1352c8 merge of MDEV-5356 5.1->5.3 (with more fixes and test suite).
THD::thd->activate_stmt_arena_if_needed() should be used to temporary activating statement arena instead of direct usage of THD::set_n_backup_active_arena() because possible such scenario:
  1) func1 saves current arena and activates copy1 of statement arena
  2) func2 saves copy1 of statement arena setup by func1 and activates copy2
  3) some changes made for copy 2
  4) func2 stores changed copy2 back to statenet arena and activates copy1
  5) func1 store unchanged copy1 back to statemnt arena (rewrite changed copy 2 so changes become lost) and activates arena which was before.
2014-01-23 12:05:10 +02:00
unknown
5f5f7befe3 MDEV-5356: Server crashes in Item_equal::contains on 2nd execution of a PS
THD::thd->activate_stmt_arena_if_needed() should be used to temporary activating statement arena instead of direct usage of THD::set_n_backup_active_arena() because possible such scenario:
  1) func1 saves current arena and activates copy1 of statement arena
  2) func2 saves copy1 of statement arena setup by func1 and activates copy2
  3) some changes made for copy 2
  4) func2 stores changed copy2 back to statenet arena and activates copy1
  5) func1 store unchanged copy1 back to statemnt arena (rewrite changed copy 2 so changes become lost) and activates arena which was before.
2014-01-23 11:11:01 +02:00
Sergei Golubchik
d28d3ba40d 10.0-base merge 2013-12-16 13:02:21 +01:00
Sergei Golubchik
6bf10fac44 5.5 merge 2013-12-15 15:57:26 +01:00
Sergei Golubchik
e68bccc743 5.3 merge 2013-12-13 13:00:38 +01:00
Sergey Vojtovich
2d7c073852 MDEV-5388 - Reduce usage of LOCK_open: unused_tables
Removed unused_tables, find LRU object by timestamp instead.
2013-12-12 21:49:14 +04:00
Sergey Vojtovich
98c4f167c5 MDEV-4956 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.used_tables
- tc_acquire_table and tc_release_table do not access
  TABLE_SHARE::tdc.used_tables anymore
- in tc_acquire_table(): release LOCK_tdc after we relase LOCK_open
  (saves a few CPU cycles in critical section)
- in tc_release_table(): if we reached table cache threshold, evict
  to-be-released table without moving it to unused_tables. unused_tables
  must be empty at this point.
2013-12-10 19:00:36 +04:00
Igor Babaev
ccf5871d7b Fixed bug mdev-5382
When marking used columns the function find_field_in_table_ref() erroneously
called the walk method for the real item behind a view/derived table field
with the second parameter set to TRUE.
This erroneous code was introduced in 2006.
2013-12-05 11:13:20 -08:00
Sergei Golubchik
38c91c067b MDEV-4955 discover of table non-existance on CREATE
when frm file exists, but the table does not.

In recover_from_failed_open(), allow the discovery
to fail without an error, when
open_strategy == TABLE_LIST::OPEN_IF_EXISTS.
2013-11-28 16:39:17 +01:00
Sergei Golubchik
c6d30805db 5.5 merge 2013-11-23 00:50:54 +01:00
Sergei Golubchik
fa3f8a18b2 mysql-5.5.34 merge
(some patches reverted, test case added)
2013-11-19 13:16:25 +01:00
Michael Widenius
de9d9792ab Fixed things missing in merge between 10.0-base and 10.0
Updated --help text to declare --slave-parallel-threads as an alpha feature

mysql-test/r/mysqld--help.result:
  Updated --help text
sql/slave.cc:
  Added missing trans_retries++ that caused rpl_deadlock_innodb.test to fail.
  This is safe as this part is never run in parallel.
sql/sql_base.cc:
  Fixed temporary table handling (part of merge)
sql/sys_vars.cc:
  Updated --help text to declare --slave-parallel-threads as an alpha feature
2013-11-03 22:26:44 +02:00
unknown
57a267a8c0 Merge from 10.0-base to 10.0 the feature MDEV-4506: Parallel replication.
The merge is still missing a few hunks related to temporary tables and
InnoDB log file size. The associated code did not seem to exist in
10.0, so the merge of that needs more work. Until this is fixed, there
are a number of test failures as a result.
2013-11-01 12:00:11 +01:00
unknown
cb86ce60b9 Merge MDEV-4506: Parallel replication into 10.0-base. 2013-11-01 09:17:06 +01:00
Sergei Golubchik
0fdb3bcfdb 10.0-base merge (roles) 2013-10-29 15:08:44 +01:00
Sergei Golubchik
7fba8e5164 find() method for Hash_set<>.
Move key function from template parameter to the constructor
2013-10-18 12:36:03 -07:00
Igor Babaev
ddc46740a5 Merge 5.5->10.0-base 2013-10-13 18:10:19 -07:00
Michael Widenius
2e100cc5a4 Fixes for parallel slave:
- Made slaves temporary table multi-thread slave safe by adding mutex around save_temporary_table usage.
  - rli->save_temporary_tables is the active list of all used temporary tables
  - This is copied to THD->temporary_tables when temporary tables are opened and updated when temporary tables are closed
  - Added THD->lock_temporary_tables() and THD->unlock_temporary_tables() to simplify this.
- Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code.
- Added is_part_of_group() to mark functions that are part of the next function. This replaces setting IN_STMT when events are executed.
- Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code.
- If slave_skip_counter is set run things in single threaded mode. This simplifies code for skipping events.
- Updating state of relay log (IN_STMT and IN_TRANSACTION) is moved to one single function: update_state_of_relay_log()
  We can't use OPTION_BEGIN to check for the state anymore as the sql_driver and sql execution threads may be different.
  Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts
  is_in_group() is now independent of state of executed transaction.
- Reset thd->transaction.all.modified_non_trans_table() if we did set it for single table row events.
  This was mainly for keeping the flag as documented.
- Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it.
- Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
- Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
- Changed some functions to take rpl_group_info instead of Relay_log_info to make them multi-slave safe and to simplify usage
  - do_shall_skip()
  - continue_group()
  - sql_slave_killed()
  - next_event()
- Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure.
- set_thd_in_use_temporary_tables() removed as in_use is set on usage
- Added information to thd_proc_info() which thread is waiting for slave mutex to exit.
- In open_table() reuse code from find_temporary_table()

Other things:
- More DBUG statements
- Fixed the rpl_incident.test can be run with --debug
- More comments
- Disabled not used function rpl_connect_master()

mysql-test/suite/perfschema/r/all_instances.result:
  Moved sleep_lock and sleep_cond to rpl_group_info
mysql-test/suite/rpl/r/rpl_incident.result:
  Updated result
mysql-test/suite/rpl/t/rpl_incident-master.opt:
  Not needed anymore
mysql-test/suite/rpl/t/rpl_incident.test:
  Fixed that test can be run with --debug
sql/handler.cc:
  More DBUG_PRINT
sql/log.cc:
  More comments
sql/log_event.cc:
  Added DBUG statements
  do_shall_skip(), continue_group() now takes rpl_group_info param
  Use is_begin(), is_commit() and is_rollback() functions instead of inspecting query string
  We don't have set slaves temporary tables 'in_use' as this is now done when tables are opened.
  Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
  Use IN_TRANSACTION flag to test state of relay log.
  In rows_event_stmt_cleanup() reset thd->transaction.all.modified_non_trans_table if we had set this before.
sql/log_event.h:
  do_shall_skip(), continue_group() now takes rpl_group_info param
  Added is_part_of_group() to mark events that are part of the next event. This replaces setting IN_STMT when events are executed.
  Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code.
sql/log_event_old.cc:
  Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
  do_shall_skip(), continue_group() now takes rpl_group_info param
sql/log_event_old.h:
  Added is_part_of_group() to mark events that are part of the next event.
  do_shall_skip(), continue_group() now takes rpl_group_info param
sql/mysqld.cc:
  Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it.
  Relay_log_info::sleep_lock -> Rpl_group_info::sleep_lock
  Relay_log_info::sleep_cond -> Rpl_group_info::sleep_cond
sql/mysqld.h:
  Updated types and names
sql/rpl_gtid.cc:
  More DBUG
sql/rpl_parallel.cc:
  Updated TODO section
  Set thd for event that is execution
  Use new  is_begin(), is_commit() and is_rollback() functions.
  More comments
sql/rpl_rli.cc:
  sql_thd -> sql_driver_thd
  Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
  Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
  Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts.
  Reset table->in_use for temporary tables as the table may have been used by another THD.
  Use IN_TRANSACTION instead of OPTION_BEGIN to check state of relay log.
  Removed IN_STMT flag setting. This is now done in update_state_of_relay_log()
sql/rpl_rli.h:
  Changed relay log state flags to bit masks instead of bit positions (most other code we have uses bit masks)
  Added IN_TRANSACTION to mark if we are in a BEGIN ... COMMIT section.
  save_temporary_tables is now thread safe
  Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock
  Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond
  Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code
  is_in_group() is now independent of state of executed transaction.
sql/slave.cc:
  Simplifed arguments to io_salve_killed(), sql_slave_killed() and check_io_slave_killed(); No reason to supply THD as this is part of the given structure.
  set_thd_in_use_temporary_tables() removed as in_use is set on usage in sql_base.cc
  sql_thd -> sql_driver_thd
  More DBUG
  Added update_state_of_relay_log() which will calculate the IN_STMT and IN_TRANSACTION state of the relay log after the current element is executed.
  If slave_skip_counter is set run things in single threaded mode.
  Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure.
  Added information to thd_proc_info() which thread is waiting for slave mutex to exit.
  Disabled not used function rpl_connect_master()
  Updated argument to next_event()
sql/sql_base.cc:
  Added mutex around usage of slave's temporary tables. The active list is always kept up to date in sql->rgi_slave->save_temporary_tables.
  Clear thd->temporary_tables after query (safety)
  More DBUG
  When using temporary table, set table->in_use to current thd as the THD may be different for slave threads.
  Some code is ifdef:ed with REMOVE_AFTER_MERGE_WITH_10 as the given code in 10.0 is not yet in this tree.
  In open_table() reuse code from find_temporary_table()
sql/sql_binlog.cc:
  rli->sql_thd -> rli->sql_driver_thd
  Remove duplicate setting of rgi->rli
sql/sql_class.cc:
  Added helper functions rgi_lock_temporary_tables() and rgi_unlock_temporary_tables()
  Would have been nicer to have these inline, but there was no easy way to do that
sql/sql_class.h:
  Added functions to protect slaves temporary tables
sql/sql_parse.cc:
  Added DBUG_PRINT
sql/transaction.cc:
  Added comment
2013-10-14 00:24:05 +03:00
Igor Babaev
c7db46a242 Merge 5.3-5.5 2013-10-13 13:43:29 -07:00
Igor Babaev
7c87385e30 Fixed bug mdev-5105.
The bug caused a memory overwrite in the function update_ref_and_keys()
It happened due to a wrong value of SELECT_LEX::cond_count. This value
historically was calculated by the fix_fields method. Now the logic of
calling this method became too complicated and, as a result, this value
is calculated not always correctly.
The patch changes the way how and when  the values of SELECT_LEX::cond_count
and of SELECT_LEX::between_count are calculated. The new code does it just at
the beginning of update_ref_and_keys().
2013-10-10 10:08:26 -07:00
Alexander Barkov
a9240dce9e Merge 10.0-base -> 10.0 2013-10-15 10:26:08 +04:00
Dmitry Lenev
fc2c669297 Fix for bug#14188793 - "DEADLOCK CAUSED BY ALTER TABLE DOEN'T CLEAR
STATUS OF ROLLBACKED TRANSACTION" and bug #17054007 - "TRANSACTION
IS NOT FULLY ROLLED BACK IN CASE OF INNODB DEADLOCK".

The problem in the first bug report was that although deadlock involving
metadata locks was reported using the same error code and message as InnoDB
deadlock it didn't rollback transaction like the latter. This caused
confusion to users as in some cases after ER_LOCK_DEADLOCK transaction
could have been restarted immediately and in some cases rollback was
required.

The problem in the second bug report was that although InnoDB deadlock
caused transaction rollback in all storage engines it didn't cause release
of metadata locks. So concurrent DDL on the tables used in transaction was
blocked until implicit or explicit COMMIT or ROLLBACK was issued in the
connection which got InnoDB deadlock.

The former issue has stemmed from the fact that when support for detection
and reporting metadata locks deadlocks was added we erroneously assumed
that InnoDB doesn't rollback transaction on deadlock but only last statement
(while this is what happens on InnoDB lock timeout actually) and so didn't
implement rollback of transactions on MDL deadlocks.

The latter issue was caused by the fact that rollback of transaction due
to deadlock is carried out by setting THD::transaction_rollback_request
flag at the point where deadlock is detected and performing rollback
inside of trans_rollback_stmt() call when this flag is set. And
trans_rollback_stmt() is not aware of MDL locks, so no MDL locks are
released.

This patch solves these two problems in the following way:

- In case when MDL deadlock is detect transaction rollback is requested
  by setting THD::transaction_rollback_request flag.

- Code performing rollback of transaction if THD::transaction_rollback_request
  is moved out from trans_rollback_stmt(). Now we handle rollback request
  on the same level as we call trans_rollback_stmt() and release statement/
  transaction MDL locks.
2013-08-20 13:12:34 +04:00
Sergey Vojtovich
b7f9c89423 MDEV-4702 - Reduce usage of LOCK_open
Following variables do not require LOCK_open protection anymore:
- table_def_cache (renamed to tdc_hash) is protected by rw-lock
  LOCK_tdc_hash;
- table_def_shutdown_in_progress doesn't need LOCK_open protection;
- last_table_id use atomics;
- TABLE_SHARE::ref_count (renamed to TABLE_SHARE::tdc.ref_count)
  is protected by TABLE_SHARE::tdc.LOCK_table_share;
- TABLE_SHARE::next, ::prev (renamed to tdc.next and tdc.prev),
  oldest_unused_share, end_of_unused_share are protected by
  LOCK_unused_shares;
- TABLE_SHARE::m_flush_tickets (renamed to tdc.m_flush_tickets)
  is protected by TABLE_SHARE::tdc.LOCK_table_share;
- refresh_version (renamed to tdc_version) use atomics.
2013-08-14 12:48:50 +04:00
Sergei Golubchik
9747fbb411 MDEV-4786 merge 10.0-monty -> 10.0
remove TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE
2013-08-12 15:46:35 +02:00
Sergei Golubchik
eafb11c821 MDEV-4786 merge 10.0-monty -> 10.0
use get_table_def_key() instead of create_table_def_key() where appropriate
2013-08-12 14:17:51 +02:00
Sergey Vojtovich
e33e9825a4 MDEV-4786 - merge 10.0-monty - 10.0
Fixed main.innodb_mysql_sync failure.

TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE is not same as
TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE. Comment in sql_table.cc says:

      Storage engine has requested exclusive lock only for prepare phase
      and we are not under LOCK TABLES.
      Don't mark TABLE_SHARE as old in this case, as this won't allow opening
      of table by other threads during main phase of in-place ALTER TABLE.

At this moment we hold exclusive metadata lock, all we should do
is purge unused TABLE objects.

sql/sql_base.cc:
  Restore 5.6 behavior of TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE.
sql/sql_base.h:
  Restore 5.6 behavior of TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE.
2013-07-25 13:40:18 +04:00
Sergei Golubchik
b7b5f6f1ab 10.0-monty merge
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
2013-07-21 16:39:19 +02:00
Sergei Golubchik
5f6380adde 10.0-base merge 2013-07-18 16:46:57 +02:00