Commit graph

175914 commits

Author SHA1 Message Date
Marko Mäkelä
341c375d4b Merge 10.1 into 10.2 2016-12-30 08:53:54 +02:00
Marko Mäkelä
f2fe65106f MDEV-11679 Remove redundant function fsp_header_get_crypt_offset()
fsp_header_get_crypt_offset(): Remove.

xdes_arr_size(): Remove.

fsp_header_get_encryption_offset(): Make this an inline function.

The correctness of this change was ensured with the following patch
that ensures that the two functions returned the same value, only
differing by FSP_HEADER_OFFSET (38 bytes):

diff --git a/storage/innobase/fsp/fsp0fsp.cc b/storage/innobase/fsp/fsp0fsp.cc
index f2a4c6bf218..e96c788b7df 100644
--- a/storage/innobase/fsp/fsp0fsp.cc
+++ b/storage/innobase/fsp/fsp0fsp.cc
@@ -850,6 +850,7 @@ fsp_parse_init_file_page(
 	return(ptr);
 }

+static ulint fsp_header_get_encryption_offset(const page_size_t&);
 /**********************************************************************//**
 Initializes the fsp system. */
 void
@@ -868,6 +869,31 @@ fsp_init(void)
 #endif

 	/* Does nothing at the moment */
+
+	for (ulint sz = 4096; sz <= 65536; sz *= 2) {
+		ulint m;
+		if (sz <= 16384) {
+			for (ulint ph = 1024; ph <= sz; ph *= 2) {
+				const page_size_t ps(ph, sz, true);
+				ulint maria = fsp_header_get_crypt_offset(ps, &m),
+					oracle = fsp_header_get_encryption_offset(ps);
+				if (maria != oracle + 38) {
+					ib::error() << "zip size mismatch: "
+						    << maria << "!=" << oracle
+						    << "(" << ph <<","<<sz<<")"
+						    << m;
+				}
+			}
+		}
+		const page_size_t p(sz, sz, false);
+		ulint maria = fsp_header_get_crypt_offset(p, &m),
+			oracle = fsp_header_get_encryption_offset(p);
+		if (maria != oracle + 38) {
+			ib::error() << "size mismatch: "
+				    << maria << "!=" << oracle
+				    << "(" <<sz<<")" << m;
+		}
+	}
 }

 /**********************************************************************//**
2016-12-29 15:27:24 +02:00
Marko Mäkelä
7bcae22bf1 Merge branch 'bb-10.2-mdev-6076' into 10.2 2016-12-29 15:05:04 +02:00
Sergei Golubchik
ab89359dde enable tests that were skipped because of have_xtradb
* some of these tests run just fine with InnoDB:
   -> s/have_xtradb/have_innodb/
* sys_var tests did basic tests for xtradb only variables
   -> remove them, they're useless anyway (sysvar_innodb does it better)
* multi_update had innodb specific tests
   -> move to multi_update_innodb.test
2016-12-29 13:23:49 +01:00
Sergei Golubchik
b3d6cbc25a cleanup: binlog.binlog_killed_simulate 2016-12-29 13:23:42 +01:00
Sergei Golubchik
0d897c2ceb cleanup: binlog.binlog_row_annotate 2016-12-29 13:23:35 +01:00
Sergei Golubchik
4a5d25c338 Merge branch '10.1' into 10.2 2016-12-29 13:23:18 +01:00
Sergei Golubchik
48dc7cc66e cleanup: redundant memcmp() 2016-12-29 11:29:28 +01:00
Oleksandr Byelkin
100f721c0a MDEV-11584: GRANT inside an SP does not work well on 2nd execution
Allocate password hash in statment memory
2016-12-28 18:07:39 +01:00
Elena Stepanova
dc9f5dfcbb Replication tests fail on valgrind due to waiting-related timeouts
MTR raises default wait_for_pos_timeout from 300 to 1500 when tests
are run with valgrind. The same needs to be done for other
replication-related waits
2016-12-27 20:41:32 +02:00
Nirbhay Choubey
143d512a22 MDEV-11035: Re-enable galera_var_innodb_disallow_writes test 2016-12-24 11:31:04 -05:00
Daniel Bartholomew
f5c4f1eb38 bump the VERSION 2016-12-24 09:54:23 -05:00
Alexey Botchkov
3a1772798d MDEV-11573 JSON_LENGTH returns incorrect results.
Item_func_json_length fixed.
2016-12-24 11:40:31 +04:00
Alexey Botchkov
4d10273b4f MDEV-11571 JSON_EXTRACT returns wrong results.
Array counter didn't increment after an item was found.
2016-12-24 10:51:43 +04:00
Marko Mäkelä
bbb3fb318e Follow-up for MDEV-11630 Call mutex_free() before freeing the mutex list
fil_tablespace_iterate(): Call fil_space_destroy_crypt_data() to
invoke mutex_free() for the mutex_create() that was done in
fil_space_read_crypt_data(). Also, remember to free
iter.crypt_io_buffer.

The failure to call mutex_free() would cause sync_latch_meta_destroy()
to access freed memory on shutdown. This affected the IMPORT of
encrypted tablespaces.
2016-12-23 09:19:39 +02:00
Marko Mäkelä
08f79bdeda MDEV-11635 innodb.innodb_mysql test hangs
Copy and adapt the test from MySQL 5.7.17.
2016-12-22 16:48:49 +02:00
Marko Mäkelä
d6a1f9f10f MDEV-11630 Call mutex_free() before freeing the mutex list
fil_space_crypt_cleanup(): Call mutex_free() to pair with
fil_space_crypt_init().

fil_space_destroy_crypt_data(): Call mutex_free() to pair with
fil_space_create_crypt_data() and fil_space_read_crypt_data().

fil_crypt_threads_cleanup(): Call mutex_free() to pair with
fil_crypt_threads_init().

fil_space_free_low(): Invoke fil_space_destroy_crypt_data().

fil_close(): Invoke fil_space_crypt_cleanup(), just like
fil_init() invoked fil_space_crypt_init().

Datafile::shutdown(): Set m_crypt_info=NULL without dereferencing
the pointer. The object will be freed along with the fil_space_t
in fil_space_free_low().
Remove some unnecessary conditions (ut_free(NULL) is OK).

srv_shutdown_all_bg_threads(): Shut down the encryption threads
by calling fil_crypt_threads_end().

srv_shutdown_bg_undo_sources(): Do not prematurely call
fil_crypt_threads_end(). Many pages can still be written by
change buffer merge, rollback of incomplete transactions, and
purge, especially in slow shutdown (innodb_fast_shutdown=0).

innobase_shutdown_for_mysql(): Call fil_crypt_threads_cleanup()
also when innodb_read_only=1, because the threads will have been
created also in that case.

sync_check_close(): Re-enable the invocation of sync_latch_meta_destroy()
to free the mutex list.
2016-12-22 15:25:23 +02:00
Marko Mäkelä
545c912696 Remove an unnecessary comparison. 2016-12-22 15:10:39 +02:00
Marko Mäkelä
7e02fd1f71 MDEV-11630 Call mutex_free() before freeing the mutex list
Make some global fil_crypt_ variables static.

fil_close(): Call mutex_free(&fil_system->mutex) also in InnoDB, not
only in XtraDB. In InnoDB, sync_close() was called before fil_close().

innobase_shutdown_for_mysql(): Call fil_close() before sync_close(),
similar to XtraDB shutdown.

fil_space_crypt_cleanup(): Call mutex_free() to pair with
fil_space_crypt_init().

fil_crypt_threads_cleanup(): Call mutex_free() to pair with
fil_crypt_threads_init().
2016-12-22 14:33:58 +02:00
Jan Lindström
55eb7120a0 MDEV-11218: encryption.innodb_encryption_discard_import failed in buildbot
Try to stabilize test cases. These test behave badly when run in certain order.
2016-12-22 14:02:51 +02:00
Sergey Vojtovich
0c3791caae MDEV-11227 - mysqlimport -l doesn't issue UNLOCK TABLES
Merged fix for innodb_mysql from 5.7.
2016-12-22 14:02:27 +04:00
Vladislav Vaintroub
9b10b4181f update libmariadb, and fix debian packaging for client libs 2016-12-22 10:17:26 +01:00
Monty
c51c885dee Fixed compiler warning 2016-12-21 22:41:07 +02:00
Monty
c33c638f39 MDEV-7558 analyze_stmt_slow_query_log fails sporadically in buildbot
The reason was that the test was reusing the same log file without deleting it between tests.
Fixed by creating a new log file as part of the test
2016-12-21 22:40:52 +02:00
Sergey Vojtovich
27f20d192c Updated test results 2016-12-21 13:21:38 +04:00
Sergey Vojtovich
8774a02364 MDEV-11227 - mysqlimport -l doesn't issue UNLOCK TABLES
Implementation of MDEV-7660 introduced unwanted incompatible change:
modifications under LOCK TABLES with autocommit enabled are rolled back on
disconnect. Previously everything was committed, because LOCK TABLES didn't
adjust autocommit setting.

This patch restores original behavior by reverting some changes done in
MDEV-7660:
- sql/sql_parse.cc: do not reset autocommit on LOCK TABLES
- sql/sql_base.cc: do not set autocommit on UNLOCK TABLES
- test cases: main.lock_tables_lost_commit, main.partition_explicit_prune,
  rpl.rpl_switch_stm_row_mixed, tokudb.nested_txn_implicit_commit,
  tokudb_bugs.db806

But it makes InnoDB tables under LOCK TABLES ... READ [LOCAL] not protected
against DML. To restore protection some changes from WL#6671 were merged,
specifically MDL_SHARED_READ_ONLY and test cases.

WL#6671 merge highlights:
- Not all tests merged.
- In MySQL LOCK TABLES ... READ acquires MDL_SHARED_READ_ONLY for all engines,
  in MariaDB MDL_SHARED_READ is always acquired first and then upgraded to
  MDL_SHARED_READ_ONLY for InnoDB only.
- The above allows us to omit MDL_SHARED_WRITE_LOW_PRIO implementation in
  MariaDB, which is rather useless with InnoDB. In MySQL it is needed to
  preserve locking behavior between low priority writes and LOCK TABLES ... READ
  for non-InnoDB engines (covered by sys_vars.sql_low_priority_updates_func).
- Omitted HA_NO_READ_LOCAL_LOCK, we rely on lock_count() instead.
- Omitted "piglets": in MariaDB stream of DML against InnoDB table may lead to
  concurrent LOCK TABLES ... READ starvation.
- HANDLER ... OPEN acquires MDL_SHARED_READ instead of MDL_SHARED in MariaDB.
- Omitted SNRW->X MDL lock upgrade for IMPORT/DISCARD TABLESPAECE under LOCK
  TABLES.
- Omitted strong locks for views, triggers and SP under LOCK TABLES.
- Omitted IX schema lock for LOCK TABLES READ.
- Omitted deadlock weight juggling for LOCK TABLES.

Full WL#6671 merge status:
- innodb.innodb-lock: fully merged
- main.alter_table: not merged due to different HANDLER solution
- main.debug_sync: fully merged
- main.handler_innodb: not merged due to different HANDLER solution
- main.handler_myisam: not merged due to different HANDLER solution
- main.innodb_mysql_lock: fully merged
- main.insert_notembedded: fully merged
- main.lock: not merged (due to no strong locks for views)
- main.lock_multi: not merged
- main.lock_sync: fully merged (partially in MDEV-7660)
- main.mdl_sync: not merged
- main.partition_debug_sync: not merged due to different HANDLER solution
- main.status: fully merged
- main.view: fully merged
- perfschema.mdl_func: not merged (no such test in MariaDB)
- perfschema.table_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
- sys_vars.sql_low_priority_updates_func: not merged
- include/thr_rwlock.h: not merged, rw_pr_lock_assert_write_owner and
  rw_pr_lock_assert_not_write_owner are macros in MariaDB
- sql/handler.h: not merged (HA_NO_READ_LOCAL_LOCK)
- sql/mdl.cc: partially merged (MDL_SHARED_READ_ONLY only)
- sql/mdl.h: partially merged (MDL_SHARED_READ_ONLY only)
- sql/lock.cc: fully merged
- sql/sp_head.cc: not merged
- sql/sp_head.h: not merged
- sql/sql_base.cc: partially merged (MDL_SHARED_READ_ONLY only)
- sql/sql_base.h: not merged
- sql/sql_class.cc: fully merged
- sql/sql_class.h: fully merged
- sql/sql_handler.cc: merged partially (different solution in MariaDB)
- sql/sql_parse.cc: partially merged, mostly omitted low priority write part
- sql/sql_reload.cc: not merged comment change
- sql/sql_table.cc: not merged SNRW->X upgrade for IMPORT/DISCARD TABLESPACE
- sql/sql_view.cc: not merged
- sql/sql_yacc.yy: not merged (MDL_SHARED_WRITE_LOW_PRIO, MDL_SHARED_READ_ONLY)
- sql/table.cc: not merged (MDL_SHARED_WRITE_LOW_PRIO)
- sql/table.h: not merged (MDL_SHARED_WRITE_LOW_PRIO)
- sql/trigger.cc: not merged
- storage/innobase/handler/ha_innodb.cc: merged store_lock()/lock_count()
  changes (in MDEV-7660), didn't merge HA_NO_READ_LOCAL_LOCK
- storage/innobase/handler/ha_innodb.h: fully merged in MDEV-7660
- storage/myisammrg/ha_myisammrg.cc: not merged comment change
- storage/perfschema/table_helper.cc: not merged (no MDL support in MariaDB PFS)
- unittest/gunit/mdl-t.cc: not merged
- unittest/gunit/mdl_sync-t.cc: not merged

MariaDB specific changes:
- handler.heap: different HANDLER solution, MDEV-7660
- handler.innodb: different HANDLER solution, MDEV-7660
- handler.interface: different HANDLER solution, MDEV-7660
- handler.myisam: different HANDLER solution, MDEV-7660
- main.mdl_sync: MDEV-7660 specific changes
- main.partition_debug_sync: removed test due to different HANDLER solution,
  MDEV-7660
- main.truncate_coverage: removed test due to different HANDLER solution,
  MDEV-7660
- mysql-test/include/mtr_warnings.sql: additional cleanup, MDEV-7660
- mysql-test/lib/v1/mtr_report.pl: additional cleanup, MDEV-7660
- plugin/metadata_lock_info/metadata_lock_info.cc: not in MySQL
- sql/sql_handler.cc: MariaDB specific fix for mysql_ha_read(), MDEV-7660
2016-12-21 13:18:45 +04:00
Sachin Setiya
9e032d6150 MDEV-11490 Galera_3nodes test suite does not suppress Warnings.
Problem:- While running individual tests of Galera_3nodes ,
  We get warnings like '[Warning] WSREP: Could not open state file
  for reading: '. And because of this individual tests fails.

  Solution:- We change suite.pm of Galera_3nodes to supress these warnings.
2016-12-21 09:37:58 +05:30
Marko Mäkelä
561b6d213c Revert "Merge pull request #275 from grooverdan/10.2-MDEV-11075-crc32-runtime-detect-getauxval"
This reverts commit edf4cc7519, reversing
changes made to 9320d8ae30.
2016-12-20 22:46:29 +02:00
Marko Mäkelä
229dd711d4 MDEV-11585 Hard-code the shared InnoDB temporary tablespace ID
Try hard-coding the ID as -2 instead of -1, so that they will not
be confused with ULINT_UNDEFINED on 32-bit platforms.
2016-12-20 22:42:13 +02:00
Nirbhay Choubey
75ab65aece Fix failing galera tests. 2016-12-20 15:31:18 -05:00
Igor Babaev
9b27d3e86e Fixed bug mdev-11608.
The fix for bug mdev-11488 introduced the virtual method
convert_to_basic_const_item for the class Item_cache.
The implementation of this method for the class Item_cache_str
was not quite correct: the server could crash if the cached item
was null.
A similar problem could appear for the implementation of
this method for the class Item_cache_decimal. Although I could not
reproduce the problem I decided to change the code appropriately.
2016-12-20 11:08:50 -08:00
Alexey Botchkov
95228dc80b MDEV-11570 JSON_MERGE returns incorrect result.
JSON merging fixed.
2016-12-20 17:32:08 +04:00
Marko Mäkelä
195241e125 Port the test innodb.doublewrite from MySQL 5.7. 2016-12-20 15:03:56 +02:00
Marko Mäkelä
83dbb2d43a MDEV-11487 Revert InnoDB internal temporary tables from WL#7682
Post-push fix: Remove the orphaned file sess0sess.h.
2016-12-20 12:07:33 +02:00
Igor Babaev
1152b076d4 Corrected a test from func_date_add.test 2016-12-20 00:42:46 -08:00
Igor Babaev
5c69879f3b Fixed bug mdev-11593.
When a condition containing NULLIF is pushed into a materialized
view/derived table the clone of the Item_func_nullif item must
be processed in a special way to guarantee that the first argument
points to the same item as the third argument.
2016-12-20 00:42:46 -08:00
Nirbhay Choubey
2c734e7be3 MDEV-10993: wsrep.mdev_10186 result depends on location of
... galera library

Update test case.
2016-12-19 16:12:26 -05:00
Nirbhay Choubey
b800264e49 MDEV-11152: wsrep_replicate_myisam: SELECT gets replicated
... using TO

Fixed the 'wsrep_replicate_myisam' check to allow only limited
set of commands. Added a debug assert to discover such cases.
2016-12-19 16:11:27 -05:00
Nirbhay Choubey
d51e7f90bf MDEV-10957: Assertion failure when dropping a myisam table
... with wsrep_replicate_myisam enabled

Internal updates to system statistical tables could wrongly
trigger an additional total-order replication if wsrep_repli
-cate_myisam is enabled.
Fixed by adding a check to skip total-order replication for
stat tables.
2016-12-19 15:55:55 -05:00
Nirbhay Choubey
9d8ec1468a MDEV-11082: Fix mysql_client_test.c (by Elena) 2016-12-19 15:50:33 -05:00
Nirbhay Choubey
da4babb86c Fix failing galera tests. 2016-12-19 15:47:01 -05:00
Marko Mäkelä
a01bfc9fc2 MDEV-11602 InnoDB leaks foreign key metadata on DDL operations
Essentially revert MDEV-6759, which addressed a double free of memory
by removing the freeing altogether, introducing the memory leaks.
No double free was observed when running the test suite -DWITH_ASAN.

Replace some mem_heap_free(foreign->heap) with dict_foreign_free(foreign)
so that the calls can be located and instrumented more easily when needed.
2016-12-19 17:27:15 +02:00
Marko Mäkelä
44da95e5ed Merge branch '10.0' into 10.1 2016-12-19 17:15:25 +02:00
Marko Mäkelä
8375a2c1ce MDEV-11585 Hard-code the shared InnoDB temporary tablespace ID at -1
MySQL 5.7 supports only one shared temporary tablespace.

MariaDB 10.2 does not support any other shared InnoDB tablespaces than
the two predefined tablespaces: the persistent InnoDB system tablespace
(default file name ibdata1) and the temporary tablespace
(default file name ibtmp1).

InnoDB is unnecessarily allocating a tablespace ID for the predefined
temporary tablespace on every startup, and it is in several places
testing whether a tablespace ID matches this dynamically generated ID.

We should use a compile-time constant to reduce code size and to avoid
unnecessary updates to the DICT_HDR page at every startup.

Using a hard-coded tablespace ID will should make it easier to remove the
TEMPORARY flag from FSP_SPACE_FLAGS in MDEV-11202.
2016-12-19 16:24:10 +02:00
Marko Mäkelä
9f863a15b0 MDEV-11602 InnoDB leaks foreign key metadata on DDL operations
Essentially revert MDEV-6759, which addressed a double free of memory
by removing the freeing altogether, introducing the memory leaks.
No double free was observed when running the test suite -DWITH_ASAN.

Replace some mem_heap_free(foreign->heap) with dict_foreign_free(foreign)
so that the calls can be located and instrumented more easily when needed.
2016-12-19 15:57:41 +02:00
Daniel Black
c35b8c46b4 Travis: parallel_jobs=3 2016-12-17 09:56:21 +04:00
Marko Mäkelä
6f4f9f2843 MDEV-6076 Adjust a test result.
This test and result had been modified in WL#6204, but I missed it,
because the test had been renamed from
innodb_fts.innodb_fts_plugin to innodb_fts.plugin.
2016-12-16 17:15:40 +02:00
Alexey Botchkov
ce55094f4f MDEV-11572 JSON_DEPTH returns wrong results.
JSON depth calculation fixed.
2016-12-16 14:06:12 +04:00
Alexey Botchkov
30c231b03a MDEV-11569 JSON_ARRAY_INSERT produces an invalid result.
String insertion fixed.
2016-12-16 13:51:35 +04:00
Alexey Botchkov
beded4350f MDEV-JSON_CONTAINS_PATH returns incorrect results and produces wrong warning.
The Item_func_json_contains_path was mistakenly set with the
        no '*' paths limitation.
2016-12-16 12:43:44 +04:00