Commit graph

165171 commits

Author SHA1 Message Date
Sergei Golubchik
2300fe2e0e Identical key derivation code in XtraDB/InnoDB/Aria
* Extract it into the "encryption_scheme" service.
* Make these engines to use the service, remove duplicate code.
* Change MY_AES_xxx error codes, to return them safely
  from encryption_scheme_encrypt/decrypt without conflicting
  with ENCRYPTION_SCHEME_KEY_INVALID error
2015-05-15 18:12:01 +02:00
Alexander Barkov
632f2307f7 MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
Step#5: changing the function remove_eq_conds() into a virtual method in Item.
It removes 6 virtual calls for Item_func::type(), and adds only 2
virtual calls for Item***::remove_eq_conds().
2015-05-14 17:42:40 +04:00
Sergey Vojtovich
fb3e9352a4 MDEV-7999 - PROFILING routines take 0.2% when profiling disabled
This is an addition to original patch. Embedded server does extra calls of
PROFILING::start_new_query() and PROFILING::finish_current_query(), which
cause DBUG_ASSERT(!current) failure.

Removed these extra calls: dispatch_command() does all needed job.
2015-05-14 12:50:23 +04:00
Sergey Vojtovich
18f88d6d94 MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
Avoid calling current_thd from thd_kill_level(). This reduces number of
pthread_getspecific() calls from 776 to 354.

Also thd_kill_level(NULL) is not permitted anymore: this saves one condition.
2015-05-14 12:50:23 +04:00
Jan Lindström
f8cacd03a7 MDEV-8143: InnoDB: Database page corruption on disk or a failed file read
Analysis: Problem was that we did create crypt data for encrypted table but
this new crypt data was not written to page 0. Instead a default crypt data
was written to page 0 at table creation.

Fixed by explicitly writing new crypt data to page 0 after successfull
table creation.
2015-05-14 11:32:24 +03:00
Alexander Barkov
be2038e3cb MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
Step 4 (there will be more)
2015-05-13 17:08:24 +04:00
Sergei Golubchik
16b6ec2e37 MDEV-8130 Wrong error code/message while encrypting a partitioned InnoDB table
when checking for a flag, use & not ==
2015-05-13 14:27:18 +02:00
Sergei Golubchik
def48e623a MDEV-8141 InnoDB: background encryption thread uses FIL_DEFAULT_ENCRYPTION_KEY
* check key version per key id (that is, per tablespace).
* wake encryption thread when a tablespace needs re-encryption
2015-05-13 14:27:17 +02:00
Sergei Golubchik
6e4c22af7c example encryption plugin supports key ids
show that two keys can be different if they have different ids
or different versions.
2015-05-13 14:27:17 +02:00
Sergei Golubchik
acd992d4b7 MDEV-8022 Assertion `rc == 0' failed in ma_encrypt on dropping an encrypted Aria table
fix encryption of the last partial block

* now really encrypt it, using key and iv
* support the case of very short plaintext (less than one block)
* recommend aes_ctr over aes_cbc, because the former
  doesn't have problems with partial blocks
2015-05-13 14:27:17 +02:00
Sergei Golubchik
bea3f30d46 move AES_CTR to its own greatly simplified function
* don't use do_crypt() for stream cipher AES_CTR
* rename do_crypt to block_crypt to emphasize its specialization
2015-05-13 14:27:17 +02:00
Sergei Golubchik
a583976e78 MDEV-8015 InnoDB: Failing assertion: new_state->key_version != ENCRYPTION_KEY_VERSION_INVALID
InnoDB: refuse to start if encryption is requested but no
encryption plugin is available
2015-05-13 14:27:17 +02:00
Sergei Golubchik
b4777bfc11 cleanup, use encryption_key_id_exists() where appropriate 2015-05-13 14:27:17 +02:00
Sergei Golubchik
66380916ee MDEV-7993 file_key_management_filekey doesn't work as expected with FILE:
chomp the secret, as read from the file. remove trailing
CR and LF bytes.
2015-05-13 14:27:16 +02:00
Sergei Golubchik
a35b538bde MDEV-8040 make aria encryption use real keys
applied upstream patch
2015-05-13 14:27:16 +02:00
Sergei Golubchik
ab8415d983 move encryption tests to a dedicate suite
remove few tests for variables that never existed (merge error)
2015-05-13 14:27:16 +02:00
Sergey Vojtovich
b22959903b MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
Added THD argument to select_result and all derivative classes.
This reduces number of pthread_getspecific calls from 796 to 776 per OLTP RO
transaction.
2015-05-13 15:56:56 +04:00
Alexander Barkov
8345bc6921 MDEV-8152 is_columns_is_embedded test fails
Recording --ps and --embedded tests (a postfix for MDEV-7807)
2015-05-13 15:34:20 +04:00
Sergey Vojtovich
9851a8193f MDEV-8001 - mysql_reset_thd_for_next_command() takes 0.04% in OLTP RO
Removed yet more mysql_reset_thd_for_next_command(). Call
THD::reset_for_next_command() directly instead.
2015-05-13 15:28:34 +04:00
Sergey Vojtovich
4d1ccc4289 MDEV-7951 - sql_alloc() takes 0.25% in OLTP RO
sql_alloc() has additional costs compared to direct mem_root allocation:
- function call: it is defined in a separate translation unit and can't be
  inlined
- it needs to call pthread_getspecific() to get THD::mem_root

It is called dozens of times implicitely at least by:
- List<>::push_back()
- List<>::push_front()
- new (for Sql_alloc derived classes)
- sql_memdup()

Replaced lots of implicit sql_alloc() calls with direct mem_root allocation,
passing through THD pointer whenever it is needed.

Number of sql_alloc() calls reduced 345 -> 41 per OLTP RO transaction.
pthread_getspecific() overhead dropped 0.76 -> 0.59
sql_alloc() overhed dropped 0.25 -> 0.06
2015-05-13 10:43:14 +04:00
Sergey Vojtovich
c4d2c4e844 MDEV-7964 - delete_dynamic() takes 0.12% in OLTP RO
delete_dynamic() was called 9-11x per OLTP RO query + 3x per BEGIN/COMMIT.

3 calls were performed by LEX_MASTER_INFO. Added condition to call those only
for CHANGE MASTER.

1 call was performed by lock_table_names()/Hash_set/my_hash_free(). Hash_set was
supposed to be used for DDL and LOCK TABLES to gather database names, while it
was initialized/freed for DML too. In fact Hash_set didn't do any useful job
here. Hash_set was removed and MDL requests are now added directly to the list.

The rest 5-7 calls are done by optimizer, mostly by Explain_query and friends.
Since dynamic arrays are used in most cases, they can hardly be optimized.

my_hash_free() overhead dropped 0.02 -> out of radar.
delete_dynamic() overhead dropped 0.12 -> 0.04.
2015-05-13 10:43:14 +04:00
Sergey Vojtovich
7cfa803d8e MDEV-8001 - mysql_reset_thd_for_next_command() takes 0.04% in OLTP RO
Removed mysql_reset_thd_for_next_command(). Call THD::reset_for_next_command()
directly instead.

mysql_reset_thd_for_next_command() overhead dropped 0.04% -> out of radar.
THD::reset_for_next_command() overhead didn't increase.
2015-05-13 10:43:14 +04:00
Sergey Vojtovich
5cfb6b479e MDEV-7999 - PROFILING routines take 0.2% when profiling disabled
PROFILING::start_new_query() optimizations:
- no need to check "current": added assertion instead
- "enabled" now means "is enabled currently" instead of "was enabled at query
  start". Old meaning was useless, new meaning echoes OPTION_PROFILING so
  that start_new_query() can be defined in sql_profile.h.
- remnants of start_new_query() moved to sql_profile.h so it can be inlined

PROFILING::start_new_query() overhead dropped 0.08% -> out of radar.

PROFILING::set_query_source() optimizations:
- no need to check "enabled": !enabled && current is impossible
- remnants of set_query_source() moved to sql_profile.h so it can be inlined

PROFILING::set_query_source() overhead dropped 0.02% -> out of radar.

PROFILING::finish_current_query() optimizations:
- moved "current" check out to sql_profile.h so it can be inlined

PROFILING::finish_current_query() overhead dropped 0.10% -> out of radar.
2015-05-13 10:43:13 +04:00
Sergey Vojtovich
55d5af733d MDEV-7945 - THD::enter_stage() takes 0.48% in OLTP RO
THD::enter_stage() optimizations:
- stage backup code moved to THD::backup_stage(), saves one condition
- moved check for "new_stage" out to callers that actually need it
- remnants of enter_stage() moved to sql_class.h so it can be inlined

THD::enter_stage() overhead dropped 0.48% -> 0.07%.

PROFILING::status_change() optimizations:
- "status_arg" is now checked by QUERY_PROFILE::new_status()
- no need to check "enabled": !enabled && current is impossible
- remnants of status_change() moved to sql_profile.h so it can be inlined

PROFILING::status_change() overhead dropped 0.1% -> out of radar.
2015-05-13 10:43:13 +04:00
Nirbhay Choubey
c8ad5b2f12 MDEV-8151 : wsrep.foreign_key, wsrep.pool_of_threads,
.. wsrep.binlog_format, wsrep.mdev_6832 fail in buildbot

Galera-3.9 logs an additional warning in the error log if
it fails to find gvwstate.dat file. Update wsrep/suite.pm.
2015-05-12 17:15:16 -04:00
Alexey Botchkov
8c54182ec4 MDEV-7926 Server crashes in get_geometry_column_record on concurrent SELECT FROM I_S.GEOMETRY_COLUMNS with join and DDL.
The bug was that open_tables() returned error in case of
        thd->killed() without properly calling thd->send_kill_message()
        to set the correct error. This was fixed already in get_schema_column_record,
        so the code was just copied to get_geometry_column_record.
2015-05-12 15:09:28 +05:00
Jan Lindström
58e8db2eb3 MDEV-7942: InnoDB: abuse of UNIV_LIKELY()/UNIV_UNLIKELY()
UNIV_LIKELY()/UNIV_UNLIKELY() hints are supposed to improve branch prediction.
Currently, they're expected to work only if cond evaluates to TRUE or FALSE.

However there're a few conditions that may evaluate to different values, e.g.:

page/page0zip.cc:		if (UNIV_LIKELY(c_stream->avail_in)) {
page/page0zip.cc:			if (UNIV_LIKELY(c_stream->avail_in)) {
dict/dict0mem.cc:		if (UNIV_LIKELY(i) && UNIV_UNLIKELY(!table->col_names)) {

Fixed these conditions so that they evaluate TRUE/FALSE.
2015-05-11 14:31:42 +03:00
Kristian Nielsen
6e49201644 Fix compilation warnings in -DWITH_WSREP=OFF build. 2015-05-11 12:47:43 +02:00
Kristian Nielsen
8bedb638d7 MDEV-8113: Parallel slave: slave hangs on ALTER TABLE (or other DDL) as the first event after slave start
In optimistic parallel replication, it is not safe to try to run a following
transaction in parallel with a DDL statement, and there is code to prevent
this.

However, the code was missing the case where the DDL is the very first event
after slave start. In this case, following transactions could run in
parallel with the DDL, which can cause the slave to hang or even corrupt
slave in unlucky cases.
2015-05-11 12:43:38 +02:00
Jan Lindström
ecfc3de57e MDEV-8129: Compilation warnings in log0crypt.cc
Fix incorrect types and compiler warnings.
2015-05-11 12:22:13 +03:00
Sergei Golubchik
5fdb14542a MDEV-8021 "InnoDB: Tablespace id 4 encrypted but encryption service not available. Can't continue opening tablespace" on server restart when there are encrypted tables
key id was written in the wrong place on the tablespace first page
(thus its value could not be read back later)
2015-05-09 11:19:36 +02:00
Jan Lindström
d259376fd8 MDEV-8041: InnoDB redo log encryption
Merged new version of InnoDB/XtraDB redo log encryption from Google
provided by Jonas Oreland.
2015-05-09 11:13:00 +03:00
Vicentiu Ciorbaru
ab54f5a8b9 Fix win/ files to be stored with LF in repository
On Windows, the files get checked out with CRLF thanks
to .gitattributes.
2015-05-08 17:31:54 +03:00
Vicentiu Ciorbaru
5ae8d06df4 Update .gitattributes 2015-05-08 17:27:41 +03:00
Alexander Barkov
91ee98a8c8 MDEV-7807 information_schema.processlist truncates queries with binary strings
Adding a new column INFORMATION_SCHEMA.PROCESSLIST.INFO_BINARY.
2015-05-08 00:34:06 +04:00
Sergey Vojtovich
0fcc350f73 MDEV-7922 - ERROR 1939 (HY000): Engine PERFORMANCE_SCHEMA failed to discover
table

Performance schema discovery fails if connection has no active database set.

This happened due to restriction in SQL parser: table name with no database name
is ambiguous in such case.

Fixed by temporary substitution of default database with being discovered table
database.
2015-05-07 20:45:46 +04:00
Jan Lindström
3832bda1ba Fix compiler error if compiler does not support c99 style
initializers.
2015-05-07 18:30:42 +03:00
Sergey Vojtovich
7ed673f35c MDEV-7793 - Race condition between XA COMMIT/ROLLBACK and disconnect
XA COMMIT/ROLLBACK of XA transaction owned by different thread may access
freed memory if that thread disconnects at the same time.

Also concurrent XA COMMIT/ROLLBACK of recovered XA transaction were not
serialized properly.
2015-05-07 16:48:30 +04:00
Jan Lindström
b975685036 Merge pull request #54 from openquery/MDEV-8053-c99-style-for-structure-members-10.1
c99 style for assigning structure members
2015-05-06 16:48:17 +03:00
Jan Lindström
bad81f23f6 MDEV-8046: Server crashes in pfs_mutex_enter_func on select from I_S.INNODB_TABLESPACES_ENCRYPTION if InnoDB is disabled
Problem was that information schema tables innodb_tablespaces_encryption and
innodb_tablespaces_scrubbing where missing required check is InnoDB enabled
or not.
2015-05-06 15:16:28 +03:00
Jan Lindström
2f25c653ad MDEV-8074: Failing assertion: mutex->magic_n == MUTEX_MAGIC_N in file sync0sync.cc line 508
Problem was that e.g. on crash recovery fil_space_crypt_close_tablespace
and fil_space_crypt_mark_space_closing access mutex that is not yet
initialized. Mutex is naturally initialized only if encryption is
configured.
2015-05-06 14:11:30 +03:00
Kristian Nielsen
31c069eb4d Merge MDEV-8103 into 10.1 2015-05-06 12:46:33 +02:00
Kristian Nielsen
ef99edf1a8 MDEV-8103: Missing DBUG_RETURN in open_table_uncached() 2015-05-06 12:24:15 +02:00
Alexander Barkov
539b3ca87d - Moving Item_func_spatial_mbr_rel from Item_bool_func2 to Item_bool_func,
as Item_func_spatial_mbr_rel needs nothing from Item_bool_func2.
- Renaming Item_func_spacial_rel (the class that implements precise spacial
  relations) to Item_func_spatial_precise_rel
- Adding a new abstract class Item_func_spatial_rel as a common parent
  for Item_func_spatial_precise_rel and Item_func_spatial_mbr_rel.
2015-05-05 09:30:17 +04:00
Alexander Barkov
2fe4d0e6cd MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
Step #3: Splitting the function check_equality() into a method in Item.
Implementing Item::check_equality() and Item_func_eq::check_equality().
Implement Item_func_eq::build_equal_items() in addition to
Item_func::build_equal_items() and moving the call for check_equality()
from Item_func::build_equal_items() to Item_func_eq::build_equal_items().
2015-05-05 01:09:47 +04:00
Alexander Barkov
9090c3efca Adding a test for Item_default_value in WHERE condition:
WHERE DEFAULT(col)
It seems this was not covered yet.
2015-05-04 18:31:24 +04:00
Alexander Barkov
462bca365a MDEV-7950 Item_func::type() takes 0.26% in OLTP RO
Step 2c:

After discussion with Igor, it appeared that Item_field and Item_ref
could not appear in this context in the old function build_equal_item_for_cond:

  else if (cond->type() == Item::FUNC_ITEM ||
           cond->real_item()->type() == Item::FIELD_ITEM)

The part of the condition checking for Item_field::FIELD_ITEM was a dead code.
- Moving implementation of Item_ident_or_func_or_sum::build_equal_items()
to Item_func::build_equal_items()
- Restoring deriving of Item_ident and Item_sum_or_func from Item_result_field.
  Removing Item_ident_or_func_or_sum.
2015-05-04 18:12:31 +04:00
Jan Lindström
53382ac128 MDEV-8079: Crash when running MariaDB Debug with InnoDB on Windows
Problem was that std::vector was allocated using calloc instead of
new, this caused vector constructor not being called and vector
metadata not initialized.
2015-05-01 14:23:08 +03:00
Sergey Vojtovich
ff1e0821d1 MDEV-8073 - Build error in sql/mdl.cc on OS X 10.10
C++03 seem to require default constructor for const objects even though they're
fully initialized. There is defect report for this:
http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#253

Apparently some compilers (e.g. gcc) addressed this defect report, while some
(e.g. clang) did not.

Added empty default constructor to MDL_scoped_lock and MDL_object_lock classes.

Also replaced lf_hash_search_using_hash_value() with lf_hash_search(). The
latter will call mdl_key->hash_value() anyway.
2015-04-30 12:36:39 +04:00
Alexander Barkov
499deca7fc A clean-up for c8141f5314
MDEV-7950 Item_func::type() takes 0.26% in OLTP RO (Step#2)

- Item_ref was doing unnecessary extra job after the "MDEV-7950 Step#2" patch.
  Fallback to Item::build_equal_items() if real_type() is not FIELD_ITEM.
  Note, Item_ref::build_equal_items() will most likely be further simplified.
  Waiting for Sanja and Igor to check a possibly dead code.

- Safety: Adding Item_sum::build_equal_items() with ASSERT, as Item_sum
  should never appear in build_equal_items() context.
2015-04-29 17:53:27 +04:00