Commit graph

3447 commits

Author SHA1 Message Date
Sergei Golubchik
5dffda3ccc Merge branch 'bb-10.1-jan-encryption' into bb-10.1-serg
With changes:

* update tests to pass (new encryption/encryption_key_id syntax).
* not merged the code that makes engine aware of the encryption mode
  (CRYPT_SCHEME_1_CBC, CRYPT_SCHEME_1_CTR, storing it on disk, etc),
  because now the encryption plugin is handling it.
* compression+encryption did not work in either branch before the
  merge - and it does not work after the merge. it might be more
  broken after the merge though - some of that code was not merged.
* page checksumming code was not moved (moving of page checksumming
  from fil_space_encrypt() to fil_space_decrypt was not merged).
* restored deleted lines in buf_page_get_frame(), otherwise
  innodb_scrub test failed.
2015-04-09 19:27:40 +02:00
Sergei Golubchik
129e960179 fix log_blocks_crypt() to actually decrypt the encrypted log
It used to double-encrypt it, relying on the fact that second
encrypt() call was (like XOR) negating the effect of the
first one.
2015-04-09 19:06:11 +02:00
Sergei Golubchik
d6b912c64d update XtraDB/InnoDB plugin maturity to match the server
because of encryption changes - make it beta and let
it mature together with the server
2015-04-09 18:44:02 +02:00
Sergei Golubchik
f130da7b00 clarify/simplify new innodb sysvars: help texts 2015-04-09 18:42:45 +02:00
Sergei Golubchik
87cf86519e clarify/simplify new innodb sysvars: innodb-scrub-force-testing
rename to innodb-debug-force-scrubbing
2015-04-09 18:42:45 +02:00
Sergei Golubchik
19e76814b7 clarify/simplify new innodb sysvars: innodb-scrub-log-interval
rename to innodb-scrub-log-speed
2015-04-09 18:42:45 +02:00
Sergei Golubchik
72c8b3fcb2 small cleanups as per review 2015-04-09 18:42:44 +02:00
Sergei Golubchik
4d40a7d133 remove now-empty my_aes.{h,cc}
move remaning defines to my_crypt, add MY_ namespace prefix
2015-04-09 18:42:44 +02:00
Sergei Golubchik
65e7826070 renames to follow single consistent naming style
with namespace prefixes
2015-04-09 18:42:44 +02:00
Sergei Golubchik
bb1b61b312 encryption plugin controls the encryption
* no --encryption-algorithm option anymore
* encrypt/decrypt methods in the encryption plugin
* ecnrypt/decrypt methods in the encryption_km service
* file_km plugin has --file-key-management-encryption-algorithm
* debug_km always uses aes_cbc
* example_km changes between aes_cbc and aes_ecb for different key versions
2015-04-09 18:42:43 +02:00
Sergei Golubchik
9ccafffc29 rename "encryption key management plugin" to "encryption plugin"
because it's going to do more than just key management
2015-04-09 18:42:43 +02:00
Sergei Golubchik
6a3932fdd0 use key derivation procedure for all encryption algorithms 2015-04-08 10:58:51 +02:00
Sergei Golubchik
ef5b4889c2 optimize encryption api
only one encryption key lookup in most cases instead of three
(has_key, get_key_size, get_key).
2015-04-08 10:58:50 +02:00
Jan Lindström
4865fd105a InnoDB/XtraDB Encryption cleanup
Step 5:
-- Rename encryption_key -> encryption_key_id
-- Remove unnecessary code
-- Fix few bugs found
-- Fix test cases and results files
2015-04-07 23:44:57 +02:00
Jan Lindström
af768c2f22 InnoDB/XtraDB Encryption code cleanup
Step 4:
-- Review fixes
-- Rename pages_page_* to pages_*
-- Remove unnecessary code
2015-04-07 23:44:57 +02:00
Jan Lindström
71ec0463af Encryption cleanup
Step 3:

-- Make encrytion_algorithm changeable by SUPER
-- Remove AES_ECB method from encryption_algorithms
-- Support AES method change by storing used method on InnoDB/XtraDB objects
-- Store used AES method to crypt_data as different crypt types
-- Store used AES method to redo/undo logs and checkpoint
-- Store used AES method on every encrypted page after key_version
-- Add test
2015-04-07 23:44:57 +02:00
Jan Lindström
0ba9fa35bc InnoDB/XtraDB Encryption cleanup
Step 2:

-- Introduce temporal memory array to buffer pool where to allocate
temporary memory for encryption/compression
-- Rename PAGE_ENCRYPTION -> ENCRYPTION
-- Rename PAGE_ENCRYPTION_KEY -> ENCRYPTION_KEY
-- Rename innodb_default_page_encryption_key -> innodb_default_encryption_key
-- Allow enable/disable encryption for tables by changing
 ENCRYPTION to enum having values DEFAULT, ON, OFF
-- In create table store crypt_data if ENCRYPTION is ON or OFF
-- Do not crypt tablespaces having ENCRYPTION=OFF
-- Store encryption mode to crypt_data and redo-log
2015-04-07 23:44:56 +02:00
Jan Lindström
b4a4d82396 InnoDB/XtraDB Encryption cleanup.
Step 1:
-- Remove page encryption from dictionary (per table
encryption will be handled by storing crypt_data to page 0)
-- Remove encryption/compression from os0file and all functions
before that (compression will be added to buf0buf.cc)
-- Use same CRYPT_SCHEME_1 for all encryption methods
-- Do some code cleanups to confort InnoDB coding style
2015-04-07 23:44:56 +02:00
Sergei Golubchik
2f8d101f9e unify my_{en|de}crypt_{cbc|ecb|ctr}. no yassl support yet. 2015-04-05 13:14:37 +02:00
Sergei Golubchik
6a7ee5a5a5 encryption cleanup: small changes
* comments
* move my_bytes_to_key() and my_aes_hex2uint() into file_key_management_plugin
* rename HA_INSERT_ORDER -> HA_PRESERVE_INSERT_ORDER
* remove unused variables and declarations
* fix casts
* don't link innodb with pcre
* remove redundant entries from aria's TARGET_LINK_LIBRARIES
2015-04-05 13:14:37 +02:00
Jan Lindström
cbc5157feb MDEV-7878: innodb_scrub_background fails sporadically in buildbot (Failing assertion: offset > 0 && offset < UNIV_PAGE_SIZE)
Problem was that when encrypted space was flushed page 0 is
also updated but crypt data offset was sometimes not
calculated.
2015-03-31 09:00:01 +03:00
Jan Lindström
3578419b77 MDEV-7797: file_key_management_plugin uses static IV for a key
Currently crypt data is written to file space always. Use
that to obtain random IV for every object (file).

Beatify code to confort InnoDB coding styles.

Conflicts:
	storage/innobase/fil/fil0crypt.cc
	storage/xtradb/fil/fil0crypt.cc
2015-03-20 13:09:07 +02:00
Jan Lindström
b3438f24a1 MDEV-7803: Encryption metadata (crypt_data) is written to InnoDB file space page 0 even when encryption is not enabled 2015-03-19 14:09:49 +02:00
Jan Lindström
bab0bdbb5c Introduce two debug crash points while writing crypt data to test
redo operations.
2015-03-19 07:13:07 +02:00
Jan Lindström
c4b268add0 InnoDB cleanup. Remove empty statements i.e. extra ; characters. 2015-03-16 09:14:13 +02:00
Jan Lindström
4d0e52189c MDEV-7770: Online DDL progress output contains incorrectly printed index names
Index names were not formatted correctly.
2015-03-13 09:38:08 +02:00
Jan Lindström
9f4ee16bf3 MDEV-7619: Improve long semaphore wait output
Add holder thread_id, file_name and line to output.
2015-03-11 07:31:57 +02:00
Jan Lindström
c8035da9b8 Add more information if page state is not correct. 2015-03-10 14:05:57 +02:00
Sergei Golubchik
20cacb0064 fix a crash of innodb.innodb_mutexes,innodb_plugin
add forgotten initialization of the mutex name
2015-03-07 13:27:49 +01:00
Sergei Golubchik
2db62f686e Merge branch '10.0' into 10.1 2015-03-07 13:21:02 +01:00
Sergei Golubchik
5f510a9175 Merge branch '5.5' into 10.0 2015-03-06 18:41:32 +01:00
Sergei Golubchik
17a37796e1 after innodb/xtradb merge: use the correct visibility for internal functions
otherwise innodb plugin might invoke xtradb function with the same name,
and that might crash (./mtr --emb innodb.strict_mode)
2015-03-06 18:13:06 +01:00
Jan Lindström
206b111b11 MDEV-7672: Crash creating an InnoDB table with foreign keys
Analysis: after a red-black-tree lookup we use node withouth
checking did lookup succeed or not. This lead to situation
where NULL-pointer was used.

Fix: Add additional check that found node from red-back-tree
is valid.
2015-03-06 11:19:23 +02:00
Sergey Vojtovich
e13459a11e MDEV-7148 - Recurring: InnoDB: Failing assertion: !lock->recursive
Re-applied lost in the merge revision:
commit ed313e8a92
Author: Sergey Vojtovich <svoj@mariadb.org>
Date:   Mon Dec 1 14:58:29 2014 +0400

    MDEV-7148 - Recurring: InnoDB: Failing assertion: !lock->recursive

    On PPC64 high-loaded server may crash due to assertion failure in InnoDB
    rwlocks code.

    This happened because load order between "recursive" and "writer_thread"
    wasn't properly enforced.
2015-03-05 15:30:11 +04:00
Jan Lindström
f66fbe8ce0 MDEV-7578 :Slave is ~10x slower to execute set of statements compared to master when using RBR
Analysis: On master when executing (single/multi) row INSERTs/REPLACEs
InnoDB fallback to old style autoinc locks (table locks)
only if another transaction has already acquired the AUTOINC lock.
Instead on slave as we are executing log_events and sql_command
is not correctly set, InnoDB does not use new style autoinc
locks when it could.

Fix: Use new style autoinc locks also when
thd_sql_command(user_thd) == SQLCOM_END i.e. this is RBR event.
2015-03-05 12:05:59 +02:00
Jan Lindström
143f5d9172 MDEV-7061: Augment innochecksum to give insight of fragmentation
Merged 615dd07d90 from https://github.com/facebook/mysql-5.6/
authored by rongrong. Removed C++11 requirement by using
std::map instead of std::unordered_set.

Add analysis to leaf pages to estimate how fragmented an index is
and how much benefit we can get out of defragmentation.
2015-03-04 16:57:31 +02:00
Jan Lindström
7047bef1ef Use standard InnoDB error mechanism on compression and encryption
error messages.
2015-03-02 10:55:48 +02:00
Sergei Golubchik
c3f80a2bff fix new innodb warnings to use the standard innodb warning syntax 2015-03-01 16:53:31 +01:00
Jan Lindström
2eae6848d9 MDEV-7572: InnoDB: Assertion failure in log_init_crypt_key if
file_key_management_plugin is used

Fixed error handling and added disabling InnoDB redo log encryption
if encryption key management plugin is not there.
2015-02-26 10:17:23 +02:00
Sergei Golubchik
6c09a72af5 Merge remote-tracking branch 'origin/10.0' into 10.0 2015-02-24 20:52:37 +01:00
Jan Lindström
8799f87075 MDEV-7623: Add lock wait time and hold time to every record/table lock in
InnoDB transaction lock printout.
2015-02-24 10:33:49 +02:00
Jan Lindström
90635c6fb5 MDEV-7620: Transaction lock wait is missing number of lock
waits and total wait time.
2015-02-23 11:24:19 +02:00
Sergei Golubchik
f2cb45daf3 Merge remote-tracking branch 'origin/10.0' into 10.0 2015-02-22 21:45:24 +01:00
Jan Lindström
1cc7befc14 MDEV-7109: Add support for INFORMATION_SCHEMA.INNODB_SEMAPHORE_WAITS
MDEV-7399: Add support for INFORMATION_SCHEMA.INNODB_MUTEXES
    MDEV-7618: Improve semaphore instrumentation

    Introduced two new information schema tables to monitor mutex waits
    and semaphore waits. Added a new configuration variable
    innodb_intrument_semaphores to add thread_id, file name and
    line of current holder of mutex/rw_lock.
2015-02-21 21:45:16 +02:00
Jan Lindström
9152b83973 Merged from 10.0-FusionIO:
Added support for compression method snappy for page compression.
2015-02-19 17:42:18 +02:00
Sergei Golubchik
6b05688f6d innodb 5.6.23 2015-02-18 17:59:21 +01:00
Sergei Golubchik
d7e7862364 Merge branch '5.5' into 10.0 2015-02-18 15:16:27 +01:00
Jan Lindström
4040bf18cf MDEV-7593: Default encryption key does not work correctly for page
encrypted tables

Introduced a new innodb_default_page_encryption_key configuration
variable to allow user to set the default key identifier.
2015-02-18 08:32:01 +02:00
Sergei Golubchik
a0e93bceb7 innodb/xtradb: update nonnull attributes to match the new semantics 2015-02-17 23:49:02 +01:00
Jan Lindström
a1a32f8f25 Revert file space allocation change on row0merge.cc. 2015-02-18 06:59:28 +02:00