Commit graph

165008 commits

Author SHA1 Message Date
Sergei Golubchik
da06baa11f mtr: *.opt files, always enable innodb-encrypt-log
because one is supposed always to use it with innodb-encrypt-tables
2015-04-10 02:51:35 +02:00
Sergei Golubchik
dab6c83744 allow srv_encrypt_tables and ENCRYPTED=YES to be used together 2015-04-10 02:51:06 +02:00
Sergei Golubchik
bc9f118e29 rename table attribute ENCRYPTION=ON/OFF to ENCRYPTED=YES/NO 2015-04-10 02:51:01 +02:00
Sergei Golubchik
ea764f5f52 s/innodb_default_encryption_key/innodb_default_encryption_key_id/
and make it a session variable, not global, as any decent
default sysvar for a table attribute should be
2015-04-10 02:40:10 +02:00
Sergei Golubchik
92ff523619 change ENCRYPTION_KEY_ID to be HA_TOPTION_SYSVAR
instead of manually implenting "default from a sysvar" in the code
2015-04-10 02:39:36 +02:00
Sergei Golubchik
dd8f931957 be less annoying about sysvar-based table attributes
do not *always* add them to the create table definition,
but only when a sysvar value is different from a default.

also, when adding them - don't quote numbers
2015-04-10 02:36:54 +02:00
Sergei Golubchik
eb29a63e45 SET STATEMENT timestamp=xxx ....
fix sys_var->is_default() method (that was using default_val property
in a global sys_var object to track per-session state):
* move timestamp to a dedicated Sys_var_timestamp class
  (in fact, rename Sys_var_session_special_double to Sys_var_timestamp)
* make session_is_default a virtual method with a special implementation
  for timestamps
* other variables don't have a special behavior for default values
  and can have session_is_default() to be always false.
2015-04-09 19:41:59 +02:00
Sergei Golubchik
0a9052f591 Store the key id in the tablespace and read it back 2015-04-09 19:36:12 +02:00
Sergei Golubchik
97d5de4ccf Add encryption key id to the API as a distinct concept
which is separate from the encryption key version
2015-04-09 19:35:40 +02:00
Sergei Golubchik
5dffda3ccc Merge branch 'bb-10.1-jan-encryption' into bb-10.1-serg
With changes:

* update tests to pass (new encryption/encryption_key_id syntax).
* not merged the code that makes engine aware of the encryption mode
  (CRYPT_SCHEME_1_CBC, CRYPT_SCHEME_1_CTR, storing it on disk, etc),
  because now the encryption plugin is handling it.
* compression+encryption did not work in either branch before the
  merge - and it does not work after the merge. it might be more
  broken after the merge though - some of that code was not merged.
* page checksumming code was not moved (moving of page checksumming
  from fil_space_encrypt() to fil_space_decrypt was not merged).
* restored deleted lines in buf_page_get_frame(), otherwise
  innodb_scrub test failed.
2015-04-09 19:27:40 +02:00
Sergei Golubchik
129e960179 fix log_blocks_crypt() to actually decrypt the encrypted log
It used to double-encrypt it, relying on the fact that second
encrypt() call was (like XOR) negating the effect of the
first one.
2015-04-09 19:06:11 +02:00
Sergei Golubchik
d6b912c64d update XtraDB/InnoDB plugin maturity to match the server
because of encryption changes - make it beta and let
it mature together with the server
2015-04-09 18:44:02 +02:00
Sergei Golubchik
3a2ec3fca6 make innodb_encryption_debug test more robust 2015-04-09 18:43:36 +02:00
Sergei Golubchik
f130da7b00 clarify/simplify new innodb sysvars: help texts 2015-04-09 18:42:45 +02:00
Sergei Golubchik
87cf86519e clarify/simplify new innodb sysvars: innodb-scrub-force-testing
rename to innodb-debug-force-scrubbing
2015-04-09 18:42:45 +02:00
Sergei Golubchik
19e76814b7 clarify/simplify new innodb sysvars: innodb-scrub-log-interval
rename to innodb-scrub-log-speed
2015-04-09 18:42:45 +02:00
Sergei Golubchik
72c8b3fcb2 small cleanups as per review 2015-04-09 18:42:44 +02:00
Sergei Golubchik
4d40a7d133 remove now-empty my_aes.{h,cc}
move remaning defines to my_crypt, add MY_ namespace prefix
2015-04-09 18:42:44 +02:00
Sergei Golubchik
65e7826070 renames to follow single consistent naming style
with namespace prefixes
2015-04-09 18:42:44 +02:00
Sergei Golubchik
c0878f64c5 remove wrappers in encryption_keys.cc
invoke plugin methods directly
2015-04-09 18:42:43 +02:00
Sergei Golubchik
bb1b61b312 encryption plugin controls the encryption
* no --encryption-algorithm option anymore
* encrypt/decrypt methods in the encryption plugin
* ecnrypt/decrypt methods in the encryption_km service
* file_km plugin has --file-key-management-encryption-algorithm
* debug_km always uses aes_cbc
* example_km changes between aes_cbc and aes_ecb for different key versions
2015-04-09 18:42:43 +02:00
Sergei Golubchik
9ccafffc29 rename "encryption key management plugin" to "encryption plugin"
because it's going to do more than just key management
2015-04-09 18:42:43 +02:00
Sergei Golubchik
6d3dace7d9 mtr: don't disable tests in suite.pm unnecessary
only those tests should be disabled in suite.pm that
cannot be disabled from mysqltest files (e.g. when the
server is started with a special command-line option -
unknown option will abort the server before mysqltest
will start its checks).
2015-04-09 18:42:42 +02:00
Sergei Golubchik
6a3932fdd0 use key derivation procedure for all encryption algorithms 2015-04-08 10:58:51 +02:00
Sergei Golubchik
ef5b4889c2 optimize encryption api
only one encryption key lookup in most cases instead of three
(has_key, get_key_size, get_key).
2015-04-08 10:58:50 +02:00
Sergei Golubchik
c91e3260e2 tests for file_key_management plugin key file parser 2015-04-08 10:58:50 +02:00
Sergei Golubchik
e02749aaf5 completely rewrote file_key_management plugin
numerous issues fixed:
* buffer overflows
* error conditions aren't checked (crash if file doesn't exist)
* accessing random unallocated memory
* hard-coded password
* arbitrary hard-coded key id limit
* incomprehensible error messages (for key_id == 0 it reported
  "The key could not be initialized", for syntax errors the message was
  "Wrong match of the keyID, see the template", for a key id
  larger than hard-coded limit the message was "No asked key", and there
  was an error "Is comment" for a comment).
* tons of small mallocs, many are freed few lines down in the code
* malloc(N) and new char[N] are used both, even in the same function
* redundant memory copies
* pcre - "I can solve it with regular expressions" - with incorrect regexes
* parser context stored in a singleton
* keys are stored as strings and are strlen-ed and hex2bin-ed on every
  get_key() request
* lots of useless code (e.g. sprintf instead of a pointer assignment,
  checking of the file length to read a part of it in a fixed buffer,
  multiplying by sizeof(char) in many places, etc)
* this list is not exhaustive
2015-04-08 10:58:50 +02:00
Sergei Golubchik
9bda4bc52a report a plugin loading offset at dlopen time
not an address of some arbitrarily chosen symbol;
not when plugin is successfully loaded (which means - no info
  when plugin->init failed);
2015-04-08 10:58:49 +02:00
Sergei Golubchik
beea778695 copy-paste bug in service_sha1.h 2015-04-08 10:58:49 +02:00
Sergei Golubchik
32e5304bff mtr: fix testname,combination syntax to work in many-combination case
in particular, "innodb.innodb-page_encryption,xtradb" should
select these three tests:

innodb.innodb-page_encryption 'cbc,xtradb' [ pass ]  35563
innodb.innodb-page_encryption 'ctr,xtradb' [ pass ]  36858
innodb.innodb-page_encryption 'ecb,xtradb' [ pass ]  36741

and deselect all innodb_plugin tests. This was not the case, because
the %test_combs hash was destructively modified in the loop
2015-04-08 10:58:49 +02:00
Sergei Golubchik
5fcba6eb26 small cleanup
* my_aes.h doesn't compile without my_global.h
* typo in a comment
* redundant condition
* if encryption plugin fails, there's no encryption_key_manager
  at plugin deinit time
* encryption plugin tests must run when plugin.so is present,
  not when a plugin is active (otherwise the test will be skipped
  when plugin fails to initialize).
2015-04-08 10:58:49 +02:00
Sergei Golubchik
88632726e8 rename plugins to remove "_plugin" from the plugin name 2015-04-08 10:58:48 +02:00
Sergei Golubchik
817a63f273 pass the correct key length to encryption routines 2015-04-08 10:58:48 +02:00
Sergei Golubchik
2643cc545f Don't crash the server if en-/decryption fails in Aria 2015-04-08 10:58:48 +02:00
Sergei Golubchik
f379c9a820 remove Aria-only handler flag 2015-04-08 10:58:48 +02:00
Sergei Golubchik
39865322c6 remove a google specific script 2015-04-08 10:58:47 +02:00
Sergei Golubchik
7a387c0158 table->keep_row_order no longer need to force DYNAMIC_RECORD
for temporary Aria tables. It can use a faster BLOCK_RECORD
2015-04-08 10:58:47 +02:00
Sergei Golubchik
5d8dbee97b remove get_iv() from the key management plugin API
now IVs are always handled internally
2015-04-08 10:58:47 +02:00
Sergei Golubchik
c238e68d96 move debug_use_static_encryption_keys and debug_encryption_key_version to a plugin 2015-04-08 10:58:46 +02:00
Sergei Golubchik
b937574293 remove old my_aes_encrypt/decrypt
and simplify Item_func_aes_encrypt/decrypt
2015-04-08 10:58:46 +02:00
Sergei Golubchik
91f7363e4b yassl padding 2015-04-08 10:58:46 +02:00
Sergei Golubchik
f444d13a3b my_aes* functions: support for different key lengths
to: different key lengths
2015-04-08 10:58:29 +02:00
Jan Lindström
d2b87801b6 Fix test failure on these scrub tests. 2015-04-07 23:44:57 +02:00
Jan Lindström
4865fd105a InnoDB/XtraDB Encryption cleanup
Step 5:
-- Rename encryption_key -> encryption_key_id
-- Remove unnecessary code
-- Fix few bugs found
-- Fix test cases and results files
2015-04-07 23:44:57 +02:00
Jan Lindström
af768c2f22 InnoDB/XtraDB Encryption code cleanup
Step 4:
-- Review fixes
-- Rename pages_page_* to pages_*
-- Remove unnecessary code
2015-04-07 23:44:57 +02:00
Jan Lindström
71ec0463af Encryption cleanup
Step 3:

-- Make encrytion_algorithm changeable by SUPER
-- Remove AES_ECB method from encryption_algorithms
-- Support AES method change by storing used method on InnoDB/XtraDB objects
-- Store used AES method to crypt_data as different crypt types
-- Store used AES method to redo/undo logs and checkpoint
-- Store used AES method on every encrypted page after key_version
-- Add test
2015-04-07 23:44:57 +02:00
Jan Lindström
0ba9fa35bc InnoDB/XtraDB Encryption cleanup
Step 2:

-- Introduce temporal memory array to buffer pool where to allocate
temporary memory for encryption/compression
-- Rename PAGE_ENCRYPTION -> ENCRYPTION
-- Rename PAGE_ENCRYPTION_KEY -> ENCRYPTION_KEY
-- Rename innodb_default_page_encryption_key -> innodb_default_encryption_key
-- Allow enable/disable encryption for tables by changing
 ENCRYPTION to enum having values DEFAULT, ON, OFF
-- In create table store crypt_data if ENCRYPTION is ON or OFF
-- Do not crypt tablespaces having ENCRYPTION=OFF
-- Store encryption mode to crypt_data and redo-log
2015-04-07 23:44:56 +02:00
Jan Lindström
b4a4d82396 InnoDB/XtraDB Encryption cleanup.
Step 1:
-- Remove page encryption from dictionary (per table
encryption will be handled by storing crypt_data to page 0)
-- Remove encryption/compression from os0file and all functions
before that (compression will be added to buf0buf.cc)
-- Use same CRYPT_SCHEME_1 for all encryption methods
-- Do some code cleanups to confort InnoDB coding style
2015-04-07 23:44:56 +02:00
Sergei Golubchik
3bbe2057da yassl support 2015-04-05 13:14:38 +02:00
Sergei Golubchik
2f8d101f9e unify my_{en|de}crypt_{cbc|ecb|ctr}. no yassl support yet. 2015-04-05 13:14:37 +02:00