mirror of
https://github.com/MariaDB/server.git
synced 2025-09-10 11:30:17 +02:00
cc277a7d24
The innodb_encrypt_log=ON subformat of FORMAT_10_8 is inefficient, because a new encryption or decryption context is being set up for every log record payload snippet. An in-place conversion between the old and new innodb_encrypt_log=ON format is technically possible. No such conversion has been implemented, though. There is some overhead with respect to the unencrypted format (innodb_encrypt_log=OFF): At the end of each mini-transaction, right before the CRC-32C, additional 8 bytes will be reserved for a nonce (really, log_sys.get_flushed_lsn()), which forms a part of an initialization vector. log_t::FORMAT_ENC_11: The new format identifier, a UTF-8 encoding of 🗝 U+1F5DD OLD KEY (encryption). In this format, everything except the types and lengths of log records will be encrypted. Thus, unlike in FORMAT_10_8, also page identifiers and FILE_ records will be encrypted. The initialization vector (IV) consists of the 8-byte nonce as well as the type and length byte(s) of the first record of the mini-transaction. Page identifiers will no longer form any part of the IV. The old log_t::FORMAT_ENC_10_8 (innodb_encrypt_log=ON) will be supported both by mariadb-backup and by crash recovery. Downgrade from the new format will only be possible if the new server has been running or restarted with innodb_encrypt_log=OFF. If innodb_encrypt_log=ON, only the new log_t::FORMAT_ENC_11 will be written. log_t::is_recoverable(): A new predicate, which holds for all 3 formats. recv_sys_t::tmp_buf: A heap-allocated buffer for decrypting a mini-transaction, or for making the wrap-around of a memory-mapped log file contiguous. recv_sys_t::start_lsn: The start of the mini-transaction. Updated at the start of parse_tail(). log_decrypt_mtr(): Decrypt a mini-transaction in recv_sys.tmp_buf. Theoretically, when reading the log via pread() rather than a read-only memory mapping, we could modify the contents of log_sys.buf in place. If we did that, we would have to re-read the last log block into log_sys.buf before resuming writes, because otherwise that block could be re-written as a mix of old decrypted data and new encrypted data, which would cause a subsequent recovery failure unless the log checkpoint had been advanced beyond this point. log_decrypt_legacy(): Decrypt a log_t::FORMAT_ENC_10_8 record snippet on stack. Replaces recv_buf::copy_if_needed(). recv_sys_t::get_backup_parser(): Return a recv_sys_t::parser, that is, a pointer to an instantiation of parse_mmap or parse_mtr for the current log format. recv_sys_t::parse_mtr(), recv_sys_t::parse_mmap(): Add a parameter template<uint32_t> for the current log_sys.format. log_parse_start(): Validate the CRC-32C of a mini-transaction. This has been split from the recv_sys_t::parse() template to reduce code duplication. These two are the lowest-level functions that will be instantiated for both recv_buf and recv_ring. recv_sys_t::parse(): Split into ::log_parse_start() and parse_tail(). Add a parameter template<uint32_t format> to specialize for log_sys.format at compilation time. recv_sys_t::parse_tail(): Operate on pointers to contiguous mini-transaction data. Use a parameter template<bool ENC_10_8> for special handling of the old innodb_encrypt_log=ON format. The former recv_buf::get_buf() is being inlined here. Much of the logic is split into non-inline functions, to avoid duplicating a lot of code for every template expansion. log_crypt: Encrypt or decrypt a mini-transaction in place in the new innodb_encrypt_log=ON format. We will use temporary buffers so that encryption_ctx_update() can be invoked on integer multiples of MY_AES_BLOCK_SIZE, except for the last bytes of the encrypted payload, which will be encrypted or decrypted in place thanks to ENCRYPTION_FLAG_NOPAD. log_crypt::append(): Invoke encryption_ctx_update() in MY_AES_BLOCK_SIZE (16-byte) blocks and scatter/gather shorter data blocks as needed. log_crypt::finish(), Handle the last (possibly incomplete) block as a special case, with ENCRYPTION_FLAG_NOPAD. mtr_t::parse_length(): Parse the length of a log record. mtr_t::encrypt(): Use log_crypt instead of the old log_encrypt_buf(). recv_buf::crc32c(): Add a parameter for the initial CRC-32C value. recv_sys_t::rewind(): Operate on pointers to the start of the mini-transaction and to the first skipped record. recv_sys_t::trim(): Declare as ATTRIBUTE_COLD so that this rarely invoked function will not be expanded inline in parse_tail(). recv_sys_t::parse_init(): Handle INIT_PAGE or FREE_PAGE while scanning to the end of the log. recv_sys_t::parse_page0(): Handle WRITE to FSP_SPACE_SIZE and FSP_SPACE_FLAGS. recv_sys_t::parse_store_if_exists(), recv_sys_t::parse_store(), recv_sys_t::parse_oom(): Handle page-level log records. mlog_decode_varint_length(): Make use of __builtin_clz() to avoid a loop when possible. mlog_decode_varint(): Define only on const byte*, as ATTRIBUTE_NOINLINE static because it is a rather large function. recv_buf::decode_varint(): Trivial wrapper for mlog_decode_varint(). recv_ring::decode_varint(): Special implementation. log_page_modify(): Note that a page will be modified in recovery. Split from recv_sys_t::parse_tail(). log_parse_file(): Handle non-page log records. log_record_corrupted(), log_unknown(), log_page_id_corrupted(): Common error reporting functions. |
||
|---|---|---|
| .. | ||
| aria_tiny.test | ||
| bulk_insert.opt | ||
| bulk_insert.test | ||
| compressed_import_tablespace.opt | ||
| compressed_import_tablespace.test | ||
| corrupted_during_recovery.combinations | ||
| corrupted_during_recovery.test | ||
| create_or_replace.opt | ||
| create_or_replace.test | ||
| create_or_replace_big.opt | ||
| create_or_replace_big.test | ||
| debug_key_management.opt | ||
| debug_key_management.test | ||
| doublewrite_debug.opt | ||
| doublewrite_debug.test | ||
| encrypt_and_grep.opt | ||
| encrypt_and_grep.test | ||
| encryption_force.opt | ||
| encryption_force.test | ||
| encryption_key_corruption.combinations | ||
| encryption_key_corruption.opt | ||
| encryption_key_corruption.test | ||
| file_creation.opt | ||
| file_creation.test | ||
| filekeys-data-too-long.key | ||
| filekeys-data.enc | ||
| filekeys-data.key | ||
| filekeys-tooshort.enc | ||
| filekeys_badtest.inc | ||
| filekeys_emptyfile.opt | ||
| filekeys_emptyfile.test | ||
| filekeys_encfile.opt | ||
| filekeys_encfile.test | ||
| filekeys_encfile_bad.opt | ||
| filekeys_encfile_bad.test | ||
| filekeys_encfile_badfile.opt | ||
| filekeys_encfile_badfile.test | ||
| filekeys_encfile_file.opt | ||
| filekeys_encfile_file.test | ||
| filekeys_encfile_no.opt | ||
| filekeys_encfile_no.test | ||
| filekeys_goodtest.inc | ||
| filekeys_nofile.test | ||
| filekeys_plugin.inc | ||
| filekeys_plugin.opt | ||
| filekeys_plugin_exists.inc | ||
| filekeys_secret_openssl_rand_128bits.enc | ||
| filekeys_secret_openssl_rand_128bits.key | ||
| filekeys_secret_openssl_rand_128bits.opt | ||
| filekeys_secret_openssl_rand_128bits.test | ||
| filekeys_secret_too_long.opt | ||
| filekeys_secret_too_long.test | ||
| filekeys_syntax.opt | ||
| filekeys_syntax.test | ||
| filekeys_tooshort.opt | ||
| filekeys_tooshort.test | ||
| filekeys_unencfile.opt | ||
| filekeys_unencfile.test | ||
| import_4k.opt | ||
| import_4k.test | ||
| innochecksum.opt | ||
| innochecksum.test | ||
| innodb-bad-key-change.combinations | ||
| innodb-bad-key-change.opt | ||
| innodb-bad-key-change.test | ||
| innodb-bad-key-change2.combinations | ||
| innodb-bad-key-change2.opt | ||
| innodb-bad-key-change2.test | ||
| innodb-bad-key-change3.combinations | ||
| innodb-bad-key-change3.test | ||
| innodb-bad-key-change4.combinations | ||
| innodb-bad-key-change4.opt | ||
| innodb-bad-key-change4.test | ||
| innodb-checksum-algorithm.test | ||
| innodb-compressed-blob.opt | ||
| innodb-compressed-blob.test | ||
| innodb-discard-import-change.combinations | ||
| innodb-discard-import-change.test | ||
| innodb-discard-import.combinations | ||
| innodb-discard-import.test | ||
| innodb-encr-threads.test | ||
| innodb-encryption-alter.test | ||
| innodb-encryption-disable.opt | ||
| innodb-encryption-disable.test | ||
| innodb-first-page-read.opt | ||
| innodb-force-corrupt.opt | ||
| innodb-force-corrupt.test | ||
| innodb-key-rotation-disable.opt | ||
| innodb-key-rotation-disable.test | ||
| innodb-missing-key.opt | ||
| innodb-missing-key.test | ||
| innodb-page_encryption-32k.opt | ||
| innodb-page_encryption-32k.test | ||
| innodb-page_encryption.test | ||
| innodb-page_encryption_compression.test | ||
| innodb-page_encryption_log_encryption.opt | ||
| innodb-page_encryption_log_encryption.test | ||
| innodb-read-only.opt | ||
| innodb-read-only.test | ||
| innodb-redo-badkey.combinations | ||
| innodb-redo-badkey.opt | ||
| innodb-redo-badkey.test | ||
| innodb-redo-nokeys.opt | ||
| innodb-redo-nokeys.test | ||
| innodb-remove-encryption.test | ||
| innodb-spatial-index.opt | ||
| innodb-spatial-index.test | ||
| innodb_encrypt_freed.opt | ||
| innodb_encrypt_freed.test | ||
| innodb_encrypt_key_rotation_age.opt | ||
| innodb_encrypt_key_rotation_age.test | ||
| innodb_encrypt_log.opt | ||
| innodb_encrypt_log.test | ||
| innodb_encrypt_log_corruption.opt | ||
| innodb_encrypt_log_corruption.test | ||
| innodb_encrypt_temporary_tables.combinations | ||
| innodb_encrypt_temporary_tables.opt | ||
| innodb_encrypt_temporary_tables.test | ||
| innodb_encryption-page-compression.opt | ||
| innodb_encryption-page-compression.test | ||
| innodb_encryption.opt | ||
| innodb_encryption.test | ||
| innodb_encryption_discard_import.opt | ||
| innodb_encryption_discard_import.test | ||
| innodb_encryption_filekeys.opt | ||
| innodb_encryption_filekeys.test | ||
| innodb_encryption_is.opt | ||
| innodb_encryption_is.test | ||
| innodb_encryption_row_compressed.opt | ||
| innodb_encryption_row_compressed.test | ||
| innodb_encryption_tables.opt | ||
| innodb_encryption_tables.test | ||
| innodb_first_page.opt | ||
| innodb_first_page.test | ||
| innodb_import.combinations | ||
| innodb_import.opt | ||
| innodb_import.test | ||
| innodb_onlinealter_encryption.opt | ||
| innodb_onlinealter_encryption.test | ||
| innodb_page_encryption_key_change.test | ||
| instant_alter_import.test | ||
| key_version_rotation.opt | ||
| key_version_rotation.test | ||
| recovery_memory.opt | ||
| recovery_memory.test | ||
| second_plugin-12863.test | ||
| tempfiles.combinations | ||
| tempfiles.opt | ||
| tempfiles.test | ||
| tempfiles_encrypted.opt | ||
| tempfiles_encrypted.test | ||