mariadb/extra
Marko Mäkelä d08d750b21 MDEV-36024: Redesign innodb_encrypt_log=ON
The innodb_encrypt_log=ON subformat of FORMAT_10_8 is inefficient,
because a new encryption or decryption context is being set up for
every log record payload snippet.

An in-place conversion between the old and new innodb_encrypt_log=ON
format is technically possible. No such conversion has been
implemented, though. There is some overhead with respect to the
unencrypted format (innodb_encrypt_log=OFF): At the end of each
mini-transaction, right before the CRC-32C, additional 8 bytes will be
reserved for a nonce (really, log_sys.get_flushed_lsn()), which forms
a part of an initialization vector.

log_t::FORMAT_ENC_11: The new format identifier, a UTF-8 encoding of
🗝 U+1F5DD OLD KEY (encryption). In this format, everything except the
types and lengths of log records will be encrypted. Thus, unlike in
FORMAT_10_8, also page identifiers and FILE_ records will be encrypted.
The initialization vector (IV) consists of the 8-byte nonce as well as
the type and length byte(s) of the first record of the mini-transaction.
Page identifiers will no longer form any part of the IV.

The old log_t::FORMAT_ENC_10_8 (innodb_encrypt_log=ON) will be supported
both by mariadb-backup and by crash recovery. Downgrade from the new
format will only be possible if the new server has been running or
restarted with innodb_encrypt_log=OFF. If innodb_encrypt_log=ON,
only the new log_t::FORMAT_ENC_11 will be written.

log_t::is_recoverable(): A new predicate, which holds for all 3
formats.

recv_sys_t::tmp_buf: A heap-allocated buffer for decrypting a
mini-transaction, or for making the wrap-around of a memory-mapped
log file contiguous.

log_decrypt_mtr(): Decrypt a mini-transaction in recv_sys.tmp_buf.
Theoretically, when reading the log via pread() rather than a read-only
memory mapping, we could modify the contents of log_sys.buf in place.
If we did that, we would have to re-read the last log block into
log_sys.buf before resuming writes, because otherwise that block could be
re-written as a mix of old decrypted data and new encrypted data, which
would cause a subsequent recovery failure unless the log checkpoint had
been advanced beyond this point.

log_decrypt_legacy(): Decrypt a log_t::FORMAT_ENC_10_8 record snippet
on stack. Replaces recv_buf::copy_if_needed().

recv_sys_t::get_backup_parser(): Return a recv_sys_t::parser, that is,
a pointer to an instantiation of parse_mmap or parse_mtr for the current
log format.

recv_sys_t::parse_mtr(), recv_sys_t::parse_mmap(): Add a parameter
template<uint32_t> for the current log_sys.format.

log_parse_start(): Validate the CRC-32C of a mini-transaction.
This has been split from the recv_sys_t::parse() template to
reduce code duplication. These two are the only functions
that will be instantiated for both recv_buf and recv_ring.

recv_sys_t::parse(): Split into ::log_parse_start(), parse_tail() and
rewind_prepare(). Add a parameter template<uint32_t format> to specialize
for log_sys.format at compilation time.

recv_sys_t::parse_tail(): Operate on pointers to contiguous
mini-transaction data. Use a parameter template<bool ENC_10_8>
for special handling of the old innodb_encrypt_log=ON format.
The previous recv_buf::get_buf() is being inlined here.

log_crypt: Encrypt or decrypt a mini-transaction in place in the
new innodb_encrypt_log=ON format.

log_crypt::append(): Invoke encryption_ctx_update() in MY_AES_BLOCK_SIZE
(16-byte) blocks and scatter/gather shorter data blocks as needed.

log_crypt::finish(), Handle the last (possibly incomplete) block as a
special case, with ENCRYPTION_FLAG_NOPAD.

mtr_t::parse_length(): Parse the length of a log record.

mtr_t::encrypt(): Use log_crypt instead of the old log_encrypt_buf().

recv_buf::crc32c(): Add a parameter for the initial CRC-32C value.

recv_sys_t::rewind(): Operate on pointers to the start of the
mini-transaction and to the first skipped record.

recv_sys_t::trim(): Declare as ATTRIBUTE_COLD so that this rarely
invoked function will not be expanded inline in parse_tail().

recv_sys_t::parse_init(): Handle INIT_PAGE or FREE_PAGE while scanning
to the end of the log.

recv_sys_t::parse_page0(): Handle WRITE to FSP_SPACE_SIZE and
FSP_SPACE_FLAGS.

mlog_decode_varint_length(): Make use of __builtin_clz() to avoid a loop
when possible.

mlog_decode_varint(): Remove the template parameter, and
Specialize each instantiation. Define
ATTRIBUTE_NONINLINE mlog_decode_varint(const byte*) because
it is a rather large function.

more private mlog_decode
2025-08-18 12:51:45 +03:00
..
aws_sdk MDEV-30831 Cannot compile AWS KMS Plugin 2025-06-10 15:18:28 +02:00
mariabackup MDEV-36024: Redesign innodb_encrypt_log=ON 2025-08-18 12:51:45 +03:00
readline Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
wolfssl Workaround WolfSSL issue #9004 to fix the build on Windows. 2025-07-25 12:28:30 +02:00
charset2html.c Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
CMakeLists.txt MDEV-36316/MDEV-36327/MDEV-36328 Debug msan 2025-05-28 16:28:34 +10:00
comp_err.c MDEV-30810 errmsg-utf8.txt no longer uses charsets 2023-03-10 08:53:58 +11:00
innochecksum.cc Merge 10.6 into 10.11 2024-11-29 13:43:17 +02:00
my_print_defaults.c Merge 10.6 into 10.11 2024-11-29 13:43:17 +02:00
mysql_waitpid.c Added 'const' to arguments in get_one_option and find_typeset() 2021-02-08 12:16:29 +02:00
mysqld_safe_helper.c mysqld_safe_help - remove warning 2017-04-30 15:06:01 +04:00
perror.c MDEV-34714 perror-win test failure on localized Windows 2024-08-09 01:15:45 +02:00
replace.c Merge 10.4 into 10.5 2022-07-01 14:42:02 +03:00
resolve_stack_dump.c MDEV-22189: Change error messages inside code to have mariadb instead of 2021-05-24 11:38:13 +05:30
resolveip.c Added 'const' to arguments in get_one_option and find_typeset() 2021-02-08 12:16:29 +02:00